[00:51] <hallyn> xnox: sadly, that didn't actually help
[00:51] <xnox> hallyn: =/ oh well.
[00:52] <xnox> hallyn: i can try to dig into it, but deffinatly not tonight.
[00:52] <hallyn> now really i expected it to make it worse (since noone is using it :)
[00:53] <hallyn> xnox: np, it's late there - ttyl
[00:53] <hallyn> thanks! gn
[00:53] <ParkerR> Or as another alternative... is the kernel in the new builds managed by a deb file?
[00:55] <ParkerR> *new touch builds
[11:38] <apw> ParkerR, the kernel is built in a .deb yes though typically upgrades in touch images are not done directly
[11:39] <ogra_> the kernel deb is only used by the android package 
[11:39] <ogra_> (which generates boot.img and system.img (the latter has the modules))
[11:44] <ogra_> just on a sidenote, forcing performance from initrd on boot on mako gains exactly nothing :(
[11:55] <apw> ogra_, worth a try
[11:55] <ogra_> yeah
[11:56] <ogra_> it actually takes a few milliseconds longer with performance ... i guess thats due to the echoing "performance" into sysfs
[12:13] <ogra_> AHA !
[12:13] <ogra_> root@ubuntu-phablet:/# ls -l /var/lib/ureadahead/debugfs/
[12:13] <ogra_> total 0
[12:13] <ogra_> root@ubuntu-phablet:/# mount | grep debugfs                              
[12:13] <ogra_> none on /sys/kernel/debug type debugfs (rw,relatime)
[12:13] <ogra_> so i guess thats why ureadahead doesnt work anymore
[12:13] <apw> aha?
[12:14] <ogra_> well, ureadahead reads from /var/lib/ureadahead/debugfs/
[12:14] <ogra_> doesnt it ?
[12:14] <apw> isn't that normal?  doesn't ureadahead mount it temporarily on its own mointpoint during its run and unmount it after?
[12:15] <ogra_> hmm
[12:15] <apw> as it may need it before it is mounted by the system
[12:16] <ogra_> hmm
[12:17] <apw> ogra_, so actually it uses /sys if it is mounted, else it mounts its own
[12:17] <ogra_> ah, k 
[12:21] <apw> ogra_, is there a bug for this ?
[12:22] <ogra_> nope, still researching 
[12:22] <ogra_> funnily it seems ot have profiled something at some point 
[12:22] <ogra_> *to
[12:22] <ogra_> but it is never executed on boot after that 
[12:22] <apw> is the packs directory writable ?
[12:23] <apw> or ... the place that the "reprofile" marker is stored
[12:23] <ogra_> yes
[12:23] <ogra_> i see packs that have been generated on first boot 
[12:23] <ogra_> http://paste.ubuntu.com/7113774/
[12:23] <ogra_> even specoific ones for the container 
[12:24] <apw> have you tried just running it to see if it works?
[12:24] <ogra_> it doesnt 
[12:24] <ogra_> root@ubuntu-phablet:/# start ureadahead
[12:24] <ogra_> ureadahead stop/waiting
[12:25] <ogra_> callong "ureadahead --daemon" directly immediately returns ... i dont see it in the processlist
[12:33] <apw> ogra_, and without --daemon and with --verbose ?
[12:34] <ogra_> looks ok http://paste.ubuntu.com/7113819/
[12:36] <apw> ogra_, ok so you don't see it because it only takes 0.02s to do the readahead, as this is flash it is not that supprising
[12:37] <ogra_> it would still show up in a bootchart 
[12:37] <ogra_> (it used to)
[12:37] <apw> so that implies it is not starting, which says the start condition is wrong
[12:38] <apw> i assume we ahve a mountall job still ?
[12:40] <apw> ogra_, ^
[12:40] <ogra_> yep
[12:41] <ogra_> http://people.canonical.com/~ogra/touch-bootcharts/ubuntu-phablet-trusty-243.png
[12:44] <zequence> infinity: No update to the precise kernel?
[12:45] <apw> ogra_, there is a level at which things get collapsed when they are tooo short
[12:45] <ogra_> hmm
[12:46] <apw> which you can turn off when generating hte graph
[12:47] <ogra_> ah, there is a HZ= setting 
[12:48]  * ogra_ sets it from 25 to 50Hz
[12:54] <ogra_> so that gets me a lot more small processes listed (and makes the tgz of the bootchart twice as big) ... but still no trace of ureadahead ... 
[12:55] <ogra_> now i wonder ... 
[12:55] <apw> ogra_, could still be tooo small to see i be reconing
[12:56] <ogra_> yes, the point is that the interesting pack files arent the default pack 
[12:56] <ogra_> (see my ls output of /var/lib/ureadahead above) 
[12:56] <apw> arn't those loaded by the other job?
[12:56] <ogra_> and these mountpoinnts arent handled by mountall 
[12:56] <ogra_> but before in the initrd
[12:57] <ogra_> i.e. they are already mounted when mountall starts 
[12:57] <apw> ok then they won't get loaded anyhow
[12:57] <ogra_> start on mounted DEVICE=[/UL]* MOUNTPOINT=/?*
[12:57] <apw> see /etc/init/ureadahead-other.conf which loads them based on the mount announcements
[12:58] <apw> so i suspect you want your own job which runs ureadahead against the special mounts
[12:58] <ogra_> so the question is, does mountall emit "mounted" for already mounted devices in fstab
[12:58] <ogra_> jodh, do you know ?^^^
[12:59] <apw> i would doubt it, you can look at the emitted events right?
[12:59] <ogra_> i think we dropped "initrcl events" at some point 
[13:00] <apw> "The  mounted event is generated by the mountall(8) daemon after it has mounted a filesystem."
[13:00] <apw> i read that as for things it has mounted itself only
[13:01] <ogra_> bah, k 
[13:01] <ogra_> so i need something that loops through fstab and calls ureadahead for each mountpoint regardless
[13:01] <ogra_> and starts on startup or so 
[13:01] <apw> you know your special ones right?
[13:01] <apw> they are always the same phone wise ?
[13:02] <ogra_> well, i guess we want everything but rootfs
[13:02] <ogra_> no, but fstab is assembled by initrd on boot 
[13:02] <ogra_> (on every boot)
[13:24] <ogra_> bah, and now i killed my remote device :(
[13:53] <jodh> ogra_: yep, apw is correct.
[13:55]  * ogra_ tries ureadahead-other with a hardcoded value for MOUNTPOINT ... but that doesnt seem to work either  
[13:57] <apw> ?
[13:57] <apw> what did you put in there
[13:57] <apw> ogra_, ^
[13:57] <ogra_> /android/system
[13:58] <ogra_> my main concern is to get the container up faster (since most of the other bits have to wait for it)
[13:59] <ogra_> i made the ureadahead-other job start on "starting mountall" and replaced the $MOUNTPOINT in the exec line with the path 
[14:00] <ogra_> i was assuming this should just work 
[14:00] <ogra_> but i neither see it in the bootchart nor do i notice any speedup ... 
[14:06] <apw> perhaps boot it --debug on the kernle command line to see if it runs
[14:07] <jodh> ogra_: try adding '--force-trace'
[14:07] <jodh> ogra_: also, what's in /var/log/upstart/ureadahead-other.log ?
[14:07] <ogra_> jodh, but that will create a new pack ... i have the packs from first boot already
[14:07] <ogra_> i just want it to make use of them :) 
[14:08] <jodh> ogra_: on those new mountpoints?
[14:08] <ogra_> i emptied the log a few boots ago (before i started hacking the job) and there is now nothing in it 
[14:08] <ogra_> root@ubuntu-phablet:/# ls /var/lib/ureadahead/android*
[14:08] <ogra_> /var/lib/ureadahead/android.firmware.pack
[14:08] <ogra_> /var/lib/ureadahead/android.persist.pack
[14:08] <ogra_> /var/lib/ureadahead/android.system.pack
[14:09] <ogra_> so there is one pack for /android/system ... if i read that correctly 
[14:09] <jodh> ogra_: ok, have you added --verbose atleast then to get some output?
[14:09] <ogra_> not yet 
[14:09] <ogra_> will do that now
[14:09]  * ogra_ reboots with --verbose
[14:11] <ogra_> root@ubuntu-phablet:/# cat /var/log/upstart/ureadahead-other.log 
[14:11] <ogra_> nothing ...
[14:12]  * ogra_ moves the start condition to "started mountall"
[14:12] <ogra_> still no log 
[14:12] <ogra_> aha
[14:13] <ogra_> manually starting it produces something in the log
[14:13] <ogra_> http://paste.ubuntu.com/7114261/
[14:13] <ogra_> smells like mountall is never emitting "started" or so 
[14:14]  * ogra_ moves to "start on startup" 
[14:14] <ogra_> lets see
[14:15] <ogra_> funny, nothing new in the log
[14:15] <jodh> ogra_: is ureadahead-other.log empty or non-existent?
[14:15] <jodh> ogra_: try running "sudo initctl notify-disk-writeable" and check the log again
[14:15] <ogra_> it was empty (because i did "echo > ... " ) ... 
[14:15] <ogra_> it got conent when i manually started the job 
[14:16] <jodh> ogra_: that's cheating :)
[14:16] <ogra_> ok, got it 
[14:16] <ogra_> "start on filesystem" works 
[14:16] <ogra_> now it writes to the log on every boot
[14:17]  * ogra_ dropes the --verbose and creates a bootchart 
[14:17] <ogra_> *drops
[14:18] <ogra_> i dont get why "start on startup" didnt work though 
[14:18] <apw> maybe you don't have the ureadahead daemon availabe then
[14:19] <apw> and it would want to write to teh ro filessytems anyhow
[14:19] <ogra_> nope
[14:20] <ogra_> the filesystem is assembled from initrd
[14:20] <ogra_> it is fully set up before we even get to the rootfs
[14:20] <ogra_> with all writable bits 
[14:20] <ogra_> and now i see ureadahead in my bootchart \o/
[14:20] <ogra_> finally
[14:21] <ogra_> but it doesnt speed up anything :(
[14:22] <ogra_> ah, no, it gains me .5 sec 
[14:22] <ogra_> lxc-start runs half a sec earlier now
[14:23] <ogra_> what a great gain for half a day of work :P
[14:27] <apw> ogra_, thats pretty good :)
[14:29] <ogra_> well, lets see, i guess i can write a loop script that processes all packs (apart from the default one)
[14:29] <ogra_> that might in the end get me 2sec then :P
[14:47] <apw> yeah it might
[14:54] <jsalisbury> **
[14:54] <jsalisbury> ** Ubuntu Kernel Team Meeting - Today @ 17:00 UTC - #ubuntu-meeting
[14:54] <jsalisbury> **
[14:58] <mdeslaur> cking: so is thermald a dependency of something? what pulls it in? should I install it?
[14:59] <cking> mdeslaur, just install it
[14:59] <mdeslaur> cking: is something going to install it by default on our images?
[14:59] <cking> for t+1 we will make it a dependency on the kernel
[14:59] <cking> but for now it's an opt-in
[15:00] <cking> it still is a "maturing" daemon
[15:00] <mdeslaur> cking: ok, thanks!
[15:09] <apw> hallyn, ok this is occuring because you have to have read/execute access to all layers in a directory "sandwich" to read the directory
[15:10] <apw> hallyn, i am trying to figure out if this is an oversite of a deliberate decision
[15:13] <hallyn> apw: yeah it seemed possible to be on purpose, but i couldn't find where in the code itw as doing it
[15:15] <apw> hallyn, you are hitting the opendir behaviour there, where it opens each dir in each layer and merges the contents to make the virtual dir you see
[15:15] <apw> so if you make the bottom directory rx the you can see in there as well
[15:16] <apw> and i asusme the under in your scenario is empty as well
[15:18] <apw> hallyn, and the permissions check is specifically checking all layers, it is not an accident
[15:18] <apw> hallyn, it has a little loop for when we are talking about a read from a directory
[15:21] <apw> hallyn, i assume this comes from using an unmodified directory image as a lower level here ?
[15:21] <hallyn> apw: it may nto be an accident, but is it worthwhile?
[15:21] <hallyn> yes
[15:21] <hallyn> overlayfs doesn't do it...
[15:22] <apw> overlayfs is much less mature, so that tells me little about which is right
[15:24] <apw> what is it stopping you doing
[15:24] <hallyn> apw: juju creates a clone of a stock ubuntu image, then wants to chown ubuntu: /etc/ssl/private
[15:25] <hallyn> apw: thinking sensibly about the behavior, I really think aufs is the one in the wrong
[15:25] <apw> i think you need to justify that
[15:25] <hallyn> sure,
[15:25] <hallyn> to do the chown you have to be root in the first place,
[15:26] <hallyn> you can't end up corrupting the underlyign fs, you can only 'leak' data through the upper fs,
[15:26] <hallyn> but since you're root, you *could* just as well do it to the underlying fs.  there is no reasonable case i can think of where you werne't meant to 'leak' the data in the file you're chowning
[15:30] <infinity> zequence: Nope.
[15:43] <apw> hallyn, ok that needs some thinking about and upstream discussion i am sure, i'll see if i can fix it as a basis or that discussion
[15:48] <hallyn> apw: ok, thanks.  i'll watch for the thread.
[15:49] <hallyn> apw: if upstream nacks it, then i guess we'll just have to issue a warning somewhere in lxc
[15:49] <hallyn> (or drop aufs from lxc clone support - stgraber :)
[16:19] <infinity> hallyn: Well, juju could hack around the issue with 'cp -a /etc/ssl/private /etc/ssl/private.new && rm -r /etc/ssl/private && mv /etc/ssl/private.new /etc/ssl/private'
[16:19] <infinity> With a chown in there somewhere.
[16:19] <infinity> Not that that's ideal, but would get you over the hump for now.
[16:19] <hallyn> infinity: for that particular case, yeah.  but if we're goign to suggest that aufs is usable for lxc clones, that's not sufficient
[16:20] <hallyn> like i say every backing store has its deficiencies for clones, but that's just not usable
[16:20] <infinity> hallyn: Not usable is a bit extreme.  How often does one actually try to do that (traverse a directory they can't read)?
[16:21] <infinity> Is that chown happening in an unprivileged container, or outside?
[16:21] <infinity> (ie: do you have real root?)
[16:21] <infinity> If you have real root, I agree that the behaviour seems entirely wrong.
[16:21] <infinity> If you don't, I'm not sure how you're reading it at all, seems like a different bug being leveraged. :P
[16:22] <hallyn> infinity: i don't think real root matters.
[16:22] <hallyn> if i create a rootfs,
[16:22] <hallyn> then create an aufs clone container from that unprivileged, it shoudl be the same situation
[16:22] <hallyn> uid 100000 (my root) chowns a dir to uid 1001000,
[16:22] <hallyn> but uid 1001000 can't read it despite owning it
[16:23] <hallyn> maybe instead of not suable i should have said not supportable :)
[16:24] <hallyn> (as in we'll get tons of weird bug reports)
[16:24] <infinity> I'm struggling to even find a 750 directory in my schroots to experiment with. :P
[16:25] <infinity> Ahh, hello /var/cache/ldconfig
[16:26] <infinity> Okay, yeahp, has nothing to do with containers at all, that's what I wanted to make sure of.
[16:26] <infinity> (trusty-amd64)adconrad@cthulhu:/var/cache$ ls -al | grep ldconfig
[16:26] <infinity> drwx------  2 adconrad adconrad   60 Mar 18 10:25 ldconfig
[16:26] <infinity> (trusty-amd64)adconrad@cthulhu:/var/cache$ ls ldconfig/
[16:26] <infinity> ls: cannot open directory ldconfig/: Permission denied
[16:26] <infinity> That seems fundamentally broken to me.
[16:27] <hallyn> the only way it would be needed is if aufs wrongly allowed non-root to chown the dir
[16:27] <hallyn> or, does aufs support some fancy uid translation gorp?  /me checks
[16:28] <hallyn> not sure waht the 'different uid/gid/permission' in mount.aufs manpage is referring to
[16:53] <apw> most of the docs are in janglish
[16:54] <jsalisbury> ##
[16:54] <jsalisbury> ## Ubuntu Kernel Team Meeting - in 5 minutes - #ubuntu-meeting
[16:54] <jsalisbury> ##
[17:06]  * ppisati -> chest press
[17:50] <UserError> Why is the Lustre FS in staging?
[17:51] <UserError> It can be used from userspace and none of the desktop or notebook users will ever use it. You have a greater chance of someone using the tdfx support.
[17:51] <ogra_> why do you ask downstream ? 
[17:52] <ogra_> thats a question for the people that decided to put it in stagint (the linux kernel ML for example)
[17:52] <UserError> Ubuntu has made additions and subtractions
[17:52] <UserError> Don't play that game.
[17:53] <UserError> This is pure bullshit bloat for something that should be in userspace on non-niche distros
[18:03] <apw> UserError, why are we worrying about 1.3M ?
[18:03] <UserError> 10MB
[18:04] <UserError> for a feature no-one will ever use and will also be in VM kernels by default
[18:04] <apw> i can only find one module, which is 1.3M, and why would it be in VMs
[18:04] <UserError> according to the latest staging on 3.13.5
[18:04] <UserError> it is nearing 10MB on 64bit
[18:05] <UserError> which is the only one worth using since 32bit still targets i686
[18:06] <UserError> apw, Ubuntu stated the reason it wasn't including that vpadu stuff was because of ISO size
[18:06] <UserError> So i started figuring out that the reasoning was bullshit
[18:06] <UserError> By going through every single file that was useless. Now i'm on the kernel.
[18:07] <UserError> Way to keep PandaBoard and Tegra shit in an x86 FS
[18:07] <UserError> and exynos in the kernel
[18:12] <apw> so build your own kernel with what you want in it
[18:13] <apw> not that the size of them in the filesystem tells you waht they are on the iso
[18:14] <UserError> Ignore the bloat behind the curtain until you make your own curtain
[18:14] <UserError> Nevermind that Exynos = ARM
[18:15] <apw> it is a trade off between the greatest usability against the effort to work out what is completely unusable
[18:15] <UserError> It isn't usability if it is impossible to use
[18:15] <UserError> Please, please use an x86 kernel on a exynos SoC
[18:16] <apw> if the config system says something is usable, turning it on is therefore the default position, if the thing is actually configurable but useless would require individual items to be investigated
[18:16] <apw> so i am sure we have a number of things turned on which cannot be used, due to the lack of correct dependancies
[18:16] <UserError> ARM =/= X86
[18:17] <UserError> what is the problem
[18:17] <UserError> Nothing is going to change that
[18:17] <apw> you are not hearing me, i am telling you how one ends up in this position
[18:17] <apw> you may have found a clear case indeed and if someone files a bug for it, we may indeed turn it off
[18:18] <apw> i am saying that these cases are not clear wthout investigation
[18:18] <UserError> Ok fine, FS wise, off the top of my head
[18:18] <UserError> alsa package
[18:18] <UserError> kernel wise, hw arch exynos
[18:19] <UserError> Also some TI stuff in there i have the path to somewhere
[18:20] <apw> yep and if someone has the configuration options and the justificaiton, and puts them in a bug
[18:20] <apw> there is a high probability it will get looked at and turned off
[18:21] <UserError> Right but the flaw is the fact that it could've been added from the beginning when the same SoC is powering the phones of the people who should know better
[18:21] <apw> when every kernel update has hundreds of new options one has to have a default position
[18:21] <apw> and that is to turn on things which the config system says are applicable to an arch
[18:22] <apw> and if we have 100 engineers looking at the config then it would likely be perfect
[18:23] <UserError> you could literally just use grep on the config with the major arm licensee models
[18:23] <apw> i could go and save space on a machine whiich on average doesn't care
[18:23] <apw> or i could go and work on some new feature we need
[18:23] <apw> which is it
[18:24] <UserError> right, which is going to harm the idea of ubuntu touch and other modile ideas
[18:24] <apw> nope they have non-generic kernels which are much much smaller
[18:24] <UserError> when you are having that many writes on every security update
[18:24] <UserError> every hyrbid device with similar nand/emmc to a phone
[18:25] <UserError> all those writes
[18:25] <apw> and touch devices have a minimal config, so this does not apply
[18:25] <UserError> all the VMs and appliance containers
[18:25] <UserError> you assume they are minimal, i beg to differ
[18:25] <apw> and the VMs do not have the entier kernel installed
[18:25] <apw> there is a split, between the core bits needed on a virtual system, and the rest
[18:26] <UserError> you're right, they only have joystick nonsense by default
[18:26] <UserError> and sound
[18:26] <apw> i am sure they have some bits they should not indeed, but they are less than a third of the size of the full ones
[18:27] <apw> and if you have some concrete config changes you thing are viable, feel free to propose them
[18:27] <apw> along with a testing plan
[18:27] <UserError> The thing is that this has a commutative impact
[18:28] <UserError> what is the launchpad for this
[18:28] <apw> i asume you are asking what package to file bugs against, that would be 'linux'
[18:29] <apw> but "your package is crap" isn't going to get much attention, concrete changes would likely
[18:29] <UserError> pretty much removal
[18:29] <UserError> the package is crap
[18:30] <UserError> i went through every file on sunday
[18:32] <apw> what is vpadu
[18:33] <UserError> vdpau*
[18:33] <UserError> http://www.phoronix.com/scan.php?page=news_item&px=MTYwNzU
[18:34] <UserError> "too big"
[18:34] <UserError> yet the kernel can grow by tens of MB in a day
[18:35] <UserError> yet panda and tegra can just chill in alsa never to be used ever by anyone
[18:35] <apw> frankly though i may even agree that we have a lot of bloat, your attitude is getting to me
[18:35]  * ogra_ wonders if UserError is like that in real life too 
[18:40] <apw> i dunno, i am nicer on irc than in real life in general
[18:41] <ogra_> heh
[18:42] <apw> to which i am sure you can attest :)
[18:42] <ogra_> well, when i meet you in person you are usually drunk already :P 
[18:43] <apw> i tend to mellow after enough of that
[18:43] <ogra_> (but i'm usually in the same state) 
[18:43] <UserError> Is there any plan to make the 32bit kernel or X11 / toolkit / MIR visual packages useful by targeting anything other than the PentiumPro platform with zero optimizations for hardware that actually fits the ram and CPU minimums? 
[18:43] <infinity> UserError: Why would we build a 32-bit kernel that doesn't run on 32-bit hardware?
[18:44] <infinity> (Hint, you can install the amd64 kernel with a 32-bit userspace, if that's what you're after)
[18:44] <UserError> i686 with a single chipset that supports a theoretical minimum on one hand is insanity 
[18:44] <UserError> not what i am talking about
[18:45] <UserError> If the recommended minimum for many packages is 800-1Ghz, and only one i686 era CPU can meet those marks on a quad setup that threads horribly...
[18:46] <UserError> Intel even proved that the major reason 64bit was faster was optimizations
[18:46] <infinity> Err, they did?  Neat trick.
[18:46] <UserError> yeh, they did
[18:46] <UserError> in 2012
[18:46] <infinity> The major reason x86_64 is faster is lack of register pressure.
[18:47] <UserError> http://software.intel.com/en-us/blogs/2012/09/26/gcc-x86-performance-hints
[18:47] <UserError> cough
[18:47] <infinity> For some things like string functions, optimisation can make a huge difference, but you get that regardless.
[18:47] <infinity> Because IFUNC is a beautiful thing.
[18:47] <mjg59> UserError: That page appears to contain nothing to support your assertion
[18:48] <UserError> People are holding back performance on 32bit because of a single chipset
[18:48] <apw> indeed it seems to support the contention that increased registers is a ke factor
[18:48] <UserError> mjg, the gain difference with the same CPU
[18:48] <apw> key
[18:48] <UserError> because of initial targeting of SSE2 vs nothing
[18:48] <UserError> i686 is nothing
[18:56] <UserError> In a few months it will be the birthday of the MMX instruction set.
[18:56] <UserError> Still lacking, since 1997 http://download.intel.com/support/processors/pentiummmx/sb/24318504.pdf
[22:25] <infinity> zequence: You happy with the kernels in your PPA?
[22:27] <infinity> zequence: Nevermind, they look good to me, copying.
[22:39] <UserError> submitted the exynos and lustre paths
[22:39] <UserError> on to the nitty gritty
[22:59] <infinity> UserError: Is all this really fallout from one article quoting one guy as to why he's not shipping and supporting a mesa driver?
[22:59] <UserError> No
[22:59] <infinity> Sure sounded like it.
[23:00] <UserError> I'm just removing the excuse of the diff as icing on the cake. I actually am super anal about bloat
[23:00] <UserError> but their words did get plenty of my side-eye