[02:39] <memguy> i don't understand one thing if udev creates/destroys the mknod device file and loads/unloads the LKM for new devices . The what links the LKM to the device file when one does say cat  something > /dev/xxx  
[02:40] <memguy> From what know the LKM has to register the /dev devices and provide the structure like cdev methods read,ioctl,write,..etc
[02:42] <memguy> So udev may create the device files and load the LKM but it would be in the LKM that the linking of the device file would be. If there is no LKM code for the device file then it is useless only  useful for user defined stuff but not like to any system char or block device
[02:44] <memguy> and when bash executes the code like cat something > /dev/xxx ... it must translate into a execv syscall or something which then  some how reaches the LKM code that is my issue the bash to the LKM and back for device files ... haven't got real into it yet but if anybody know let me know. My first thing is using debuggers or strace utility 
[03:02] <memguy> right now i just don't see how the cat /dev/sda gets link to say the LKM for the ata hdd driver code. Get it translates to execv command but from there not sure at some level it must call a read function that some how calls the LKM
[03:03] <mjg59> The driver registers a major and minor number, and the kernel dispatches the access to the driver
[03:10] <memguy> its the &fops in the register_chrdev for example that will control what read function is called for a device file but how does the actual read() user level function get to that
[03:10] <memguy> LKM
[03:10] <mjg59> The kernel dispatches it to whatever registered that major and minor number
[03:12] <memguy> read( fd , , ) takes a file descriptor number so how does that call get to the correct LKM
[03:12] <memguy> after its at the LKM then i get from the fops will call the correct read function for the device file.
[03:15] <memguy> plus the file descriptor doesn't even have to corrospond to a /dev/xxx file in the first place so i am assuming the userland code has tons of checks on what type of file descriptor this corrosponds to. And if it is  a /dev/xxx file it dispatches it to some syscall to dispatch it to the correct LKM code ... this is my guess mjp59 if you know more then by all means 
[03:17] <mjg59> The kernel dispatches it to whatever registered that major and minor number
[03:17] <mjg59> I don't really know how else to explain it
[03:17] <mjg59> The kernel knows that the file descriptor refers to a device node
[03:17] <mjg59> It knows the major and minor numbers of that device node
[03:17] <mjg59> It knows what registered those numbers
[03:17] <mjg59> And so it calls that code
[03:18] <mjg59> Userland doesn't need to do anything special
[03:21] <memguy> so read converts or looks up if the file descriptor fd has a major and minor number  from inode or filesystem info then how does userland know about the registered device thought only kernel level knows about that
[03:21] <memguy> it would have to syscall with a major and minor number or something
[03:21] <mjg59> Userland doesn't need to care
[03:22] <memguy> for the read function at some level
[03:22] <mjg59> The kernel created the file descriptor
[03:22] <mjg59> It knows what it refers to
[03:24] <memguy> O so the open () function requests this int fd  from kernel  not at userspace when opening a file it fd is created in the kernel
[03:25] <memguy> never mind kind of stupid should have thought thanks for clearing that up
[03:27] <memguy> so then it is always in the LKM or built into the kernel the code for linking a device file with the read, writting, ioctl ,...etc . And all udev is doing is mknod and insmod LKM plug in play wises
[03:27] <memguy> But it is the LKM that gives the device file life
[03:44] <memguy> class_create ,device_create ,register_chrdev and from these the kernel can associate the fd , major/minor , and  LKM methods to call
[04:07] <MoPac> Hello. I'm trying to understand more about (kernel-level?) control of CPU freq in this era of intel_pstate. My CPUs keep (at random-ish intervals) getting stuck at 800MHz until I suspend/resume or restart. But there's nothing obvious happening in kern.log/syslog/dmesg/thermald, and the governor's available frequencies always show the correct range (unlike old issues Googling shows).
[04:07] <MoPac> Am I even correct in thinking this is a kernel issue?
[04:09] <MoPac> (It's 16.04 /i7-4510U)
[08:00] <apw> NoPac (N,BFTL), that is cirtainly non-trivial to determine
[09:38] <Evlb> hey
[09:38] <Evlb> any1 here a tomoyo ninja?
[09:43] <apw> Evlb, always best to just ask your question, we might know, we might not
[09:46] <Evlb> In my case, i need to be able to disable tomoyo to be able to install updates, how ever, down-time is not feasible in this project.. So the question really is how to be able to install new software/updates without restarting the hardware? I am not able to install updates as of now due to Tomoyo
[13:31] <Amine_> Hello all, I am running a Linux kernel version 3.14.32 under a remote Ubuntu 14.04 LTS linux machine. After a recent upgrade and a reboot the machine refused to boot and freeze at some point. I was able to reboot the machine in system rescue and also read the content of /var/log/dmesg to see what happened.
[13:31] <Amine_> here is the dump of dmesg http://codepad.org/llFGvb62
[13:32] <Amine_> can anyone help me please to interpret what's going on ?
[14:42] <apw> Amine_, that pastebin doesn't look to be a dump from the failure that looks to be a failsafe boot
[14:42] <apw> Amine_, is this a grsecurity kernel ?
[14:48] <Amine_> apw, hmmm 
[14:48] <Amine_> apw, are you sure ?
[14:48] <apw> init: failsafe main process (1130) killed by TERM signal
[14:48] <Amine_> I am a true newbie where can I found such logs please ?
[14:49] <apw> logs for a previous boot are normally in syslog
[14:52] <Amine_> apw, hmm ok
[14:52] <apw> assuming of course they got there before the machine imploded beyond begin able to write
[14:52] <Amine_> but I just monted a var partition where the logs has been recorded so I don't see if it concerns the mode rescue 
[14:53] <Amine_> the current dmesg that I dumped was here
[14:54] <Amine_>  /mnt/var/log/dmesg
[14:54] <Amine_> the dmesg of the current rescue mode is in /var/log/dmesg
[14:54] <jtaylor> hi perf is broken in xenial due to the binutils update :(
[14:54] <jtaylor> includes the -31 in proposed
[14:54] <jtaylor> needs a rebuild to pickup the new libbfd soname
[14:55] <jtaylor> apw: I think you handled that last time ^
[14:56] <apw> jtaylor, that might imply we need to promote the libbfd update to -security or something
[14:56] <apw> jtaylor, could you file me a bug so i can track it
[14:56] <jtaylor> k
[14:57] <apw> jtaylor, and we might not be able to fix kernels already released either, hrm
[14:58] <apw> jtaylor, i assume you are saying if you downgrade binutils to the version in -security it starts working ?
[14:59] <jtaylor> apw: probably, the libbfd library changed named with the update
[14:59] <jtaylor> 2.26 to 2.26.1
[14:59] <jtaylor> don't know if that change really broke abi
[14:59] <apw> bah, really they should not be allowed to do that
[14:59] <infinity> Erk.
[14:59] <apw> erk indeed
[14:59] <infinity> That means a transition in xenial, other things depend on that too.
[14:59] <infinity> How annoying.
[15:00] <jtaylor> if it did not updating binutils to add a compat symlink might work
[15:00] <apw> jtaylor, which was the first kernel you noticed this with
[15:01] <jtaylor> apw: 28, but it should be all where perf links with libbfd
[15:01] <jtaylor> which probably goes back to before xenial release
[15:01] <jtaylor> don'T remember when exactly we fixed this (the link is needed so perf can demangle c++)
[15:03] <infinity> Depends: binutils (>= 2.26), binutils (<< 2.27)
[15:03] <infinity> So, that's a lie.
[15:03] <infinity> And it's binutils' fault.
[15:03] <infinity> The fix belongs there, if it's still compatible.
[15:04] <jtaylor> so should I file a bug against binutils instead?
[15:07] <apw> jtaylor, that was prolly when we first compiled against that version of binutil
[15:07] <infinity> jtaylor: Yeah.  Either the SONAME shouldn't bump, or the shlibs need to change to (>= 2.26.1), (<< 2.26.2)
[15:07] <infinity> jtaylor: In the latter case, we need a transition in -updates for all the rdeps. :/
[15:08] <jtaylor> apw: when bug 1248289 was closed I think, so 4.4.0-24.43
[15:08] <infinity> slangasek: ^-- FYI
[15:09] <infinity> It's not just a perf issue anyway, there are several packages that depend on binutils for libbfd.
[15:09] <infinity> IIRC.
[15:15] <apw> jtaylor, ok
[15:18] <apw> infinity, ok those deps come from ${misc:Depends} how are those calculated
[15:18] <infinity> apw: shlibs:Depends, from binutils' shlibs file.
[15:19] <apw> ok so either way it is binutils... sigh
[15:20] <infinity> Yeah, you've done nothing wrong here.
[15:20] <infinity> It's all doko.
[15:20] <infinity> Who's on VAC for a week or two. :P
[15:21] <apw> infinity, oh derp, thats helpful
[15:21] <jtaylor> I guess best option is to run the abi tool on libbfd and if its still compatible add a symlink
[15:21] <jtaylor> its probably reasonably likely this minor release did not break the abi
[15:21] <jtaylor> fwiw perf does not explode if you simply use the new soname ;)
[15:22] <infinity> ABI tracking on binaries is an inexact science, mind you.
[15:22] <apw> infinity, it does look from the way the shlibs is contructed that the lib name is not meant ot need to change for X.Y.Z Z++
[15:22] <infinity> There's no way from the binary to know if a symbol is compatible, only if it was dropped or added.
[15:22] <jtaylor> infinity: I mean this tool that looks at the source code
[15:22] <infinity> jtaylor: Yeah, the header mangling tools are more exact, to be sure.  Just a pain to use.
[15:23] <infinity> jtaylor: If you feel the urge, please do?
[15:23]  * infinity is meant to be out today with a head-exploding cold, but was here for a meeting happening in parallel.
[15:24] <slangasek> infinity: so the problem is we have a new bfd soname in 2.26.1, and a shlibs there which appears to be correct, but the previous binutils package declared a broader versioned dep which is no longer applicable?  Oh, and also this binutils was SRUed into xenial, brilliant
[15:24] <infinity> The other "simple" way could just be to diff the header files manually and look for prototype changes.
[15:24] <infinity> They could be identical.
[15:24] <infinity> slangasek: Is the new shlibs correct?
[15:24] <slangasek> infinity: it's "correct" in that it matches the files in the new package
[15:25] <slangasek> I don't know if it's correct that the soname should have changed
[15:25] <infinity> Oh, so it is.
[15:25] <infinity> slangasek: Right, I meant the former.
[15:25] <infinity> So, that's been "fixed", but in fixing it, doko didn't consider it meant a transition.
[15:25] <infinity> \o/
[15:25] <slangasek> so we basically need rebuilds + a binutils with Breaks: added
[15:26] <infinity> Alternately, if they *are* compatible, we could fix it in a hurry with a symlink, then unwind the mess.
[15:26] <slangasek> yeah, I can look at that
[15:27] <infinity> apw: None of that should prevent promotion of your current kernel, nothing's more broken than it was before.
[15:27] <apw> infinity, agreed
[15:28] <apw> slangasek, thanks
[15:29] <apw> jtaylor, would you file a bug if you haven't on binutils and let us know the number so we cna track this
[15:36] <slangasek> apw, infinity: there's a single symbol dropped between 2.26 and 2.26.1 libbfd, "_bfd_elf_strtab_restore_size", which looks like an internal symbol name that shouldn't impact ABI. and libopcodes has no changed symbols.  I can dig a little deeper to make sure none of the symbols have changed ABI, but it looks like a compat symlink is going to be the aswer
[15:36] <slangasek> jtaylor: ^^
[15:36] <jtaylor> apw: bug 1603137
[15:37] <apw> slangasek, it does feel like we are switching to a stable branch which might explain the .1 suffix on the overall version
[15:37] <infinity> slangasek: In that case, changing the shlibs to (>= 2.26), (<< 2.26.2) along with the symlink addition would avoid the breaks-and-rebuild mess.
[15:37] <jtaylor> slangasek: grepping perf code it uses no symbol starting with _bfd
[15:37] <jtaylor> slangasek: only a small handful of bfd_ functions
[15:38] <slangasek> yes, _bfd is a strong indicator that this is an internal symbol that should be ignored for ABI
[15:38] <apw> infinity, then we get the same problem with the next one ?
[15:38] <slangasek> apw: next one we stand at the entrance to the SRU queue with canes and make the SRUer get it right before it's accepted ;)
[15:38] <infinity> apw: Yeah, but at least we'll know, because the deps will be correct. :P
[15:39] <slangasek> right, someone know where I should look for binutils upstream VCS?
[15:39] <slangasek> since debian/copyright points me at cvs://:pserver:anoncvs@sources.redhat.com:/cvs/src and I desperately want that to not be correct
[15:39] <apw> infinity, right but if the intent was never to change the .so, then we _do_ have to rebuild everything with the tight restriction
[15:39] <infinity> slangasek: That symbol doesn't show in the headers, so I concur with your internal-only assessment.  Vaguely entertaining that the binutils maintainers don't know how to make private symbols hidden. :P
[15:40] <infinity> slangasek: https://sourceware.org/git/?p=binutils.git;a=summary
[15:40] <infinity> Err, no.
[15:40] <infinity> That's dead.
[15:41] <slangasek> https://www.gnu.org/software/binutils/ says git://sourceware.org/git/binutils-gdb.git instead
[15:41] <infinity> Ahh, it moved, yes.
[15:41] <infinity> Found it when you did.
[15:42] <infinity> apw: Well, kernels will rebuild organically but, indeed, we'd need to transition other rdeps if we want to keep the right restriction.  Point made.
[15:42] <jtaylor> or we convince upstream to not release point releases which break abi anymore
[15:42] <jtaylor> then the restriction could stay correct
[15:43] <infinity> It may well be that's upstream's intent already, but because it's a private library with a full-ver SONAME, they fail even when they succeed.
[15:45] <infinity> Would be tempting to submit patches to both make it a proper shared library and mark all the internal symbols hidden.
[15:45] <infinity> But I suspect the first would be rejected, and the second would be an annoying amount of fiddly work.
[15:46] <infinity> Especially if the first wasn't considered.
[15:46] <slangasek> I, too, am often tempted when I see a rathole to get out my yak clippers
[15:50] <infinity> slangasek: I'm going to go find chickens and oranges.  If you need an emergency SRU review later for this mess, text me, and I'll show up if I'm not out like a light.
[16:21] <unixabg> apw: Greetings, we talked about a year ago about overlayfs and casper. I am trying to add additional partial.squashfs to live media
[16:22] <unixabg> so a filesystem.squafhs and a partial.squashfs file.
[16:22] <unixabg> I believe I see what the issue is and I am testing with 16.04. Should I test and work with 16.10 daily to do more debug
[16:23] <unixabg> work. The issue (or so I think) is that the partial.squashfs file gets mounted but does not get added to the lowerdir
[16:24] <apw> unixabg, blimey, that was a long time ago, but yes, i remember vaguely
[16:24] <unixabg> stack. 
[16:24] <unixabg> here http://install.fyeox.com/scripts/live-partial-squashfs-updates is how I make my partial updates.
[16:26] <unixabg> This is probably a better link to share: https://github.com/unixabg/live-medium-install-tools/blob/master/scripts/live-partial-squashfs-updates  
[16:28] <apw> unixabg, you should work on the yakkety one yes, as we can rotate that quicker to get fixes in
[16:28] <unixabg> Not sure where in casper that happens but in my partial squashfs update script PSU, I am appending see line 35-37 on link from githup.
[16:28] <unixabg> s/githup/github/g
[16:29] <unixabg> Ok my time is like vapor just as I suspect yours is. Ok to just download daily.iso and start from there ?
[16:30] <unixabg> I would like to get to where you can have the prestine original filesystem.squashfs and a psu.squashfs for easy
[16:30] <unixabg> remaster and a clean path of updates.
[16:30] <unixabg> s/updates/updates for custom iso/g
[16:33] <apw> sounds reaosnable indeed
[17:39] <dmj_s76> kamal: So I've tested the patch, and it fixes the installer in uefi mode.
[17:41] <dmj_s76> kamal: Bios mode seems a little more complicated.
[17:41] <dmj_s76> There's a difference in behavior regarding vga modes at install time vs first boot
[17:45] <kamal> dmj_s76, the ISO kernel just went out the door so if there are further issues we'll have to tackle them in the next cycle
[17:46] <kamal> dmj_s76, so pls start up a fresh bug report to track that, when you get to that point
[17:50] <apw> kamal, indeed, they are publishing right now
[17:51] <kamal> apw, launchpad has been flooding my inbox with the news :-)
[17:51] <apw> kamal, i bet ... :)  mine is deluged too
[17:57] <dmj_s76> kamal: Well, it's a huge improvement
[17:57] <dmj_s76> kamal: Might not be a kernel specific issue either
[17:59] <kamal> dmj_s76, my favorite kind of issue: not the kernel's problem ;-)
[17:59] <dmj_s76> kamal: ...seems that bios mode only works if you set a vga mode manually
[18:00] <dmj_s76> (in the live install)
[18:00] <dmj_s76> (first boot doesn't need a vga mode set)
[18:17] <unixabg> apw: Ok just a bit of overlayfs mounting help? When I setup overlay for partial squashfs creation, I compose a list and mount all
[18:18] <unixabg> in one. In casper it looks like it wants to sort of compose them together one at a time. So I am not sure about that. Is there
[18:19] <unixabg> somewhere I can read to help me better understand how to mount overlay in on top of $rootmnt ?
[18:20] <unixabg> mount -t overlay overlay -olowerdir="${_LSTACK}",upperdir=./psu_overlay_rw,workdir=./psu_overlay_work ./psu_overlay
[18:20] <unixabg> is what I am used to using and I have the ${_LSTACK} confiured with all the mounts in the correct order.
[18:23] <apw> unixabg, so i think we didn't use to be able to mount multiple underlays, can ou do that now with overlay ?
[18:23] <apw> if so then the code is suboptimal as it is
[18:27] <unixabg> apw: well I do it so, I will say yes. Can you take a quick look at my utility script to create psu and rejoin work?
[18:27] <unixabg> https://github.com/unixabg/live-medium-install-tools/blob/master/scripts/live-partial-squashfs-updates
[18:27] <unixabg> It is not long and it does work, so I _think_ it not a large jump to just compose the lower mount list and 
[18:28] <unixabg> pass that for a single mount command. Or at least that is what I _think_.
[18:29] <unixabg> casper currently does mount the partial-squashfs-update.squashfs file, but it just does not get mounted in 
[18:29] <unixabg> the lower stack. I hope this helps apw and I am willing to test.
[18:35] <slangasek> infinity, jtaylor, apw: I've satisfied myself that the only ABI change in 2.26.1 is the dropped "internal" symbol (not exported in headers); will take care of fixing up the backwards-compatibility
[18:36] <apw> slangasek, excellent news
[19:43] <memguy> I am wondering if the is a file in /proc/pid that tells you stuff about the fs_struct locks for a particular process i know there is /proc/locks or lsof one can uses but was just curious if on dumped that info into each process folder seems it should be there but haven't found it
[19:48] <memguy> /proc/pid/fd tells you what files via file descriptors but not anything about the file locks or file attributes of the descriptor
[19:49] <memguy> /proc/locks is the only thing i really see for locks was thinking it would be under each process the locking for a process in each pid folder as well 
[19:52] <memguy> as /proc should contain all the info for any process so i may just be forgetting where to look for this anybody know
[20:01] <memguy> Seems to me proc is missing alot of info on things that have to do with locks there should be for each process a memlocks , filelocks , process_locks /thread_lock files ( maybe the latter of the 3 should be broken down into semephores, mutex, spinlocks info files )
[20:02] <apw> memguy, not everything is out there indeed
[20:03] <memguy> well it seems /proc should contain anything held in the task_struct and substruct info that one could view and in most cases it does just right now not really good with locks
[20:03] <memguy> at least for me that is
[20:04] <infinity> memguy: /proc/<pid>/fdinfo/<fd> maps to /proc/locks.  Same info, different view.
[20:06] <memguy> well that tells you the filenames and descriptors the process uses but where does it tell you what region is locked , what type of lock,...etc
[20:06] <infinity> memguy: http://paste.ubuntu.com/19406450/
[20:07] <infinity> For example.
[20:11] <memguy> hum weird i only get pos:	0
[20:11] <memguy> flags:	01 maybe its something with my lock because i get it showing up in /proc/locks
[20:12] <memguy> Another thing though is this is just for file locking where does one get the info for thread/process locks 
[20:16] <memguy> for memory locks i have similar to file locks map_files and maps which i imagine the 
[20:16] <memguy> b751b000-b76c4000 r-xp 00000000 07:00 3792       /lib/i386-linux-gnu/libc-2.19.so
[20:16] <infinity> By thread locks, you mean pthread_mutex_lock()?
[20:17] <memguy> r-xp  p means protected memory  
[20:17] <infinity> Cause I can't see how or why that would be exported in /proc
[20:17] <memguy> I mean all of the locks spinlock , semphores , or the binary ones called mutex all of it would depend on the process to say i was looking at kernel level kworker or 
[20:19] <memguy> normally it would just be mutex i get that but i it also on the rare occasion provide the things when spinlocking or other semephores occur
[20:19] <apw> the kernel does not export info on its internal locks no
[20:19] <infinity> You're unlikely to find ways to introspect that without gdb and a knowledge of how those bits are implemented on your platform.
[20:20] <infinity> It's a libc/kernel tag-team, and very hard (impossible on some platforms, I'd suggest) to export.
[20:24] <memguy> ok but for pthreads and userlevel multitasking that needs to lock parts of code i imagine it is mostly all done with mutex or semephores not so much busy waiting spinlocks so is there a file to look up that info for those for each /proc/pid/xxx
[20:31] <apw> those are implemented with futex's normally, no idea if they are exposed in uspace, though by definition you only get half the story
[20:32] <apw> as they only implement the "not spinning and wasting cpu" parts
[20:47] <memguy> how about more on the memory attributes  r-xp "p or s " tell me its shard but nothing about if it is locked or other attributes that the MMU imposes on it?
[20:51] <memguy> I know i could write an LKM to create a file in each /proc/pid printing any info i want from task_struct or sub structures so in theory i can do it all but if its already there i would be reinventing the wheel plus my point of view right now is understanding how to trouble shoot any problem that could happen with a process/scheduler, filesystem , or MMU . Think of it more like I am finishing up with super sysadmin and moving 
[20:51] <memguy> back to developer soon. I fluctuate between software engineer/program to sysadmin.
[21:01] <unixabg> apw: ok I think this is the answer of support of multiple lower layers:
[21:01] <unixabg> https://kernelnewbies.org/Linux_4.0#head-2274894c6e481a014a984600daeb55d8e5d6d91a
[21:03] <unixabg> Which in fact is similar to what my script does. Now how to move casper in to this format I think is in the
[21:03] <unixabg> setup_unionfs function line 385. 
[21:51] <memguy> also what is futex its seems like it is a type of mutex at the syscall level all most. But is it only for processor locking or can it do memory locking and file locking because kind of confused about this and flock or mlock where they all stand
[21:55] <memguy> never mind its for process's /code /shared memory locking 
[21:55] <memguy> fast user space mutexes