[13:54] <koni_> o
[14:59] <the_hydra> ok, I think I should just start now :)
[15:00] <the_hydra> allow me to introduce myself. I am Mulyadi Santosa, 31 years old
[15:00] <the_hydra> currently resides in Indonesia
[15:00] <the_hydra> and as you can easily guess, I use Linux daily ....
[15:01] <the_hydra> ok, let's start with easy one "what is /proc/meminfo"?
[15:01] <the_hydra> and to answer that, you need to know what /proc is....
[15:01] <the_hydra> it's a pseudo filesystem
[15:02] <the_hydra> meaning, it doesn't exist in your disk...be it your hard disk, SSD, floppy etc
[15:02] <the_hydra> it is entirely created in memory...in run time,by the Linux kernel
[15:02] <the_hydra> so you see the content of /proc, just like any other normal directory
[15:03] <the_hydra>    /proc/meminfo holds almost all information you need about your virtual memory condition
[15:04] <the_hydra> and, perhaps it  might surprise you, tools such as top, vmstat, atop, htop and others parse /proc/meminfo to gather information about virtual memory
[15:04] <the_hydra> up to this point, any question?
[15:05] <the_hydra> none? ok let's continue
[15:05] <the_hydra> MemTotal....what is this field supposed to mean?
[15:05] <the_hydra> your entire RAM?
[15:06] <the_hydra> anybody wanna bravely guess?
[15:09] <koni_> total RAM
[15:09] <the_hydra> close...
[15:09] <the_hydra> else?
[15:10] <the_hydra> ok, it's actually total amount of RAM recognized and "mappable" by your kernel
[15:10] <rsajdok> +swap
[15:10] <the_hydra> this might sound like a rocket science for seconds :)
[15:10] <mozer> 2^32
[15:10] <the_hydra> what if I told you, that initially, linux kernel could only manage up to 896 MiB?
[15:11] <the_hydra> this is due to ...what the geek call...virtual memory spliy
[15:11] <the_hydra> split
[15:11] <the_hydra> in 32 bit machine, in total, we could address up to 4 GiB...2^32
[15:12] <the_hydra> in this address space, linux kernel split it into two
[15:12] <the_hydra> 3 GiB for user space, 1 GiB for the kernel itself
[15:12] <the_hydra> so within this 1 GiB space, kernel must fit everything regarding memory allocation
[15:12] <the_hydra> and you might further ask "ok 1 GiB..but you say 896 MiB? how's that?"
[15:13] <the_hydra> kernel...reserve more or less 128 for special needs
[15:13] <the_hydra> so, enough to say that, without additional "hacks", Linux kernel...initially could only address up to 896 MiB
[15:14] <the_hydra> so, for those of you who owns aging distro like redhat 5.2, don't scream if you put 4 GiB RAM module in your machine and scream "why on earth, it's just showing 896 MB? crap"
[15:15] <the_hydra> luckily, modern distro now equipped with default kernel which is able to address up to 4 GiB
[15:15] <the_hydra> this is called highmem
[15:15] <the_hydra> how about machine with RAM up to 64 GiB?
[15:16] <flan> Ooh! Virtual MMU!
[15:16] <the_hydra> assuming your processor has PAE (Processor Address Extension) flag ( grep -i pae /proc/meminfo), just install pae kernel package
[15:16] <the_hydra> and kazaam, 64 GiB room for you
[15:17] <the_hydra> but wait, you said 64 GiB..but uhm, this is 32 bit you know?
[15:17] <the_hydra> yeah, that's what PAE is...to "extend" addressing
[15:17] <the_hydra> slight pause here...any question?
[15:18] <the_hydra> this is a generic knowledge, so it's not limited for Ubuntu and its family only...
[15:18] <the_hydra> every Linux distro apply the same thing here...
[15:18] <the_hydra> flan, yeah MMU....
[15:18] <the_hydra> flan, you got it right :)
[15:19] <the_hydra> allow me to gently remind you, it's all described in my article...in Full circle magazine issue 39...page 14 to 20
[15:20] <the_hydra> so, in case you need longer explanation, you might download and read it there
[15:20] <the_hydra> ok guys...I move again..
[15:20] <flan> Technical question, asked only out of curiosity and not for practical relevance: would PAE emulate MMU functionality for devices that have no real MMU of their own?
[15:20] <flan> Like ancient Palms.
[15:20] <the_hydra> nope AFAIK
[15:21] <the_hydra> PAE is PAE
[15:21] <the_hydra> MMU is MMU
[15:21] <the_hydra> in other word, without MMU, there won't be virtual memory
[15:21] <the_hydra> PAE just deal with addressing
[15:21] <flan> Okay, so it's a pre-requisite.
[15:21] <the_hydra> think like this: MMU is like your office rooms
[15:22] <the_hydra> it segments a floor into smaller...ehm, how to say that..cubicles
[15:22] <mhall119> where are you going to get intel hardware without an MMU anyway?
[15:22] <flan> I was thinking ancient ARM.
[15:22] <the_hydra> flan, nice question anyway
[15:22] <flan> But this is getting off-topic.
[15:22] <mhall119> PAE is an x86 only thing isn't it?
[15:22] <the_hydra> ok, shall I continue?
[15:22] <the_hydra> mhall119, AFAIK yes
[15:23] <the_hydra> mhall119, but it could be named something else in other platform
[15:23] <the_hydra> ok, memfree..this one is also confusing sometimes
[15:23] <the_hydra> again, let me ask, what's your understanding about it?
[15:23] <the_hydra> one person at a time, ok? :D
[15:24] <the_hydra> nobody? ok.... memfree is total amount of your RAM...100% unused for nothing!
[15:24] <flan> The amount of memory not occupied by anything cached or loaded by the kernel.
[15:24] <the_hydra> sorry i mean, unused for every purpose
[15:25] <the_hydra> flan, thanks...that's almost similar like mine :)
[15:26] <the_hydra> ok, so, in memfree, not just your application's allocated memory determines it, but also how much kernel does caching and any other kind of allocation
[15:27] <the_hydra> so if you see it and think "what? i just by my super dooper 16 GiB RAM stick and it is chewed down to 8 GiB in just few hours while I just sit and explore web using  my great firefox?"
[15:28] <the_hydra> that's the answer....firefox allocates...kernel might do caching..and so on
[15:28] <the_hydra> but that's good
[15:28] <the_hydra> and you might say "are you nuts???"
[15:28] <the_hydra> no I am not
[15:29] <the_hydra> think like this: suppose you really have 16 GiB...and kernel use it to cache your recent read files for about 10 GiB (ok, this might sound exaggerating..)
[15:29] <the_hydra> then, for next access, kernel will simply read from in-RAM cache instead of reading your disk
[15:29] <the_hydra> isn't that nice?
[15:30] <the_hydra> you get speed up, people! :D
[15:30] <flan> But I like hearing my HDD thrash. :(
[15:31] <the_hydra> flan, a strong magnetic iron will do that much quicker :D
[15:31] <the_hydra> ehheheh
[15:31] <the_hydra> coupling that, is buffer and cache
[15:31] <the_hydra> actually, they are more or less same thing
[15:32] <the_hydra> "page cache" a.k.a disk cache
[15:32] <the_hydra> so you sum them both, and you get the amount of RAM used for disk caching...
[15:32] <the_hydra> pretty simple...
[15:33] <the_hydra> ok...no question so far?
[15:33] <the_hydra> I shall jump to Committed_AS
[15:33] <the_hydra> if you wanna know how much RAM your applications ask for memory, this is the indicator
[15:34] <the_hydra> please notice, there is different between "how much an application ask" and "how much it is really allocated for now"
[15:34] <the_hydra> committed_as reflect the first one...the "request"
[15:35] <the_hydra> thus, the higher this field, you know there is a memory chewing application running..especially when it constantly increase
[15:35] <the_hydra> but it never exceed commitlimit
[15:35] <the_hydra> this is the upper limit, dynamically set by kernel...that denotes how much virtual memory you can ask
[15:36] <the_hydra> so "great, you make me dizzy, please show me how much RAM is really allocated by my applications...."
[15:36] <the_hydra> AnonPages tells you that
[15:37] <the_hydra> Anon...short of Anonymous...is a technical way to say "a page that has no physical disk/file backing"
[15:37] <the_hydra> ever use malloc() in C/C++, people?
[15:37] <the_hydra> malloc() is one of the function that ask for anonymous pages
[15:38] <flan> Also new and del for C++, mirrored by higher-level languages, like Java.
[15:38] <the_hydra> page itself is a smallest unit of RAM that a processor+MMU deal with your RAM
[15:38] <the_hydra> just like sector in disk...
[15:38] <the_hydra> flan, yeap
[15:39] <the_hydra> but...let me tell you, malloc() isn't directly allocate memory for you
[15:39] <the_hydra> it's just "asking"
[15:39] <the_hydra> the real deal happen when you access the allocated memory for the first, hence triggerring what the kernel geek calls "page fault"
[15:40] <the_hydra> swap total and swap free...easy....swap total denotes your total *active* swap area
[15:41] <the_hydra> the active ones...please notice that
[15:41] <the_hydra> so if you swapoff a swap area...it will be out of ring
[15:41] <the_hydra> swap free....the unallocated swap space from the active swaps
[15:41] <flan> QUESTION: What happens if you swapoff a partition while it's in use?
[15:41] <the_hydra> 'til now, questions?
[15:42] <the_hydra> flan, first, kernel check if there is sufficient of free RAM that can contain the bits in there
[15:42] <the_hydra> so say, it holds 128 MiB of data
[15:42] <the_hydra> your free RAM, must at least have that amount
[15:43] <the_hydra> because, if not, it must be kicked out again to swap....
[15:43] <the_hydra> agree here?
[15:43] <flan> Yes, thus far.
[15:43] <flan> But I'm really asking because I've never tried it.
[15:43] <the_hydra> and...dynamically, it will be swapped in to RAM
[15:43] <the_hydra> relax, i did it hundreds of time :D
[15:44] <the_hydra> and read the codes in linux kernel...so I can tell :)
[15:44] <the_hydra> once it all in RAM, that swap is detached
[15:44] <the_hydra> but your application "don't feel it"
[15:44] <the_hydra> you just feel few glitches....if there's any
[15:45] <the_hydra> in fact, doing swapoff -a followed by swapon -a (but please, do it carefully) is an easy way to bring all your swapped out pages back to RAM :D
[15:46] <flan> So what if there isn't enough free RAM/other swap? Will the kernel refuse to complete the command?
[15:46] <the_hydra> linux kernel has tendency to swap out inactive pages...slowly to swap areas
[15:46] <flan> (Yay for vm.swappiness)
[15:46] <the_hydra> thus freeing precious RAM..for more useful things
[15:46] <the_hydra> flan, yes...correct
[15:46] <the_hydra> flan, correct again...
[15:47] <the_hydra> the lower vm.swappiness is, everything will last longer in RAM
[15:47] <the_hydra> OTOH, the bigger it is (max is 100), it will be swapped out fairly fast
[15:48] <the_hydra> which one is ideal? up to you...but IMHO 60..the default...is already a nice number
[15:48] <the_hydra> did I say inactive?
[15:48] <the_hydra> active?
[15:48] <the_hydra> yes, kernel marks pages which has been inactive for certain interval
[15:48] <the_hydra> and they are candidates to be swapped out
[15:49] <flan> You were talking about active swap areas.
[15:49] <the_hydra> nope, in-RAM
[15:49] <flan> Oh, in terms of swap-propagation.
[15:49] <the_hydra> for swaps, there is no active-inactive :)
[15:49] <the_hydra> active..denotes portion of RAM that frequently accessed
[15:50] <flan> I should probably stop derailing your lesson.
[15:50] <the_hydra> the field itself is splitted into 2, anon..and file
[15:50] <the_hydra> flan, we can continue in other related ubuntu chat room L)
[15:50] <the_hydra> :)
[15:50] <the_hydra> anon...is the anonymous
[15:51] <the_hydra> and ...file? ok, remember when I talk about cache? this is it..along with the so called file memory mapping
[15:51] <the_hydra> this sounds geek ...but really you see it everyday
[15:51] <the_hydra> library loading...primarily :)
[15:51] <the_hydra> so, big anon...aggresive malloc()
[15:52] <the_hydra> big file (both active and inactive), could be aggresive library loading etc
[15:53] <the_hydra> unevictable...mlocked..are portion of RAM that is made unswappable
[15:53] <the_hydra> most likely, it is allocated by kernel for special need
[15:53] <the_hydra> or if you're multimedia people who works on audio recording etc, this number might rise...
[15:53] <the_hydra> it is a standard way to lessen latency
[15:54] <the_hydra> by locking allocated memory to be always in RAM, it's guaranteed that the access time is fast...because if it hits swap..well you know the consequences :D
[15:54] <the_hydra> ok, quite fast I guess...questions?
[15:54] <the_hydra> i selected the most important fields so far...
[15:55] <the_hydra> this is really linux memory management 101 :D
[15:55] <the_hydra> i feel like mel gorman junior :D
[15:56] <the_hydra> ok ...i guess time still permits
[15:56] <the_hydra> "dirty"
[15:56] <the_hydra> oh my god, dirty? dirt in my RAM
[15:56] <the_hydra> enough to say that, when you write something to disk, it's not done right away
[15:56] <the_hydra> most likely, it's deferred
[15:57] <the_hydra> portion of RAM that contains bits that haven't been pushed back to the related files..is said to be dirt
[15:57] <the_hydra> y
[15:57] <the_hydra> thus, the higher this number might mean : your disk is busy...or something is not optimal with write back operation
[15:58] <the_hydra> questions?
[15:58] <flan> Nope.
[15:58] <the_hydra> others?
[16:01] <flan> Thank you very much, the_hydra. I definitely learned a few things I didn't know about today. :)
[16:01] <the_hydra> thanks
[16:01] <the_hydra> :)
[16:01] <the_hydra> that would be a great honour