| jair | Hello all | 02:10 |
|---|---|---|
| jair | it's me again with that memory leak issue in Ubuntu 17.10 but now removed it and installed the latest update of ubuntu 16.04, however, the memory leak is still present | 02:10 |
| jair | Is there any command I can try other than reboot to make the OS to release the memory? | 02:12 |
| sarnold | jair: what problem are you *really* seeing? | 02:13 |
| jair | see paste > https://ibb.co/cfCOpc | 02:14 |
| jair | sarnold: the server start consuming memory and after 22 hours it keeps consuming it without releaseing it | 02:14 |
| jair | until it crashes | 02:14 |
| jair | Here is the ouput of the console I took after the crash | 02:15 |
| jair | see this other paste > https://ibb.co/gw5hhx | 02:15 |
| jair | the first paste is the current status, the memory does not goes down, it keeps going until it crashes | 02:16 |
| jair | the only thing running on this server is BIRD 1.6.3 we are using the server as a BGP router | 02:16 |
| sarnold | wow, OOM killing things that take one megabyte of ram .. | 02:16 |
| jair | hahaha yep | 02:17 |
| sarnold | jair: what else does this system do? | 02:18 |
| jair | sarnold: nothing else really, we have LXD installed and snapd but I believe those were there just because installed and thought about the usage of LXCs but with this issue I doubt it | 02:19 |
| jair | sarnold: there is nothing we can do to force this system to release memory? | 02:19 |
| jair | it is going to crash in a few minutes from now :( | 02:20 |
| sarnold | jair: if you're desparate maybe kill or purge lxd / snapd | 02:21 |
| jair | sarnold: https://paste.ubuntu.com/p/YGw6njmzrf/ | 02:21 |
| sarnold | jair: what's slabtop say? | 02:21 |
| sarnold | jair: you might be able to buy some time by adding in a swapfile | 02:21 |
| jair | interesting | 02:21 |
| sarnold | jair: do you have any apparmor profiles in complain mode? | 02:22 |
| sarnold | jair: what's dmesg output look like? | 02:22 |
| jair | here > https://paste.ubuntu.com/p/XTHg28hRky/ | 02:22 |
| jair | apparmor not that I know of, we are using for sure ip tables | 02:23 |
| jair | and to be honest I don't think lxd or snapd are the issues | 02:23 |
| jair | sarnold: let me paste the dmesg | 02:23 |
| sarnold | jair: sync; echo 3 > /proc/sys/vm/drop_caches .. that ough to take a big bite out the dentry slab | 02:24 |
| sarnold | man that's only going to free up something like 100 megs, I think. Hrm. | 02:25 |
| jair | sarnold: here dmesg > https://paste.ubuntu.com/p/bVFBVhNcWp/ | 02:25 |
| sarnold | ohho conntrack | 02:25 |
| sarnold | I have a very vague memory of seeing systems fall over due to too many conntrack entries | 02:26 |
| jair | sarnold: I ran the command > sync; echo 3 > /proc/sys/vm/drop_caches but didn't seem to do much | 02:27 |
| sarnold | jair: you could check the results with slabtop again, but .. it probably only bought you a hundred megs or so :( | 02:27 |
| jair | sarnold: OK | 02:28 |
| jair | sarnold: this is really bad :( | 02:33 |
| jair | I honestly being keeping myself away from pure ubuntu releases because of this type of issues | 02:33 |
| jair | I stick to debian for servers and so far other than weird stuff or delays with systemd when restarting the services or the server buyt no something like this with memory | 02:34 |
| sarnold | jair: how about a ps auxw pastebin? | 02:35 |
| jair | ok checking | 02:35 |
| jair | sarnold: sorry for the delay > https://paste.ubuntu.com/p/bF7x7pD7mk/ | 02:38 |
| sarnold | wow, there's so little there. hrm. | 02:41 |
| jair | right, this is really worrying | 02:44 |
| jair | I am not feeling confortable to use ubuntu as a server... I will still prefer Debian | 02:44 |
| jair | I believe | 02:44 |
| jair | but I will need to find the drivers for the RAID controller | 02:44 |
| jair | sarnold: https://paste.ubuntu.com/p/56GXBtbF9r/ | 02:46 |
| jair | I need to make sure debian supports that raid controller | 02:47 |
| sarnold | jair: seems likely, megaraid_sas appears to have been around foreever https://cateee.net/lkddb/web-lkddb/MEGARAID_SAS.html | 02:49 |
| sarnold | jair: before you tear this machine down, can you run 'ubuntu-bug linux'? I hope this won't be the allocation that pushes the machine beyond it's limits, but it would be nice to capture this | 02:50 |
| sarnold | the only real advice I've got is to add a swap file or swap partition so the kernel has some place to shove data it doesn't need. But with 16 gigs ram and the machine doing so little, that feels like it wouldn't really help, just buy some more time. | 02:51 |
| jair | sarnold: | 02:52 |
| jair | sure | 02:52 |
| jair | sarnold: done > https://bugs.launchpad.net/ubuntu/+source/linux-hwe/+bug/1757037 | 02:59 |
| ubottu | Launchpad bug 1757037 in linux-hwe (Ubuntu) "Ubuntu crashes after running out of memory 16G total" [Undecided,New] | 02:59 |
| sarnold | jair: *thanks* :D can you paste in the slabtop, ps auxw, and meminfo files? | 03:02 |
| jair | sure | 03:02 |
| sarnold | jair: I've got to run.. good luck :) | 03:03 |
| jair | sarnold: soiunds good, thank you for the help | 03:03 |
| Wolf_Y | Hi, is anyone online ? | 09:50 |
| Wolf_Y | i have some noob-ish questions to ask | 09:50 |
| MitchT | Odd_Bloke: I see it! 18.04.201803180 | 12:01 |
| MitchT | building now | 12:03 |
| MitchT | err deploying | 12:03 |
| === JanC_ is now known as JanC | ||
| MitchT | Odd_Bloke: it works again. Thanks so much. | 12:37 |
| MitchT | commented on the bug report. | 12:40 |
| aoam | hello guys, doesnt anybody of you know, how to automate command "lxd init", it setup networks etc | 12:49 |
| Odd_Bloke | MitchT: \o/ | 12:54 |
| Odd_Bloke | aoam: `lxd init --auto`? | 12:54 |
| aoam | Odd_Bloke: i has it this way as you say in my script, lxdbr0 does not have ipv4 so containers have no internet | 12:56 |
| Odd_Bloke | aoam: Have you looked at `lxd --help`? There are options to modify network setup. | 13:00 |
| aoam | perfect, i’ll try, thanks | 13:03 |
| cpaelzer | rbasak: your mongo MP has no diff on the LP view | 13:10 |
| cpaelzer | rbasak: I don't care and will check the changes one by one, but wanted to let you know | 13:10 |
| cpaelzer | rbasak: also is that in your experimental ppa again? | 13:11 |
| aoam | just for record, if anyone looking for solution, this could be right one: https://github.com/lxc/lxd/blob/master/doc/preseed.md | 13:12 |
| cpaelzer | rbasak: I see a (different) 1:3.4.7-1ubuntu3~ppa2 in there | 13:13 |
| rbasak | cpaelzer: yeah ppa:racb/experimental | 13:17 |
| rbasak | cpaelzer: it is identical to the MP except changelog wording and versions, but feel free to verify that | 13:17 |
| cpaelzer | rbasak: I did already | 13:17 |
| cpaelzer | fine | 13:18 |
| cpaelzer | rbasak: while I test you can explain the deps you chose for breaks/replaces - see MP comment | 13:18 |
| rbasak | cpaelzer: thanks for reviewing! Replied. | 13:20 |
| cpaelzer | rbasak: mongodb-server (<= 1:3.4.7-1ubuntu3~) | 13:26 |
| cpaelzer | rbasak: that should be the correct thing, don#t you tihnk | 13:26 |
| rbasak | cpaelzer: no, because ubuntu3~ would be derived from ubuntu3, which is after the change. | 13:26 |
| cpaelzer | rbasak: no, because "3~..." is everything sorted before "3" | 13:27 |
| rbasak | Yes, 3~ would sort before 3 | 13:27 |
| cpaelzer | so if one had an experimental 3~changeindocs | 13:27 |
| rbasak | But we want the breaks/replaces to match against neither 3 nor 3~ | 13:27 |
| rbasak | We do want it to match against 2 and 2~ | 13:27 |
| cpaelzer | we want it to match 3~ for my example above | 13:28 |
| cpaelzer | I agree you do not want it to match against 3, but it would not with my suggestion above | 13:28 |
| rbasak | Why would you have a 3~changeindocs? | 13:28 |
| cpaelzer | was an example for any change that might exist having a 3 but not being yours | 13:28 |
| rbasak | Any published (not development) version of 3~ that might exist would be derived from 3. | 13:29 |
| rbasak | The 3 that I will publish | 13:29 |
| rbasak | And in this case, it shouldn't match against the breaks/replaces | 13:29 |
| rbasak | Since both 3 and the derived-from-3 3~ will be after the rearrangement. | 13:30 |
| cpaelzer | rbasak: no it won't be derived of 3, ~ versions are created as pre-next | 13:30 |
| maddawg2 | so i'm FINALLY sitting down to learn about all this hype of containers lol | 13:30 |
| cpaelzer | if you have 2 today, and one tests something he will create 3~ppa1 or such | 13:30 |
| cpaelzer | and eventually this or some other change will become the real "3" | 13:30 |
| rbasak | cpaelzer: the use of the ~ in published versions is for backports etc. | 13:30 |
| cpaelzer | rbasak: so 3~ is derived from something before 3 | 13:30 |
| rbasak | A backport derived from 2 will be 2~something | 13:30 |
| rbasak | A backport derived from 3 will be 3~something | 13:31 |
| maddawg2 | i'm thinking of experimenting with it on a virtual machine, but I'm curious if anyone has any recommendations on where i should start... should i start with docker? | 13:31 |
| maddawg2 | or this LXD thing in Ubuntu | 13:31 |
| cpaelzer | rbasak: yeah I didn't talk about backports in this case, but only usual devel versions | 13:31 |
| rbasak | cpaelzer: the fact that 3~ orders before 3 is separate. | 13:31 |
| cpaelzer | maddawg2: start here https://stgraber.org/2016/03/11/lxd-2-0-blog-post-series-012/ | 13:31 |
| rbasak | cpaelzer: once 3 is published, nobody will make a 3~ devel version anyway. In any case, we don't usually consider those when choosing version strings. | 13:32 |
| cpaelzer | rbasak: hrm, I hate letting go habits | 13:33 |
| cpaelzer | rbasak: at least I can blame you for setting me up that way to begin with | 13:33 |
| cpaelzer | :-) | 13:33 |
| rbasak | Sorry :) | 13:33 |
| maddawg2 | thanks cpaelzer... i'm also interested in learning chef and/or puppet type stuff as well... would LXD still be a good option for that | 13:33 |
| rbasak | The only reason ~ ends up in these things in the archive is to accomodate existing or future backports. | 13:33 |
| maddawg2 | or should i use something like docker | 13:33 |
| rbasak | (published, official backports) | 13:33 |
| rbasak | I suppose it also does help with local unpublished backports. | 13:34 |
| rbasak | Using 2 here will break none of these cases, since backports derive from a published version and append ~ | 13:34 |
| cpaelzer | maddawg2: you are getting to the land of opinions here, for me docker sucks and LXD gives me all that I ever need | 13:34 |
| cpaelzer | maddawg2: but for others it might be just vice versa | 13:34 |
| rbasak | maddawg2: docker and lxd have very different use cases. | 13:34 |
| cpaelzer | exactly | 13:34 |
| cpaelzer | maddawg2: never the less I think to experiment with chef and similar a system level container (LXD) is more appropriate | 13:34 |
| rbasak | maddawg2: lxd is easier conceptually. lxd gives you nested standard distribution installations. | 13:35 |
| rbasak | maddawg2: an Ubuntu inside an Ubuntu. | 13:35 |
| rbasak | maddawg2: Docker is a much larger paradigm shift. | 13:35 |
| rbasak | maddawg2: there's really very little to learn with lxd if you consider it that way. | 13:35 |
| rbasak | maddawg2: most things are exactly the same. You only need to know how to launch instances. And understand that the guest is mostly isolated by default, so if you don't have permission on the host, you won't have permission in the guest, etc. | 13:36 |
| cpaelzer | maddawg2: if you are up for a read start with any of http://lmgtfy.com/?q=system+vs+application+level+container | 13:36 |
| cpaelzer | rbasak: review complete, I've added a testing suggestion | 13:37 |
| rbasak | Thanks! | 13:37 |
| maddawg2 | cpaelzer, i'm looking to get make a career shift from IT management and system engineering over into a devops type position, but i'm not sure which product is most used in the enterprise that would be most beneficial to learn | 13:42 |
| aoam | in this command `lxc image copy some_remote:some_container local: --alias local_copy --auto-update` im wondering if auto update really updates my local image | 13:46 |
| sdeziel | maddawg2: try both, lxd can run docker in a container (https://stgraber.org/2016/04/13/lxd-2-0-docker-in-lxd-712/) :) | 13:46 |
| rbasak | cpaelzer: is that pastebin test in a dep8 test anywhere? | 13:50 |
| rbasak | Sounds like it should be in pymongo? | 13:50 |
| cpaelzer | rbasak: it is in the qa tests that we use before bigger uploads but not fit dep8 | 14:09 |
| cpaelzer | rbasak: git+ssh://git.launchpad.net/qa-regression-testing | 14:09 |
| adac | hmm the newest kernel I get when i dis-upgrade my xenial server installation is 4.4.0-89-generic | 14:15 |
| adac | locally I have also 16.04 and there I have 4.13.0-37-generic | 14:16 |
| patdk-lap | yesand? | 14:17 |
| patdk-lap | and? | 14:17 |
| patdk-lap | that sounds right | 14:17 |
| patdk-lap | !hwe | 14:17 |
| ubottu | The Ubuntu LTS enablement stacks provide newer kernel and X support for existing LTS releases, see https://wiki.ubuntu.com/Kernel/LTSEnablementStack | 14:17 |
| adac | patdk-lap, ok do I understand it correctly: if i install LTS point release (currently 16.04.4) I do immediately get HWE but if I dist-upgrade from a lower point release then I have to add it manually with: | 14:23 |
| adac | sudo apt-get install --install-recommends linux-generic-hwe-16.04 | 14:23 |
| patdk-lap | I have no idea how it works on upgrades | 14:27 |
| adac | patdk-lap, when I provisioned a server from my hoster with ubunut 16.04 minimal I got the 4.13.0-37-generic kernel as well | 14:29 |
| adac | with my "old" virtual hosts I constantly upgraded I only get 4.4.0-89-generic | 14:29 |
| dpb1 | adac: 4.13 is the "HWE" kernel | 14:29 |
| dpb1 | https://wiki.ubuntu.com/Kernel/LTSEnablementStack | 14:30 |
| dpb1 | adac: new installs (depending on cloud, etc) often will have that enabled by default | 14:30 |
| dpb1 | but for upgrades, you have to opt-in to it | 14:31 |
| adac | dpb1, yes looks like that. Ok i will install it on all my machine | 14:31 |
| adac | ansible comes into the game :D | 14:31 |
| adac | thanks for your help guys! | 14:31 |
| maddawg2 | strange... i just installed ubuntu server 16.04 and ran a sudo apt-get update but it's failing.... | 14:33 |
| maddawg2 | it thinks the stuff is on a CD-Rom :-\ | 14:33 |
| === dgadomski_ is now known as dgadomski | ||
| maddawg2 | lesson here is not to use vmware's quick install | 14:38 |
| maddawg2 | lol | 14:38 |
| maddawg2 | that was stupid | 14:38 |
| MitchT | cpaelzer: i think docker is .. overhyped - it probably has its place but in my line of work, setting up and maintaining our magento store, for something that acts more like a service and less like an "app" that can spool up and shut down... well idk. Maybe i'm too old. | 14:46 |
| adac | dpb1, patdk-lap the kernel version i got from my hoster was: 4.13.0-36-generic | 14:47 |
| adac | there was no hwe mentioned there | 14:48 |
| MitchT | couple that with the 'always online' windows service that couples the site to our crm / inventory system and you have yourself a need for a regular old server. | 14:48 |
| dpb1 | adac: do `dpkg -l |grep hwe`, does it show anything? | 15:04 |
| adac | dpb1, i downgraded there to a 4.4.x kernel since I had a problem with this kernel. | 15:09 |
| adac | dpkg -l |grep hwe | 15:09 |
| adac | ii linux-headers-generic-hwe-16.04 4.13.0.36.55 amd64 Generic Linux kernel headers | 15:09 |
| adac | so i guess i forgot to remove the headers and it was indeed a hwe kernel then | 15:09 |
| adac | or? | 15:09 |
| adac | dpb1, this was froma grep when I had the 4.13.x still installed: https://gist.github.com/anonymous/1cae9e709dd448164e5f37b00acaac99 | 15:11 |
| adac | ok generic is also shown for my local installation with this grep/awk | 15:13 |
| dpb1 | adac: basically, there are two separate "tracks". normal and HWE, but you could certainly have both kernels installed on your system at the same time. | 15:46 |
| adac | dpb1, but I need always have this hwe package from your link installed to have kernel bigger then 4.4.x right? | 15:51 |
| dpb1 | basically, this package: linux-generic-hwe-16.04 is a meta package that will get you on the hwe track. It will then get updated to point to newer and newer 4.13 kernels as they are released. | 15:53 |
| ahasenack | hm, in my package I'm renaming a couple of files in /etc/update-motd.d from 99-<something> to 80-<something> | 17:39 |
| ahasenack | turns out dpkg doesn't take care of removing 99-<something> | 17:39 |
| ahasenack | so I end up with both versions after an upgrade | 17:40 |
| ahasenack | how is this solved? I suspect some maintainer script in pre/post | 17:40 |
| nacc | ahasenack: those are conffiles? | 17:41 |
| nacc | ahasenack: dpkg-maintscript-helper mv_conffie | 17:41 |
| nacc | *mv_conffile | 17:41 |
| ahasenack | nacc: they are conf files because they live in /etc. Other than that, the user has no incentive to touch them | 17:42 |
| nacc | ahasenack: sure, but i meant form the package's perspective | 17:42 |
| ahasenack | just because they live in /etc | 17:42 |
| nacc | ahasenack: no, that'snot the definition | 17:43 |
| nacc | one sec | 17:43 |
| nacc | https://www.debian.org/doc/manuals/maint-guide/dother.en.html#conffiles | 17:43 |
| nacc | e.g. | 17:43 |
| nacc | oh maybe it's all files under /etc nm | 17:43 |
| nacc | so then you can use the above helper to move it | 17:44 |
| nacc | *should* use | 17:44 |
| ahasenack | let me check its docs | 17:44 |
| CarComp | oops. forgot what the /quit command oes | 17:50 |
| CarComp | does | 17:50 |
| ahasenack | nacc: yay, it worked | 18:08 |
| ahasenack | with no drama :) | 18:08 |
| ahasenack | $ cat ubuntu-advantage-script/debian/ubuntu-advantage-tools.maintscript | 18:09 |
| ahasenack | mv_conffile /etc/update-motd.d/99-livepatch /etc/update-motd.d/80-livepatch 15~ | 18:09 |
| ahasenack | mv_conffile /etc/update-motd.d/99-esm /etc/update-motd.d/80-esm 15~ | 18:09 |
| nacc | ahasenack: nice | 18:41 |
| nacc | ahasenack: yeah that's the best way to do it, as it is supposed to handle the corner cases | 18:41 |
| ahasenack | good | 18:42 |
| === whaley is now known as hugh_jim_bissell | ||
| MACscr | this seems to be the way apache2.4 and php-fpm are configured by default when installing using apt. Is this really the right way to be enabling php-fpm for php files? http://paste.debian.net/1015758/ | 20:31 |
| nacc | teward: jinx (re: ubuntu-devel-discuss) | 20:32 |
| sarnold | MACscr: if you're using Ondrej's packages it might be worth asking him, or if he's got a community around his packages, then someone from that specifically | 20:33 |
| MACscr | sarnold ugh, ok. I figured he was just providing updated versions, not really config changes | 20:35 |
| tomreyn | MACscr: FilesMatch evaluates regular expressions,t his seems more complicated than necessary. i'd assume AddType would be a better way. | 20:39 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!