gbkerseyJamesBenson: I'm using BCM57711 10-Gigabit in quite a few R710/R610 all running Ubuntu 16.04 with no problems.00:44
oskieis there a way to control when the system does "apt-get update" automatically?05:11
lordievaderGood morning06:36
=== jelly-home is now known as jelly
naccoskie: well, it generally doesn't15:34
fooI'm having memory issues that have occured 3 times in the past month. It would seem something is going on... the system mainly has nginx + gunicorn + various python scripts + postgres. I suspect nginx and postgres tuning for system specs is a good place to start, agreed?15:35
naccfoo: descirbe "memory issues"?15:36
tewardfoo: *usually* I'd be looking at gunicorn or the python scripts as your culprits, but describe what you mean by "memory issues"?15:44
foonacc: I can see MemoryError getting thrown in python. Although, upon further inspection this time... I see various issues: 2019-04-10 02:02:39,956 connectionpool 13279 - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError( ... socket.gaierror: [Errno -3] Temporary failure in name resolution ... OSError: [Errno 101] Network is15:46
foounreachable ... MemoryError ... hmmm.15:46
fooI'm beginning to blame python scripts and something it uses. Thanks teward15:46
fooThere's an optimizing change we can make that I've been meaning to make. Now might be a good time. Also, someone suggested installing sysstat and have it run every minute in cron15:47
tewardfoo: unreachable means a network problem resolutoin failures are DNS< and MemoryError from *python* means your Python Scripts / gunicorn backend are consuming memory15:47
tewardnginx just hands stuff off to gunicorn, and PostgreSQL doesn't really take up *that* much memory depending on what DB commands you're running15:47
footeward: two separate things, right? What's strange is this both happened at the same time15:47
footeward: yeah, relatively small data set too.15:47
tewardgunicorn is not 'light' by the way for running Python things, so it's entirely possible that that needs tuned better.15:50
tewardbut yeah you'll be looking at your python stuff and your unicorn backend for the memory errors15:51
tomreynsuch events can be related, you'll need to work out which one occurred first. maybe the network link was lost and a process which depends on it was spawned many times, consuming more memory than it would if the network link had been there.15:51
tewardsince its likely using all your system resources15:51
tewardalso what tomreyn said15:51
footomreyn: thank you, I'm thinking something like that happened15:51
foorad, appreciate it ya'll!15:52
tomreynthis is just a theory i just magically brought up out of a magicians hat, so dont rely on that to be what happened. check + compare timestamps in logs, and, yes, do what teward said ;)15:52
footomreyn: nope, i see u as my god. /me bows15:57
footomreyn: thanks ;)15:58
tomreynthat's a bad combo, since the only thing i believe in is the existence of aliens (and i don't mean rpm)16:00
footomreyn: the challenge with that is we'll never see aliens. I mean, would you visit our solar system if we only had 1 star? Maybe they have another rating system16:02
* tomreyn chuckles16:02
yossarianukHi - I am planning to install ubuntu-server in a restricted network  - where outgoing traffic is restricted- in order to connect to apt (gb.archive + security) what ip's do I need to whitelist for? Also do I need to enable the port for GPG ?16:03
yossarianuki.e is it just a case of doing a nslookup on  gb.archive.ubuntu.com + security.ubuntu.com  and whitelist those ips ?16:09
tomreynyossarianuk: i don't think these A records are guaranteed to be static16:10
sdezielyossarianuk: it's a good use case for an HTTP proxy16:11
tomreyni.e. you'll need a local proxy of sorts, which is allowed to connect to * firewall-wise. or, if this is not acceptable, work based off point release ISOs16:11
yossarianuktomreyn: thanks16:11
tomreynyossarianuk: i didn't understand the GPG port question16:12
tomreynGPG, as in GNU Privacy Gueard, does not run as a daemon which listens on the Internet16:13
yossarianukhmm I thought you needed access to a port16:14
yossarianukits the HKP port16:14
rypervencheyossarianuk: That's only if you are wanting to access a key server.16:15
tomreynif you'll use utilities such as apt-add-repository which look up apt repository signing keys automatically when a PPA is added, yes. otherwise, i don't think so.16:15
yossarianukI thought that is what apt did by default ?16:15
yossarianukok thanks16:15
tomreynubuntu's archive server apt gpg signing keys are packaged, so there should be no need to look those up. either way, you could deploy apt signing keys manually to your systems.16:16
tomreynsee /etc/apt/trusted.gpg.d/16:16
tomreynalso "apt-key list"16:17
fooI have ideas... but... what do you see? Looks like something is pegging CPU and RAM, agree?17:45
foouptime load average: 1.77, 3.88, 2.4217:45
sarnolda load average doesn't mean too much in isolation17:58
sarnoldI've seen nearly idle machines with a load average of ~32 and machines doing strenuous work with load average of ~217:58
sarnoldinstead use top or htop or similar to see which processes are using cpu; vmstat 1 to see bi and bo, si and so columns, to see how much disk io and swap io there is17:59
sarnoldthat'll give you much better indicator of what the machine is doing17:59
foosarnold: oh, whoops, I forgot to link it, heh... https://paste.ofcode.org/p8dTGJX5AKZYzAtJsgAD9t17:59
sarnoldhey there we go! :)18:00
sarnoldsadly I odn't know this tool.. can you scrape the si and so columns?18:01
foosarnold: sure.18:02
* foo checks vmstat18:02
foosarnold: https://paste.ofcode.org/39eLF9awfa9qpXJx4RagY2Q18:03
sarnoldfoo: cool, thanks; minimal disk IO, no swapping, cpu spending a lot of time idle. it feels like a lightly-loaded network server to me. how'd I do? :)18:04
foosarnold: ha! thank you :) In that case, I'll pay less attention to https://paste.ofcode.org/Vtqr9abPzHsWbELp5zJfLS - eg. %memused and %commit seemed high (they're red when viewed in terminal)18:07
foosarnold: I also just rolled in a fix so that may have helped. Will keep an eye on it, thanks for the two cents! I've made notes of this, too. Been a bit rusty with troubleshooting... glad to keep notes with this now18:07
sarnoldfoo: a friend once said "unused memory is wasted memory"  :)18:08
sarnoldfoo: hold on a sec..18:08
sarnoldfoo: there's weeks of excellent reading on http://www.brendangregg.com/linuxperf.html and linked pages18:09
compdocI wish I had friends :(18:09
foosarnold: thanks!18:09
sarnoldcompdoc: aww :(18:09
gislavedwhen I have a nic on 2 hosts crossconnected, make a bridge for it, attach it to a VM to that bridhe, create a nic in both VM's within the same subnets, shouldn't they be able to ping ?18:13
sdezielgislaved: in theory yes. Make sure you don't have br_netfilter loaded on any of the hosts though18:14
gislavedsdeziel as far as I see that is not the issue, it are two vyos vm's on a proxmox box and they cannot ping eachother18:14
gislaved2 proxmox boxes18:15
sdezielgislaved: I don't know proxmox but surely tcpdump should give you some visibility inside those bridges18:15
gislavedthat is possible indeed :)18:16
gbkersey JamesBenson: I'm using BCM57711 10-Gigabit in quite a few R710/R610 all running Ubuntu 16.04 with no problems.21:10
JamesBensongbkersey: thanks, any special drivers?21:11
JamesBensonor just install ubuntu 16 and go?21:11
JamesBensonwhat kernel do you use?21:11
gbkerseycurrently Linux palm 4.4.0-142-generic #168-Ubuntu SMP Wed Jan 16 21:00:45 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux21:12
gbkerseyusing default params with bnx2x module21:13
gbkerseythis is a filesystem network for a vm hosting cluster - we're primarily doing drbd over these connections using jumbo packets21:14
gbkerseyyou aren't going to see 10GB out of these on 11th gen servers - the pci bus isn't quite fast enough - but we do get 7-8 Gbps21:15
gbkerseywe used the BCM5709 for the pxe installs though, so I'm not sure if the installer supports bnx2x21:16
gislavedwhat bond mode is best supported with 2 direct links between 2 servers ? so to say 2 crossconnects ? active/backup is no option I think23:20

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!