[00:02] sarnold: well i was capturing for udp pkts from this server .. nothing going out [00:02] no pkts even trying to send to remote.. pure quite [00:03] axisys: how about trying nc to send udp to the remote host? [00:03] ok.. let me check [00:04] axisys: that would let you sort out if it is a matter of configuring rsyslog or fixing firewall rulesets or routing or something similar [00:04] right.. how do I generate a udp pkt with nc [00:05] got it.. [00:07] ok nc 192.168.1.100 works .. remote server gets the pkts.. so it is issue with rsyslog then [00:08] nc 192.168.1.100 514 [00:08] are you using -u ? or not [00:09] nc -u 192.168.1.100 514 works too.. pkt recvd on remote side [00:09] oh okay [00:09] so that leaves troubleshooting rsyslog :/ [00:09] right.. [00:13] hrm, do you need to restart or reload rsyslog to pick up changes? have you started or restarted rsyslogd since adding this line? [00:13] I did .. let me do it again [00:15] I killed it and it came back up right away [00:16] ah.. upstart [00:18] this is pre-start /lib/init/apparmor-profile-load usr.sbin.rsyslogd and /etc/default/rsyslog has RSYSLOGD_OPTIONS="" and then just an exec rsyslogd $RSYSLOGD_OPTIONS .. [00:20] do you have anything in dmesg? [00:22] [20646747.288516] init: rsyslog main process ended, respawning [00:23] heh. handy :) but sadly not much help [00:23] lol [01:03] nacc: pity this was marked private security .. it might have been easier to address ~ten days ago. Is this something for your team? 1753018 [02:54] sarnold: looking [02:54] sarnold: yes [02:55] sarnold: i'll make sure it gets noticed tmrw === Desktop_ is now known as allyru [06:28] good morning #server [07:24] Good morning === _cpaelzer is now known as cpaelzer === Frankfurt_SoupAU is now known as Frankfurt_Soup === slashd- is now known as slashd === Kamilion|ZNC is now known as Kamilion === broder_ is now known as broder === soren_ is now known as soren === diddledan_ is now known as diddledan === LaserAllan is now known as Guest25362 === y0sh_ is now known as y0sh === frickler_ is now known as frickler === tyhicks` is now known as tyhicks === tyhicks is now known as Guest65850 [12:22] i'm facing quite an interesting case with sas storage in hp server. the storage is configured to be 12Tb, but the ubuntu server sees with blockdev only 1.1Tb. Any tips ? [12:26] Mava: wrong partition table? [12:27] OpenTokix: should it affect the information blockdev reports ? [12:29] Mava: It's where the sectors and sector-size is stored. [12:30] OpenTokix: good point, unfortunately converting it to gpt did not fix anything. [12:32] Does the num sectors and sector size add up to wrong or right? [12:32] once you said: both are wrong [12:33] like. it calculates it right [12:34] but the amount of sectors and the sector size is not the ones specified in the array configuration utility [12:37] now i've got a clue. thanks OpenTokix ! [12:43] how do people monitor mdadm arrays? ilke, I have 2 different arrays, a 2 drive raid 1 for / and /boot and an 8 drive raid 6 for /mnt/storage - I know I can look at mdadm --detail and check the state is clean anddisks are active, but is there a best practice way of keeping on top of it in case a disk fails? [12:44] I just let munin do it for me [12:47] you can also configure MAILADDR in mdadm.conf for alerting [12:48] ooh, that'd probably be good enough [12:49] although i presume it expects there to be a local MTA rather than being able to specify an SMTP server? [13:10] ahasenack: rbasak: did you ever realize that system looses "fast" error messages? [13:10] not really [13:11] what did you (not) see? [13:11] Mava: cool, good luck [13:11] Am I missing some context here? [13:11] slow: http://paste.ubuntu.com/p/2nybTkkm3h/ [13:11] fast: http://paste.ubuntu.com/p/mSfYPHssfT/ [13:12] essentially echo + exit vs echo + sleep + exit [13:12] result: http://paste.ubuntu.com/p/TT38mQ83Mf/ [13:12] you must be debugging something interesting :) [13:13] many services that have wrappers do like "initial check, error out if failed" [13:13] I realized while working on one that the message isn't there [13:13] ahasenack: this is what got me to this [13:14] for what is worth, I do find that the logs from systemd are lacking detail [13:14] ahasenack: rbasak: do you see that the "fast" output is not in the status info [13:14] maybe because of that [13:14] Interesting [13:14] ahasenack: I find the logs better accessible than before sytemd, but this is odd [13:14] I just wondere - might this be a general flaw [13:14] or do I miss something [13:14] I can't think of any architectural reason why there has to be a race like that [13:14] but I feel my example is not simplified a lot, and still reproduces [13:14] are they also not in the output of journalctl? [13:15] ahasenack: the are in the journal [13:15] just not in the status output? [13:15] just not on systemctl status, which is where people look first [13:15] ohh [13:15] that just got more interesting [13:15] an i mean, sleep fixes it c'mon we are not in 1998 [13:16] sleep fixes so many things [13:16] I tried to echo to stderr (unbuffered) but that didn't change anything [13:16] how do you echo unbuffered? [13:16] or is that stderr's default behavior [13:16] the default of stderr [13:17] ahasenack: rbasak: but none of you points out obvious flaws right - so I might ask #systemd then [13:17] and even after a while systemctl status still won't show it? [13:17] right [13:17] ahasenack: not after 7 minutes [13:17] good enough [13:17] what about the <3> prefix, does that mean anything special? [13:18] or just log level [13:18] (which would be special) [13:18] ahasenack: that makes log levels for journal/systemd [13:18] but yeah, lets try without [13:19] removing the log level has no effect [13:19] but was worth a try [13:19] ok [13:20] sleep 0.01 is enoug [13:20] h [13:21] so just any sort of interruption [13:29] rbasak: ahasenack: xenial not affected, but showing in bionic [13:33] Is there a recommended way to set the FQDN on a pure IPv6 setup. Equivalent to the IPv4 /etc/hostname "127.0.1.1 hostname.domain.tld hostname" ? The "::1" is generally set to a similar list as '127.0.0.1' but do is there an IPv6 version of '127.0.1.1' ? [13:34] no [13:34] but I dunno why your using loopback for fqdn, that just doesn't work [13:34] That explains why my search-fu has been failing :) [13:34] you could always use ::127.0.1.1 [13:35] I'm not, I've been investigating a pure IPv6 deployment/config with IPv4 totally disabled in kernel to provoke bugs and other problems [13:36] Recommendation is not but FQDN in /etc/hostname; if sticking to that then there ought to be a recommended location for the domain [13:36] s/but/put/ [13:36] Anyone know what the point of the 127.0.1.1 address is? - From what I understand it's just a smeantic differnce. No technical difference. [13:37] OpenTokix: i /think/ originally it was to prevent errors where 'localhost' was removed from the '127.0.0.1' entry [13:37] since the entire network 127.0.0.0/8 is the same interface. [13:43] no [13:43] it's so you can have multible things going on [13:43] I bind a lot of things to different loopback ip's [13:45] rbasak: ahasenack: FYI that is https://github.com/systemd/systemd/issues/2913 [13:46] I was following your discussion in #systemd [13:46] nice that you found the bug [13:46] it essentically fails to associate with the unit before it is gone [13:46] and therefore missing on systemctl status and journal -u output [13:47] that's an old bug :/ [13:47] it does not affect my xenial system it seems === Guest65850 is now known as tyhicks === tyhicks is now known as Guest1006 [13:50] as I understand it we might loose any late message [13:50] as long as it comes very shortly before the PID goes away [13:50] :-/ [13:51] But quite often the last message before something dies is the most important one [13:52] patdk-lap: yes, but they all bind to the same interface. [13:53] isn't that the whole point? [13:53] if it didn't, useless [13:55] it is very useful, but also a magic interface [13:56] nothing magic about it [13:56] now, dummy, that is a magical interface :) [13:56] It binds to a whole /8, but only one of those ips is showing. [13:57] no it doesn't [13:57] it binds to exactly one ip [13:57] but it *routes* the rest [13:57] just like anything else [14:01] rbasak: ahasenack: for the sake of awareness on the Ubuntu side I filed bug 1756081 [14:01] bug 1756081 in systemd (Ubuntu) "journald is unable to attribute messages incoming from processes that exited their cgroup" [Undecided,New] https://launchpad.net/bugs/1756081 [14:05] ok [14:25] hello, I have an issue with the HWE kernel for xenial "linux-image-4.13.0-37-generic", it's freezing my server some minutes after the boot, is this a known issue? [14:28] Slashman: try #ubuntu-kernel [14:28] rbasak: thx [14:44] rbasak: hi, are you reviewing https://code.launchpad.net/~paelzer/ubuntu/+source/chrony/+git/chrony/+merge/341461 or was that comment just a drive-by? [14:45] ahasenack: I claimed the review as I was curiousu to look anyway [14:45] cool [14:45] cpaelzer: what does EFF_ stand for OOI? [14:45] EFFECTIVE [14:45] a rename can be done [14:45] Ah [14:45] just let men know [14:45] me [14:46] I just replied for the default conffile change === ptx0_ is now known as ptx0 [16:28] https://gist.github.com/61a118ed1c8437e2b480c6049cee07d7 [16:28] not sure why i got those errors after running update && upgrade [16:28] try running it as root [16:45] run commands it suggests or run update && upgrade again? [16:46] update and upgrade again as a first step [16:46] its odd because i have 4 identical servers and only two of them had these errors. [16:47] and i always update them at the same time. [17:05] madLyfe, what happens if you try `touch /var/lib/apt/lists/test.txt` ? [17:06] well it looks like the updates are going through at this point with root. === kees_ is now known as kees === mikal_ is now known as mikal