| sergeant | while connecting to the mongo shell on an ubuntu server. I am getting this error http://paste.ubuntu.com/p/SS7VBxFdck/. Please help!!!!!!! | 06:08 |
|---|---|---|
| lotuspsychje | sergeant: please idle a bit here, volunteers might be still waking up ok | 06:10 |
| sergeant | yeah sorry | 06:12 |
| igordc | or going to sleep :( | 06:17 |
| igordc | sergeant, I haven't particularly dealt with mongo much but it looks like it isn't running on that specified port | 06:18 |
| igordc | sergeant, double check the server | 06:18 |
| sergeant | how do i check that ? | 06:19 |
| === cpaelzer__ is now known as cpaelzer | ||
| eject_ck | After installing packages updates on 5 Ubuntu 16.04 servers one wont start (just stuck during kernel boot), https://imgur.com/a/Nukuvk9 | 08:20 |
| eject_ck | Anybody had such an issue ? | 08:20 |
| eject_ck | how can I collect details for such probelm? | 08:22 |
| eject_ck | kernel 4.4.0.141 | 08:24 |
| lotuspsychje | !info linux-image-generic xenial | 08:31 |
| ubottu | linux-image-generic (source: linux-meta): Generic Linux kernel image. In component main, is optional. Version 4.4.0.141.147 (xenial), package size 2 kB, installed size 14 kB | 08:31 |
| lotuspsychje | eject_ck: did you try booting a previous kernel yet? | 08:31 |
| eject_ck | yes, no luck | 08:32 |
| eject_ck | I tried to boot with dis_ucode_ldr [X86] Disable the microcode loader. and it worked | 08:33 |
| eject_ck | interesting now why it caused problems | 08:33 |
| lotuspsychje | eject_ck: maybe provide us some dpkg logs from the installed updates recently, maybe volunteers can find a link | 08:34 |
| eject_ck | ok | 08:35 |
| lordievader | Good morning | 08:36 |
| eject_ck | lotuspsychje: where to send ? | 08:55 |
| lotuspsychje | !paste | eject_ck | 08:55 |
| ubottu | eject_ck: For posting multi-line texts into the channel, please use https://paste.ubuntu.com | To post !screenshots use https://imgur.com/ !pastebinit to paste directly from command line | Make sure you give us the URL for your paste - see also the channel topic. | 08:55 |
| eject_ck | https://paste.ubuntu.com/p/4f4rfg2Xvk/ | 08:57 |
| lotuspsychje | eject_ck: that seems like a big dpkg list, is that a normal update or did you upgrade or wait long? | 08:59 |
| eject_ck | normal update | 09:01 |
| eject_ck | after long break | 09:01 |
| lordievader | You want the apt history log 😉 | 09:01 |
| lotuspsychje | lordievader: he got 5 xenial servers not booting anymore | 09:02 |
| eject_ck | no no | 09:02 |
| lordievader | `/var/log/apt/history.log` to be precise. | 09:02 |
| eject_ck | I got 1 server out of 5 not bootable | 09:02 |
| lotuspsychje | ah | 09:02 |
| lordievader | `dpkg -l` just shows what is installed. The apt history log shows what it did, what it updated, installed, etc. | 09:03 |
| eject_ck | adding dis_ucode_ldr to boot options helped to start server | 09:03 |
| eject_ck | then I ran apt update && initramfs -u | 09:03 |
| Mudchains | Good morning all. I have a old 8.04 ubuntu server with mysql databases on it. I want to setup a new 18.04 server and migrate the databases. Is there any guide lines about scsci controllers and disk design ? I am running VMware Vcenter/ESX 6.5 | 09:03 |
| Mudchains | hi lotuspsychje ;) | 09:04 |
| eject_ck | then I downloaded latest microcode from Intel and put it into /var/firmware and restarted | 09:04 |
| eject_ck | it started with no problems | 09:04 |
| lordievader | Mudchains: What you could do is boot a live-usb/iso of 18.04 and check if everything works. | 09:10 |
| Mudchains | lordievader: I am running 18.04 on multiple VM's and they are working fine. They have only a 'old' disk design and scsci controller attached. | 09:12 |
| Mudchains | For the next VM's I want to make a template :) | 09:13 |
| lordievader | If you only have databases on there, wouldn't it be a better idea to setup a new VM with 18.04 install maria-db and transfer the databases to the new VM? | 09:13 |
| Mudchains | lordievader: thats my idea also | 09:14 |
| Mudchains | lordievader : do you have experience with the vmware paravirtual scsci controller and performance? | 09:17 |
| lordievader | No, I use kvm/qemu. | 09:17 |
| Mudchains | ah ok :) | 09:17 |
| lordievader | I try to stay away from vmware 😉 | 09:17 |
| Mudchains | we are running 310+ machines on vmware atm :) | 09:18 |
| Mudchains | google doesnt say anything about optimized ubuntu templates for vmware unfortunally | 09:19 |
| === jelly-home is now known as jelly | ||
| === lotuspsychje_ is now known as lotuspsychje | ||
| Mudchains | lordievader: the new server is up and running, now the most time taking job..mysql, optimalisation and db migration.. :) | 09:59 |
| Mudchains | lordievader: why choosing maria-db btw? | 10:17 |
| lordievader | Because mysql is Oracle now. | 10:18 |
| lordievader | Maria-db is drop-in replacement. | 10:18 |
| Mudchains | lordievader: thats the only reason? :) | 10:19 |
| lordievader | For me it is, but I've moved away from mysql alltogether. | 10:20 |
| lordievader | A more indepth comparison: https://blog.panoply.io/a-comparative-vmariadb-vs-mysql | 10:21 |
| Mudchains | lordievader: I just readed it haha :D | 10:23 |
| ahasenack | good morning | 11:03 |
| rbasak | o/ | 11:04 |
| ahasenack | hi rbasak | 11:05 |
| ahasenack | kstenerud: did you see my notes about schleuder? | 11:07 |
| ahasenack | I added a bug to the exim4 card comments | 11:07 |
| kstenerud | Yes, so we need to fix schleuder to unblock exim4 right? | 11:09 |
| ahasenack | yep, for a loose definition of "fix" | 11:10 |
| ahasenack | might need kicking out too, I asked in #ubuntu-release yesterday, didn't get a response | 11:10 |
| Mudchains | lordievader: pff what a job, migrating the databases xD | 11:18 |
| lordievader | Is it? | 11:19 |
| lordievader | Dump, scp, import. Right? | 11:19 |
| Mudchains | at least the new ubuntu 18.04 server is up and running :) | 11:19 |
| Mudchains | lordievader: yes thats correct, also found a query to copy the mysql users | 11:20 |
| Mudchains | the most annoying part is all the application/odbc connections | 11:22 |
| awalende | Hi there, is it possible to write iptable rules for vlans? | 14:41 |
| awalende | Id like to block all incoming on a vlan interface of mine | 14:42 |
| sdeziel | awalende: yes, -i and -o support any interface name | 14:42 |
| awalende | sooo "iptables -P INPUT DROP -i vlan118" should do the trick | 14:43 |
| sdeziel | awalende: well, -P doesn't accept -i | 14:44 |
| sdeziel | awalende: -P is to set the chain policy (aka default faith of a packet reaching the end of the chain) | 14:45 |
| sdeziel | awalende: but any -I/-A rules that you have can use -i vlan118 | 14:45 |
| sdeziel | awalende: ex: "iptables -A INPUT -i vlan118 -j DROP" | 14:46 |
| awalende | ah okay, I'll try this. Thanks! | 14:46 |
| awalende | mhh weird, "iptables -L" shows me that I have a new DROP rule. However this list does not show my any information on which vlan this rule is enforced. | 14:51 |
| awalende | "DROP all -- anywhere anywhere " | 14:52 |
| sdeziel | awalende: could you pastebin "iptables-save" ? | 14:52 |
| TJ- | awalende: "iptables -nvL" | 14:52 |
| awalende | https://pastebin.com/k5YhK1RZ | 14:54 |
| awalende | ah I believe "iptables -nvL" did the trick, I see the rule for vlan118 now | 14:55 |
| awalende | -nvl - > https://pastebin.com/WViuyD5G | 14:56 |
| awalende | thanks for your help folks :) | 14:57 |
| sdeziel | awalende: np. FYI, you can use prefix matching for input/output devices like this "-i vlan+" | 14:59 |
| sdeziel | I find this quite useful at times so I thought I'd mention it ;) | 14:59 |
| awalende | :) | 15:00 |
| === ossurayynot is now known as tonyyarusso | ||
| herald85 | hi, i keep having issues during updates with downloading the required version of linux-headers. When I manually browse to http://security.ubuntu.com/ubuntu/pool/main/l/linux/ and click on linux-headers-4.4.0-141_4.4.0-141.167_all.deb it also fails to download. Anyone know how I can work around this? | 16:15 |
| ansyeb | hello. how it that possible? https://pastebin.com/YgJttRpw | 16:16 |
| ansyeb | what is on 22001? | 16:16 |
| ansyeb | <SerajewelKS> ansyeb: almost certainly a -R forward from one of your users | 16:16 |
| ansyeb | could someone provide a link to the corresponding manual page? | 16:16 |
| jelly | ansyeb: man ssh_config, search for RemoteForward | 16:20 |
| ansyeb | I found this: http://www.unixwiz.net/techtips/ssh-agent-forwarding.html#fwd | 16:20 |
| jelly | ssh has about 4-5 different forwardings and agent forwarding or X11 forwarding is not relevant for -R | 16:21 |
| jelly | -R is a ssh client command line option that opens a listener on the remote side and tunnels tcp connections somewhere visible to the client side | 16:22 |
| ansyeb | oh man.. | 16:22 |
| jelly | serajewelks suggested you were seeing a remote listener side of such a setup | 16:23 |
| jelly | the* remote listener side | 16:23 |
| jelly | if you want web search keywords: ssh remote forwarding | 16:24 |
| ansyeb | щл ен | 16:24 |
| ansyeb | ok ty | 16:24 |
| jelly | it's a way to enable access to a service that ssh server you connected to can't otherwise reach; somethimes used as a workaround instead of having to punch holes in firewalls | 16:26 |
| jelly | ansyeb: what does "ps -fp 1973" say? it might be interesting to see what is its parent process, that might confirm the -R theory. However,but it is somewhat unusual for a sshd process with a -R listener socket open to be running as root | 16:32 |
| baffle | I have a server with 2*L3 uplinks; The uplinks has a /31 for basic connectivity, and the same /32 on both interfaces+loopback. 0.0.0.0/0 is routed via the /31 on both interfaces, and src set to the /32. Packets gets sent randomly out via both interfaces, but there is asymmetrical routing so replies might come to the other interfaces; If this happens, the package seems to just disappear. I have | 17:29 |
| baffle | rp_filter set to 0, what else have I forgotten? Iptables INPUT/FORWARD is set to ACCEPT... | 17:29 |
| baffle | Anyone have any ideas what it could be? | 17:29 |
| sarnold | baffle: rp_filter on *all* interfaces? or just the global config? | 19:32 |
| baffle | sarnold: It's set to 2 on all interfaces.. | 19:50 |
| sarnold | baffle: this guide suggests assymetric routing uses would benefit from '1' https://www.tldp.org/HOWTO/Adv-Routing-HOWTO/lartc.kernel.rpf.html | 19:52 |
| baffle | sarnold: That's weird, this does not match documentation in https://www.kernel.org/doc/Documentation/networking/ip-sysctl.txt <- 1 is strict, 2 is loose. 0 is disabled. | 19:58 |
| baffle | (I have it set to 2, not 0, so that was wrong on my part) | 19:58 |
| baffle | Set to 0 on all/default/<interfaces> now, same behaviour. | 20:00 |
| baffle | And I have enabled log_martians too, and there are no entries in any logs. :-/ | 20:00 |
| sarnold | baffle: huhn. that is more or less exactly opposite of the what the guide said :( | 20:02 |
| baffle | sarnold: Any tips on how to debug further? | 20:14 |
| sarnold | baffle: nothing good, I'm insanely rusty on router kinds of things :/ .. maybe firewall packet counts? perhaps they're being blocked by rules? | 20:15 |
| TJ- | baffle: I may be confused but your original question seems to be talking about *replies* coming into this server on a different interface - rp_filter is about sending replies *out* from this system | 20:17 |
| sdeziel | rp_filter is a source validation mechanism so should apply to inbound traffic I think | 20:19 |
| baffle | sdeziel: But maybe in the forwarding path? | 20:19 |
| baffle | I.e. for packets passing. | 20:19 |
| baffle | (Typically a router) | 20:19 |
| sdeziel | baffle: AFAIK, rp_filter is applied on traffic reception even in the forwarding case | 20:20 |
| sdeziel | baffle: I don't have the time to look myself but surely sharing "ip a; ip ro" would help folks have a better idea of your setup | 20:21 |
| baffle | sdeziel: Oh. I forgot the paste-link I prepared... https://paste2.org/FLH32Z5F | 20:22 |
| baffle | (rp_filter is now set to 0, not 2 as in the paste) | 20:22 |
| sdeziel | baffle: is 10.200.0.1/32 supposed to be some kind of HA IP or Virtual IP? | 20:22 |
| baffle | sdeziel: Yes, it is supposed to be a HA IP. | 20:23 |
| sdeziel | baffle: then I wouldn't expect to see it configured on the 2 NICs at the same time. | 20:23 |
| baffle | sdeziel: Well, I only had it on loopback before, then packets could be sent to it from an external host.. I.e. to reach a service bound to that IP. But for the host to source packets to that IP, and not the /31 linknets, one both needs to set the src in ip route + set IP on the interfaces. If IP is not set on the interfaces, packets will not be sent out by the kernel.. | 20:25 |
| sdeziel | baffle: looks like you are running bgp which is unexplored territory for me, sorry ;) | 20:25 |
| sdeziel | baffle: I would have think that you could set just an ip route with source specification without needing to actually have the IP configured on the real outbound NIC (just lo0) | 20:26 |
| sarnold | btw how does a /31 work? there's two addresses, and the all-ones-equiv would be broadcast.. leaving the all-zeros for the one host? | 20:27 |
| baffle | sdeziel: Yes, it's BGP, but all it does it populate the routing table, and announce the /32 to the switches. | 20:28 |
| baffle | sdeziel: If I remove the IP from the outbound NICs, and just put it in lo0, and have src set to the IP, no packets go out.. | 20:29 |
| sdeziel | baffle: your default route with 2 nexthops looks good to me. Have you confirmed with tcpdump what's going on? | 20:30 |
| sdeziel | sarnold: I'm suspecting some kind of p2p setup | 20:31 |
| sdeziel | ipcalc says there are no broadcasts for /31: https://paste.ubuntu.com/p/yhR2dkxS8B/ | 20:32 |
| sdeziel | ipcalc also mentions https://tools.ietf.org/html/rfc3021 for /31 ranges :) | 20:33 |
| sarnold | ha! of course there's an rfc to answer my exact question :) | 20:34 |
| sarnold | thanks sdeziel | 20:34 |
| baffle | sarnold: It works great for linknets between routers, and there is no network/broadcast address. | 20:37 |
| baffle | sdeziel: Yeah, if I source icmp packets from interface ens1f0 I get echo+reply back on ens1f0. If I source icmp packets from interface ens1f1 the echo is sent out from ens1f1, but the reply comes back on ens1f0 (due to what I ping being a few hops away, and having a best path via the switch ens1f0 is connected to) | 20:40 |
| sdeziel | baffle: have you tried "ping -I 10.200.0.1 10.100.1.5" ? | 20:43 |
| sdeziel | baffle: I'd expect it to work and load balance the echo requests evenly between the 2 NICs since they have the same weight | 20:44 |
| sdeziel | baffle: the echo replies might all come via ens1f0 though if the switch behaves that way | 20:45 |
| sdeziel | baffle: out of curiosity, why deal with this at L3 instead of L2 (LACP, bonds, etc)? | 20:46 |
| baffle | sdeziel: Uh, that worked. One minute, I'll check something.... | 20:51 |
| baffle | sdeziel: Whaddayaknow. Facepalm time. It works fine, and probaly has been all along, I think I was looking the wrong place all along. All day. | 20:54 |
| sdeziel | baffle: hehe | 20:54 |
| sarnold | uhoh | 20:56 |
| sarnold | whatr exactly was the wrong thing in question? :) | 20:56 |
| baffle | sdeziel: The reason for going with L3 instead of L2 is to avoid having MLAG on the switches, I've seen that (and stacking) fail too many times.. | 20:59 |
| blackflow | sarnold: oh hey, you're a ZFS fan amirite? | 20:59 |
| sdeziel | baffle: OK | 20:59 |
| sarnold | blackflow: yeah | 20:59 |
| blackflow | sarnold: is the ZFS wiki page editable only by ubuntu devs, or community? because the uses cases are blatantly lying :) https://wiki.ubuntu.com/ZFS | 21:00 |
| baffle | sdeziel: So, instead of using L2 that we all know and love, I'm introducing more complexity with routing instead.. It is probably a bad idea.. But at least it is standardized, and you can use whatever vendor.. | 21:00 |
| blackflow | that really needs some correction, because it's very much false. | 21:00 |
| sarnold | blackflow: I'd expect anyone in the right launchpad group would be able to edit it | 21:00 |
| blackflow | should I open a bug report then? | 21:01 |
| blackflow | eg. Jack's use case is fiction. ZFS does no such thing. | 21:01 |
| sdeziel | baffle: I've heard good things about L3 redundant setups so I guess it's just a matter of fully understanding this new paradigm | 21:01 |
| sarnold | blackflow: hah yeah that looks way wrong | 21:01 |
| blackflow | so is Ari's use case, ZFS does not do that :) | 21:01 |
| sarnold | ha | 21:02 |
| baffle | sarnold: I think what I originally observed, but failed to catch, was that outgoing connections from a container got masqueraded (randomly) to linknet IPs on interfaces + the "HA" IP. | 21:02 |
| lordcirth_ | Yeah, Jack's is handled by btrfs, I'm not aware of any filesystem that just grabs storage devices lol | 21:03 |
| sarnold | blackflow: if you want to edit the wiki, this is the group to join https://launchpad.net/~ubuntu-wiki-editors -- many other groups are already included on the thing, so maybe it'd make sense to join one of the other groups instead of this one | 21:04 |
| sarnold | blackflow: I've got to run for lunch.. if you'd rather not bother, just let me know and I'll happily delete those usecases :) | 21:04 |
| blackflow | sarnold: thanks, I'll see what I can do first. | 21:04 |
| sdeziel | baffle: the masquerading shouldn't be random since your default route says to go out with 10.200.0.1, no? | 21:05 |
| sarnold | blackflow: thanks | 21:05 |
| blackflow | sarnold: bon apetit! | 21:05 |
| sarnold | :D | 21:05 |
| baffle | sdeziel: Yes, but it is still very hard to know what is the correct way to design a spine/leaf design with full redundancy on hosts.. Some designs seems to think that spine/leaf should be core, with ToR switches connected to the leafs, and host using L2 to one ToR switch. Or LACP/MLAG to two ToR switchces. Some designs use ToR switches as leafs (as I do).. But that both in a rack should use iBGP and | 21:05 |
| baffle | bgp and be in the same AS.. Some have the same AS on spines.. It's very confusing.. | 21:05 |
| baffle | sdeziel: That's what I tought... | 21:05 |
| sdeziel | baffle: the only semi-random (round robin I think) portion would be the outbound NIC the kernel picks | 21:06 |
| baffle | sdeziel: But I'll modify the rule to have --to-source.. | 21:06 |
| sdeziel | baffle: out of curiosity, if you run this multiple time, do you see the kernel alternating the outbound NIC: ip ro get 1.1.1.1 | 21:07 |
| baffle | sdeziel: No, that returns same IP consistently. And I've set sys/net/ipv4/fib_multipath_hash_policy to 1 (L4).. But 1 sec, I'll see what happens. | 21:11 |
| sdeziel | baffle: yeah, same source IP but what about the dev? | 21:11 |
| baffle | sdeziel: Same device, same link IP, same source IP. I.e. -> 1.1.1.1 via 10.20.128.32 dev ens1f0 src 10.200.0.1 uid 1000 | 21:12 |
| baffle | sdeziel: But I assume that is just cached. If I actually generate TCP traffic to the same host now, the flows round-robin. | 21:13 |
| baffle | sdeziel: I'll modify the masquerade rule and test now.. | 21:13 |
| sdeziel | baffle: probably but I would have appreciate the kernel telling you about the round robin thing | 21:13 |
| sdeziel | "ip route get fibmatch 1.1.1.1" maybe? | 21:15 |
| baffle | sdeziel: That works, returns both path. | 21:20 |
| baffle | s | 21:20 |
| baffle | sdeziel: Also, manually replacing MASQUERADE with -J SNAT --to-source works a treat. | 21:21 |
| sdeziel | baffle: thanks good to know | 21:21 |
| sdeziel | baffle: I don't understand why MASQUERADE would do the wrong thing though | 21:22 |
| teward | masquerade uses the primary IP address on the system, if I'm not mistaken | 21:23 |
| teward | and not "alternative IPs" (secondary, tertiary, extra, etc.) | 21:23 |
| sdeziel | teward: it should make a decision based on the info from routing table, or at least that would be a logical (to me) way of doing it | 21:25 |
| teward | while I agree with you, i'm also coming in late. | 21:25 |
| teward | so I'm not up to speed :P | 21:25 |
| baffle | teward: What is the "primary" IP anyway? | 21:27 |
| sdeziel | that's what the routing table tells you it is | 21:27 |
| teward | unless your routing tables are screwed, the 'default route' according to the routing table typically | 21:28 |
| teward | usually the first IP address on an interface if you don't have any custom routing tables in play | 21:28 |
| baffle | Hmm, wonder what happens if I reorder IP addresses in netplan. | 21:29 |
| teward | just as an FYI I came in late, did you share your configuration? Do you have custom policy-based route rules set up? | 21:30 |
| teward | (which would therefore alter the 'default routes') | 21:30 |
| sdeziel | teward: 2 nexthops with same weight as default gw | 21:30 |
| sdeziel | https://paste2.org/FLH32Z5F | 21:30 |
| teward | hah BGP is at play I see | 21:31 |
| teward | sdeziel: I usually consider in a Multi IP scenario SNAT/DNAT is better than the MASQUERADE functionality in iptables | 21:32 |
| teward | just from experience | 21:32 |
| baffle | sdeziel: I totally agree. | 21:32 |
| sdeziel | teward: agreed but I would still expect MASQUERADE to do the right thing in such scenario | 21:32 |
| teward | sdeziel: my two cents is I call masquerade a 'hackish' way to SNAT/DNAT automagically. | 21:33 |
| teward | just my thoughts on it :P | 21:33 |
| sdeziel | this automagic should be reliable ;) | 21:34 |
| teward | sdeziel: when is anything networking related EVER reliable :p | 21:34 |
| sdeziel | teward: lo is pretty reliable but that's the exception | 21:36 |
| baffle | Now the /32 is the first IP on both interfaces, but MASQUERADE still chooses the link-net as NAT source. ¯\_(ツ)_/¯ | 21:37 |
| baffle | Guess I'll have to disable the automatic creation of NAT rules in Docker. Maybe it's time to check if they've added more functionality.. | 21:38 |
| teward | sdeziel: well other than that lol | 21:38 |
| sdeziel | baffle: oh well, I was wrong (again) :P | 21:39 |
| teward | baffle: to be fair in my containerized environments (EXCEPT for this laptop, because it has only 1 IP lol), i never trust MASQUERADE to do what I want lol | 21:39 |
| teward | always SNAT everything :P | 21:39 |
| teward | my two cents. | 21:39 |
| baffle | teward: I don't think I have a choice. | 21:42 |
| Mudchains | i love it when a old optimized my.cnf of mysql5.0 fixes the new slow installed mysql5.7 server | 21:42 |
| Mudchains | first i changed the scsi controllers, but then ubuntu didnt start up anymore haha | 21:43 |
| baffle | sdeziel/sarnold/teward++: Thanks for all the help! | 21:44 |
| sdeziel | baffle: yw | 21:46 |
| === lifeless_ is now known as lifeless | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!