| === BenderRodriguez is now known as RobertMuellerDid | ||
| === RobertMuellerDid is now known as BenderRodriguez | ||
| === cpaelzer__ is now known as cpaelzer | ||
| Skyrider | Greetings all | 15:05 |
|---|---|---|
| Skyrider | I am in need of some help.. think my firewall, is... kinda? busted. It appears all allow rules in ufw works perfectly fine.. though, the deny/reject doesn't work at all. | 15:05 |
| Skyrider | I used a simple rule to deny my mobile IP to connect to the server, yet I can still load sites just fine. | 15:06 |
| teward | we'd need to see your rules you're trying to set up | 15:07 |
| teward | also UFW is specifically based on first-match so if your ALLOW rules are before your DROP/REJECT rules it will still allow | 15:07 |
| Skyrider | Anywhere DENY 143.179.70.207 - is above the allow rules | 15:08 |
| teward | and you're *sure* your phone is using that IP actively? (send your phone to https://ifconfig.co to find out) | 15:09 |
| teward | if it's NOT when it goes out to the 'net then that's why your rule doesn't work | 15:09 |
| Skyrider | According to that site, my mobile is using 143.179.70.207 | 15:10 |
| Skyrider | Which matches with the deny rule. | 15:10 |
| Skyrider | https://i.imgur.com/ZCQglP5.png | 15:11 |
| Skyrider | Even when I reload ufw, doesn't make much difference. | 15:14 |
| teward | `ufw status numbered` please - i think you actually do have a rule ordering issue | 15:16 |
| teward | (AND include the headers please) | 15:16 |
| teward | (Because reading that output is pain) | 15:16 |
| teward | just want to verify the numerical ordering is right | 15:16 |
| teward | because I just spun a container up with ufw rules, and did an allow rule for SSH but also a REJECT rule for my IP and it puts the SSH Allow above | 15:17 |
| teward | (which is important to note because it will match the 22 ALLOW rule first) | 15:18 |
| teward | also make sure your phone isn't using IPv6 if v6 is on your infra | 15:19 |
| Skyrider | https://paste.ubuntu.com/p/DhHby6f5F4/ | 15:19 |
| Skyrider | Not sure why there are 2x 22'c, I'll erase that later on. But, I do believe the order is correct. | 15:20 |
| teward | probably because at one point you specified just port 22 which is any protocol | 15:21 |
| Skyrider | All reject/denies are above the allowed rules.. so.. I'm, honestly confused. | 15:21 |
| teward | so am I let me do some poking | 15:21 |
| teward | what's your box's version? 18.04? | 15:22 |
| teward | just to make sure i'm testing the right version :p | 15:22 |
| lordievader | Skyrider: Could you pastebin the output of `sudo iptables-save`? | 15:23 |
| teward | that as well | 15:23 |
| teward | because ufw might be doing a weirdness behind the scenes | 15:24 |
| teward | Skyrider: how're you testing the DENY rule? | 15:25 |
| teward | out of curiousity | 15:25 |
| teward | because `ping` isn't denied by ufw by default | 15:25 |
| teward | so you will always get false info from `ping` | 15:25 |
| teward | (just confirmed this in an 18.04 box) | 15:25 |
| Skyrider | ya, 18.04 and lemme grab the iptables rules. As testing, using mobile device to check my domains. Should be rejected. | 15:26 |
| teward | so, accessing it via HTTP and such then | 15:27 |
| Skyrider | yup | 15:28 |
| teward | ahhhhh okay i replicated your issue | 15:28 |
| teward | I *think* the ALLOW rules get placed in a higher priority in iptables | 15:28 |
| teward | which is stupid | 15:28 |
| Skyrider | https://paste.ubuntu.com/p/s8tMkSPzpx/ | 15:29 |
| teward | somehow | 15:29 |
| lordievader | I guess it is the ufw-user-input chain | 15:33 |
| lordievader | Line 99 | 15:33 |
| Skyrider | not local? | 15:34 |
| lordievader | Which happens before line 60 (reject), due to line 57 calling the ufw-user-input chain. | 15:34 |
| teward | lordievader: i think there may be other bugs... | 15:37 |
| teward | perhaps this is because i'm running in a container BUT | 15:37 |
| teward | with a REJECT on my IP and an ALLOW on 80 with apache2 installed it still permits me in | 15:37 |
| Skyrider | Interesting fact I just discovered. | 15:38 |
| Skyrider | The deny/rejection on IP didn't allow my mobile to connect to SSH. | 15:38 |
| teward | but did for HTTP? | 15:38 |
| Skyrider | Indeed | 15:38 |
| teward | i assume *caching* was eliminated as a factor | 15:38 |
| teward | because browsers are evil | 15:38 |
| teward | (my tests are with curl :) | 15:38 |
| teward | (which seem to suggest the HTTP Allow rule takes precedence) | 15:39 |
| Skyrider | Once I erased my cells IP rule, I was able to connect again. | 15:39 |
| Skyrider | I assumed all reject/deny were systemwide, even for http(s). | 15:40 |
| teward | they should be | 15:41 |
| teward | this seems like a bug | 15:41 |
| teward | but i need to verify somehow | 15:41 |
| teward | (ufw isn't doing logging right even at full levels) | 15:41 |
| teward | okay i think something's actually odd on your env | 15:45 |
| teward | because I realized I fatfingered a rule on my side | 15:45 |
| teward | and it's now working | 15:45 |
| teward | https://paste.ubuntu.com/p/H3p6F5PnH6/ | 15:45 |
| teward | and a curl rejects to it | 15:45 |
| teward | (curl doesn't have a cache) | 15:45 |
| teward | Skyrider: they should be | 15:46 |
| teward | what you aren't doing is verifying if your browser is caching or not | 15:46 |
| lordievader | If you want a detailed answer why something went through or not look into the trace functionality of netfilter. | 15:46 |
| teward | nuke your browser cache | 15:46 |
| teward | and try again | 15:46 |
| teward | because browsers are NOTORIOUS with caching if you've gone to the page at least once | 15:46 |
| teward | Skyrider: readd your REJECT rule in the right spot and then erase your browser cache and try again on your phone | 15:46 |
| teward | chances are it'll reject | 15:46 |
| teward | and your browser was simply caching content | 15:46 |
| teward | (which is usually the case for browsers) | 15:47 |
| Skyrider | I shall try.. however, would like to add that last week I mentioned that a deny/reject rule I added with ip range kept scanning my sites. | 15:48 |
| Skyrider | Was forced to block the IP range through nginx instead. | 15:48 |
| teward | hmmm | 15:48 |
| teward | just as an FYI | 15:48 |
| teward | this is why I don't use ufw | 15:49 |
| teward | it has some... nuances... | 15:49 |
| teward | I just control everything via iptables directly :P | 15:49 |
| teward | ufw is nice, but it can be weird and glitchy | 15:49 |
| Skyrider | Ya.. I find iptables a bit... complicated. | 15:49 |
| teward | most users do unfortunately, hence why they use ufw. | 15:49 |
| teward | the problem I find is, ufw does a lot of other 'stuff' with its rules that can be glitchy | 15:49 |
| teward | and ufw logging is poor :P | 15:49 |
| jdstrand | if there are weird glitchy issues in ufw, please file bugs | 15:49 |
| teward | jdstrand: you mean like `ufw logging full` not actually logging anything anywhere? :P | 15:50 |
| teward | (at least, in LXD containers) | 15:50 |
| Skyrider | I re-added the the IP, though no success. Gotta admit, I didn't clean the cache. But I went to forum pages that I never went to before on mobile, even private area's. Wasn't blocked a single time. | 15:50 |
| jdstrand | teward: that isn't ufw's fault. unprivileged containers don't have access to kmsg and that is where the kernel logs | 15:50 |
| teward | jdstrand: this is a privileged container ;) | 15:50 |
| teward | and it still can't access | 15:50 |
| jdstrand | again, that is the kernel, not ufw | 15:51 |
| Skyrider | Even created a new thread on the forums with my mobile :p, you can't cache that. | 15:51 |
| teward | hmm | 15:51 |
| teward | let me spin up a VM | 15:51 |
| teward | remove the kernel headaches from the equation for testing | 15:51 |
| weedmic | why not use iptables, then you can save it to a file and at any time run it to instantly set it the way you want? | 15:51 |
| teward | and get some niiice auditing to try and replicate this | 15:51 |
| teward | weedmic: because they find it 'complicated' :P | 15:51 |
| teward | most users do | 15:51 |
| jdstrand | logging is also complicated by journald | 15:52 |
| jdstrand | so depending on the release in the container, there may be some stuff going on with journald | 15:52 |
| weedmic | well, it's just a language (of sorts) - it's only got like 5 things in it - but I could see that - I find it easier than suse's gui for firewall | 15:52 |
| Skyrider | If there's something that is similar to ufw (in simplicity), but adds rules as iptables would.. I don't mind. | 15:52 |
| Skyrider | Never had any issues with ufw until a few weeks ago. | 15:53 |
| * jdstrand also notes that ufw uses iptables-restore syntax under the hood | 15:53 | |
| jdstrand | (man ufw-framework) | 15:53 |
| jdstrand | Skyrider: please file bugs if there are issues | 15:53 |
| teward | wellllllllllll | 15:53 |
| teward | I was writing a python frontend to `iptables` that does much like what `ufw` does... | 15:54 |
| teward | but I got busy/bored/distracted | 15:54 |
| Skyrider | I could report a bug, but need to be sure that this isn't my fault XD | 15:54 |
| teward | jdstrand: well rsyslogd dumps it to /var/log/ufw.log and that oddly enough works | 15:54 |
| teward | Skyrider: let me set up a test env with logging, etc. give me a minute then i'll coopt you to do some testing | 15:54 |
| teward | *goes to the Cloud* | 15:55 |
| Skyrider | ^_^ | 15:56 |
| jdstrand | teward: its gonna depend on what is available in the container, how the container is configured, how logging overall is configured, the kernel, etc. ufw does ship an rsyslog snippet, yes | 15:56 |
| teward | right | 15:56 |
| teward | but this is why i'm going to the Cloud and VMs for a full env | 15:56 |
| teward | thank you cloud VPS deployment xD | 15:56 |
| jdstrand | but the imklog | 15:56 |
| jdstrand | module from rsyslog might not be able to read the kernel logs | 15:57 |
| jdstrand | so, it can't put it in ufw.log | 15:57 |
| jdstrand | but again, lots of variables | 15:57 |
| jdstrand | the container host should still see it though | 15:58 |
| teward | yep which is why i'm spinning a cloud instance up to test with :p | 15:59 |
| teward | should help | 15:59 |
| Skyrider | Hopefully with results matching my issue... makes things easier. | 16:01 |
| teward | just takes a while to launch xD | 16:03 |
| Skyrider | Cloud aint that fast ey | 16:03 |
| teward | not as fast as I'd like but I don't use predeployed images :P | 16:03 |
| teward | I boot the ISOs and deploy my own settings :P | 16:03 |
| teward | Skyrider: check your PMs I sent you a location to go to | 16:19 |
| teward | jdstrand: i think we've narrowed it down | 16:42 |
| teward | but it's not a ufw bug | 16:42 |
| teward | it's how their android phone behaves with Internet | 16:42 |
| teward | (everything goes via Google Proxies for some reason) | 16:42 |
| teward | which makes sense if it's using stock Google Chrome on the Android :P | 16:42 |
| teward | Skyrider: ^ | 16:42 |
| teward | as to why the /16 wasn't blocking I am unsure, because by all intents and purposes it shouold have | 16:57 |
| teward | can't rule out proxies at play though | 16:57 |
| jdstrand | teward: oh, interersting. yeah, clouds, their kernels and their logging infra can certainly affect things. good luck tracking it down! | 17:15 |
| teward | KVM VPS on OpenStack though... | 17:15 |
| teward | Thats full virtualization ;) | 17:15 |
| setuid | Need some Android Emulator running on top of an instance inside that OpenStack though :D | 17:18 |
| Skyrider | Guess my headless server is busted then | 17:18 |
| JanC | the "google proxies" = AMP stuff ? | 17:30 |
| teward | JanC: Not sure, but Skyrider had Google Data Saver on in their Chrome browser | 17:39 |
| teward | that was probably a factor | 17:39 |
| teward | jdstrand: but Skyrider DOES have something wonky up with their environment, even with the rules in the proper places it still allows connections. Can't rule out caching, but something's up with their system | 17:52 |
| teward | not a ufw bug per se, maybe something specific to their config | 17:52 |
| jdstrand | Skyrider: note that connection tracking may be an issue. usually this indicates either other firewall software is installed and is competing with ufw or something is not configured correctly in the kernel. you can run 'sudo /usr/share/ufw/check-requirements' for the latter | 18:03 |
| Skyrider | jdstrand: Command requests to create rules. | 18:22 |
| Skyrider | teward: Does ufw override iptables rules by any chance? | 18:22 |
| teward | ufw *is* iptables rules | 18:22 |
| teward | it will overwrite any other loaded netfilter rules | 18:22 |
| teward | (which is iptables) | 18:23 |
| Skyrider | That, I know.. But I noticed ufw lines being used in iptables rule list. | 18:23 |
| teward | that's normal | 18:23 |
| Skyrider | Maybe just for naming purposes? | 18:23 |
| teward | because ufw *uses* netfilter/iptables under the hood | 18:23 |
| teward | it's UFW specifics for how it does its tables/namings/etc | 18:23 |
| teward | but it's still ultimately iptables, just a user-friendly frontend to manage the iptables rules | 18:23 |
| Skyrider | Maybe I should try to create an iptables rule denying my own ip address, directly from iptables. | 18:23 |
| Skyrider | Rather than ufw.. see if that works. | 18:23 |
| teward | disable ufw first, then apply a ruleset similarly. I can help you write a ruleset as such if you want | 18:23 |
| teward | ufw will overwrite any iptables rules you add | 18:24 |
| teward | (for the record I use iptables exclusively, but i have reasons xD) | 18:24 |
| Skyrider | I assume "iptables -A INPUT -s IP-ADDRESS -j DROP" would work fine. Not sure the order it would add it though. | 18:24 |
| Skyrider | Maybe I should just use iptables, and use aliasses to make it more simpler for me. | 18:24 |
| teward | `iptables -I INPUT 1 -s IPADDRESS -j REJECT --reject-with icmp-host-unreachable` would be more instantaneously identifiable as a truly effective rule | 18:25 |
| teward | of course with sudo :P | 18:25 |
| Skyrider | ^_^ | 18:25 |
| Skyrider | oki, ufw disabled. | 18:25 |
| Skyrider | What about the order? | 18:26 |
| Skyrider | Is there a command to set the order like ufw? sudo ufw insert x .. etc, etc, etc. | 18:26 |
| teward | the rule I provided adds to the top of the list and there's a default ALLOW by default in `iptables` | 18:26 |
| teward | `iptables -I` - `-I CHAIN INDEX` Insert rule into specified chain at specified index/position | 18:26 |
| teward | in this case, INPUT is the chain (as expected) and INDEX is 1, meaning top of the chain] | 18:27 |
| teward | Skyrider: i'd be happy to give you a crash course in `iptables` syntax at some point :P | 18:28 |
| teward | if it turns out this deny/reject actually WORKS | 18:29 |
| teward | ... ops? | 18:29 |
| teward | ooops* | 18:29 |
| teward | i think they killed themselves... if they did i'll have to help them fix it | 18:29 |
| * teward reminds himself that ufw can be evil | 18:29 | |
| Skyrider | *shrugs* | 18:36 |
| Skyrider | Forgot, that it would block me from .. .well, everything | 18:36 |
| Skyrider | lol | 18:36 |
| Skyrider | Was forced to reboot and reset the rules. | 18:36 |
| Skyrider | Interesting fact though that blocking my IP in iptables disconnected me almost instantly from everything.. while if I do the same on ufw, it didn't. | 18:37 |
| Skyrider | So my guess is.... ufw.. is, busted on my end. | 18:37 |
| jdstrand | teward, Skyrider: actually, ufw will *not* overwrite rules you have unless you specify MANAGE_BUILTINS=yes in /etc/default/ufw, which is not the default | 18:41 |
| teward | jdstrand: interesting. so it'll what, append rules? | 18:42 |
| tds | Skyrider: that depends on where exactly you're adding the drop/reject rules - if they go before a rule using conntrack allowing existing connections, that will terminate all open connections, while after will only make them affect new connections | 18:42 |
| teward | last i checked ufw overwrites whatever I put in `iptables` when ufw is disabled | 18:42 |
| jdstrand | teward: no, it works on its own chains that are all prepended with ufw-* | 18:42 |
| jdstrand | teward: this is how it plays well with libvirt, etc which adds their own rules. but something like iptables-persistent or other things that assume they own the firewall can interfere with ufw | 18:43 |
| Skyrider | The rule I was provided to add in iptables disconnected all my connections. | 18:43 |
| Skyrider | Let me try again, but only adding the web ports instead rather than a global deny. | 18:44 |
| jdstrand | still, the advice is sound to disable ufw and see if something simple is working | 18:44 |
| Skyrider | "iptables -A INPUT -p tcp --destination-port {PORT-NUMBER-HERE} -j DROP" ? | 18:45 |
| Skyrider | err, no.. lacking ip | 18:45 |
| teward | Skyrider: `sudo iptables -I INPUT 1 -s IPADDRESS -p tcp --dport 80 -j REJECT --reject-with icmp-host-unreachable` | 18:45 |
| teward | same for 443 (though you can replace the index with `2` | 18:45 |
| teward | and that'll ONLY block traffic matching the source IP address heading to port 80 on TCP | 18:45 |
| Skyrider | Thanks :) lemme try | 18:45 |
| teward | (standard HTTP) | 18:45 |
| teward | my apologies for fubaring your access, but at least we know THAT worked xD) | 18:46 |
| Skyrider | Simple reboot fixed it XD.. why isn't iptables saved though? | 18:46 |
| Skyrider | With that, I mean.. saved on reboot ,etc. | 18:46 |
| jdstrand | teward: it depends on how your are disabling ufw. if you disable and do /lib/ufw/ufw-init flush-all or other low level things (or have MANAGE_BUILTINS=yes set), then yes, it will blow stuff away | 18:46 |
| teward | because it's not stored in a file anywhere, ufw when it reenables loads data from stored rules/data and reloads it into netfilter | 18:47 |
| teward | similar to how `iptables-persistent` does the same on boot (but assumes firewall ownership and blasts things apart) | 18:47 |
| Skyrider | oki, I dropped the connection on 80 and 443.. didn't work. | 18:47 |
| tds | Skyrider: iptables itself doesn't do anything for saving on boot, if you want that you need to install something like the iptables-persistent package | 18:47 |
| Skyrider | Well, it got added.. just didn't block my access on my web stuff. | 18:47 |
| jdstrand | do note that just because ufw doesn't stomp on the rules doesn't mean that rule ordering isn't in play. iptables rules are order-dependent. if the ufw chains are first, then ufw will work, if after, they may not (ie, -A (append) vs -I (insert)) | 18:47 |
| teward | Skyrider: did you use DROP or REJECT? | 18:48 |
| teward | test REJECT | 18:48 |
| teward | instead of DROP | 18:48 |
| teward | this said, DROP should've worked too | 18:48 |
| teward | Skyrider: is this a VPS? Is it containerized (LXD, OpenVZ) or full virtual (KVM)? | 18:48 |
| teward | Skyrider: also, make sure you're actually blocking the right IP (remember the google proxy evil?) | 18:49 |
| Skyrider | Virtualization: kvm | 18:49 |
| jdstrand | Skyrider: how are you testing the connection? is the traffic going through the interface you expect to be firewalling? did you kill off any connection tracking (eg, sudo conntrack -D ...) | 18:49 |
| Skyrider | Not sure what conntrack is. | 18:50 |
| jdstrand | Skyrider: did you add the rules with -A (append) or -I (insert)? | 18:50 |
| teward | jdstrand: would that apply even if the first rule were a global matchall? If they did `-I INPUT 1` like i suggested it'd have put it at first position in INPUT | 18:50 |
| jdstrand | Skyrider: it is also possible there are vps rules that are in front of your added rules | 18:50 |
| jdstrand | I see Skyrider used iptables -A INPUT... | 18:51 |
| teward | ah that might break it | 18:52 |
| Skyrider | Copy/pasted, ya :P | 18:52 |
| teward | Skyrider: did you use `iptables -A INPUT` or `iptables -I INPUT 1` like i suggested? | 18:52 |
| Skyrider | I copy/pasted your line | 18:52 |
| Skyrider | So.. with input 1. | 18:52 |
| jdstrand | Skyrider: with INPUT 1, it worked and blocked the connection? | 18:53 |
| Skyrider | -A INPUT -s xxxxxx/32 -p tcp -m tcp --dport 443 -j REJECT --reject-with icmp-host-unreachable - and another line with port 80 according to iptables -S | 18:53 |
| Skyrider | And no, it didn't block my web connection. | 18:53 |
| teward | show us the `iptables-save` output (Pastebin it please) | 18:53 |
| Skyrider | sure | 18:54 |
| tds | `iptables-save -c` ideally :) | 18:54 |
| teward | ^ that | 18:54 |
| Skyrider | >_> | 18:54 |
| teward | ... ooooooooooops i just broke my own email, stupid firewalls and NAT... *goes to fix* | 18:55 |
| Skyrider | https://paste.ubuntu.com/p/49hZ53dx4d/ | 18:55 |
| teward | the only reason it'd be doing *that* is because your IP(s) accessing the site aren't what we think it is | 18:56 |
| jdstrand | Skyrider: how did you disable ufw? | 18:56 |
| teward | which is what was happening earlier with our tests skyrider | 18:56 |
| Skyrider | sudo ufw disable | 18:56 |
| Skyrider | Status: inactive | 18:56 |
| jdstrand | Skyrider: and you rebooted? | 18:56 |
| Skyrider | Reboot kept ufw disabled. | 18:56 |
| teward | jdstrand: shouldn't the first to input lines (24, 25) have caught regardless if the source IP matching was correct? | 18:57 |
| teward | 0:0 suggests that it never triggered/matched on sourceIP | 18:57 |
| teward | and since those rules are ABOVE the ufw rules, those should be matched regardless of UFW active/inactive | 18:57 |
| teward | at least per my nf understanding | 18:57 |
| jdstrand | Skyrider: that isn't what a disabled ufw looks like after a reboot | 18:59 |
| Skyrider | Don't look at me XD | 18:59 |
| jdstrand | (if it is disabled, after a reboot, it should have no ufw chains) | 18:59 |
| jdstrand | Skyrider: what release of ubuntu is this? | 19:00 |
| Skyrider | 18.04 | 19:00 |
| Skyrider | I removed ufw and purged it. | 19:00 |
| Skyrider | ufw rules in iptables-save are still there. | 19:00 |
| Skyrider | At least, the ufw lines. | 19:00 |
| teward | jdstrand: just ran ufw in the command line in a container | 19:00 |
| teward | aaaaaaaaaand, the ufw rules are still there when disabled | 19:00 |
| teward | even on reboot | 19:00 |
| jdstrand | Skyrider: this is a container? | 19:00 |
| teward | except on reboot* | 19:00 |
| teward | jdstrand: confirming on a VM currently as well | 19:01 |
| teward | standby... | 19:01 |
| jdstrand | teward: ufw disable pre-reboot will leave chains in place, but empty them. on reboot, they won't be added | 19:01 |
| teward | that part i confirm | 19:01 |
| teward | Skyrider: you did `sudo ufw disable` and then restarted the server yes? | 19:02 |
| jdstrand | Skyrider: what is ENABLED set to in /etc/ufw/ufw.conf? | 19:02 |
| teward | because jdstrand *is* right - it won't add those rules if `ufw` was disabled when you rebooted | 19:02 |
| tds | Skyrider: did you install iptables-persistent or anything similar? | 19:02 |
| tds | if you did, that may have saved the ufw rules for you | 19:02 |
| teward | looks to me more like you just rebooted then did `sudo ufw disable` | 19:02 |
| jdstrand | tds: yes | 19:02 |
| jdstrand | tds: that is often a problem | 19:02 |
| tds | `grep ufw /etc/iptables/rules.v4` would tell you :) | 19:02 |
| teward | (and won't exist if it's not installed or manually created) | 19:03 |
| * jdstrand is reminded he needs to add iptables-persistent detection to /usr/share/ufw/check-requirements | 19:03 | |
| teward | jdstrand: what's the underlying language for it? | 19:03 |
| teward | i might be bored enough to try :p | 19:03 |
| jdstrand | python3 for the ufw command, shell for early boot setup and iptables-restore syntax for rules | 19:05 |
| Skyrider | Ya, iptables-persistent is installed. | 19:05 |
| jdstrand | Skyrider: iptables-persistent and ufw do not work well together. choose one or the other | 19:05 |
| Skyrider | I only added it a few hours ago, when I was trying to fix all of this. | 19:05 |
| teward | remove iptables-persistent and reboot | 19:06 |
| teward | then add the `iptables` rule I stated | 19:06 |
| teward | this said | 19:06 |
| Skyrider | Do I have to reboot? :P | 19:06 |
| teward | jdstrand: my previous question stands. | 19:06 |
| teward | jdstrand: shouldn't the first to input lines (24, 25) have caught regardless if the source IP matching was correct? | 19:06 |
| teward | because of the ordering in INPUT | 19:06 |
| jdstrand | Skyrider: that's fine, but we can only debug what is happening right now | 19:06 |
| Skyrider | One moment, will disable/enable some services and reboot. | 19:07 |
| jdstrand | teward: yes, they should have | 19:07 |
| teward | jdstrand: one of the issues we ran into earlier with Skyrider was their device(s) were being routed via Google's Proxies | 19:07 |
| teward | as a result of datasaver, etc. | 19:07 |
| teward | when we blocekd the IP of their actual address that was in the access logs from the webserver they couldn't get to it | 19:07 |
| Skyrider | This time however, using my own browser on my main desktop system. | 19:07 |
| jdstrand | teward: so the incoming ip is NAT'd or something? that is going to make a source ip infeasible | 19:08 |
| Skyrider | Wait a sec... | 19:08 |
| teward | jdstrand: there's a few different factors | 19:08 |
| Skyrider | once I removed persistent, everything started to work | 19:08 |
| Skyrider | nvm.. | 19:08 |
| teward | when their main computer connected and I blacklisted that in a live system it blocked them | 19:08 |
| Skyrider | Just slow connection XD | 19:08 |
| teward | but the tricky part here is that i can't verify what's going on on their main desktop | 19:08 |
| teward | jdstrand: i also can't rule out Proxy settings at play by ISP or some other VPN or proxy system at play | 19:09 |
| teward | in which case if they ARE indeed nat'd we'd have some... headaches with that and you're rightw e'd have to ignore source IP as a valid identifier (because NAT breaks it) | 19:09 |
| teward | case in point my IP currently, it shows as my home's IP instead of ym work's IP because I'm VPN'd through home, but HTTP Proxies would behave similarly for HTTP/HTTPS connections in some cases | 19:09 |
| teward | Skyrider: confirm you're still rejected on the test IP I provided earlier as well | 19:10 |
| jdstrand | sudo tcpdump -i <interface name> -s 0 -n -vv port 80 | 19:10 |
| Skyrider | Not atm (not on my desktop)_ | 19:10 |
| jdstrand | that will also be helpful on the server ^ | 19:10 |
| jdstrand | Skyrider: just for your own debugging. you'll then see the IPs involved | 19:11 |
| teward | jdstrand: i'd agree with what jdstrand says :P | 19:11 |
| Skyrider | **rebooting** | 19:11 |
| teward | then attempt connecting yourself and see the IP(s) | 19:11 |
| teward | blehrewrherw9re | 19:11 |
| teward | *yawns* | 19:11 |
| teward | jdstrand: ewwww check-requirements is bash >.< | 19:14 |
| teward | /bin/sh * | 19:14 |
| jdstrand | oh your question was about that. yes, shell | 19:14 |
| teward | one of my more hated languages | 19:14 |
| teward | behind Ruby and Perl | 19:14 |
| Skyrider | Okay, so.. | 19:15 |
| Skyrider | Reboot done. | 19:15 |
| teward | (Ruby, Perl, pure Dash/sh, C, C++...) | 19:15 |
| Skyrider | iptables only shows 3 lines now. | 19:15 |
| Skyrider | input, forward and output. | 19:15 |
| jdstrand | heh, well, it works ok for that particular debugging purpose :) | 19:15 |
| jdstrand | Skyrider: ok, so you now have a completely clean slate | 19:15 |
| teward | jdstrand: oh fun fact: you might need to remove the python 2.* checks in this thing for focal if we're sure we're blasting Py2 away | 19:16 |
| teward | :P | 19:16 |
| Skyrider | I re-added the 80/443 rules. | 19:17 |
| * jdstrand notes that the ufw rules that were there before were all 'pass through' but it complicated the output. please, trying to reason around ufw and iptables-persistent at the same time is never good | 19:17 | |
| jdstrand | teward: yes, ufw itself is all py3, etc, etc. I do need to drop the py2 packaging which I'll do before ff. note that check-requirements looks for any python, it just happened to find py2 on the system | 19:19 |
| jdstrand | Skyrider: and the connection is still not blocked? | 19:19 |
| Skyrider | Not yet.. just reminded server is using cloudflare.. lemme disable that. | 19:20 |
| Skyrider | sudo tcpdump -i <interface name> -s 0 -n -vv port 443 - Can I ignore the the machines IP? | 19:21 |
| Skyrider | Getting spammed with it. | 19:21 |
| Skyrider | A lot of them with cksum 0x8d69 (incorrect -> 0x0873), seq 1:194, ack 1, win 229, length 193 | 19:21 |
| Skyrider | Specifically the "incorrect" part. | 19:22 |
| teward | jdstrand: https://paste.ubuntu.com/p/k5NPprmBv3/ in case you're curious *might* help, since those're the scripts iptables-persistent currently installs - if either of them exists then the package is installed :P | 19:25 |
| teward | it's a diff btw | 19:25 |
| teward | > server uses cloudflare | 19:26 |
| teward | well that explains a lot | 19:26 |
| teward | CloudFlare is equivalent to Proxies and VPN, it futzes the endpoint Source IPs | 19:26 |
| Skyrider | Side question.. if I allow a port to be used on a specific IP with iptables, does it block the rest? | 19:26 |
| teward | so you can't use `iptables` or `ufw` to filter on SourceIP if you're behind CloudFlare | 19:26 |
| teward | Which explains the problems you're having | 19:26 |
| teward | Skyrider: if you put the default policy to DROP or add a REJECT rule at the end of the rules as a default, then it will | 19:27 |
| Skyrider | Okay, that makes it different over UFW. | 19:27 |
| teward | no | 19:27 |
| teward | because iptables defaults is ALLOW | 19:27 |
| jdstrand | teward: re check-requirement, thanks :) | 19:27 |
| teward | ufw's 'default' policy is deny incoming ;) | 19:27 |
| Skyrider | Adding an accept port / ip rule through ufw, auto denies it for other ip's. | 19:27 |
| teward | jdstrand: yep, it's likely to need minor refinement. | 19:27 |
| teward | Skyrider: because the default policy is DROP | 19:27 |
| teward | in Ufw | 19:27 |
| Skyrider | Which I find, better.. in my opinion | 19:28 |
| teward | iptables doesn't have that unless you specify it as that, but that's a different discussion | 19:28 |
| teward | because you can *set* iptables to default-Deny | 19:28 |
| teward | which can cause issues if you're not careful | 19:28 |
| jdstrand | Skyrider: you can add 'and host <ip addr>' | 19:28 |
| Skyrider | Ah, that makes it better :D | 19:29 |
| Skyrider | Also... still doesn't block my IP >_> | 19:29 |
| Skyrider | ****web wise. | 19:29 |
| teward | Skyrider: if you're using Cloudflare then it won't | 19:29 |
| teward | to reiterate WHY: | 19:29 |
| teward | CloudFlare is equivalent to Proxies and VPN, it futzes the endpoint Source IPs | 19:29 |
| teward | because the traffic goes through CloudFlare's nginx proxy to try and mitigate DDoS and provide minor CDN functionality | 19:29 |
| Skyrider | Oh, I know. :) | 19:29 |
| teward | so... :P | 19:30 |
| Skyrider | Set it to DNS only | 19:30 |
| teward | and you're sure your computer sees the updated DNS? | 19:30 |
| Skyrider | Chrome / Firefox appears to be blocked now.. Internet explorer.. not so much. | 19:30 |
| teward | ... ewww IExplorer | 19:30 |
| teward | *shivers* | 19:30 |
| Skyrider | XD | 19:30 |
| Skyrider | Using it just for testing. | 19:30 |
| teward | Internet Explorer is evil and relies on Windows' DNS cache | 19:30 |
| teward | Chrome/FFox both do their own queries lol | 19:30 |
| Skyrider | Okay, soo... works now.. so all this time, cloudflare was the issue. | 19:30 |
| Skyrider | Really wish iptables would be smarter to check "behind" the proxy... | 19:31 |
| jdstrand | it can't. the info is gone from the packet | 19:32 |
| Skyrider | Fun times.. can't even use cloudflare XD | 19:33 |
| jdstrand | the T in NAT is Transalation. the host doing the translating rewrites the address before sending the packet on its way | 19:33 |
| teward | jdstrand: regarding iptables-persistent if that patch looks good I can turn that into an actual quilt patch and upload it. but i don't want to touch things that're major-critical without second sets of eyes/testing :P | 19:34 |
| jdstrand | Skyrider: but the host doing the translating sees it. perhaps there is something in the cloudfare UI that allows you to block that ip | 19:34 |
| teward | also my shell is rusty so :P | 19:34 |
| teward | jdstrand: CF will pass a header called Ray-ID | 19:34 |
| teward | and XFF | 19:35 |
| teward | nginx can check XFF | 19:35 |
| teward | iptables can't | 19:35 |
| Skyrider | iptables sounds outdated :o | 19:35 |
| teward | i think you're confusing 'firewall' with 'proxy filtration' | 19:35 |
| teward | 'firewall' is good at the packets level | 19:35 |
| Skyrider | And yea, using multiple methods online to convert the proxy ip to the users real IP. | 19:35 |
| teward | NOT at the application level | 19:35 |
| jdstrand | teward: I'm doing uploads for Debian and Ubuntu in the coming weeks. thanks for the offer | 19:35 |
| Skyrider | Set it up in nginx as well. | 19:35 |
| teward | which is what XFF / RayID involves | 19:35 |
| teward | jdstrand: ack | 19:35 |
| teward | jdstrand: just, if you use the diff verbatim, please credit me :) | 19:35 |
| jdstrand | teward: of course | 19:36 |
| teward | (it's inspiration though if you have a better way, I just fast-hashed it together) | 19:36 |
| teward | ... DAMN YOU EMAIL | 19:36 |
| teward | ... seriously hate email sometimes, I broke it bad... | 19:36 |
| teward | *goes to fix his postfix and dovecot mail system* | 19:36 |
| jdstrand | and if I modify it, I'll still reference you 'based on...' or whatever makes sense | 19:36 |
| Skyrider | I do appreciate all you guys help and efforts btw! | 19:36 |
| teward | jdstrand: ack, thank you very much :) | 19:37 |
| teward | Skyrider: glad we could help, sorry that it took so long to debug | 19:37 |
| teward | Skyrider: yeah, if you're using CloudFlare you need to use web server level filtering/blocking for the traffic on those IP(s) | 19:37 |
| teward | if you're not, you can use `iptables` | 19:37 |
| teward | it's unfortunately the case of different levels of the TCP/IP stack being at play because of CF | 19:37 |
| jdstrand | or ufw ;) | 19:37 |
| teward | true :P | 19:38 |
| * teward has a habit of typing `iptables` :P | 19:38 | |
| jdstrand | that is unfortunate, yes | 19:38 |
| teward | `iptables`/`ufw` work at one level of the TCP/IP stack (packet level), your webserver will operate at a more application-level filtration (so XFF header, etc. from Cloudflare so it IDs properly at endpoints for filtration) | 19:38 |
| teward | the second of those which is what you have to do when Cloudflare is in the equation | 19:39 |
| jdstrand | I suspect it makes since to use iptables(-persitent), ufw, or something on the server to block most stuff (ie, with rules to allow all 80, 443, etc) and then use cloudfare UI to blacklist certain IPs | 19:39 |
| jdstrand | s/since/sense/ | 19:39 |
| teward | yep | 19:40 |
| teward | though unless you pay for CF your filtration there is limited | 19:40 |
| Skyrider | I find webmin.. weird. | 19:42 |
| Skyrider | The ufw chains still exists in its firewall rules. | 19:42 |
| teward | webmin is evil don't rely on it | 19:43 |
| Skyrider | At least rules are working.. | 19:44 |
| Skyrider | What about nftables ? :P | 19:48 |
| jdstrand | Skyrider: did you 'dpkg --purge iptables-persistent'? maybe there are some lingering files that still fire on reboot | 19:50 |
| jdstrand | Skyrider: (or at the time of the removal, apt-get remove --purge iptables-persistent) | 19:51 |
| * jdstrand is confused. I thought Skyrider said after reboot it was empty... | 19:51 | |
| Skyrider | Ya, the iptables -S said it did.. just webmin for some reason cached the old ufw chains. | 19:52 |
| Skyrider | I erased webmin though. | 19:52 |
| jdstrand | oh, I see. I don't know what it was doing | 19:52 |
| Skyrider | Wow iptables is sooo limited >_> | 20:01 |
| teward | for the record | 20:03 |
| teward | to properly protect systems | 20:03 |
| teward | you need IDS/IPS *and* firewalls | 20:04 |
| teward | and WAFs | 20:04 |
| teward | IDS/IPS to protect from signature based threats, firewalls for specific IPs, WAFs for HTTP/HTTPS level application filtering when CF and stuff are in play | 20:04 |
| teward | it's a lot more complex than just "Use a firewall" | 20:04 |
| teward | esp. when Cloudflare is involved | 20:04 |
| Skyrider | Indeed, sounds complicated :o | 20:08 |
| mybalzitch | whats a waf | 20:09 |
| Skyrider | dog? :p | 20:09 |
| mybalzitch | woof? | 20:09 |
| teward | Web Application Firewall | 20:09 |
| teward | something that operates at the HTTP/HTTPS 'application' level | 20:09 |
| mybalzitch | neat | 20:10 |
| Skyrider | eg ModSecurity.. seen that around. | 20:10 |
| Skyrider | Though, kinda sucks I have to compile nginx to include MC into nginx. | 20:14 |
| === jelly-home is now known as jelly | ||
| === randomthoughts is now known as tops | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!
