| Now_its_Broken | Hi, can anyone here point to some instructions on getting openvswitch working with netplan? | 03:07 |
|---|---|---|
| lordievader | Good morning | 07:12 |
| ahasenack | rostam: try the dig tool, dig @<ip-of-dns-server> <name-you-want-to-resolv> | 11:13 |
| ahasenack | rostam: start with using the ip from /etc/resolv.conf | 11:13 |
| ahasenack | if that's 127.0.1.1 or some such, move on to your real dns server, then try 8.8.8.8 (google's), etc | 11:14 |
| ahasenack | rbasak: hi, I'm getting this from g-u merge start on samba: | 11:53 |
| ahasenack | $ git ubuntu merge start pkg/ubuntu/devel | 11:53 |
| ahasenack | 01/22/2018 09:53:01 - ERROR:pkg/ubuntu/devel version (2:4.7.3+dfsg-1ubuntu1) is after debian/sid version (2:4.7.1+dfsg-1). Are you sure you want to merge? (Pass -f to force the merge). | 11:53 |
| ahasenack | rbasak: debian/sid is at 4.7.4, though | 11:53 |
| ahasenack | seems like the debian branch(es?) are behind in the importer? | 11:53 |
| rbasak | AFAICT, sid is at 2:4.7.3+dfsg-1 and bionic-devel is at 2:4.7.3+dfsg-1ubuntu1 | 12:01 |
| rbasak | Do you see something different? | 12:01 |
| rbasak | I was looking at https://code.launchpad.net/~usd-import-team/ubuntu/+source/samba/+git/samba | 12:01 |
| ahasenack | rbasak: rmadison, and the merges page | 12:21 |
| ahasenack | both show sid at 4.7.4-+dfsg-1 | 12:21 |
| ahasenack | rbasak: sorry, the vpn dropped, this was the last I saw: | 12:29 |
| ahasenack | <ahasenack> both show sid at 4.7.4-+dfsg-1 | 12:29 |
| ahasenack | * Disconnected () | 12:29 |
| rbasak | ahasenack: ah. OK. So the problem is that the importer is straightforwardly out of date rather than inconsistently out of date in different branches I think? | 13:30 |
| rbasak | I need to sort out my VPN connection. That's blocking me from looking right now :-/ | 13:30 |
| ahasenack | rbasak: I don't know, but "out of date" sounds right | 13:31 |
| eoli3n | Hi | 13:50 |
| eoli3n | i'm trying to preseed (with ubuntu kickstart + some preseed lines) a ubuntu install on a dualboot | 13:50 |
| eoli3n | each kickstart make Win7 not bootable | 13:50 |
| eoli3n | i need to repair Win7 with CDROM | 13:50 |
| eoli3n | fact is that i need to automate 800 nodes deploy | 13:50 |
| eoli3n | here is the kickstart file -> https://ptpb.pw/F5ts | 13:51 |
| eoli3n | here's some checksums on what changed at each steps -> https://ptpb.pw/0Y6b.png | 13:51 |
| eoli3n | and here the "view details" log from win7 repair tool -> https://ptpb.pw/fxvz | 13:51 |
| eoli3n | the fact is that PARTUUID seems changing after kickstart | 13:51 |
| eoli3n | i think its due to partman | 13:51 |
| eoli3n | and i'm searching for a way to keep same sda1 PARTUUID | 13:51 |
| eoli3n | to not to have to repair WIN7 after kickstarting | 13:51 |
| eoli3n | the strange thing is that kickstarting edit only MBR's bootloarder part -> and repairing with WIN7 CDROM changes only C:\Boot\BCD | 13:51 |
| eoli3n | and every loop i do a kickstart deploy gives the same | 13:51 |
| eoli3n | while true; do ; kickstart change bootloarder ; WIN7 boot broken ; repairing with CDROM, it edits date inside sda1 ; done | 13:51 |
| eoli3n | (sorry for that huge paste) | 13:51 |
| Ussat | dont dual boot | 13:54 |
| eoli3n | are you serious ? | 13:58 |
| Ussat | very | 13:58 |
| eoli3n | cool | 13:58 |
| eoli3n | any serious help here ? | 13:58 |
| Ussat | There is, in my opinion, almost no reason to dual boot anymore with modern hypervisors | 13:59 |
| eoli3n | you doesn't have my needs | 13:59 |
| eoli3n | and i'm not asking for that help | 13:59 |
| eoli3n | that's cool from you | 13:59 |
| eoli3n | but not my ask | 13:59 |
| mason | eoli3n: Wait, you're taking a checksum of sda1 there? | 14:09 |
| mason | eoli3n: I'm assuming sda1 is your ESP, yes? | 14:09 |
| mason | I'm not completely understanding here, but I'd be more inclined to think you're seeing a problem with your efibootmgr entries. | 14:10 |
| eoli3n | mason: i'm using legacy not uefi bios | 14:10 |
| eoli3n | i took some checksum to see what kickstart change | 14:10 |
| eoli3n | sda1 is windows part | 14:10 |
| mason | eoli3n: Ah... Nowadays I wouldn't consider dual-booting without UEFI, but I'm not sure Windows 7 can deal with UEFI. Can it? | 14:11 |
| eoli3n | mason: i use old nodes | 14:11 |
| eoli3n | i don't know, problem is that i will node renew my 800 nodes now :) | 14:11 |
| eoli3n | i will not | 14:11 |
| eoli3n | * | 14:11 |
| mason | right | 14:11 |
| eoli3n | my problem needs a bit reflexion to understand what i do | 14:12 |
| eoli3n | sorry to bot be able to make it easy undestandable | 14:12 |
| mason | just noted that you've included multiple pastes - looking | 14:12 |
| eoli3n | thx, ask if any question | 14:12 |
| eoli3n | the important paste is the table | 14:12 |
| eoli3n | a bit hard to understand | 14:12 |
| eoli3n | i started at line1 with a dualboot working | 14:13 |
| eoli3n | then i reinstall with kickstart at line 2 | 14:13 |
| eoli3n | my kickstart installation needs to keep win7 working | 14:13 |
| eoli3n | after kickstart, i'm not able to boot win7 without win7 CDROM repair tool | 14:13 |
| mason | #Clear the Master Boot Record | 14:13 |
| mason | zerombr yes | 14:13 |
| mason | Guessing that's why. | 14:13 |
| eoli3n | nomp | 14:13 |
| eoli3n | nop | 14:13 |
| eoli3n | asked to dev of ubuntu kickstart | 14:14 |
| eoli3n | it does nothing | 14:14 |
| rbasak | ahasenack: I import samba by hand and it worked fine. | 14:14 |
| rbasak | Not sure what happened in the past. | 14:14 |
| rbasak | ahasenack: so it should be good for you now. | 14:14 |
| rbasak | Sorry it didn't work before. It could have been due to a gap when the importer was running. | 14:15 |
| rbasak | We haven't got the full "catch up everything" thing running yet. | 14:15 |
| rbasak | nacc: ^ | 14:15 |
| eoli3n | mason: he pasted me that -> https://paste.ubuntu.com/26409694/ | 14:15 |
| mason | eoli3n: Hm, well. Hopefully someone who's done this will come around with ideas, or there's always the mailing list. I don't run Windows anywhere so I'm not entirely clear on what it wants and what's changed out from under it. | 14:18 |
| TJ- | eoli3n: I only just came in, but it sounds to me like GRUB needs to create a menu entry for the Windows install. If you want Windows to remain the primary boot-strap bootloader then you have to prevent GRUB from writing it's bootstrap code | 14:19 |
| eoli3n | TJ-: i don't want to prevent it | 14:21 |
| eoli3n | chainloader +1 is working | 14:21 |
| eoli3n | problem is that win7, while booting, ask for repairing | 14:21 |
| eoli3n | TJ-: https://ptpb.pw/0Y6b.png | 14:21 |
| eoli3n | please check and tell me if its clear | 14:22 |
| eoli3n | when installing with kickstart, nothing changed on the disk exept ubuntu install on sda2 and bootloader 0>446 on sda | 14:22 |
| eoli3n | the strange part is | 14:22 |
| eoli3n | when i repair win7 | 14:22 |
| eoli3n | it repair by changing BCD on win7 install | 14:22 |
| eoli3n | i don't know more what it does -> here's the log file -> https://ptpb.pw/fxvz | 14:23 |
| eoli3n | grub is working | 14:23 |
| TJ- | eoli3n: is the partitioning using only MBR ? | 14:24 |
| eoli3n | what do you mean ? "using only MBR" ? | 14:24 |
| TJ- | eoli3n: it's possible to have GPT hybrid that also has a valid MBR | 14:25 |
| eoli3n | i'm using mbr not gpt | 14:25 |
| eoli3n | https://ptpb.pw/F5ts | 14:25 |
| eoli3n | https://ptpb.pw/F5ts | 14:25 |
| TJ- | OK, and which sector does sda1 start at? | 14:25 |
| eoli3n | oups sorry for double | 14:25 |
| eoli3n | 2018 | 14:25 |
| eoli3n | ahhh | 14:25 |
| eoli3n | 2048 | 14:25 |
| eoli3n | please look at the paste | 14:26 |
| eoli3n | at PRE part | 14:26 |
| eoli3n | you will see that i use sfdisk to restore part table | 14:26 |
| TJ- | eoli3n: OK, so under normal circumstances GRUB will write it's boot-strap into sector 0 and it's core image into sector #1 through 2047 | 14:26 |
| eoli3n | not from 0>446 ? | 14:27 |
| eoli3n | hm i didn't knew there was more that "boot-strap" | 14:27 |
| eoli3n | my question is | 14:27 |
| TJ- | eoli3n: 446-509 is the partition table, 510-511 is the signature | 14:27 |
| eoli3n | ok | 14:28 |
| eoli3n | so why | 14:28 |
| eoli3n | installing grub breaks win7 boot | 14:28 |
| eoli3n | as win7 bootloard is at start of sda1 | 14:28 |
| eoli3n | bootloader | 14:28 |
| eoli3n | so at 2028 | 14:28 |
| eoli3n | 2048 | 14:28 |
| TJ- | eoli3n: the process goes PC >BIOS > read sector 0 > execute code from offset 0. This is the bootloader's boot-strap code. In GRUB it then uses BIOS services to read sector #1-2047 into memory and continues executing - that is GRUB's core image, which then finds GRUB's root file-system and accesses that, loads the normal.mod and executes 'normal' command which reads /grub/grub.cfg and processes it | 14:30 |
| TJ- | (menu, wait for key press, launch OS, etc) | 14:30 |
| eoli3n | i get it | 14:31 |
| TJ- | As I recall, Windows boot-strap code in sector 0 looks for the partition that is flagged as Bootable, then reads boot code from that partition, which then reads Windows bootmgr code | 14:31 |
| eoli3n | the partition sda1 is marked as bootable | 14:31 |
| eoli3n | with "boot" flag i mean | 14:31 |
| eoli3n | so installing grub can not break any win7 install right ? | 14:32 |
| TJ- | eoli3n: right, from your table it looks as if the Win7 repair is writing something into the 'spare' sectors from sector 1 onwards - is that correct? | 14:32 |
| eoli3n | which are spare sectors | 14:33 |
| eoli3n | what i can say, is that repair tools write between 2048 and end of partition | 14:34 |
| eoli3n | i know that it edit BCD file | 14:34 |
| TJ- | eoli3n: Installing GRUB will break Windows every time 100%, since it has to write it's boot-strap code into sector 0. However, it uses os-prober to locate the Windows OS during "update-grub" and adds a menuentry for Windows | 14:34 |
| eoli3n | TJ-: i used a previous installation method which install grub with a custom script without breaking win7 part | 14:35 |
| eoli3n | win7 install i mean | 14:35 |
| TJ- | eoli3n: it sounds more likely Windows is breaking itself - by 'thinking' it needs a repair when it doesn't simply because sector 0 changed, then during the repair it goes on to change things it doesn't need to change | 14:35 |
| eoli3n | so how to make it works without repair ? dd backup then restore ? | 14:36 |
| eoli3n | i need to automate the process | 14:36 |
| eoli3n | but still, i'm not understanding on how my previous deploying method differs | 14:36 |
| eoli3n | in my previous installation/deploy method, i uncompressed a huge tar.xz on disk, then install grub in chroot with this script -> https://ptpb.pw/DR9s | 14:38 |
| eoli3n | that didn't make win7 needs a repair | 14:38 |
| eoli3n | why ? | 14:38 |
| eoli3n | my previous installation working method, complete is -> boot debian bootstrap pxe -> sfdisk to restore part table -> detar.xz sda2 (/boot) and sda3 (/) -> chroot -> install grub with the script in chroot -> reboot | 14:39 |
| eoli3n | win7 still working after that | 14:39 |
| eoli3n | what differ in kickstart method ? | 14:39 |
| eoli3n | weird, isn't it ? | 14:43 |
| TJ- | I'm trying to determine what exactly "grub-installer/with_other_os" is supposed to fo | 14:45 |
| TJ- | I think that should be set to "true" when you expect another OS to be installed; I don't think that'll affect the issue you are having though | 14:50 |
| eoli3n | that was my question before all of that on #debian | 14:50 |
| eoli3n | i tried every options TJ- | 14:50 |
| eoli3n | with_other_os and only_debian, set to true or false | 14:51 |
| eoli3n | all false was my last try | 14:51 |
| eoli3n | just tried to install grub with my custom script | 14:51 |
| eoli3n | as https://ptpb.pw/n5Av | 14:51 |
| eoli3n | lets try, i tell you in 20min | 14:51 |
| eoli3n | grub-installer/skip boolean true | 14:51 |
| eoli3n | then generate a chroot grub.sh installer | 14:51 |
| eoli3n | cat not echo -> fixed : https://ptpb.pw/ao3y | 14:54 |
| TJ- | eoli3n: Are you sure when Win7 'repairs' it's not writing a GPT to the disk? Because in your table for lines (3) and (4) you show PARTUUID - that will only be available for GPT, MBR scheme has nowhere to store a /partition UUID/ (whereas the file-system in the partition can/does have a UUID) | 15:08 |
| TJ- | eoli3n: GPT uses sectors 1-33 which would explain why in line (3) you have a different checksum for 0>1024 | 15:11 |
| eoli3n | i repair, and recheck | 15:12 |
| TJ- | use "gdisk" to check before and after | 15:15 |
| eoli3n | i can't, no xserver | 15:15 |
| TJ- | gdisk is console | 15:17 |
| eoli3n | ah | 15:17 |
| eoli3n | huhu | 15:17 |
| TJ- | "gdisk -l /dev/sda" | 15:17 |
| eoli3n | need to redeploy, my custom grub.sh breaks | 15:18 |
| eoli3n | tell you in some minutes | 15:18 |
| eoli3n | hmm that could be the trick | 15:20 |
| eoli3n | dump and restore with sgisk | 15:20 |
| eoli3n | TJ-: before -> http://ix.io/EvA, after -> http://ix.io/EvC | 15:49 |
| eoli3n | GUID change each time i run gdisk | 15:50 |
| eoli3n | http://ix.io/EvD | 15:50 |
| eoli3n | http://ix.io/EvE | 15:50 |
| TJ- | OK, so not GPT then. So why is Win7 changing something in sector 1+? Does it also hide recovery info there? | 15:51 |
| eoli3n | hide recovery info ? | 15:51 |
| eoli3n | where ? | 15:51 |
| eoli3n | i don't know why, i think, maybe it changes only BCD boot file | 15:51 |
| eoli3n | in log of repair tool, it just says that it edit the entry in BCD | 15:51 |
| eoli3n | suppress previous one then replacing by a new matching one | 15:52 |
| eoli3n | as i my previous paste | 15:52 |
| eoli3n | https://ptpb.pw/fxvz | 15:52 |
| eoli3n | https://ptpb.pw/0Y6b.png | 15:52 |
| eoli3n | 3 collumns in orange are same | 15:53 |
| TJ- | eoli3n: oh! I misread your table "0>1024" as being the first 4 sectors of the disk, but that's actually the 1st partition | 15:53 |
| eoli3n | i mean first one is 2048 > (1024*50) | 15:53 |
| eoli3n | yep | 15:53 |
| TJ- | eoli3n: you should take a checksum of sectors 1-2047 | 15:53 |
| eoli3n | hm but how to cut it | 15:53 |
| mason | eoli3n: This is where UEFI is much more orderly. There aren't random things slipping their tentacles around different undocumented bits of disk. | 15:53 |
| eoli3n | 0>446, 446>510, 512> 2047 ? | 15:54 |
| TJ- | as in "dd if=/dev/sda skip=1 count=2047 | md5sum" | 15:54 |
| eoli3n | ok but it will tell nothing | 15:55 |
| eoli3n | https://ptpb.pw/0Y6b.png | 15:55 |
| eoli3n | i already test lower part | 15:55 |
| eoli3n | 0>446 , 446>510 | 15:55 |
| eoli3n | 1>2047 will have part table and bootloader into it no ? | 15:55 |
| eoli3n | the range is too high ? you know what i mean ? | 15:56 |
| eoli3n | sadly, i have to go :( , i really want to find out that problem, i will try to diff md5sum 512>2047 too tommorow | 15:56 |
| TJ- | eoli3n: no, 1-2047 are 'spare' sectors which GRUB puts its core image in | 15:57 |
| eoli3n | ? | 15:57 |
| eoli3n | https://fr.wikipedia.org/wiki/Master_boot_record | 15:57 |
| eoli3n | i don't get what you mean | 15:57 |
| eoli3n | sorry | 15:57 |
| eoli3n | wikipedia says that 0>446 is part table | 15:58 |
| eoli3n | so how 1>2047 could be grub core image | 15:58 |
| eoli3n | 1 is a byte, yes ? | 15:58 |
| eoli3n | byte "1" to byte "2047" | 15:58 |
| eoli3n | ? | 15:58 |
| TJ- | No, it's sectors of 512 bytes | 15:59 |
| TJ- | dd uses 512 byte blocks by default | 15:59 |
| eoli3n | ohhhhh | 16:00 |
| eoli3n | i will take a look tomorow morning at 8h (GMT+1) | 16:01 |
| eoli3n | thx a lot for your help | 16:01 |
| DammitJim | is there an ubuntu repo for tomcat 8.5? | 18:10 |
| sarnold | DammitJim: 8.5 appears to be in artful and forthcoming bionic | 18:11 |
| DammitJim | I guess I've got to learn what bionic and artful is | 18:12 |
| DammitJim | I have ubuntu 16.04 LTS servers with tomcat 8 and apparently Apache Tomcat is making tomcat 8 EOL in September | 18:13 |
| nacc | DammitJim: 18.04 (unreleased) and 17.10, respectively | 18:13 |
| DammitJim | trying to start getting off that version | 18:13 |
| DammitJim | gosh, it looks like I'm going to have to just uninstall tomcat8 | 18:14 |
| sarnold | oh | 18:14 |
| DammitJim | and download the apache=tomcat-8.5.zip | 18:14 |
| DammitJim | and work it from that angle | 18:14 |
| sarnold | so you don't actually have an application that requires 8.5? | 18:14 |
| DammitJim | no repos | 18:14 |
| DammitJim | I do not | 18:15 |
| DammitJim | we are purely doing it because it's EOL | 18:15 |
| sarnold | just upgrade to 18.04 LTS when you're comfortable with the change | 18:15 |
| sarnold | 18.04 will be released before september. | 18:15 |
| DammitJim | yeah, I think that'll be an option in my proposal | 18:15 |
| sdeziel | DammitJim: also, Tomcat 8 being in main, it should be supported for the full lifetime of 16.04 even if upstream reaches its EOL | 18:22 |
| nacc | (supported by canonical/ubuntu) | 18:22 |
| nacc | *not* by upstream, to be clear :) | 18:22 |
| sdeziel | yeah, main is a canonical thing :) | 18:23 |
| DammitJim | thanks for clarifying that, sdeziel | 18:30 |
| DammitJim | so, if there was a problem with tomcat 8, Canonical would fix it and release an update? | 18:30 |
| nacc | DammitJim: yeah | 18:31 |
| nacc | generally speaking, it does depend on 'the problem', as we still need to follow SRU rules | 18:31 |
| nacc | but preusming you mean CVEs or so, then yes | 18:31 |
| DammitJim | awesome | 18:31 |
| DammitJim | I need to find the documentation that explains that, because that is AMAZING | 18:32 |
| nacc | https://help.ubuntu.com/community/Repositories#Main | 18:32 |
| nacc | iirc | 18:32 |
| DammitJim | thanks | 18:49 |
| DammitJim | nacc, I'm reading the Main section | 19:04 |
| DammitJim | that's where Tomcat would fall under, right? | 19:04 |
| nacc | DammitJim: correct | 19:04 |
| sarnold | you can use 'apt-cache policy tomcat8' to see | 19:05 |
| sarnold | not all binary packages built from a source package are in main, so it doesn't hurt to check all the binary packages you care about | 19:06 |
| nacc | sarnold: good point | 19:06 |
| DammitJim | oh gosh | 19:07 |
| DammitJim | the devil is in the details, but thanks! | 19:07 |
| DammitJim | so, it seems that if I want to use tomcat 8.5, I'll have to upgrade to Ubuntu 18.04? | 19:07 |
| nacc | DammitJim: i think everying is in main except libtomcat8-embed-java nad tomcat8-user | 19:08 |
| nacc | DammitJim: once released, yes, or 17.10 in the meanwhile | 19:09 |
| Ussat | sigh | 19:45 |
| Olanzapin | &j ssacc.net | 21:34 |
| keithzg | So here's something that's been baffling me, and it's arguably appropriate since the router in question runs Ubuntu ;) | 21:54 |
| keithzg | A bunch of random sites seem to have started blocking HTTP traffic from my office, with what appear to be Apache "Access Denied" messages. This extends to curl/wget from the router itself . . . but somehow *not* to traffic through the OpenVPN instance? | 21:55 |
| TJ- | keithzg: is your public IP on a blacklist? | 21:56 |
| keithzg | TJ-: That was my first thought, but if it is, I can't seem to find any publically-accessible listing thereof | 22:00 |
| TJ- | keithzg: what's the public ip address/mask ? | 22:01 |
| keithzg | TJ-: It's 184.70.164.246, aka gmcl.com | 22:02 |
| keithzg | (My current working theory remains that it's some sort of private corporate blacklist; that doesn't explain why VPN'd traffic doesn't get block but I can sortof hand wave that away with "routing is complicated, I'm probably not understanding something") | 22:03 |
| TJ- | keithzg: if you are using openvpn to tunnel out to another host that then routes, it's IP address will be different | 22:04 |
| keithzg | TJ-: The router is also the VPN server, though, so shouldn't sites see that as the IP address of the traffic? | 22:05 |
| TJ- | keithzg: you mean you connect from LAN clients using openvpn to your gateway router? | 22:06 |
| TJ- | keithzg: checked the IP, not blocked anywhere | 22:07 |
| keithzg | TJ-: Specifically I mean that the router for the office LAN is also the VPN server that external clients use to get into our LAN remotely. | 22:07 |
| TJ- | keithzg: Oh, I thought you meant you link your gateway to another location and tunnel /out/ through it | 22:08 |
| TJ- | keithzg: then I can only thing your gateway is messing with the traffic, are you sure your network doesn't have a transparent proxy? | 22:09 |
| TJ- | keithzg: does it affect HTTPS connections or only HTTP? | 22:09 |
| TJ- | keithzg: i'd suspect the Apache message you see is from your own network | 22:10 |
| keithzg | TJ-: Funny story about that, due to this I've noticed that https://thebay.com doesn't have a valid cert ;) but yeah it appears to affect both HTTPS and HTTP traffic, for instance https://tools.usps.com has a valid HTTPS connection but tells me "You don't have permission to access "http://tools.usps.com/" on this server" | 22:11 |
| TJ- | keithzg: I think you've got an internal redirection issue in the gateway. Possibly the rules that were set for incoming openvpn tunnel traffic are breaking regular forwarded traffic | 22:12 |
| TJ- | keithzg: Ask yourself: 1) when did this start? 2) What did I change just before I noticed the issue? | 22:13 |
| keithzg | TJ-: That's the problem, other than the standard security patches I haven't touched anything on the router in ages now. | 22:13 |
| TJ- | keithzg: it's an Ubuntu server actiing as gateway? | 22:14 |
| keithzg | TJ-: Yup. | 22:14 |
| TJ- | keithzg: check /var/log/syslog and /var/log/kern.log for clues | 22:14 |
| TJ- | keithzg: also, if it has apache2 web-server installed, check it's logs in /var/log/apache2/ in case it indicates it's responsible for the messages | 22:15 |
| keithzg | TJ-: No messages in kern.log for several days now, I can't see anything that seems remotely relevant in either syslog or the apache logs, and nothing seems to shwo up if I tail them while trying to access a site :( | 22:18 |
| TJ- | keithzg: has it run out of space? "df -h" | 22:19 |
| TJ- | also try "df -ih" (for inodes) | 22:19 |
| sarnold | pity there's no way to get both in one invocation :( | 22:20 |
| keithzg | TJ-: Naw, the 256GB M.2 SSD that the router runs is only at 4% space usage (the only higher is /run at 8%, and inode usage for everything is being rounded to 1% | 22:20 |
| TJ- | keithzg: good. Are you comfortable sharing the netfilters rules? ("pastebinit <( sudo iptables-save )" | 22:22 |
| keithzg | TJ-: https://paste.ubuntu.com/26440410/ | 22:29 |
| tomreyn | if you request httpS://tools.usps.com amdd get to see an error stating that you may not access HTTP://tools.usps.com/ (so non-encrypted) then this is a pretty obvious hint that your TLS connection was stripped towards the receiving end. | 22:29 |
| tomreyn | s/amdd/and/ | 22:29 |
| TJ- | tomreyn: that's why I suspect a local proxy | 22:29 |
| tomreyn | right | 22:29 |
| TJ- | keithzg: are the local clients using Ubuntu/Linux ? | 22:29 |
| keithzg | Yeah I wonder . . . I'm going to try turning off the Apache server (which is proxying to a VM running the *actual* company website) | 22:29 |
| keithzg | TJ-: Not all of them, the first person who noticed this and has continued to notice things is on Windows 8.1, and I randomly tried a macOS VM at one point. I myself have been mostly testing this from Kubuntu. | 22:30 |
| keithzg | Well, shutting down the Apache server on the router didn't change anything. | 22:31 |
| TJ- | keithzg: I'm also surprised, if that is a gateway router, that the INPUT chain doesn't have a DROP policy and then specific rules for allowing VPN/SSH traffic in | 22:31 |
| TJ- | keithzg: can you do one of those 'wget' ops that gets denied and show us the output in a pastebin? | 22:32 |
| TJ- | keithzg: from the gateway itself | 22:32 |
| TJ- | keithzg: also, "pastebinit <( ip -4 -6 route show )" | 22:33 |
| keithzg | TJ-: I have SSH blocked with `-A INPUT -i external0 -p tcp -m tcp --dport 22 -j REJECT --reject-with icmp-port-unreachable` and that I thought worked fine for blocking SSH traffic (I actually run a port knocker, and that's how rules for allowing SSH in get added) | 22:34 |
| sdeziel | "ip -4 -6 ro" only returns v6 routes here (on Xenial) | 22:38 |
| TJ- | keithzg: right, but if there are other services on the gateway they may be exposed. | 22:38 |
| TJ- | sdeziel: it returns both here on 16.04 | 22:38 |
| keithzg | TJ-: Ah, fair enough. | 22:38 |
| TJ- | sdeziel: oh, no, you're correct! Sorry, I misread! | 22:38 |
| keithzg | Anyways the denied requests are just single-line HTTP responses, ex. | 22:38 |
| keithzg | <html><head><title>Request Rejected</title></head><body>The requested URL was rejected. Please consult with your administrator.<br><br>Your support ID is: 16159986499229415207</body></html> | 22:38 |
| sdeziel | TJ-: too bad, I would have like it to work ;) | 22:38 |
| TJ- | keithzg: so, "pastebinit <( ip route show; ip -6 route show )" | 22:38 |
| TJ- | sdeziel: yeah, it annoys me that without switches it defaults to IPv4 | 22:38 |
| keithzg | TJ-: https://paste.ubuntu.com/26440446/ | 22:38 |
| keithzg | Oh how I wish we had IPv6 | 22:38 |
| TJ- | keithzg: I wanted to see the entire wget messages. Do "pastebinit <( wget http:///whatever 2>&1 )" | 22:38 |
| keithzg | TJ-: Oh, fair enough, I just saw that it was going "200 OK" and nothing else other than the actual *contents* of index.html indicates any problem. But here: https://paste.ubuntu.com/26440457/ | 22:38 |
| keithzg | Oh oops, never mind that | 22:39 |
| keithzg | Although maybe that's a clue to some degree, many initial URLs are fine, but then what they're actually redirecting to isn't OK. | 22:39 |
| keithzg | Actual wget output for one of these 403s: https://paste.ubuntu.com/26440460/ | 22:40 |
| sdeziel | "wget -SO - http:///whatever 2>&1 )" | 22:40 |
| tomreyn | i'd rather suggest to use curl --trace /tmp/trace http(s)://... | 22:40 |
| TJ- | keithzg: thebay.com is broken for me too, and TLS connections return SSL_ERROR_BAD_CERT_DOMAIN | 22:42 |
| TJ- | keithzg: any other domains do this to you? | 22:42 |
| sdeziel | 69.192.84.206 belong to Shaw Communications | 22:42 |
| sdeziel | keithzg: would that be your ISP ^ | 22:42 |
| tomreyn | tools.usps.com seemed like a better test candidate | 22:42 |
| TJ- | For that I get "20 redirections exceeded." | 22:43 |
| sdeziel | and when resolving www.thebay.com, I get an akamai IP so it looks like an ISP getting in the way | 22:43 |
| keithzg | tomreyn: heh yup, that would be precisely what I was going to try as an alt. Here in fact is the curl trace output for that: https://paste.ubuntu.com/26440477/ | 22:44 |
| tomreyn | keithzg: that's the output of which command? | 22:44 |
| keithzg | sdeziel: Interesting, I saw Akamai when trying to figure things out before too with https://www.us-cert.gov/ (the third website giving this error) | 22:45 |
| TJ- | sdeziel: that explains it then, they've left thebay.com in the DNS zone file but it should be front-ended by cloudfront. www.thebay.com is correct with a CNAME | 22:45 |
| keithzg | tomreyn: `curl --trace tracefile https://tools.usps.com` | 22:45 |
| TJ- | sorry, akamai, not cloudfront | 22:45 |
| sdeziel | keithzg: from that paste, it seems the 403 was emitted by Akamai themselves | 22:46 |
| sdeziel | Server: AkamaiGH | 22:46 |
| sdeziel | maybe Akamai thinks your IP has a bad reputation? | 22:46 |
| TJ- | tools.usps.com also on akamai - do Akamai have a problem? | 22:46 |
| TJ- | sdeziel: I did an RBL check on the IP addres over about 30 RBLs and it's clean | 22:47 |
| sarnold | akamai may have their own database | 22:47 |
| tomreyn | akamai don't like you. | 22:47 |
| keithzg | That defintiely aligns with the vague intimations of https://community.akamai.com/community/cloud-security/blog/2016/04/07/why-is-akamai-blocking-me | 22:48 |
| keithzg | (one of the first things I found when initially noticing this error and when nmap told me that the us-cert site was hosted by Akamai) | 22:50 |
| TJ- | this is 1 of the big downsides to centralised proxies | 22:50 |
| keithzg | Yeahhh, and unfortunately that help page gives the impression that there's no way to contact Akamai and ask them precisely what's going on :( | 22:50 |
| TJ- | keithzg: has anything on your network been doing automated connections, say to access the USPS site for example? | 22:51 |
| TJ- | someone developing a tool to query shipping status maybe? | 22:51 |
| keithzg | TJ-: Nothing intentional, no; the only software development anyone other than ol' sysadmin me does is oldskool Win32 work for the software we sell (that in turn tends to run on theoretically airgapped networks, so we don't even have any sort of auto-updater or such for our tools). | 22:53 |
| TJ- | keithzg: how about via the VPN - you said connections via the VPN were OK - which I can only see happening if the VPN uses a different IP address for it's exit interface from your /30 | 22:55 |
| sdeziel | or maybe the VPN is using split tunneling ? | 22:56 |
| TJ- | keithzg: show us "pastebinit <( ip addr show )" | 22:56 |
| keithzg | TJ-: Yeah, that one baffles me as well. Here it is: https://paste.ubuntu.com/26440512/ | 22:57 |
| TJ- | keithzg: so, the external clients /are not/ using the VPN for general internet access, which explains htat | 22:58 |
| sdeziel | keithzg: on a VPN client that works, could you share "pastebinit <( ip ro)" | 22:58 |
| TJ- | sdeziel: I hate that "split tunnel" name, it's deceptive, there's no split in the tunnel, it's just standard routing! | 22:59 |
| sdeziel | TJ-: I /think/ it could be it but I haven't seen any evidence so far | 22:59 |
| keithzg | sdeziel: Well drat, I didn't bring my laptop to work today! I wonder if Android can manage that. | 22:59 |
| TJ- | sdeziel: I'm sure it is, the client will use their standard default route (and hence connect successfully to thebay.com) but will route 10.0.0.0./8 (or similar) via the openvpn tunnel | 23:01 |
| sdeziel | keithzg: you could do something server side. "sudo iptables -A FORWARD -i tun0 -o external0" | 23:01 |
| TJ- | keithzg: 1 thing you could try us assign another IP address to external0 and use it as the source address for a test, see if it is blocked too, or just your current single IP address | 23:02 |
| sdeziel | keithzg: then with the VPN client, access a site then run this on the server side: "pastebinit <(sudo iptables -nvL FORWARD)" | 23:02 |
| TJ- | my typing is going downhill, my fingers have a mind of their own! | 23:02 |
| sdeziel | TJ-: if Akamai blocks based on IP reputation, they maybe do that using wider CIDRs than /32 | 23:02 |
| TJ- | sdeziel: right, but it's worth testing, because if it's only a /32 there is a workaround | 23:03 |
| keithzg | sdeziel: Seems the command works fine from a shell on my Android phone, I think? https://paste.kde.org/pw2t6zene | 23:03 |
| sdeziel | keithzg: hmm, no default route? | 23:04 |
| tomreyn | 184.70.164.246 belongs to a /13 (!) - not ideal if your hopes are that you'll be emitting less bad traffic than the average of shaw communication (cable?) users. for business use, a much smaller address range would be recommendable. | 23:04 |
| sdeziel | keithzg: "ip ro g 8.8.8.8" ? | 23:04 |
| TJ- | which is "ip route get 8.8.8.8" | 23:04 |
| sdeziel | yeah, sorry, I'm very lazy | 23:05 |
| TJ- | :D trying to educate as well as diagnose :p | 23:05 |
| sdeziel | and it's better to have the receiving end understanding a command before running it :) | 23:05 |
| keithzg | sdeziel: 8.8.8.8 via 10.180.113.86 dev rmnet_data0 src 10.180.113.85 uid 10228 | 23:06 |
| TJ- | I keep telling that to my Huskies! | 23:06 |
| sdeziel | lol | 23:06 |
| keithzg | (I haven't tried `sudo iptables -A FORWARD -i tun0 -o external0` yet, for the record) | 23:06 |
| sdeziel | keithzg: I think that's your answer | 23:06 |
| sdeziel | keithzg: you are using split tunnelling (or routing just the remote LAN IP space over your VPN/tun0) | 23:07 |
| keithzg | Aha, fair enough (can't remember if that was intentional on my part when I ported over to the new server, or maybe that | 23:07 |
| keithzg | 's an artifact of the convoluted setup of my predecessor) | 23:07 |
| TJ- | keithzg: it makes sense, no point in routing internet bound traffic via the tunnel | 23:07 |
| sdeziel | keithzg: at least that's what I believe is going on. I'm not very familiar with the per UID routing stuff done on Android | 23:08 |
| keithzg | TJ-: Yeah, it's definitely what I'd have chosen to set up if I actually did so deliberately ;) | 23:08 |
| TJ- | keithzg: do you want to test with your 'spare' currently unused IP address? | 23:08 |
| * sdeziel wonders if per UID routing landed upstream | 23:08 | |
| keithzg | TJ-: That does sound like a plan---although I honestly don't know what it is, heh (I knew we had a second static address but I haven't bothered to look it up) | 23:10 |
| TJ- | keithzg: your 184.70.164.244/30 gives you 2 IP addresses: .245 and .246 (which you use) so we can try adding .245 and using it | 23:11 |
| TJ- | keithzg: "sudo ip addr add 184.70.164.245/30 dev external0" | 23:11 |
| sdeziel | wget --bind-address=184.70.164.245 -SO - https://tools.usps.com/ | 23:12 |
| TJ- | keithzg: then try "wget --bind-address=184.70.164.245 -S -O - http://www.thebay.com" | 23:13 |
| TJ- | Grrrr, stealing my typing :D | 23:13 |
| sdeziel | I don't trust your self-aware fingers either ;) | 23:14 |
| TJ- | LOL | 23:14 |
| keithzg | hehe | 23:15 |
| keithzg | Well I must've messed up the addition of the additional address or something, since that (either one) just hangs at "Connecting" forever | 23:15 |
| TJ- | keithzg: check with "ip addr show dev external0" | 23:15 |
| TJ- | keithzg: you should see both .245/30 and .246/30 | 23:16 |
| TJ- | We know there's no netfilter rules to get in the way :) | 23:16 |
| keithzg | TJ-: Heh. Yeah I see it listed although it's different? https://paste.kde.org/pbatt57z3/32j5t0/raw | 23:18 |
| TJ- | keithzg: that's fine | 23:19 |
| TJ- | keithzg: let's check routing: "pastebinit <( ip route show )" | 23:19 |
| keithzg | TJ-: Fair enough, I just wasn't sure, although upon reflection I assume "brd" means broadcast so that makes sense to me then | 23:19 |
| keithzg | TJ-: https://paste.ubuntu.com/26440728/ | 23:20 |
| TJ- | keithzg: did you say both .245 and .246 hung when used with wget --bind-address= ? | 23:20 |
| keithzg | TJ-: Oh, no I didn't try binding to .246 instead. Just tried it, it instantly works. | 23:21 |
| TJ- | interesting, adding the new IP changed the default via to .245 | 23:21 |
| keithzg | (well, gives the 403 from AkamaiGHost, heh) | 23:21 |
| TJ- | keithzg: I'm getting confused. which works? "wget -S -O - http://www.thebay.com" ? | 23:22 |
| keithzg | TJ-: That works (gets a 403), explicitly binding to .246 works (also 403), explicitly binding to .245 does not (hangs on the Connecting step) | 23:23 |
| TJ- | keithzg: weird, that suggests your ISP has assigned a /30 but is not routing it! Is there a modem/router from your ISP connected to the server? | 23:24 |
| TJ- | keithzg: in which case it's likely .245 is assigned to that device | 23:24 |
| TJ- | keithzg: do "sudo ip addr del 184.70.164.245/30 dev external0" to clean up the server | 23:26 |
| keithzg | Yeah they insisted they couldn't give me *just* a modem anymore :( (this is why I don't use Shaw personally!). In theory they've disabled it so that it's only acting as a modem, it's not bridged or anything, but I wouldn't be surprised then if the modem+router box also has its own IP address secretly then. | 23:26 |
| keithzg | Huh, even add that del I still see .245 as the default route. | 23:27 |
| keithzg | (Did I just not have a default before? Yeesh, I need to keep better track of these things!) | 23:27 |
| sdeziel | https://paste.ubuntu.com/26440446/ shows that you had a "default via 184.70.164.245 dev external0 onlink" | 23:28 |
| sdeziel | keithzg: that's one of your earlier paste | 23:29 |
| keithzg | sdeziel: Ah, fair enough. So it's more just a lack of comprehension of what it all means on my part then, heh. | 23:29 |
| TJ- | oh, so .245 is the ISP device | 23:29 |
| TJ- | i missed that | 23:30 |
| sdeziel | keithzg: no, we screwed up a little I'm affraid | 23:30 |
| TJ- | keithzg: so the upshot is ... Akamai | 23:30 |
| TJ- | i read the default as .246 originally, grrr | 23:30 |
| keithzg | Curse our fallable humanity! | 23:31 |
| TJ- | I blame my fingers AND my eyes :) | 23:31 |
| TJ- | basically, broken I/O | 23:31 |
| sdeziel | on the up side, you were not SSH'ed in from a remote location and didn't lose access :) | 23:31 |
| keithzg | hehe | 23:31 |
| keithzg | Yeah, seems like my next step is probably to bug my ISP and see if there's some way they can get more info or a delisting from Akamai. | 23:32 |
| keithzg | Many, many thanks! :) | 23:33 |
| keithzg | I wish all of the internet was full of people as wonderful as #ubuntu-server :D | 23:33 |
| keithzg | Fallable and human as they may be ;) | 23:34 |
| sdeziel | hehe | 23:34 |
| sdeziel | good luck! | 23:34 |
| keithzg | Thanks! | 23:34 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!
