[00:26] <drab> for some reason no joy, trying to figure out what's wrong
[00:27] <drab> I've added the one routing rule on the gw, and I can ping the gw of the lan from behind the vpn, but not other hosts on the lan
[01:03] <TJ-> does the 'other' end of the vpn 'know' about the LAN subnet?
[01:46] <drab> TJ-: the default gw does, it has a static route to it, I thought that should have been enough
[01:47] <drab> tomorrow I need to go to a startbucks, like this I can't even debug, if I connect to ovpn from inside the lan the results are skewed
[01:47] <drab> I tried to test using someone's phone data, but they are gone now
[01:48] <sarnold> you could spin up a gce or aws instance for a few bucks if you're not actually interested in the coffee :)
[01:49] <TJ-> drab: let me be clear; your problem is when connecting into the LAN through the VPN, the client that connects cannot ping hosts on the LAN sub-net?
[01:49] <drab> TJ-: correct. but they can ping the gateway of the lan and ping the internet
[01:50] <drab> sarnold: you mean installing a vpn client on it?
[01:50] <sarnold> drab: yeah, or whatever testing you needed..
[01:51] <drab> fair enough, I guess it'd be enough to figure out the routing issue
[01:51] <TJ-> drab: in the openvpn config file on the openvpn server have you got a "push "route 10.254.201.0 255.255.255.0"  " ? (with your LAN sub-net, not mine)
[01:53] <drab> TJ-: oh, eer, I don't. why does it matter? aren't the vpn clients going to send those pkts to their default gw (the ovpn server), which then knows (has a direct route) to the hosts on the lan?
[01:54] <TJ-> drab: and the client's config should have a "pull" to ensure those are accepted
[01:55] <TJ-> drab: the remote client does not know about the server's LAN sub-net, so the server has to tell the client about it
[01:55] <drab> but wouldn't a vpn, like any other network client, send pkts for hosts it doesn't recognize on its lan to the default gw? (the ovpn server)
[01:55] <drab> effectively achieving the same result
[01:56] <drab> since I'm forcing all traffic to go through the vpn
[01:56] <drab> I have that push "redirect-gateway def1 bypass-dhcp"
[01:56] <TJ-> drab: you're using redirect-gateway ?
[01:57] <drab> yeah, that's in my server conf
[01:58] <TJ-> drab: OK, is the openvpn server on the LAN also acting as the LAN's gateway to the internet, or is that done by another router?
[01:58] <drab> done by another router (also an ubuntu-server)
[01:59] <TJ-> drab: if the default gateway of the LAN is another router, then all those PCs need to have a route added for the VPN sub-net so they can *return* packets
[01:59] <drab> sdeziel earlier was mentioning icmp redirects, which I thought made sense
[01:59] <drab> won't the gw, which they will send it to since they don't know what to do with it, tell them?
[02:00] <TJ-> drab e.g: gw 192.168.0.1, ovpn-server 192.168.0.254, vpn 10.0.1.0/24 ... LAN clients will need a "ip route add 10.0.1.0/24 via 192.168.0.254"
[02:01] <drab> right, I added that route on the gw, thinking that when the clients send it pkts for the vpn host the gw would have redirected them
[02:01] <drab> I guess that doesn't work
[02:01] <TJ-> drab: drab you said the ovpn-server has 2 NICs, one on the LAN and the other to the router?
[02:02] <drab> I wasn, all hosts are on the lan, the ovpn server has the lan nic and its tun interface
[02:02] <drab> wasnt clear*
[02:02] <TJ-> drab: oh, I misread
[02:02] <TJ-> OK, that makes more sense :)
[02:03] <TJ-> so the router has effectively "ip route add 10.0.1.0/24 via 192.168.0.254"
[02:03] <drab> correct
[02:03] <TJ-> is ip4_forward enabled on the ovpn-server's interfaces?
[02:03] <drab> yep
[02:03] <drab> sysctl -p
[02:03] <drab> net.ipv4.ip_forward = 1
[02:04] <drab> but I just realized that probably yhe gw has icmp redirect disabled
[02:04] <drab> net.ipv4.conf.all.send_redirects = 0
[02:04] <TJ-> OK, well, test your routing manually. Create a dummy interface on the ovpn-server with the IP subnet the vpn usually uses. Then try to ping that IP from the clients on the LAN. Use tcpdump on the various PCs to see where it is going astray, if it doesn't work
[02:05] <drab> oh, right, such a simple and great idea, I can just test that way, brilliant
[02:06] <TJ-> you can also connect a local client with a VPN to your ovpn-server and test that :)
[02:06] <drab> I tried that, results were skewed
[02:06] <TJ-> if all that works then you've an issue in the ovpn config itself
[02:07] <TJ-> are you using UDP ?
[02:07] <drab> no, I set it up with tcp
[02:07] <TJ-> Ahhh. I've always used UDP; not had a problem in over 10 years
[02:07] <sarnold> encapsulating tcp within tcp is often a recipe for catastraphic latency when you least want it :)
[02:08] <drab> point taken
[02:08] <TJ-> not to mention fragmentation and MTU issues
[02:08] <TJ-> but basic ping should not be upset by those
[02:08] <drab> but for the local test, the issue seemed to be that routes on the android phone still show lan routes
[02:09] <drab> so I thought when I tested that way that reachability worked because some pkts "leaked" or something
[02:09] <TJ-> I prefer UDP since it's harder to DoS it, especially using the ta.crt option
[02:09] <drab> ok, I'll switch to udp, but don't think it has to do with my issues right now
[02:09] <TJ-> drab: don't switch NOW! You'll introduce another variable
[02:09] <TJ-> get your TCP config sorted first, one thing at a time
[02:10] <TJ-> 1. Prove your LAN routing works as you expect and you can ping the dummy i/f from clients on the LAN that are using the default GW
[02:11] <TJ-> 2. create VPN tunnel from a client to the ovpn-server and check each stage of the routing through it, using ping/tracepath
[02:12] <drab> roger that
[02:12] <drab> thanks
[02:16] <TJ-> drab: when I first started with openvpn I got the same kind of problems; I found running tcpdump on each hop on the network really useful for figuring out issues. The main is when you try to ping from PC1 to PC2 and it doesn't work you assume it's PC1 at fault, but often it's PC2 doesn't have a return route so it receives the ping packets but never replies. You can see that instantly with tcpdump
[02:17] <drab> true, seen that in the past debuggin other issues
[02:17] <drab> will look for that
[02:33] <drab> ok
[02:34] <drab> so indeed pings aren't getting back to the vpn subnet and get lost at the gw. if I add a route on the hosts on the lan too then it works
[02:34] <drab> but I'm not going to add a route for that everywhere
[02:34] <sarnold> drab: hrm. does the vpn even encapsulate icmp? or .. uh.. something?
[02:35] <drab> I don't understand the question. the behavior makes sense, I thought icmp redirect would have fixed that, but it's disabled by default and it seems I don't wanna enable it
[02:35] <drab> when the hosts on the lan get the pkts they don't know what to do with it and send it to the gw and they die there
[02:37] <drab> I have a great idea that i'm sure will make everybody cringe :P
[02:37] <drab> since I don't wanna do masquerading, add a route on each host or enable icmp redirect
[02:38] <drab> what if, I gave the vpn client a range that's on the same subnet as the lan but unused, and then had the ovpn server arp_proxy for it?
[02:38] <drab> that way when a host on the lan gets a pkg from a vpn client it'll think it's on the lan, arp for it, adn the ovpn proxy will respond for it
[02:55] <TJ-> drab: I'd sooner want to figure out why the router isn't using the route? Did you also add firewall pin-hole rules to ensure the vpn traffic was allowed to be forwarded/
[02:58] <drab> TJ-: yeah, I was thinking about that, but I feel I'm back on networking 101 and my brain not responding well to it...
[02:59] <drab> TJ-: shuold the router, when it sees a pkt from a client on the lan, having a route to the vpn clients, send it over to the ovpn-server?
[02:59] <TJ-> Yes
[02:59] <drab> ok, good point, lemme check on the fw part
[02:59] <TJ-> but it has to go through the FORWARD tables
[03:00] <TJ-> you may need to enable a default ACCEPT for anything arriving from the LAN side and leaving on the LAN side
[03:01] <drab> once again you're right, they are in ulog... adding a rule
[03:09] <drab> TJ-: ok, it works, and I see the icmp redirect , I thought those were disabled by default but guess they aren't and they weren't issued becuase the fw was blocking stuff
[03:10] <TJ-> drab: you're about sorted now then
[03:10] <TJ-> drab: seems like that was your problem all along
[03:10] <drab> well my problem is that I haven't got a clue anymore... if I ever had it :P
[03:11] <TJ-> well are your pings ponging?
[03:11] <drab> yeah
[03:11] <drab> everything checks out
[03:12] <drab> I'll do one last test tomorrow from the actual phone, but from the dummy itnerface everything works
[03:12] <drab> I can see the icmp and the redirects with tcpdump and all
[03:12] <drab> so I'm assuming the phone will work too tomorrow
=== albech1 is now known as albech
[06:33] <hdon_> hi all :) can i "install" ubuntu server to a ramfs/tmpfs?
[06:49] <cpaelzer> good morning
[07:05] <lordievader> Good morning
[09:25] <zioproto> good morning
[09:25] <cpaelzer> hiho zioproto
[09:30] <zioproto> anybody already faced this ? :
[09:30] <zioproto> gbp:error: Pristine-tar couldn't checkout "horizon_10.0.5.orig.tar.gz": pristine-tar: delta is version 3, newer than maximum supported version 2
[09:31] <cpaelzer> zioproto: https://bugs.launchpad.net/launchpad/+bug/1732400 ?
[09:31] <ubottu> Launchpad bug 1732400 in Launchpad itself "please upgrade pristine-tar to support version 3" [Undecided,New]
[09:32] <zioproto> looks like it
[09:33] <cpaelzer> zioproto: do you get that locally on your system?
[09:33] <zioproto> I get it on the buildroot for horizon
[09:33] <cpaelzer> hmm, yeah so likely the bug I linked
[09:33] <zioproto> I do
[09:33] <cpaelzer> but in the scope of your build then
[09:33] <zioproto> mk-sbuild xenial
[09:34] <cpaelzer> which will be xenial which is on pristine-tar 1.33 (current is 1.42)
[09:34] <zioproto> I guess that is the problem
[09:34] <zioproto> so it is just about upgrading pristine-tar in Xenial ?
[09:35] <cpaelzer> well, that would not match SRU policy right - as it is feature addition at the cost of regressions for others
[09:35] <cpaelzer> but in that case I'm not sure how it woud be handled
[09:36] <cpaelzer> zioproto: are you doing backports building that in xenial?
[09:37] <zioproto> I am building a package for Openstack Newton
[09:37] <zioproto> in Xenial
[09:37] <zioproto> horizon exactly
[09:37] <zioproto> I have to test a patch
[09:37] <cpaelzer> ok, so that could be the issue
[09:38] <cpaelzer> hmm, I'm sure the openstack Team has sometihng in place to make this not a FTBFS
[09:38] <cpaelzer> jamespage: coreycb: ^^ ?
[09:39] <cpaelzer> zioproto: are you building "through" gbp ?
[09:39] <cpaelzer> maybe that is the issue then
[09:39] <cpaelzer> as it might rely on pristine-tar in your cae but you might be able to get going with e.g. dpkg-buildpkg + sbuild or such
[09:40] <cpaelzer> I'd not have expected to need the "in target" pristine-tar for a build
[09:40] <cpaelzer> on the other hand, if your host is xenial it might be you just need it on your system
[09:41] <zioproto> I am trying to upgrade pristine-tar
[09:41] <zioproto> this tool does not even print his own version
[09:41] <jamespage> hmm that's a new one
[09:42] <cpaelzer> zioproto: I have no personal involvement to pristine-tar, just using it :-)
[09:42] <cpaelzer> zioproto: but you might find https://www.preining.info/blog/2014/06/debian-pristine-tar-packaging/ amusing
[09:42] <cpaelzer> actually it never broke me, so I'm good
[09:43] <cpaelzer> it's also of 2014 and recently seems well maintained
[09:45] <cpaelzer> jamespage: other question on planning OVS/DPDK in binoic
[09:45] <cpaelzer> jamespage: I mentioned https://code.launchpad.net/~ubuntu-server-dev/ubuntu/+source/openvswitch/+git/openvswitch/+ref/ready-for-dpdk-17.11 last week
[09:45] <cpaelzer> jamespage: 17.11-1 would be ready to sync this week
[09:45] <cpaelzer> jamespage: what is your intended OVS version in bionic?
[09:46] <jamespage> cpaelzer: 2.9.x
[09:46] <cpaelzer> jamespage: that is bug 1733325 btw
[09:46] <ubottu> bug 1733325 in openvswitch (Ubuntu) "Update in Bionic to match DPDK 17.11" [Undecided,New] https://launchpad.net/bugs/1733325
[09:46] <jamespage> cpaelzer: yeah sorry I was off on leave last week - I'm bad at catching up on backscroll
[09:46] <cpaelzer> jamespage: ok, so would it be ok for you if I upload dpdk this/next week and along that a small delta to OVS 2.8.1 to make it work for now
[09:47] <cpaelzer> jamespage: that delta will be droppable on the 2.9 upload then
[09:47] <zioproto> I dont get it for Horizon. In the pristine branch I get the files horizon_10.0.5.orig.tar.gz.delta and horizon_10.0.5.orig.tar.gz.id. But the all thing fails looking for the file horizon_10.0.5.orig.tar.gz that is not there
[09:47] <cpaelzer> zioproto: so it wants to generate the tarball via pristien-tar and that is failing you
[09:48] <zioproto> yes... this I got it
[09:48] <cpaelzer> you could work around all that by just getting the file there
[09:48] <zioproto> so the Horizon release 10.0.5 does not even exist in Openstack
[09:48] <zioproto> I cannot download the tarball
[09:48] <cpaelzer> cd ..; pull-lp-source horizon yakkety; cd -
[09:48] <cpaelzer> well
[09:48] <cpaelzer> this might overwrite your dir dpeending on what you work on
[09:49] <cpaelzer> so do the same in /tmp and copy the tarball
[09:49] <cpaelzer> or fetch it from LP
[09:49] <zioproto> wow, let me try this
[09:49] <cpaelzer> https://launchpad.net/ubuntu/+source/horizon/3:10.0.4-0ubuntu1
[09:49] <cpaelzer> oh that is 10.0.4
[09:50] <cpaelzer> zioproto: where did you base your version on?
[09:50] <zioproto> the latest ubuntu cloud archive
[09:50] <cpaelzer> maybe in cloud archive there is something newer than at the end of yakkety support
[09:50] <zioproto> 3:10.0.5-0ubuntu1~cloud1 500
[09:50] <zioproto> can I use pull-lp-source to pull stuff from the ubuntu cloud archive ?
[09:50] <cpaelzer> zioproto: then just enable the deb-src lin in /etc/apt/source.list.d/<cloudarchive>
[09:51] <cpaelzer> apt update; apt source horizon
[09:51] <cpaelzer> that will give you the source as it is there
[09:51] <cpaelzer> zioproto: in that regard UCA is like a ppa, so not pull-lp-source, but the above with apt-source will work
[09:53] <zioproto> it worked !
[09:53] <cpaelzer> \o/
=== albech1 is now known as albech
[14:15] <adrian_1908> I still frequently see `service <servicename> <action>` being mentioned, which works with systemd services as well. Isn't this (strictly speaking) deprecated in favor of `systemctl <action> <servicename>` or will both remain correct indefinitely?
[14:16] <Ussat> No, not indefinately, and yes, depricated
[14:16] <Ussat> its there as a conveniance
[14:16] <adrian_1908> ok, thanks.
[14:22] <rbasak> Ussat: do you have a citation for it being deprecated?
[14:23] <rbasak> I'm not aware that it is.
[14:26] <Ussat> citation no, but that is the way things are moving...
[14:26] <Ussat> Its pretty logical to me, that the old way will not stick around forever since the new way has been introduced. Its only there now as a transition
[14:28] <rbasak> Debian still supports multiple init systems. That's where the wrapper comes from.
[14:28] <rbasak> It's useful because you don't need to care about which init system is in use.
[14:29] <Ussat> ....
[14:29] <rbasak> ...so I don't see it going away any time soon.
[14:29] <Ussat> I dont care about debian
[14:29] <rbasak> Not until Debian drops support for multiple init systems.
[14:29] <rbasak> Therefore it isn't deprecated.
[14:29] <Ussat> ...
[14:30] <Ussat> ok
[14:31] <rbasak> Unless some more closely associated Debian or Ubuntu developer says it is.
[14:31] <rbasak> But I'm not aware of any such position.
[14:32] <Ussat> sigh..ok
[14:43] <coreycb> cpaelzer: zioproto: 'pull-uca-source horizon newton' is also an option
[14:43] <zioproto> thanks !
[14:50] <cpaelzer> uh - pull-uca-source
[14:50] <cpaelzer> nice
[14:50] <cpaelzer> coreycb: package?
[14:51] <cpaelzer> I actually have a ~/bin/pull-uca-source.py - probably from you coreycb :-)
[14:51] <coreycb> ha
[14:51] <coreycb> cpaelzer: ubuntu-dev-tools
[14:51] <cpaelzer> ah but post xenial
[14:51] <cpaelzer> found it
[14:52] <cpaelzer> https://bugs.launchpad.net/ubuntu/+source/ubuntu-dev-tools/+bug/1661324
[14:52] <ubottu> Launchpad bug 1661324 in ubuntu-dev-tools (Ubuntu) "add pull-ca-source to ubuntu-dev-tools" [Low,Fix released]
[14:58] <coreycb> jamespage: if we were to do a snapshot of the latest upstream gnocchi commit, we could drop pandas from the queens cloud archive
[14:59] <jamespage> coreycb: I peeked at that yesterday - we could just pick the commit that drops it
[14:59] <coreycb> jamespage: yes true, looking for that
[15:03] <jamespage> oh nice I'd not realized that pull-uca-source had got into dev-tools
[15:19] <jamespage> coreycb: having a run at deploying queens :-)
[15:19] <jamespage> needed to switch keystone to v3 but that bit is working ok
[15:19] <jamespage> glance is foobar
[15:20] <jamespage> aodh don't know what queens it
[15:26] <jamespage> coreycb: prob need a  charmhelpers release for that
[15:26] <coreycb> jamespage: ok awesome about queens. want me to get charmhelpers?
[15:27] <coreycb> jamespage: i'm uploading gnocchi with the patch and will get rid of pandas
[15:27] <jamespage> coreycb: charmhelpers is ok - it just needs a release for the reactive charms
[15:27] <jamespage> coreycb: oh ok - oops
[15:27] <coreycb> jamespage: oh ok
[15:27] <jamespage> git push --force over my changes :-)
[15:27] <coreycb> jamespage: oh you got it already? no prob :)
[15:27] <jamespage> coreycb: no you take it
[15:33] <coreycb> jamespage: ok that's uploaded
[15:33] <cpaelzer> rbasak: ahasenack: is the serverguid still the on on bzr?
[15:33] <jamespage> coreycb: awesome
[15:34] <cpaelzer> rbasak: ahasenack: I always have something on the back of my mind they wanted to change, but miss to remember if that happened
[15:54] <jamespage> coreycb: where did we leave glance b1?
[15:54] <jamespage> I remember we had some sort of unit test failure right?
[15:54] <ahasenack> cpaelzer: should still be in bztr
[15:54] <ahasenack> bzr
[15:55] <coreycb> jamespage: i think we were going to skip it due to https://bugs.launchpad.net/glance/+bug/1728368
[15:55] <ubottu> Launchpad bug 1728368 in oslo.serialization "oslo.serialization 2.21.2 breaks glance" [Undecided,New]
[15:56] <jamespage> right
[15:56] <rbasak> cpaelzer: I think it may be but I'm not certain.
=== lamont` is now known as lamont
[19:22] <drab> sdeziel: TJ-: fwiw when I tested today with the actual phone from a remote connection it did not work unless I put static routes on the individual nodes
[19:22] <drab> I'm out of time so I can't look into why, I know last night I saw the icmp redirect and it worked with the dummy interface
[19:22] <drab> but it would not trying to ping the android phone
[19:23] <drab> luckily I actually don't really care to get to all the nodes on the lan from the phone, just out to the network and to my workstation and I can reach everything else from the latter
[19:24] <drab> s/network/internet/
[19:24] <sdeziel> drab: the clean fix would be to relocate the VPN server to a new network zone that is directly attached to the router, this way, you'd only need the static route on the router itself
[19:25] <drab> you mean like in a DMZ?
[19:25] <sdeziel> drab: kind of  but only for the VPN server
[19:25] <sdeziel> drab: because if you put some machines next to the VPN server, you'd run into the same problem when trying to access them from the VPN clients
[19:25] <drab> oh, I see
[19:26] <sdeziel> drab: in other words, you need to always have your router between your destination and your VPN server otherwise, machine will figure out that the router's hop is pointless
[19:26] <drab> I guess I could do that, yeah, hopefully nothing funny happens with the fw, when I was working on it last night I saw some weird things happening (I'm using firehol as a frontend to iptables)
[19:26] <drab> right, got it
[19:27] <drab> because if the ovpn is still physically on the lan interface then I have inface/outface on the gw both being lan and it seemed to trigger some issue with rule matching
[19:28] <drab> anyway, I'm about to take off so for this trip it's staying this way, but when I coem back that does sounds like a cleaner/permanent solution
[19:28] <drab> sdeziel: thanks
[19:28] <sdeziel> drab: you could probably use a dedicated vlan to get a new interface
[19:29] <drab> good point, I'll explore that
[19:29] <drab> thanks again, afk
[20:09] <ahasenack> hi, could someone please accept my nominations in bug #1732032? Thanks
[20:09] <ubottu> bug 1732032 in iproute2 (Ubuntu) "ip maddr show and ip maddr show dev enP20p96s0 show different data" [Undecided,In progress] https://launchpad.net/bugs/1732032
[20:10] <ahasenack> and a question: do we SRU FTBFS fixes?
[20:10] <ahasenack> or only when there is another needed change, then we really have to fix the ftbfs problem?
[20:10] <sarnold> ahasenack: done
[20:10] <ahasenack> sarnold: thanks
[20:11] <sarnold> ahasenack: I'm going to guess it's not worth an sru to fix ftbfs
[20:11] <ahasenack> sarnold: ok, ideas on how not to "lose" the fix, when a real sru comes by? Leave it attached to the bug?
[20:12] <ahasenack> just wondering
[20:12] <ahasenack> https://bugs.launchpad.net/ubuntu/+source/iproute2/+bug/1735158 in this case
[20:12] <ubottu> Launchpad bug 1735158 in iproute2 (Ubuntu) "FTBFS with gcc7" [Undecided,In progress]
[20:12] <sarnold> ahasenack: is it fixed in the devel release yet?
[20:12] <ahasenack> no, I"m about to do that
[20:13] <sarnold> ahasenack: my guess is that fixing it in devel is sufficient
[20:13] <ahasenack> ok
[21:44] <ahasenack> hm, I don't understand this git ubuntu lint error
[21:44] <ahasenack> E: Targetted branch distribution (devel) and changelog distribution (bionic) do not agree