[03:10] <caliculk> Does anyone here use an inventory management solution in the workplace that is compatible with Linux, OS X, and Windows and can perform SNMP scanning/polling, agent and agentless based information, has an API, and can pull serial numbers for all hard drives.
[05:47] <vbotka> caliculk, you might want to take a look at ansible.com
[09:44] <EmilienM> jamespage: hey, we're trying to deploy Mitaka on Xenial, we're having an issue with OVS
[09:44] <EmilienM> degorenko: can you past your logs here?
[09:44] <degorenko> yep
[09:44] <degorenko> http://paste.openstack.org/show/497463/
[09:45] <degorenko> http://paste.openstack.org/show/497467/
[09:45] <EmilienM> it sounds similar to https://bugs.launchpad.net/networking-ovs-dpdk/+bug/1512701
[09:47] <EmilienM> maybe do we need openvswitch-switch-dpdk ?
[09:47] <EmilienM> degorenko: can you run "dpkg -l | grep openvswitch" ?
[09:48] <degorenko> EmilienM, http://paste.openstack.org/show/497469/
[09:49] <EmilienM> jamespage: do we need something else when deploying openvswitch-switch package?
[09:51] <jamespage> EmilienM: nope
[09:51] <jamespage> EmilienM: status looks ok
[09:53] <EmilienM> jamespage: we also have an issuel with SSL deployment. The same Puppet manifests to deploy SSL certs on Trusty does not work on Xenial, see https://etherpad.openstack.org/p/puppet-openstack-xenial
[09:53] <EmilienM> we have a lot of [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)
[09:53] <EmilienM> something changed in Xenial for SSL certs?
[09:56] <EmilienM> degorenko: I'm adding the Horizon failure in the etherpad, you missed it
[09:56] <degorenko> are you about failed tempest tests?
[09:57] <EmilienM> yes
[09:57] <EmilienM> jamespage: we're trying to deploy Mitaka on Xenial, we have some issues
[09:58] <EmilienM> degorenko: I'm going to enable SSL again so we can have the failures available in logs
[09:58] <degorenko> EmilienM, what we should to do with openvswitch? :)
[09:59] <EmilienM> degorenko: I have no idea
[09:59] <degorenko> :(
[10:03] <EmilienM> jamespage: if you can help us that would be awesome, please let us know what you think about our issues in the etherpad. Thanks
[10:24] <jamespage> EmilienM: sorry juggling a few things todo so will be a bit async on responses
[10:24] <jamespage> EmilienM, degorenko: OK lets look at ovs first...
[10:25] <jamespage> you should not need the -dpdk binary for general use
[10:25] <jamespage> and the path looks just fine for the database.
[10:29] <jamespage> things to look at:
[10:29] <jamespage> check perms on /var/run/openvswitch/db.sock
[10:30] <jamespage> look in /var/log/openvswitch/*
[10:30] <jamespage> might give some further clues...
[10:36] <EmilienM> jamespage: thx for looking, I'm a bit afk for lunch, with reply in async
[10:42] <degorenko> jamespage, root@de-xenial:~/puppet-openstack-integration# ll /var/run/openvswitch/db.sock
[10:42] <degorenko> srwxr-x--- 1 root root 0 May 17 15:06 /var/run/openvswitch/db.sock=
[10:42] <degorenko> cat: /var/run/openvswitch/db.sock: No such device or address
[10:43] <jamespage> degorenko, is the trailing '=' a typo or actually whats in the directory?
[10:44] <degorenko> jamespage, http://paste.openstack.org/show/497480/
[10:44] <degorenko> i don't know why was added =
[10:44] <degorenko> :)
[10:46] <jamespage> degorenko, ok have one running as well - my /var/run/openvswitch looks comparable to yours
[10:46] <jamespage> and cat /var/run/openvswitch/db.sock does the same thing
[10:46] <jamespage> but
[10:46] <jamespage> I can add a bridge to the configuration
[10:46] <jamespage> http://paste.ubuntu.com/16487891/
[10:47] <jamespage> degorenko, anything useful in /var/log/openvswitch?
[10:47] <degorenko> jamespage, nothing, but let me look one more time
[10:47] <degorenko> jamespage, ovsdb-server has: 2016-05-18T10:46:51.182Z|01882|ovsdb_jsonrpc_server|WARN|Dropped 11 log messages in last 56 seconds (most recently, 6 seconds ago) due to excessive rate
[10:47] <degorenko> 2016-05-18T10:46:51.182Z|01883|ovsdb_jsonrpc_server|WARN|punix:/var/run/openvswitch/db.sock: connection exceeded maximum (330)
[10:48] <degorenko> and nothing interesting in ovs-vswitchd log
[10:48] <jamespage> degorenko, anything in syslog?
[10:49] <degorenko> jamespage, i see several neutron errors, but it is related to already posted error during creating bridge - and same error i see in syslog
[10:50]  * jamespage scratches his head
[10:50] <jamespage> degorenko, can you do a ovs-vsctl show?
[10:51] <jamespage> just wondering what is working and what's not
[10:51] <jamespage> also anything in /etc/default/openvswitch-switch ?
[10:51] <jamespage> other than the stock installed file
[10:52] <degorenko> jamespage, i found this in syslog http://paste.openstack.org/show/497480/
[10:52] <EmilienM> degorenko: do you have enough memory on the VM?
[10:52] <jamespage> degorenko, I think thats a previous pastebin
[10:53] <degorenko> 2 gb more free, 5.6 from 7.8
[10:53] <degorenko> EmilienM, ^
[10:54] <degorenko> jamespage, file /etc/default/openvswitch-switch is empty
[10:54] <jamespage> that's ok
[10:54] <degorenko> only comment lines
[10:56] <jamespage> on the deployment I have running we run it in a 1.5G instance for network nodes - and thats working fine...
[10:57] <degorenko> jamespage, show command also failed with same error
[10:58] <jamespage> degorenko, its related to 2016-05-18T10:46:51.182Z|01883|ovsdb_jsonrpc_server|WARN|punix:/var/run/openvswitch/db.sock: connection exceeded maximum (330)
[10:58] <jamespage> the socket is full - are both ovs processes still running?
[10:59] <degorenko> jamespage, http://paste.openstack.org/show/497483/ - yes
[11:01] <jamespage> degorenko, can I access your instance/
[11:01] <jamespage> ?
[11:01] <degorenko> no :( it is under private network
[11:02] <degorenko> jamespage, ^
[11:03] <jamespage> degorenko, what was the syslog thing you found?
[11:04] <degorenko> jamespage, well, i see a lot of request for creation bridges and then
[11:04] <degorenko> jamespage, http://paste.openstack.org/show/497484/
[11:04] <degorenko> after that i have database connection error
[11:05] <jamespage> degorenko, hmm
[11:05] <jamespage> degorenko, something is getting wedged, but I've not seen this before
[11:05] <jamespage> degorenko, can you try a restart of openvswitch-switch pls
[11:06] <degorenko> jamespage, full log for ovs commands: http://paste.openstack.org/show/497485/
[11:06] <degorenko> that's all i have from syslog related only ovs
[11:06] <jamespage> ovs was functional
[11:06] <degorenko> yes, it was
[11:06] <jamespage> and then after a few hours borked in some way
[11:07] <degorenko> i can redeploy it again
[11:07] <degorenko> and we will have fresh logs
[11:07] <degorenko> but anyway
[11:07] <degorenko> we have issue, that during second puppet run for idempotency - service was restarted
[11:07] <jamespage> degorenko, before you do "netstat -an | grep openvswitch" would be useful
[11:08] <degorenko> yeah
[11:08] <degorenko> to many connections
[11:08] <degorenko> http://paste.openstack.org/show/497486/
[11:08] <degorenko> jamespage, ^ that's not a full output
[11:09] <degorenko> i have 404 lines after this command
[11:09] <jamespage> degorenko, ok so we need to figure out what's holding those connections open
[11:09] <jamespage> sudo lsof | grep /var/run/openvswitch
[11:10] <jamespage> degorenko, I see this sort of thing:
[11:10] <jamespage> http://paste.ubuntu.com/16488179/
[11:11] <degorenko> jamespage, yep, http://paste.openstack.org/show/497487/
[11:11] <degorenko> i have a lot of db.sock
[11:11] <jamespage> from which processes?
[11:13] <degorenko> jamespage, root     17422  0.0  0.0  24732  2036 ?        S<s  May17   0:00 ovs-vswitchd: monitoring pid 17423 (healthy)
[11:13] <degorenko> root     17423  0.1  0.6 394188 50404 ?        S<Ll May17   1:37  \_ ovs-vswitchd unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir --log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach --monitor
[11:13] <degorenko> so, both processes which i posted above
[11:14] <degorenko> all db.sock from ovsdb-server /etc/openvswitch/conf.db -vconsole:emer -vsyslog:err -vfile:info --remote=punix:/var/run/openvswitch/db.sock --private-key=db:Open_vSwitch,SSL,private_key --certificate=db:Open_vSwitch,SSL,certificate --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --no-chdir --log-file=/var/log/openvswitch/ovsdb-server.log --pidfile=/var/run/openvswitch/ovsdb-server.pid --detach --monitor
[11:17] <jamespage> degorenko, might seem like an odd question but is self host resolution ok?
[11:17] <jamespage> degorenko, any spurious output on sudo XX comands?
[11:18] <degorenko> jamespage, he he, yes, it is
[11:18] <degorenko> ubuntu@de-xenial:~$ sudo -i
[11:18] <degorenko> sudo: unable to resolve host de-xenial
[11:18] <jamespage> degorenko, can you pastebin /var/log/openvswitch/ovsdb-server.log pls
[11:18] <jamespage> degorenko, hmm
[11:18] <degorenko> but i 'm not sure that we have same error on our infra slaves
[11:19] <jamespage> degorenko, are there alot of ovs-vsctl processes hanging around?
[11:19] <jamespage> or ovs-* processes - might be other cli tools
[11:19] <degorenko> jamespage, http://paste.openstack.org/show/497488/ - yes
[11:20] <degorenko> ~200
[11:20] <jamespage> degorenko, this is your problem
[11:20] <jamespage> degorenko, when there is spurious output on the sudo command, neutron-openvswitch just spins loads of those monitor processes and everything collapses...
[11:21] <jamespage> that's consuming all of the socket connections, resulting in what you see now...
[11:21] <jamespage> degorenko, EmilienM: this might be a diff in the  xenial image vs trusty image...
[11:22] <degorenko> jamespage, hmmm, ok, thank you for help, i'll rebuild my vm, then will add my host name to /etc/hosts and try to redeploy
[11:22] <jamespage> degorenko, np
[11:24] <jamespage> coreycb, hey - I switch the newton neutron builds to using ostestr - build speed picked up alot in the PPA's
[11:24] <jamespage> and seems more reliable...
[11:25] <jamespage> sort alot of sqlite racey things before - which I think is the cause of the FTBFS in yakkety for the mitaka point relase uploads...
[11:25] <jamespage> coreycb, also backported git-buildpackage for trusty for liberty and mitaka - should be able to switch --merge-mode=replace on soon
[11:35] <jamespage> coreycb, replace mode re-enabled...
[11:40] <coreycb> jamespage, awesome thanks
[11:47] <jamespage> coreycb, I also switched over keystone for newton
[11:47] <jamespage> appears to test alot faster now
[11:47] <jamespage> 19 mins vs 1hr
[11:47] <coreycb> jamespage, wow, that's a huge improvement
[12:34] <jamespage> coreycb, keystone in ppa is now 16 mins vs 52
[12:35] <coreycb> jamespage, that is really awesome.  my laptop is burning up here running neutron tests. :)
[12:46] <coreycb> beisner, these packages are ready for promotion to kilo-updates.  they've tested successfully and have aged 7 days in kilo-proposed.  http://paste.ubuntu.com/16490018/
[12:47] <beisner> coreycb, are there any pre-requisite charm upgrades tied to those package revs?
[12:47] <coreycb> beisner, no not for these
[12:48] <coreycb> beisner, while you mention it, I'm going to make a card to generalize that charm-helpers minor point release versioning so we don't have to deal with that.
[12:49] <beisner> coreycb, yes that'd be really nice.  it seems like an easy win to not potentially break folks, and ... peace of mind :-)
[12:49] <coreycb> beisner, definitely
[12:50] <beisner> coreycb, jamespage - kilo-proposed has libvirt-python and qemu.  are those needing to go as well?
[12:51] <coreycb> beisner, not yet for qemu, that's new
[12:51] <beisner> coreycb, ack
[12:51] <coreycb> beisner, not sure about libvirt-python
[13:01] <jamespage> coreycb, beisner: libvirt-python ++ yes please
[13:02] <jamespage> https://bugs.launchpad.net/ubuntu/+source/nova/+bug/1539506
[13:02] <jamespage> for reference
[13:02] <jamespage> coreycb, reverted merge-mode again
[13:02] <jamespage> apparently my backported package was foobar.
[13:02] <coreycb> jamespage, doh
[13:03] <coreycb> jamespage, need anything?
[13:10] <jamespage> coreycb, nah - back in now - testing atm
[13:11] <jamespage> coreycb, hows microversion-parse?
[13:11] <coreycb> jamespage, oh sorry, I'll work on it, I thought you said it was done
[13:18] <jamespage> coreycb, nope
[13:31] <EmilienM> jamespage: for the SSL errors, everything works fine on Trusty, but same manifests fail on Xenial, I have some logs
[13:31] <EmilienM> Error: Could not prefetch keystone_service provider 'openstack': Execution of '/usr/bin/openstack service list --quiet --format csv --long' returned 1: SSL exception connecting to https://127.0.0.1:35357/v3/services: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)
[13:32] <EmilienM> http://logs.openstack.org/30/308530/16/experimental/gate-puppet-openstack-integration-3-scenario003-tempest-ubuntu-xenial/cd44c41/console.html#_2016-05-18_10_16_38_727
[13:32] <EmilienM> apache logs: http://logs.openstack.org/30/308530/16/experimental/gate-puppet-openstack-integration-3-scenario003-tempest-ubuntu-xenial/cd44c41/logs/apache/
[13:36] <jamespage> EmilienM: deploying now to see if I can reproduce
[13:36] <EmilienM> jamespage: thanks for your help
[13:36] <jamespage> EmilienM: np
[13:36] <jamespage> want to see you unblocked so you can start testing out newton packages :-)
[13:36] <EmilienM> so here's how we deploy SSL :
[13:37] <jamespage> EmilienM: oh I have a way to deploy SSL :-)
[13:37] <EmilienM> 1) we drop /etc/ssl/certs/puppet_openstack.pem file
[13:37] <EmilienM> I mean, we put the file at this place
[13:37] <jamespage> sure
[13:37] <EmilienM> 2) we run /usr/sbin/update-ca-certificates -f
[13:38] <EmilienM> 3) we put the cert in /etc/keystone/ssl/private/cert.pem (etc for every project)
[13:38] <EmilienM> that's all I think
[13:38] <EmilienM> it used to work fine with Trusty
[13:39] <jamespage> EmilienM: is that for the CA cert?
[13:40] <EmilienM> the CA cert is put in /etc/ssl/certs/puppet_openstack.pem
[13:40] <jamespage> EmilienM: hmm ok
[13:40] <EmilienM> well
[13:40] <EmilienM> in fact we use the same cert for both
[13:40] <EmilienM> https://github.com/openstack/puppet-openstack-integration/tree/master/files
[13:41] <EmilienM> we have a crt & key for ipv4 & ipv6 (because we deploy apache and use ::1 or 127.0.0.1)
[13:41] <jamespage> EmilienM: any way you can turn up and capture the output of update-ca-certificates?
[13:41] <jamespage> -v would be useful
[13:42] <EmilienM> jamespage: yes! we can do it
[13:42] <EmilienM> degorenko: I'm going to update the patch ^
[13:43] <jamespage> EmilienM: awesome
[13:43] <jamespage> EmilienM: looking at the log I think you're writing it to /usr/local/share/ca-certificates/puppet_openstack.crt
[13:43] <jamespage> and then running update-ca-certifcates
[13:43] <EmilienM> yeah
[13:45] <EmilienM> jamespage: can I run it afterwards the puppet run?
[13:45] <jamespage> yup
[13:47] <EmilienM> ok let's try
[13:54] <jamespage> EmilienM: testing out my Juju deployed SSL openstack xenial mitaka foo now
[13:54] <EmilienM> excellent
[13:54] <EmilienM> jamespage: how do you deploy SSL?
[13:54] <EmilienM> can you point me to the charms?
[13:54] <EmilienM> (see how good is my vocabulary)
[13:55] <jamespage> EmilienM: kinda the same way you do  - the keystone charm acts as a CA for everything else, and provides signed certs back to related services
[13:55] <EmilienM> ok same yeah
[13:55] <jamespage> which then use them and update their endpoint entries todo https
[13:56] <jamespage> EmilienM: OK so I'm looking functional with https enabled...
[13:56] <jamespage> EmilienM: that's from a xenial client to a xenial cloud
[13:56] <EmilienM> ok, let's see
[13:57] <jamespage> EmilienM: that's good because it broadly means we're not looking at a code fault somewhere in the stack on installed things...
[13:57] <jamespage> EmilienM: how do you generate your CA and cert? I wonder whether its something about them that xenial does not like
[13:58] <EmilienM> jamespage: on http://www.selfsignedcertificate.com/
[13:58] <EmilienM> I'm very lazy, I know.
[13:58] <EmilienM> but it just works
[13:59] <jamespage> EmilienM: so how does the chain of trusty work with those?
[13:59] <jamespage> puppet_openstack.crt is the cert for the signing CA?
[13:59] <EmilienM> jamespage: yes
[14:00] <EmilienM> our certs are here: https://github.com/openstack/puppet-openstack-integration/tree/master/files
[14:00] <EmilienM> we have a pair of cert/key for ipv4 & ipv6
[14:01] <jamespage> hmm this rings a bell for some reason...
[14:01] <jamespage> EmilienM: I suppose those match to localhost right?
[14:02] <EmilienM> jamespage: 127.0.0.1 and ::1
[14:02] <EmilienM> we don't match on localhost because we want to test ipv4 & ipv6 endpoints
[14:02] <EmilienM> jamespage: but ipv6 tests don't run on ubuntu, so consider 127.0.0.1 only.
[14:02] <jamespage> ok
[14:03] <jamespage> EmilienM: verbose output of that update-ca-certifcates looks like it might help
[14:03] <jamespage> EmilienM: I'd like to check that your cert is getting into /etc/ssl/cert/ca-certificates.crt
[14:03] <EmilienM> yeah, CI is currently running
[14:03] <jamespage> you can grep for a line or so in that file to match
[14:08] <jamespage> EmilienM: might be relevant
[14:08] <jamespage> http://www.python.org/dev/peps/pep-0476/
[14:08] <jamespage> trusty has a pre 2.7.9 python 2.7 version
[14:09] <EmilienM> ok I have logs
[14:09] <EmilienM> a sec
[14:10] <jamespage> EmilienM: OK my SSL cloud appears completely functional, so it must be something todo with the way you're doing cert and building the trusty chain
[14:10] <jamespage> just a hunch
[14:10] <jamespage> trust chain
[14:10] <jamespage> not trusty...
[14:10] <EmilienM> ok
[14:10] <EmilienM> I'm trying to get the logs
[14:11] <EmilienM> job is still running, will show output in a few min
[14:12] <jamespage> ok stepping away for a bit...
[14:12] <jamespage> biab
[15:03] <jak2000> my network card not bringup at the start/boot, i need do a ifdown eth0 and then ifup eth0 how to fix it?
[15:29] <nacc> rbasak: i assume you're gone already, but if not o/; and if so, can we sync up tmrw AM?
[15:42] <smb> hallyn, somehow your 1.3.4 libvirt merge did upgrade without issues and it even seems to boot Xen PV and HVM guests... I wonder what I am doing wrong... ;) nice job
[15:54] <hallyn> smb: did you get a systemctl preset error on upgrade?
[15:54] <hallyn> once i finish testing my kernel patch i was going to try and reproduce that with a minimal empty pkg set
[15:57] <smb> hallyn, hm, not that I remember. I might have missed it if it did not cause a upgrade failure. But systemctl status on libvirtd was ok. Not sure whether one would now expect libvirt-bin to be the alias (or whether that actually is possible)
[15:58] <hallyn> smb: it is now the alias, and on clean installs that works
[15:59] <EmilienM> jamespage: http://logs.openstack.org/30/308530/18/experimental/gate-puppet-openstack-integration-3-scenario002-tempest-ubuntu-xenial/e17a5b4/console.html#_2016-05-18_15_26_02_283
[15:59] <smb> hallyn, ah ok. so that would be the only odd thing "systemctl status libvirt-bin" reports it as not running. Even after reboot...
[16:00] <EmilienM> jamespage: and http://logs.openstack.org/30/308530/18/experimental/gate-puppet-openstack-integration-3-scenario002-tempest-ubuntu-xenial/e17a5b4/console.html#_2016-05-18_15_26_13_161
[16:00] <hallyn> smb: yeah i think that may even be the whol eproblem - maybe systemd is trying to set up the symlink for the Alias and that fails bc something already exists
[16:02] <smb> hallyn, I believe that is what is done. But I don't know how one is supposed to cleanly do that switch-over
[16:02] <hallyn> :)  I can tell you that it worked before an init-system-helpers update in yakkety
[16:02] <hallyn> so it still may really be a bug there
[16:03] <hallyn> or, i was always doing something wrong and the old code just let it fly
[16:05] <smb> hallyn, :) I would not want to claim I knew what is right or wrong when it comes to systemd... one thing I just noticed is that status reports active running and then "bad" whatever that means
[16:14] <jeeves_moss> is the pagkage PuppetMaster still broken for 16.04?  I'm trying to install it through apt-get, and it fails
[16:15] <nacc> jeeves_moss: fails how?
[16:15] <jeeves_moss> nacc: "Job for puppetmaster.service failed because a timeout was exceeded. See "systemctl status puppetmaster.service" and "journalctl -xe" for details."
[16:16] <jeeves_moss> nacc: full error  --->  http://pastebin.com/3KNDZ4Nx
[16:17] <nacc> jeeves_moss: anything in the referred to logs (feel free to pastebin them too)
[16:17] <jeeves_moss> one sec.  I'll have a look once this wget is done
[16:19] <xnox> smoser, is it possible to do "xnox hates cloud-init networking" via cloud config drive and/or via user data and/or vendor data?
[16:22] <jeeves_moss> ugh, I miss having a box in the datacenter.  Trying to do a setup at home takes forever!
[16:27] <jeeves_moss> nacc: lol.  looks like it couldn't make the .pid file.  <rolls eyes>
[16:29] <nacc> jeeves_moss: permissions?
[16:29] <jeeves_moss> nope.
[16:30] <jeeves_moss> nacc: "puppetmaster.service: Failed to read PID from file /run/puppet/master.pid: Invalid argument"
[16:30] <nacc> jeeves_moss: does said file exist? and have valid contents?
[16:31] <jeeves_moss> the file didn't exist.  so I tried touching the file, and re-running the installer
[16:33] <nacc> jeeves_moss: it probably needs a pid in it
[16:33] <nacc> jeeves_moss: but i might be wrong
[16:33] <jeeves_moss> nacc: http://pastebin.com/WWWw7Hny
[16:34] <nacc> jeeves_moss: that still happens you mean? or was the original error?
[16:35] <jeeves_moss> nacc: orignial error.  new server, new install
[16:35] <nacc> jeeves_moss: and after `touch` it gets further?
[16:35] <nacc> jeeves_moss: just fyi, it would be good to file a bug (or see if one si filed) for puppetmaster that it fails to install due to the above
[16:36] <jeeves_moss> I had a look through the service file, and it should be "master.pid"
[16:37] <jeeves_moss> I'm going to see if it fails again.
[16:37] <jeeves_moss> would be nice if I can fil a bug.  would be the first one I've ever filed
[16:37] <nacc> !bug | jeeves_moss
[16:38] <nacc> :)
[16:39] <jeeves_moss> typical.  most things I want are always broken
[16:40] <nacc> jeeves_moss: fwiw, the version in 16.04 (and yakkety) is the same as in debian, so it's probably a debian bug, really -- would be good to verify if it happens there too
[16:41] <nacc> not strictly necessary, but it will probably get asked in teh bug :)
[16:41] <jeeves_moss> nacc: lol.  no body's got time for 'dat.   -->  http://s2.quickmeme.com/img/a0/a0ed68c2b414e58e131e7fa1c7ac66e4df4a14d30df577734812cdb95d9aaa99.jpg
[16:43] <jeeves_moss> ohhhh, new error!!!  java ins't installed.  might be a dep problem here guys
[16:43] <jeeves_moss> <rolls eyes>  all of this to manage 8 machines
[16:45] <nacc> jeeves_moss: puppet shouldn't need java? it's a ruby tool
[16:45] <nacc> i thought
[16:46] <jeeves_moss> hummm, we shall see.  I tried intstalling the latest snapshot from puppetlabs, and that's what I got.  so, we shall see
[16:46] <nacc> jeeves_moss: hrm? so not the ubuntu pacakge?
[16:46] <jeeves_moss> no, I grabbed the latest from their site.  if it installs, then the one oin the repo is suspect
[16:48] <jeeves_moss> nacc: it says puppetserver requires java
[16:49] <nacc> jeeves_moss: "it" being their version?
[16:49] <nacc> their being puppetlabs?
[16:50] <jeeves_moss> yes.  the latest from puppetlabs.  if that package works, then there is something wrong with the package in the ubuntu repo
[16:50] <nacc> well, they are different versions clearly
[16:51] <sdeziel> jeeves_moss: puppetmaster (ruby based) is not to be confused with puppetserver (java based)
[16:51] <nacc> sdeziel: thanks
[16:51] <sdeziel> nacc: np
[16:53] <jeeves_moss> sdeziel: any ideas on how to fix this install issue?
[16:54] <sdeziel> jeeves_moss: the puppetserver or the puppetmaster on?
[16:54] <sdeziel> s/on/one/
[16:54] <jeeves_moss> sdeziel: puppetmaster
[16:55] <jeeves_moss> sdeziel: this is the error we are getting.  http://pastebin.com/3KNDZ4Nx
[16:56] <sdeziel> jeeves_moss: I don't have any experience with puppetmaster directly. I've only used "puppetmaster-passenger" on Trusty
[16:57] <sdeziel> jeeves_moss: if you pastebin "systemctl status puppetmaster.service" and "journalctl -xe" I could look at them though
[16:57] <jeeves_moss> sdeziel: the outputs are in http://pastebin.com/3KNDZ4Nx
[16:58] <sdeziel> jeeves_moss: I only see the apt install failure log
[17:00] <jeeves_moss> hummm.  one sec.  it looks like even the latest pagakge from puppetlabs suffers the same issue
[17:01] <sdeziel> jeeves_moss: upstream focus seems to be on the Java based daemons (puppetserver and puppetdb) so it's possible they legacy version isn't in top shape
[17:01] <jeeves_moss> hummmm
[17:02] <jeeves_moss> at this rate, I should just abandon this mess
[17:02] <sdeziel> jeeves_moss: but for those you will get better support in their own channel
[17:03] <jeeves_moss> I've asked in there,  it's a ghost town!
[17:04] <sdeziel> jeeves_moss: hmm OK. If you want a puppetmaster/server and want to use the upstream apt repo, you are better off using Trusty because they don't yet officially support Xenial as the server role
[17:06] <jeeves_moss> sdeziel: fun.  how do I get the previous release if I'm installing with Apt-get?
[17:07] <sdeziel> jeeves_moss: apt-get install foo=<version>
[17:07] <jeeves_moss> hummm, google!!
[17:22] <coreycb> jamespage, https://git.launchpad.net/~corey.bryant/ubuntu/+source/python-microversion-parse
[17:59] <davethenoob> hi there everyone
[17:59] <davethenoob> I wonder if I could get a little advice please?
[18:00] <davethenoob> about me running my home server with Ubuntu Server installed
[18:00] <jak2000> my network card not bringup at the start/boot, i need do a ifdown eth0 and then ifup eth0 how to fix it?
[18:01] <genii> davethenoob: A more specific question might be more useful for someone to give an answer to :)
[18:01] <patdk-wk> !ask
[18:02] <patdk-wk> !poll
[18:02] <patdk-wk> !best
[18:02] <patdk-wk> damn bot
[18:02] <TJ-> !giveupwhilstyoureahead
[18:02] <davethenoob> hah. fair point. So I have my server installed and have had owncloud running on it and all good. Except that it started running slow and would upload photos to owncloud, which made me think it could have been a ddos attack. My question is can you recommend some good steps to do tonight to help minimize that risk?
[18:02] <jak2000> patdk-lap any advice?
[18:03] <davethenoob> thanks in advance
[18:03] <davethenoob> its 14.04
[18:03] <beisner> coreycb, oh i forgot to confirm that i pushed kilo-proposed to updates, except for qemu earlier.
[18:06] <coreycb> beisner, cool thanks
[18:06] <genii> davethenoob: Move ssh from port 22 to something more obscure, above 1024. Use key based authentication and not password based. Install and configure fail2ban
[18:08] <patdk-wk> genii, how does any of that help?
[18:08] <patdk-wk> he wants ddos protection
[18:08] <patdk-wk> you cannot protect yourself against a ddos unles you have enough bandwidth to handle it, or you hide behind someone that does
[18:08] <patdk-wk> you could completely firewall port 22 and not evne have ssh or any other service open
[18:08] <davethenoob> i have a home server that I want to access from outside home with owncloud and subsconic
[18:08] <patdk-wk> but your still vaunerable to a ddos
[18:09] <davethenoob> I have pasted an iptables rule into terminal
[18:09] <davethenoob> iptables -A INPUT -p tcp --dport 80 -m limit --limit 25/minute --limit-burst 100 -j ACCEPT
[18:09] <davethenoob> that one
[18:09] <patdk-wk> davethenoob, what is the goal?
[18:09] <patdk-wk> cause nothing you do on that server will help protect you from a ddos
[18:09] <patdk-wk> that limits http connects, not ddos
[18:09] <genii> patdk-wk: When someone is trying to bruteforce ssh passwords, it effectively chews up all the bandwidth and can cause outages
[18:09] <davethenoob> and used iptables persistent for it
[18:10] <patdk-wk> genii, yes, but that is hardly ddos protection
[18:10] <patdk-wk> he asked for ddos
[18:11] <patdk-wk> now to do more, stop bad/unwanted things from using your bandwidth, there are millions of things yo ucan do
[18:11] <patdk-wk> but you should find out what is using your bandwidth to find out what you should attempt in what order
[18:12] <patdk-wk> but no matter what you do, it is a simple job for someone to ddos you, just by flooding too much traffic to you
[18:12] <patdk-wk> it doesn't matter if you have an iptables rule that says DROP all incoming
[18:25] <davethenoob> thanks for advice guys.
[18:25] <davethenoob> After thinking, I think im gonna just make it a local server and not accessible outside
[18:25] <davethenoob> for safety
[18:26] <davethenoob> or start off local then maybe make it web-facing when I have some more knowledge / confidence
[18:44] <roaksoax_> ~/win 8
[19:07] <fellayaboy> can someone help me on how a thinclient connects to a vm on either xenserver or vsphere esxi?  how do i setup a thinclient to connect to a vm on a server?
[19:09] <patdk-wk> what is this *thinclient*?
[19:09] <fellayaboy> what protocol does the thinclient use to boot into a vm?  what server serves the thinclient
[19:09] <fellayaboy> a thin client machine - no cd/dvd, no hard drive.  just boots a VDI off the network.
[19:10] <fellayaboy> very small and minimal machine
[19:10] <patdk-wk> ok, but what does that have to do with a vm at all?
[19:10] <fellayaboy> i assumed a thin client connects to a vm
[19:10] <fellayaboy> how does it work then
[19:11] <patdk-wk> I guess this is more a question to your thinclient seller
[19:11] <fellayaboy> someone give me the basics pooo lease
[19:11] <patdk-wk> you need to know what your thinclient supports, what it can do
[19:11] <patdk-wk> I dunno what you mean by connects to a vm
[19:11] <fellayaboy> have u ever worked/setup a thinclient before patdk-wk
[19:12] <patdk-wk> the *standard* definition of a thin client is a basic monitor/keyboard has no, or very little programs installed
[19:12] <patdk-wk> fellayaboy, around 15years
[19:12] <fellayaboy> i thought a thin client machine boots an operating system off a server somewhere
[19:12] <patdk-wk> it might, it might not
[19:12] <patdk-wk> that depends on the thin client
[19:12] <fellayaboy> i see
[19:12] <fellayaboy> well in this case lets say it does
[19:12] <patdk-wk> it might have a 1gig cf card or something
[19:13] <fellayaboy> whats cf?
[19:13] <patdk-wk> yes, but I don't know what one oyu have, what it wants to do, does it do iscsi? nfs? http? how does it boot, what image does it expect?
[19:13] <patdk-wk> is it pxe? dhcp?
[19:14] <fellayaboy> so thin clients can boot straight to iscsi, nfs, http, pxe
[19:14] <patdk-wk> thinclients is a generic term that means nothing
[19:14] <fellayaboy> i see ok makes sense
[19:14] <patdk-wk> it is a catchall for any simple, reimagable bare-minimal workstation
[19:15] <patdk-wk> a thinclient could be custom hardware, that runs customfirmware
[19:15] <patdk-wk> it could be normal computers
[19:15] <patdk-wk> it could be normal computers that boot remotely, centrally
[19:15] <patdk-wk> but generally they are stripped down graphics, minimal ram, ...
[19:16] <fellayaboy> well, i want to create a system where i can use cheap inexpensive thin clients that connect to a centralized server, where they can run an operating system.
[19:16] <patdk-wk> normally only powerful enough generally to remote into something else, like using rdp/vnc
[19:16] <fellayaboy> through the network
[19:17] <patdk-wk> sounds like yo uwant to recreate vmware view with desktop hardware
[19:17] <patdk-wk> your going end up running a full blown desktop os on the clients, in order to run vnc, if you roll this yourself
[19:17] <fellayaboy> i guess that would be it.. ihavent used vmware view, ive used esxi server and client and rdp to connect to vm
[19:18] <fellayaboy> but never used a thin client to connect to one
[19:19] <fellayaboy> i want to have inexpensive thin clients connect to server. have the server do all the work etc.  and so that i can expand it all i want of course.  add more vm's, more endpoints. put all the load on the server.
[19:20] <fellayaboy> using 10gbps network card if have to with heavy duty switch/router etc
[19:20] <patdk-wk> you cannot put all the load on the server
[19:20] <patdk-wk> you can only put the cpu/ram load onto the server
[19:20] <patdk-wk> all graphics load will be on the client
[19:20] <patdk-wk> and on the server
[19:21] <fellayaboy> well not all, just want to have inexpensive endpoints thus thin clients
[19:21] <patdk-wk> and the vnc/rdp/... protocol processsing on the client
[19:21] <fellayaboy> that part is fine
[19:21] <fellayaboy> so theres  some thinclients that have in there bios a vnc/rdp protocol running already? or some kind of embedded os ?
[19:21] <patdk-wk> yes
[19:22] <fellayaboy> oh i see that makes sense
[19:22] <fellayaboy> and some have to option to boot off iscsi and nfs right
[19:23] <fellayaboy> what i want to do is create a Point of Sales network system.  say for a store.  using all open source technology
[19:23] <patdk-wk> maybe, those are getting not as thin then
[19:23] <fellayaboy> i see
[19:23] <fellayaboy> well now i know that some thinclients use vnc rdp in there embedded os
[19:23] <fellayaboy> that gives mea little headstart and some clarity
[19:23] <fellayaboy> thank you patdk-wk
[19:41] <mikedep333> So the Vagrant Xenial cloud images are currently broken. In the past, ultlemming fixed a Vagrant cloud image issue ASAP. Is he or someone else able to fix this?
[19:41] <mikedep333> https://bugs.launchpad.net/cloud-images/+bug/1565985
[19:49] <OerHeks> mikedep333, interesting.
[19:50] <OerHeks> dpkg -s virtualbox-guest-utils  is missing.. and post #7 about standard ssh keys missing..
[19:50] <OerHeks> i am glad about that last part, security wise
[19:51] <mikedep333> Yeah. Empirically, the ssh solution is working, but the shared folders is not.
[19:52] <OerHeks> not even after manual install, like post #5?
[19:52] <mikedep333> it looks like someone builds the virtualbox kernel modules without the package being installed
[19:52] <OerHeks> jups
[19:52] <OerHeks> corporate
[19:52] <mikedep333> the other virtualbox packages are not installed either
[19:53] <OerHeks> is it fixed v20160518.0.0 (last release 2 hours ago) ?? https://atlas.hashicorp.com/ubuntu/
[19:53] <mikedep333> So I should use those images rather than the ones on http://cloud-images.ubuntu.com/ ?
[19:54] <OerHeks> those are the same, see the last comment.
[19:54] <mikedep333> ok
[19:55] <mikedep333> I'll try them out, thanks
[22:00] <davethenoob> hello again. thanks for help earlier
[22:02] <davethenoob> I have opted for local home server with computer running ubuntu server. I know just have a pptp port open so i can vpn into home network and upload any new photos to my owncloud
[22:02] <davethenoob> are there any security measures to put in place when taking this route at all please?
[22:03] <sdeziel> davethenoob: pptp is known insecure
[22:03] <sdeziel> davethenoob: for alternatives you could check out OpenVPN or some IPsec implementations
[22:04] <davethenoob> umm. is it easy to swap?
[22:04] <davethenoob> I am complete noob to server stuff. Im just learning the ropes. Im php dev by trade
[22:04] <davethenoob> which do you suggest between the two?
[22:05] <sdeziel> davethenoob: OpenVPN as you'll be able to reuse the pptp port (TCP/1723)
[22:05] <Sling> davethenoob: why not just use SSH to manage the server?
[22:05] <sdeziel> davethenoob: this will save you a trip to your router's config
[22:06] <davethenoob> all i need it for is to access music on my subsonic, or photos on owncloud, both through android apps pointing to 192.168.x.x
[22:06] <davethenoob> I just want a vpn to access it outside
[22:07] <sdeziel> davethenoob: OpenVPN is available on Android as well
[22:07] <davethenoob> sdeziel so i just apt-get install the openvpn, disable pptp and enable openvpn?
[22:09] <sdeziel> davethenoob: that's a good starting point. Then you'll want to head to https://openvpn.net/index.php/open-source/documentation/howto.html for the configuration
[22:10] <sdeziel> davethenoob: you can also take a look at https://help.ubuntu.com/community/OpenVPN but I'd recommend to avoid bridged VPN (tap), prefer the tun/routed style
[22:12] <davethenoob> https://help.ubuntu.com/16.04/serverguide/openvpn.html
[22:12] <davethenoob> ?
[22:13] <sdeziel> davethenoob: that's actually an excellent guide
[22:14] <davethenoob> darn it
[22:14] <davethenoob> just realized im using my raspberry pi as vpn server
[22:17] <sdeziel> davethenoob: I have to go. Good luck with the VPN
[22:45] <Bae> anyone here good with the apparmor?
[22:45] <sarnold> hi Bae, what's up?
[22:49] <Bae> hey sarnold. glad i found someone who uses the project. i have a question about it. when i installed it and set it enabled what i saw was that it creates profiels for every bin/ binary out there. that means it creates some for ping etc. all that jazz. my question was about the default state of such bin files in the context of apparmors. as in, are they all set BY DEFAULT to inherit permissions/rules from its parent (calling process?) ?
[22:50] <sarnold> Bae: it depends upon the profile of the calling process. If the calling process is unconfined, then the "attachment specification" at the start of the profile says which programs to confine and with which profile
[22:50] <Bae> an example of this would be. say i made a nodejs file that calls the ping binary. would the ping binary BY DEFAULT (right after installing apparmor) be set to inherit apparmor permissions from the parent? as in if the parent is not allowed to access the network the ping service would be denied by apparmor?
[22:51] <sarnold> Bae: does your nodejs program run confined or unconfined?
[22:51] <Bae> sarnold, confined. and that will be enforced
[22:52] <sarnold> Bae: if you want to allow ping to run as expected, you could use /bin/ping Px,   rules in the nodejs program's profile
[22:53] <Bae> sarnold, a better example i thought up right now would be something like say, if my nodejs profile is not allowed to edit a file in directory say /home/someone/blockedDir/. i would add deny rules in the nodejs profile. cool. but what if nodejs calls another process to run (another binary) that is set to access that very folder? i want to make sure that that child process binary cannot also access that particular folder ONLY if nodejs calls it
[22:54] <sarnold> Bae: if you want to forbid ping's networking but still let it be run (for whatever reason...) you could use /bin/ping Cx -> ping,  rule, then add a "child profile", something like "profile ping { ... }"  to your nodejs profile..
[22:55] <sarnold> Bae: if you want to forbid ping's networking but still let it be run (for whatever reason...) you could use /bin/ping Cx -> ping,  rule, then add a "child profile", something like "profile ping { ... }"  to your nodejs profile..
[22:55] <Bae> OH i see what you mean sarnold. so that means i could leave the ping binary as is. and then in the node js {} i add a nest that shows the binary for ping {} that disallows it. cool.
[22:56] <Bae> shows the rules for ping {}*
[22:56] <sarnold> Bae: you could also use 'ix' rules instead of 'cx' -- then they'd run with the same privileges. But I like to encourage Cx where practical, since it can often be a drastic reduction in what privileges are allowed where
[22:56] <Bae> i should say
[22:57] <Bae> sarnold, if i were to use ix (inherit right? ) i would have to set ix in the ping binaries profile right ? not in the branched profile of the nodejs one
[22:58] <sarnold> Bae: btw, even though apparmor does all you to write e.g. "allow /home/** r, deny /home/sarnold/** r," it's really best to stick to _whitelisting_ as much as you can. It's often possible to construct attacks that get at data that is denied via a "deny" rule
[22:58] <Bae> yeah thats the plan. the deny thing was an example. my idea is to only whitelist certain directories and functions
[22:58] <sarnold> Bae: the nodejs' program's profile would use "/bin/ping ix," -- and the global /bin/ping profile is left alone completely
[22:58] <sarnold> Bae: okay, good, good :)
[22:59] <Bae> sarnold, the /bin/ping ix in the nodejs programs profile is not saying "inherit rules from ping" ?
[22:59] <Bae> because i want the ping to inherit from node. not node to inherit from ping
[22:59] <sarnold> Bae: correct; it is saying "when executing ping, it inherits this current profile"
[23:00] <Bae> ok let me do a quick clarification example if u dont mind
[23:00] <sarnold> or, rather, "when a program executing in this profile executes ping, ping inherits this current profile"
[23:01] <Bae> AH Yes
[23:01] <Bae> ok
[23:01] <Bae> so in here: http://pastebin.com/ArQFSCeF this is saying. when ping is born from nodejs app. ping inherits from nodejs app. right?
[23:02] <sarnold> Bae: ahh, skip the "-> child1," bit when using ix rules
[23:02] <Bae> thats from the site lol
[23:02] <Bae> sorry
[23:02] <sarnold> Bae: it is? can you link me? :)
[23:03] <Bae> maybe they did it for clarifications but here: so sarnold to ensure maximum security
[23:03] <Bae> ugh
[23:03] <Bae> http://wiki.apparmor.net/index.php/QuickProfileLanguage
[23:05] <Bae> so sarnold to ensure maximum security what i could do then is in the nodejs app i could put all the names of binaries that are profiled and tell them all to inherit rules from the nodejs app. in this way, any sort of exploit into the nodejs children processes would be thwarted. what do you think about it ?
[23:05] <sarnold> Bae: you should only need to add 'ix' rules for programs that your nodejs program actually executes
[23:06] <Bae> sarnold, so what is best way to see what programs nodejs executes? or rather, apparmor detects that the nodejs program executes ?
[23:06] <sarnold> Bae: if it only ever calls awk and sed, for example, you could just add /usr/bin/awk ix, /bin/sed ix, and be done :)
[23:06] <sarnold> Bae: aa-logprof should prompt you for them, but.. that family of tools is cranky. please file bugs as you find them. :)
[23:06] <Bae> oh ok
[23:07] <Bae> yeah i was thinking of something like valgrind that shows all the processes as you run it
[23:07] <Bae> but in apparmor :p
[23:07] <sarnold> Bae: you'll find them in the logs: apparmor="DENIED" operation="exec" profile="/tmp/bash" name="/tmp/ls" pid=21726 comm="bash" requested_mask="x" denied_mask="x" fsuid=0 ouid=0
[23:08] <sarnold> or apparmor="ALLOWED" if you're running the profile in learning mode, of course
[23:09] <Bae> so the best way for me to proceed right now to ID these processes first would be to run it in permissive mode and generate logs and unit test the full app
[23:09] <Bae> then grep those logs for all the processes generated. then add those in the nodejs app profile in apparmor with the ix
[23:09] <sarnold> you ought to be able to get apparmor to report all execs using something like (UNTESTED): "/tmp/bash { file, audit /** ix, }"   :)
[23:09] <Bae> to make sure that all the processes nodejs can ever generate, will be inheriting rules from the original one :)
[23:10] <Bae> ah thank you so much sarnold you are very helpful :D
[23:10] <sarnold> Bae: the aa-genprof tool should step you through those steps :)
[23:10] <Bae> yeah i did that lol. it just generates a boilerplate
[23:11] <Bae> i want something i can look through. that i can see those logs and see exactly what file is called and where
[23:11] <sarnold> aha then you sound like you -do- want to do things by hand. :)
[23:11] <stoned> Hello, I'm having trouble finding the xdebug config file in ubuntu server
[23:11] <Bae> i do yes sarnold
[23:11] <Bae> anyway i will be saving this conversation
[23:11] <Bae> i dont mind getting my hands dirty
[23:13] <sarnold> Bae: there's also #apparmor on irc.oftc.net in case there's no one around here who can help
[23:15] <Bae> oh i did not know that! thanks so much :)
[23:15] <sarnold> sure thing :) have fun :)
[23:15] <Bae> yeah sarnold thank you so much for all your help