[00:00] <bananapie> this is bad, all the defunct processes have parent process id of 1...
[00:00] <bananapie> and two /sbin/init running
[00:00] <sarnold> ...
[00:01] <bananapie> something has gone terribly wrong on this server.
[00:01] <sarnold> hmm, the second init might be a user session thing, I've got one of those too: init --user --restart --state-fd 25
[00:05] <Ironlenny> I have a kvm vm that is using a macvtap bridge, but I cannot get an ip address from my network dhcp server. I'm running 14.04
[00:10] <bananapie> I rebooted the server. :(
[00:10] <bananapie> this server has the worst uptime
[00:10] <bananapie> 267
[00:11] <bananapie> 267 days*
[00:48] <Ironlenny> I have a kvm vm that is using a macvtap bridge, but I cannot get an ip address from my network dhcp server. I'm running 14.04
[08:32] <nivv_> hey guys, how do I block this "user" to do stuff? root@databeredning
[08:32] <nivv_> what does the @ mean?
[08:46] <lordievader> Good morning.
[08:48] <lordievader> nivv_: The 'databeredning' is the hostname of the machine.
[08:48] <nivv_> ah ok,
[08:58] <nivv_> When I try to do "iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT" I get "iptables: No chain/target/match by that name."
[08:58] <nivv_> how come?
[08:58] <nivv_> the input chain is present
[08:59] <lordievader> nivv_: Your kernel knows conntrack?
[08:59] <lordievader> Ubuntu kernels should by the way...
[09:00] <nivv_> lordievader, no idea,
[09:01] <nivv_> Do I need to install it?
[09:02] <lordievader> nivv_: What does "sudo lsmod|grep conntrack" return?
[09:02] <nivv_> lordievader, hold on, I'll check. The tech support says that my server is being hacked
[09:02] <nivv_> using up loads of network bandwidth so I can barley ssh into the machine
[09:02] <nivv_> not seeing anything on nethogs or iftop
[09:04] <nivv_> lordievader it returned nothing, blank line
[09:06] <lordievader> nivv_: Do you run the default Ubuntu kernel?
[09:06] <nivv_> "Welcome to Ubuntu 12.04.5 LTS (GNU/Linux 2.6.32-042stab062.2 x86_64)"
[09:06] <nivv_> Don't know if it's the default
[09:07] <lordievader> !info linux-image-generic precise
[09:07] <lordievader> nivv_: It ain't.
[09:08] <nivv_> it's a hosted vps
[09:09] <lordievader> nivv_: Your kernel either does not have conntrack compiled in or loaded as a module.
[09:09] <nivv_> ok, i tried doing "sudo apt-get install conntrack"
[09:09] <nivv_> but still get nothing when doing sudo lsmod|grep conntrack
[09:11] <nivv_> if I do conntrack -L
[09:11] <lordievader> nivv_: What happens when you run "sudo modprobe nf_conntrack"?
[09:11] <nivv_> i get conntrack v1.0.0 (conntrack-tools): Operation failed: Connection refused
[09:11] <nivv_> WARNING: Deprecated config file /etc/modprobe.conf, all config files belong into /etc/modprobe.d/.
[09:11] <nivv_> FATAL: Module nf_conntrack not found.
[09:12] <lordievader> As I figured.
[09:13] <lordievader> Hate to break it to you, but custom kernels are not supported here. Running the default Ubuntu kernel solves your problem. However a workaround would be to use something other than conntrack.
[09:15] <nivv_> lordievader, thanks anyways! I really appreciate it. I don't even know what "sudo iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT" means
[09:15] <lordievader> nivv_: Then it might be a good idea to learn that first ;)
[09:15] <nivv_> of course. But at the same time I'm being "hacked" according to the tech support
[09:16] <nivv_> so I don't really have much time
[09:16] <nivv_> :(
[09:17] <lordievader> nivv_: Drop everything but port 22?
[09:18] <nivv_> Yeah, I'm trying to implement this
[09:18] <nivv_> http://paste.jesse-obrien.ca/1c5S
[09:18] <nivv_> but I'm stuck on the first one
[09:18] <lordievader> nivv_: Do you have a backup way in, if ssh fails?
[09:18] <nivv_> no
[09:18] <nivv_> not that I know of
[09:19] <lordievader> nivv_: Since firewalls can lock you out very easily ;)
[09:20] <nivv_> maybe that's a good thing...seeing how clumsy I seem to be
[09:20] <lordievader> nivv_: You can skip the first line, it will make your firewall slower but for 3 rules you won't notice it.
[09:20] <nivv_> oh sweet
[09:20] <nivv_> here goes nothing then
[09:23] <nivv_> lordievader, this is the info I got from tech support, does it tell you anything? : http://paste.jesse-obrien.ca/1c6d
[09:24] <nivv_> wtf is "lrwkqgjsb"
[09:25] <lordievader> Could be an exploit, is it still running? If so, kill it right now!
[09:27] <nivv_> how do I find it?
[09:27] <lordievader> nivv_: "ps aux|grep lrwkqgjsb"
[09:27] <nivv_> got this "albin     1103  0.0  0.0   9516   948 pts/4    S+   10:27   0:00 grep --color=auto lrwkqgjsb"
[09:28] <nivv_> now I can't do "sudo apt-get update"
[09:28] <nivv_> can't connect to the repos
[09:28] <lordievader> nivv_: Ok that is good. But still. Take a look at the other processes.
[09:29] <lordievader> nivv_: Did you also drop outgoing connections?
[09:29] <nivv_> https://www.dropbox.com/s/n2xgxdq605gl8qy/Sk%C3%A4rmklipp%202014-12-05%2010.29.17.png?dl=0
[09:29] <nivv_> dump of the "sudo iptables -L -v"
[09:34] <nivv> lordievader, sorry got disconnected
[09:34] <nivv> did you see anything wrong in the iptables?
[09:35] <lordievader> nivv: No, outgoing connections should be fine.
[09:35] <nivv> i flushed the iptables again and now it's working.
[09:37] <nivv> And when I add the rules again I can't connect to archive.ubuntu.com
[09:39] <lordievader> nivv: That doesn't make any sense...
[09:39] <nivv> found this: http://serverfault.com/questions/121309/how-to-configure-iptables-to-use-apt-get-in-a-server
[09:39] <nivv> see the top answer
[09:39] <nivv> maybe has something to do with not using the first line?
[09:39] <lordievader> Can you still perform dns lookups?
[09:40] <lordievader> nivv: You don't have a drop policy on the output chain.
[09:41] <nivv> When I add sudo iptables -A INPUT -j DROP it stops working
[09:42] <lordievader> I do hope you have your "allow ssh" above it...
[09:43] <lordievader> Anyhow can you answer my question?
[09:43] <nivv> i do it in this order
[09:43] <nivv> http://paste.jesse-obrien.ca/1c7c
[09:44] <lordievader> Can you do dns lookups?
[09:44] <nivv> sorry for being ignorant, but how do I do that?
[09:45] <lordievader> nivv: "nslookup www.ubuntu.com"
[09:45] <nivv> -bash: nslookup: command not found :D
[09:47] <lordievader> nivv: sudo apt-get install dnsutils
[09:47] <lordievader> IIRC
[09:48] <nivv> yup! thanks
[09:58] <lordievader> nivv: Does "sudo iptables -I INPUT 1 -m state --state RELATED,ESTABLISHED -j ACCEPT" fix your apt problem?
[09:59] <nivv> hold on, im on the phone with tech support
[09:59] <nivv> :)
[10:00] <JediMaster> hi, can anyone recommend any file change monitoring services? A client has recently had a wordpress installation breached and it wasn't obvious that it had been. The files were quite expertly altered, both PHP and javascript files.
[10:00] <nivv> lordivader, the tech support says that our server uses a massive amount of network bandwidth. Wouldn't I see that when looking on the activity in nethogs?
[10:01] <nivv> lordievader ^
[10:01] <lordievader> nivv: iftop would show you.
[10:01] <JediMaster> The files were altered in such a way that the timestamp on the files did not change (I'm still not sure how they pulled that off in PHP), and the commands were hidden within multi-line comments that already existed and enough spaces added to the end of the line to hide it from common command line text editors
[10:03] <lordievader> JediMaster: Zabbix can alert you of file changes, but I'm not sure if it will recursively parse through a directory if you give it one.
[10:03] <JediMaster> So really we're interested in monitoring only a subset of file changes, not timestamp based detection but maybe md5sum checks and only for certain file types, e.g. php, javascript
[10:03] <JediMaster> lordievader, May the force be with you!
[10:03] <lordievader> JediMaster: That's what zabbix does ;)
[10:04] <JediMaster> ohh shiny, I already have zabbix clients on all the servers
[10:04] <JediMaster> I wasn't awear it could monitor file changes
[10:04] <lordievader> JediMaster: https://www.zabbix.com/forum/showthread.php?t=23061
[10:05] <JediMaster> a quick glance over that seems to suggest you need to specify files, there would be tens of thousands of PHP files to monitor
[10:06] <lordievader> JediMaster: Like I said, I'm not sure if there is folder support...
[10:06] <lordievader> But there might just be ;)
[10:07] <nivv> lordievader, just hung up on the tech support of our hosting prodivder....they CONFUSED our IP with another scumbags IP
[10:07] <nivv> our server was never affected, it was another one in the same cluster
[10:08] <JediMaster> the other problem is that we do updates to files nearly every day, so it'd be good if there was some way we could update the md5sums when doing an git pull/svn update
[10:09] <lordievader> JediMaster: You could write a script that checks "git status" output ;)
[10:09] <lordievader> nivv: Doesn't take away that it is a good idea to have a firewall running.
[10:09] <nivv> lordievader, exactly
[10:09] <JediMaster> well interestingly, the way we found it was an svn status
[10:10] <nivv> so I'm still gonna see to that, soon™
[10:11] <lordievader> You could even write a cronjob that simply makes sure that there are no uncommitted changes... ofcourse if an attacker notices this he (or she ;) ) will simply commit the changes ;)
[10:11] <JediMaster> that had occured to me
[10:11] <JediMaster> both points
[10:12] <lordievader> I suppose you could block commits being made on production machines.. but that would likely be a hassle.
[10:15] <JediMaster> ah, I forgot about tripwire, but I think that'll have the hassle of warning me about every legitimate change we make
[10:18] <lordievader> A very ugly solution: put it in puppet :P
[10:56] <jamespage> gnuoy, "python-logutils	Liam Young <liam.young@canonical.com> (James Page <james.page@ubuntu.com>)" is showing up on my merge report - want to take care of that and I'll sponsor it for you?
[10:57] <jamespage> semiosis, I also see glusterfs on the list of merges - are you going to pull in the version from Debian experimental this cycle?
[11:02] <jamespage> semiosis, I'm guessing that might fixup alot of the feedback from the MIR review in 14.04
[11:02] <gnuoy> jamespage, sure, thanks
[11:42] <acmehandle> Does anyone know if stackless python is in apt repositories?  I tried searching apt-cache and it doesnt seem so
[11:48] <lordievader> acmehandle: Stackless Python, what is that?
[11:49] <acmehandle> http://www.stackless.com/
[11:51] <lordievader> Apt says about 'python3-greenlet': The greenlet package is a spin-off of Stackless
[11:51] <lordievader> !info python3-greenlet
[11:52] <acmehandle> Excellent.
[11:53] <acmehandle> Didnt know what to look for.
[11:53] <acmehandle> Its not going to replace the python3 I already have right?
[11:53] <lordievader> acmehandle: It doesn't conflict with it.
[11:54] <acmehandle> Ok, now that I installed it how do I invoke it?
[11:54] <lordievader> acmehandle: No idea...
[11:54] <acmehandle> is it now part of python3 then?
[11:54] <lordievader> acmehandle: No idea ;)
[11:54] <acmehandle> Where can I find the readme for it?
[11:55] <lordievader> acmehandle: I thought you knew the stackless thing ;) Anyhow apt-file will probably tell you what files it provides.
[11:55] <acmehandle> Nope, just discovered it.
[11:56] <acmehandle> whilst searching for an asynchronous webapp solution.
[11:56] <acmehandle> ghetto gang banging rap music at 7am.  thats class.
[12:00] <acmehandle> No, python3-greenlet is not stackless.
[12:00] <acmehandle> greenlet is a module
[12:00] <acmehandle> Its not a python interpreter
[12:02] <acmehandle> No, thats not at all what I want.
[12:52] <nivv> lordievader, you still here? :)
[12:54] <nivv> Anyone know of any alternatives to iptables?
[12:55] <mardraum> there are lots of them. Do you need them to work on ubuntu or not? :p
[12:55] <nivv> hehe yea, my kernel doesn't have conntrack, and when I add some rules the dns lookup stops working :S
[12:55] <nivv> marrdraum https://www.dropbox.com/s/3ge0czq84fbllgm/Sk%C3%A4rmklipp%202014-12-05%2013.55.48.png?dl=0
[12:56] <nivv> thats my rules right now
[12:56] <nivv> and if I try a nslookup I get nothing :S
[12:58] <mardraum> perhaps you need an OUTPUT rule, like eg iptables -A OUTPUT -m state --state NEW,RELATED,ESTABLISHED -j ACCEPT
[12:59] <mardraum> I hate firewalling with iptables though, I would always replace it with something else if that were the purpose.
[12:59] <nivv> ah, I found the problem. I needed to add the following rule: iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
[13:00] <mardraum> yeah, you do need that
[13:00] <nivv> :))
[13:02] <nivv> mardraum the guide I was following was using conntrack, my kernel didn't have that so it threw an error
[13:03] <nivv> hey mardraum, now I can't access the server via SSH
[13:03] <nivv> wtf
[13:04] <nivv> nevermind
[13:04] <nivv> works now
[13:04] <nivv> phew
[13:11] <lordievader> nivv: So my guess was right ;)
[13:12] <nivv> lordievader, what were your guess now again? ;)
[13:12] <lordievader> nivv: By the by, it is better to put the RELATED rule as the first in the chain, not as the last.
[13:12] <nivv> you mean "sudo iptables -I INPUT 1 -i lo -j ACCEPT" ?
[13:12] <lordievader> From earlier: "nivv: Does "sudo iptables -I INPUT 1 -m state --state RELATED,ESTABLISHED -j ACCEPT" fix your apt problem?"
[13:13] <nivv> ah shiet, I missed that lordie!
[13:13] <nivv> https://www.youtube.com/watch?v=l1dnqKGuezo
[13:13] <lordievader> No ^ that rule, that command puts it as the first rule.
[13:13] <nivv> ah
[13:13] <nivv> this is the rules I'm using now, in that order
[13:14] <nivv> https://gist.github.com/nivv/de0cf110131f830e37fc
[13:14] <nivv> should I change anything from that?
[13:17] <lordievader> nivv: That is allways a difficult question to answer as I have no idea what your requirements in terms of service or security are.
[13:18] <nivv> I mean the order of the lsit
[13:18] <nivv> list
[13:19] <lordievader> nivv: Line 6 could be replaced with a policy drop (-p DROP). But further more, I guess it is allright.
[13:19] <nivv> like this "sudo iptables -A INPUT -j -p DROP" ?
[13:21] <lordievader> nivv: "sudo iptables -p INPUT DROP" But make sure you allow your services first ;)
[13:22] <nivv> yea :) what's the difference between -p and -j ? (sorry again if I'm being stupid)
[13:22] <lordievader> nivv: man iptables ;)
[13:23] <Guest87920> guys, anyone have any idea how is it even possible that ubuntu/other distros doesnt have smething like mageia control center?
[13:23] <nivv> ah thanks, lordievader, really appreciate your help!
[13:24] <mardraum> Guest87920: maybe it sucks? what is it.
[13:24] <lordievader> Guest87920: Mageia control center?
[13:24] <Guest87920> this http://doc.mageia.org/mcc/3/en/content/mcc-hardware.html
[13:25] <lordievader> Blegh gui's...
[13:26] <lordievader> You have a command line, what's more to want?
[13:28] <Guest87920> lordievader: yea your grandmother is a perl monk to i suppose?
[13:29] <lordievader> You suppose wrongly.
[13:30] <Guest87920> mardraum: how can you assume something sucks if u dont even know what it is
[13:32] <nivv> Guest87920: he said maybe ¯\_(ツ)_/¯
[13:32] <Guest87920> nivv: mkay :D
[13:39] <Guest87920> nivv: all i meant to express was that if he saw word mageia ot doesnt mean it have to suck. Maybe its actually just the opposite. Who knows .. :D
[13:40] <lordievader> To each his own, eh ;)
[13:40] <nivv> I think he was trying to point out that it was impossible for us to know what you're talking about without a description of the software, which you gave immediately afterwards :)
[13:40] <Guest87920> :D
[13:41] <JediMaster> lordievader: ever come across "aide" looks something like what I want, but not quite
[13:43] <lordievader> JediMaster: Never heard of it.
[13:47] <Guest87920> so nobody got even a slight speculation why it could be so that MCC havent already been ported to any other distro?
[13:47] <Patrickdk> heh?
[13:47] <Patrickdk> this channel is about ubuntu-server, how would we know about other distro's? or about mcc?
[13:48] <Patrickdk> I would ask the #mcc channel about it
[13:48] <jamespage> zul, coreycb, gnuoy: can I get a +1 for inclusion of openvswitch 2.3.0 in the kilo CA please?
[13:48] <jamespage> ditto for ceph giant once I have it in vivid
[13:48] <zul> jamespage:  yes go ahead
[13:49] <lordievader> Guest87920: (Linux-)Server guys don't like gui's for configuring their servers...
[13:49] <Guest87920> oh right
[13:49] <lordievader> Generally at least.
[13:49] <Patrickdk> windows server guys too :)
[13:49] <Patrickdk> why we have core-server for windows now :)
[13:49] <lordievader> The Windows admins I know still like to click around ;)
[13:49] <Patrickdk> yuk
[13:50] <samba35> is it possible to add ethernet card to system (want to add 1 more nic to system)
[13:50] <Guest87920> not on newer versions of it i suppose.. :D
[13:50] <Patrickdk> we have deployed core server for most things, but a *few* apps still need the gui :(
[13:50] <Patrickdk> samba35, only if you have some place to plug it in
[13:51] <samba35> yes i have some free pci slots are there
[13:51] <Patrickdk> then just pick one with a sane driver then :)
[13:51] <Patrickdk> most cards will work
[13:51] <coreycb> jamespage, +1
[13:51] <samba35> Patrickdk: can you please tell me which card do you recommand
[13:51] <Patrickdk> intel and stuff will be painless, broadcom will be more painful
[13:52] <samba35> ok thanks
[13:52] <samba35> bye for now
[13:58] <coreycb> zul, jamespage: ceilometer 2014.2.1 is ready for review - https://code.launchpad.net/~corey.bryant/ceilometer/2014.2.1/+merge/243799
[13:58] <zul> coreycb: cak
[14:01] <coreycb> zul, too early for cake
[14:01] <zul> its never too early for cake
[14:08] <coreycb> zul, jamespage: cinder 2014.2.1 ready for review - https://code.launchpad.net/~corey.bryant/cinder/2014.2.1/+merge/243803
[14:09] <zul> coreycb: lemme know when they are all ready for review and ill do it
[14:09] <coreycb> zul, k
[14:15] <acmehandle> I'm trying to install a deb pacakge using dpkg -i and am getting the following error:  unable to open file '/var/lib/dpkg/tmp.ci//control': No such file or directory
[14:15] <acmehandle> Please advise
[14:15] <jrwren> acmehandle: sounds like it is not a deb package?
[14:17] <acmehandle> jrwren: Its this:  http://www.stackless.com/wiki/Download   In the binaries section
[14:18] <acmehandle> The 3.2 version installed.  Whereas me trying to install the 2.7 produced the above error
[14:23] <jrwren> acmehandle: did you check the md5 ?
[14:26] <acmehandle> jrwren: md5sum checks out
[14:28] <acmehandle> I did a dpkg-deb -R went in to DEBIAN, apparently the control file is a ln -s to a non existent file.
[14:29] <acmehandle> Whereas when compared to the 3.2 package the DEBIAN/control file is its own file, not a link
[14:29] <lordievader> acmehandle: Is the package made for Ubuntu?
[14:30] <acmehandle> Dont know, its made for debian.  I installed the 3.2 version but it required tcl and tk dependencies to be resolved
[14:30] <acmehandle> Still, 3.2 installed without trouble
[14:31] <lordievader> acmehandle: Then get one made for Ubuntu. Debian and Ubuntu are similar and yet they are different ;)
[14:31] <acmehandle> I dont think there is a stackless python made for ubuntu, only the derivative stackless module.
[14:31] <acmehandle> Not what I want.
[14:31] <mardraum> so use debian
[14:32] <lordievader> Or compile from source.
[14:32] <acmehandle> Sounds like I'll be compiling from source then
[14:32] <lordievader> Create your own package and throw it on launchpad.
[14:32] <acmehandle> Ok, I'll compile, how would I create my own package though?
[14:32] <acmehandle> I've compiled before using configure
[14:33] <acmehandle> just never created a package.
[14:34] <lordievader> Err, I've only made packages to distibute code. Not binary packages... But google probably knows ;)
[14:34] <acmehandle> Indeed, the internet knows everything
[14:35] <jrwren> acmehandle: maybe they have a source deb?
[14:36] <jrwren> acmehandle: nope, I don't see a source deb :(
[14:36] <jrwren> acmehandle: depending on why you want stackless, pypy may be an option.
[14:38] <acmehandle> its stackless mode, but not stackless
[15:02] <zul> jamespage:  craaaap http://paste.ubuntu.com/9383355/
[15:21] <jamespage> zul: love it -https://launchpad.net/ubuntu/+source/ceph/0.87-0ubuntu1
[15:21] <jamespage> first time that's ever happened to me with ceph...
[15:22] <zul> jamespage:  no problems
[15:22] <zul> ?
[15:44] <samba35> how do i add psi-passtroth on ubuntu 14.04.1 with kvm
[16:21] <teward> who has primary control over the serverguide documentation?
[16:21] <teward> server team or doc team?
[16:36] <lakin> Good morning folks, I'm running into some difficult to reproduce on command, but regular (a couple an hour) segfaults/aborts with apache2 on Ubuntu 14.04. I have some backtraces generated from core dumps, all seem to be crashes in libssl - I am fully up to date as of last night. How would I go about figuring out the cause of it?
[16:38] <lakin> stack traces are here: http://apaste.info/OBl http://apaste.info/FiK http://apaste.info/ptl http://apaste.info/q8X http://apaste.info/tjh http://apaste.info/Lc7
[17:20] <jcastro> gaughen, ping!
[17:20] <gaughen> jcastro, pong
[17:21] <jcastro> hi!
[17:21] <jcastro> we got a bunch of new questions on openstack and the charms: http://askubuntu.com/questions/tagged/openstack?sort=unanswered&pageSize=50
[17:21] <jcastro> can you ask people to take a look? I've tacked on some bounties as well
[17:22] <jcastro> roaksoax, there are maas questions piling up too
[17:25] <roaksoax> jcastro: haha :)
[17:27] <sarnold> lakin: has anything else been segfaulting? since the crashes are all over the place I kinda wonder if you've got some bad memory or something similar
[17:28] <sarnold> lakin: I don't know the openssl internals well enough to know if the presense of ssl3 in all those methods means actual ssl3 or if they just never got around to renaming functions for TLS; it might be worth rechecking that you've got ssl2, ssl3 disabled if you can
[17:29] <sarnold> .. details on turning off ssl3 here: http://askubuntu.com/questions/537196/how-do-i-patch-workaround-sslv3-poodle-vulnerability-cve-2014-3566
[17:53] <lakin> sarnold: I believe we have turned off ssl3
[17:54] <lakin> but double checking now
[17:55] <lakin> Confirmed, SSLProtocol All -SSLv2 -SSLv3 is in our configuration
[17:58] <lakin> sarnold: thanks for your help, I have to leave but I'll be back once I'm at the office
[18:31] <hariom> What is the best way to transfer files (size less than 150 kb) to another server in secure and fastest possible way? As soon as file comes on Server A, I want it to send to Server B in secure way
[18:33] <sarnold> hariom: there's a lot of options, you'll have to constrain the space a bit to get good answers. how many files per second? how far apart are the servers? why do you want the copies, what will the copies achieve?
[18:35] <hariom> sarnold: I need to send each file separately. Anywhere between 100 to 200 files per second (the max I can think I can get from clients). Servers are on LAN in a data center
[18:35] <hariom> sarnold: Each file is unique and not made available before
[18:36] <hariom> sarnold: Can NFS achieve that securely? or SCP/rsync etc?
[18:38] <hariom> sarnold: Are you there?
[18:38] <sarnold> hariom: scp or rsync would have extreme trouble keeping up with 100 per second. NFS ought to be able to do that, as could ceph
[18:40] <hariom> sarnold: Can NFS be secured?
[18:41] <ikonia> depends on your issues
[18:41] <hariom> ikonia: like what?
[18:41] <ikonia> like why you think it's not secure, what your limitations are
[18:41] <hariom> Ceph may have a learning curve. Never heard it before but seems very interesting
[18:42] <hariom> ikonia: Sniffing
[18:42] <ikonia> sniffing what ?
[18:42] <sarnold> it'd be worth spending two hours reading about ceph regardless if you ever use it or not, it's neat stuff
[18:43] <sarnold> hariom: you could run nfs / cifs / ceph / whatever over IPSec or openvpn or something else if you wished; I don't believe NFS has any real privacy controls..
[18:43] <LinStatSDR> Hello all.
[18:57] <hariom> sarnold: What if I create a socket client and server. As soon as I get a file, I read and send it to server. Socket server can have SSL
[18:58] <hariom> I suppose this will be quick, fast and secure?
[18:59] <sarnold> hariom: the trick is that you'd need some protocol of some sort to identify filename, file size, build in recovery mechanisms if the connection stalls, etc.
[18:59] <sarnold> hariom: it sounds simple at first but getting it right will take some effort
[19:00] <hariom> sarnold: I don't need file name and size as they are saved in db so remote server can access it from there.
[19:02] <sarnold> hariom: but you need to know when one file stops and another file starts, right? :)
[19:03] <hariom> hmm... got your point
[19:04] <hariom> sarnold: base64 encode and send json string with filename :)
[19:04] <hariom> Overhead of encoding
[19:06] <sarnold> hariom: hehehe, oof :) but that does sound like an option
[19:07] <sarnold> hariom: maybe something like 0mq or protobufs can handle this; I really haven't looked at moving larger objects with those systems but they might be well-suited
[21:22] <keithzg_> Hmm, how do I tell where dnsmasq is getting its responses from? Trying to figure out exactly what's responsible for annoyingly long DNS reply lag on my network.
[21:27] <keithzg_> ex. any ping immediately resolves the correct address, but then pauses for a very long time before actually getting replies, presumably taking forever to perform the reverse lookup if I'm understanding it correctly.
[21:38] <guntbert> keithzg_: *if* you have nm-tool:   nm-tool|grep -i dns
[21:45] <keithzg_> guntbert: hrmmm, that's interesting, nm-tool returns an IP address that is the expected DNS server, and running dig on arbitrary internal hostnames returns pretty much instantly, but pinging still has a long suspicious delay . . .
[21:47] <keithzg_> I swear I was getting a long delay with dig when not specifying a nameserver earlier, but that appears to have vanished as a symptom now.
[21:48] <guntbert> keithzg_: never heard of ping doing reverse lookups
[21:50] <keithzg_> guntbert: I'm probably getting the nomenclature wrong, but it's worth mentioning that there's no delay if I add -n as an argument
[21:50] <keithzg_> " -n     Numeric output only.  No attempt will be made to lookup symbolic names for host addresses."
[21:55] <guntbert> keithzg_: you may be right, but as I said, I've never heard about that
[21:55] <keithzg_> guntbert: fair enough.
[21:57] <keithzg_> Hmmm and weirdly one of the suspect servers is responding to pings as hostname.local rather than just hostname or hostname.our.fqdn.
[22:29] <Logos01> keithzg_: What distro?
[22:29] <Logos01> Oh. ubuntu.
[22:29]  * Logos01 needs to make a note of how many channels he's in
[22:32] <Logos01> keithzg_: regarding dig and not specifying a nameserver -- IIRC there was a bugged version of dnsmasq that had that problem.
[22:53] <JediMaster> lordievader: I couldn't find something that did exactly what I wanted to monitor the website files for changes, and in the end I wrote a program to do it for me =)
[22:53] <keithzg_> Logos01: Huh, that'd be quite the unfortunate coincidence, heh.
[22:53] <Logos01> JediMaster: inotifywait
[22:54] <JediMaster> Logos01, not realistic with half a million files
[22:54] <JediMaster> possible but silly
[22:54] <Logos01> Depends, really -- it works on system calls in the filesystem...
[22:54] <JediMaster> it'd take 0.5GB of ram to monitor that many files
[22:55] <JediMaster> and it's non-paged kernel memory too
[22:55]  * Logos01 shrugs -- that's not a huge investment tbqh
[22:55] <JediMaster> true, but this only took an hour or so to write
[22:55] <acmehandle> Is there a way to reverse an apt-get install with its dependencies?  For example if I installed tcl8.5 and its dependencies is there a way to reverse that?  So I dont keep dependencies I dont want until I really need them
[22:55] <Logos01> JediMaster: There's already shell utilities that operate on that syscall so you'd be able to do something similar in about that much time.
[22:55] <JediMaster> lordievader: it catalogues all the files of certain types (e.g. .php, .js etc.) md5sum's them all, checks against the previous run, looks for files that are new, files that have been deleted and changes. It then checks if they're in SVN, if so it shows an SVN diff if they're not just updated
[22:55] <Logos01> I mean, what did you do in lieu?
[22:57] <JediMaster> Logos01, I did try some of them out, they all failed after the OS'es default 8k monitor limit, easily changable of course, but it couldn't do all of the above with svn integration etc. so figured I might as well do it myself
[22:57] <JediMaster> they also had limits such as not being able to monitor certain file types recursively through a tree
[22:58] <JediMaster> I did it in 67 lines of code (with comments) in the end, so not too worried =)
[22:58] <JediMaster> probably more like 30 lines in total
[22:58] <Logos01> "not being able to monitor certain file types recursively through a tree" <-- curiously I didn't experience the same when I had to work with the realtime transfer of newly created pdf and xml files (but not any other filetypes)
[22:59] <Logos01> Granted I had more lines than that, but I also did more things than just transfer the files...
[22:59] <JediMaster> Logos01, what did you use in the end? 30 lines includes all the monitoring too
[23:00] <JediMaster> of course, it does mean running md5sum over all the files each time it's run, but it's suprisingly fast, only takes 2 seconds to run
[23:01] <Logos01> inotifywait.
[23:01] <JediMaster> I also didn't fancy keeping a script running all the time for inotify to tell it something had changed
[23:02] <qman> Large scale inotifywait works really well IME
[23:03] <qman> I recently wrote a script as a bandaid to netatalk permission failure on a massive web directory
[23:04] <qman> I had to increase the limits in sysctl, but it works great
[23:04] <JediMaster> still rather think that the kernel memory could be better used than 0.5+GB being used for it though
[23:04] <JediMaster> also, I didnt' really want to have an email sent out each time it was triggered, as we often upload 100+ files in one go via svn/git update
[23:05] <JediMaster> at least with running the script every 5 minutes it catches it in one bulk email, and if it's comitted changes it's a lot less verbose
[23:06] <JediMaster> I'm sure if you need instant notifications it's much better
[23:07] <qman> In my case it runs a chmod and chown every time a file is created/modified
[23:07] <qman> Because despite netatalk having parameters for that sort of thing, they don't actually work
[23:08] <JediMaster> yeah so effectively one or only a few lines of scripting needed
[23:08] <JediMaster> this needed a fair bit of logic to decided if to bug people about changes
[23:11] <Logos01> JediMaster: In my case I needed to set up a realtime file transfer process.
[23:11] <Logos01> My whole script is only about 55 lines and I introduced some extra logic on top of that.
[23:11] <Logos01> My fileset is only in the mid 4 digits low 5 digits though.
[23:14] <xcyclist> Cannot find doc on the two zeros at the end of:  192.168.X.X:/path/to/nfssharefolder             /media/nfs-foo     nfs     _netdev,defaults,user,auto,noatime,intr   0 0
[23:15] <xcyclist> in this example fstab line.  I just need to find the right doc.  man fstab doesn't do it.  Perhaps an online doc I have not yet gotten to.
[23:15] <sarnold> xcyclist: man 5 fstab, look for fs_freq and fs_passno
[23:16] <xcyclist> Never mind.  I got it on the wiki.  Sorry guys.
[23:21] <Logos01> xcyclist: Oh. Those deal with the frequency at which a filesystem is fscked, and when it is fscked, upon boot.
[23:22] <Logos01> As sarnold said, fs_freq and fs_passno in fstab are your keywords. :)
[23:40] <xcyclist> Thank you.
[23:41] <xcyclist> I don't see addr= as an option in any of the documention, but my spec uses it.  Is this just a mistake, or am I looking at the wrong docs?
[23:41] <xcyclist> I do see clientaddr=, but not addr=
[23:44] <sarnold> xcyclist: you may only see errors if you try "mount /media/nfs-foo" and let mount look it up from fstab..
[23:44] <sarnold> xcyclist: .. and even then you may only see errors in the logs rather than at the terminal
[23:46] <xcyclist> I found something about it in this doc:  http://wiki.linux-nfs.org/wiki/index.php/NewMountDesignSpec, but pretty weak.
[23:47] <xcyclist> sorry sarnold.  I am sure that will help me, but I am still rather weak myself in context on this.  Any elaboration you can make would be appreciated.
[23:51] <sarnold> xcyclist: interesting; addr= is clearly documented there, but they never say what it does or why. that's annoying.