=== dumfries_ is now known as dumfries === bitnumus is now known as Guest6432 [00:39] I found Ubu's "whoopsie" daemon installed, and spewing DNS queries, on a bunch of our Ubu servers. *I* certainly never gave it, or apport, permission ... but there you have it. I've rm'd the package from all the boxes -- QUESTION: [00:39] How do you permanenetly lock/prevent the install of package? === lifeless_ is now known as lifeless [00:50] DarylXian: nothing additionally should be installed without sudo permission [00:51] holstein: It was apparently installed, and enabled, without my permission in the 1st place. I'd rather not trust the "should", and lock out 'whoopsie' from ever being (re)installed. [00:52] afaict, you can lock UPGRADES. haven't figured out how to lock out INSTALLS. [00:52] DarylXian: if the packages come with the OS, then they will have already been installed [00:52] it is installed by default on every system. [00:52] DarylXian: i dont select "auto upagrades" at install [00:52] DarylXian: you should be able to remove anything you please,a nd nothing will be added or changed without your doing so [00:53] perhaps you want to set up a debian installer preseed file that knows to uninstall it so your own installs won't hve it [00:53] sarnold: Right. ANd apparently enabled. Which is what's got me riled. But, ignore me/that. I simply don't trust this to not happen again. [00:53] Is there no locking mechanism -- like opensuse's zypper locks -- to prevent installs? [00:53] or just add 'apt-get purge whoopsie' or whatever to an after-install script you run on systems? [00:54] DarylXian: you can always make a fake whoopsie package to install.. kinda blunt as an instrument, but there you go [00:54] DarylXian: things dont get installed without your permission.. nothing was installed.. it comes withthe pacakges you are mentioning [00:54] holstein: It's been ENABLED without my express permisson. That's at best -- shoddy. [00:55] sarnold: Not elegant, but a workaround. THanks. I can stuff this into Puppet somehow ... [00:56] DarylXian: enabled is not "installed after installation of the OS in the background without permission" [00:56] DarylXian: you should have no trouble removing anything you please, and it wont "automatically" do anything [00:57] It 'automatically' was enabled. ~10 million DNS queries later ... I'm simply interested in preventing it from doing so again. [01:07] its a good question, you could look into APT pinning? [01:11] Pin-Priority: -1 # should prevent it from being ever installed [01:11] ooh [01:11] that's easier than what I mentioned to him, the equivs package [01:14] Package: perl [01:14] Pin: version 5.10* [01:14] sorry. [01:14] clipboard fail [01:14] echo -e "Package: whoopsie\nPin-Priority: -1" | sudo tee /etc/apt/preferences.d/whoopsie [01:14] on all your servers should do it. [01:15] or in a shared cloud-init [04:01] does someone know of a good site that shows best practices for directory permissions? I have /var/www/client1/public_html/ for each of my clients... lots of permission issues with joomla [04:02] not specific enough [04:02] MavKen: welcome to the wonderful world of "trying to make mod_php secure" [04:02] the question will be, how are you running php? what user? [04:02] there are really just 3 options [04:02] solution: don't. use php fpm or cgi or such [04:03] i added all users to www-data [04:03] run it as a single user, then any user can screw with other users [04:03] then made www-data owner of each [04:03] run it as the user, then user can screw with themselfs, and anything outside their public_html folder [04:03] or make a new user for each user that the php runs as for that user [04:04] ya, your going have lots of fun there, all you have to do is see what joomla says is required though [04:05] I set permissions at 755 but no good... only 777 worked [04:05] MavKen: whatever you do, do NOT 777 [04:05] grr [04:05] yeah [04:05] that is not what I said [04:05] I said set them up as joomla tells you to [04:05] NEVER 777. [04:05] ever. [04:05] it will tell you what folder needs what permissions [04:05] I don't plan on keeping 777 [04:06] MavKen: don't ever even do it to start [04:06] 777 is for /tmp and that's it. [04:06] and ever that is 1777 [04:07] alright [04:07] it says to use 755 for all directories [04:07] but wasnt sure if i should make client1 the owner or www-data [04:08] MavKen: you need to look into php fpm [04:08] mod_php should not be used for any multi-tenancy [04:09] mod_php should never be used ever :) [04:09] I disagree [04:09] it's a memory and performance hog [04:09] oh? [04:09] weird [04:09] why weird? [04:09] maybe fpm is better now, then [04:09] I always assumed mod_php was the fastest because the interpreter was always there [04:09] mod_php never beat php fcgi [04:09] ahh, I don't think I ever ran it fcgi [04:09] and requires apache prefork, and then you have lots of wasted memory [04:10] php_fpm runs as fastcgi, but even before fpm :) [04:10] I was doing something like 97rps with mod_php and 114 with fcgi [04:10] crazy [04:10] but really it was about not wasting all that memory on php to fork apache for html files [04:10] we just ran it as cgi. worked ok [04:10] and pictures [04:11] i keep wondering if i should try something other than php [04:11] yes [04:11] you should [04:11] php should not be used [04:11] you mean, try a language that has a better reputation of developers not making stupid security mistakes? :) [04:12] oh set php's security issues aside and you still have an abomination. [04:12] between joomla, wordpress, ...., endless issues [04:12] I admit it's getting better but it still carries way too much baggage [04:12] oh, I'm only talking core [04:12] I'm not even talking php itself [04:12] people write horrible software in all languages [04:13] ya, just the liberty php gives, seems to attract them [04:13] I mean sendmail was written in C ... [04:13] my only experience is with php... what else should I look at? [04:13] I will say, that is why I do code in php, cause it's quick and dirty [04:13] MavKen: ruby and python are kinda the new hotness nowadays [04:13] yeah, its easy to throw something together using php [04:13] though, I have spent a long time *fixing* php issues, and also doing thing right [04:14] they have excellent web frameworks (rails, django, to name a few) [04:14] then things like node.js are becoming popular [04:15] never would have imagined .js being used that way [04:15] node is pretty rad. [04:16] thinking about trying www.django-cms.org [04:18] I don't know anything about it other than it has automatically created admin interfaces and is written in python and was extracted out of the CMS of a newspaper in kansas [04:19] I've known ABOUT it for like .. 8 years now? but never used it. === Rasmus`- is now known as Rasmus` === jml_ is now known as jml === tjaalton_ is now known as tjaalton === Tm_K is now known as Tm_T === rvba` is now known as rvba === ivoks_ is now known as ivoks === gnuoy` is now known as gnuoy === diplo_ is now known as diplo === psivaa_ is now known as psivaa [11:45] apw, the 3.13 changes for openvswitch look a bit terrifying and are definately beyond my capability to fixup in openvswitch [11:46] I'm going to punt this upstream and disable the DKMS package for the time being [11:50] jamespage, do we lose anything significant by doing so, you mentioned a feature [11:50] apw, just support for LISP based tunnelling which is pretty experimental [11:55] jamespage, so not the end if the world i hope then [11:55] apw, not at all - the 'supported' tunnelling mechanisms for openstack are GRE and VXLAN [11:56] both are support directly by the in-tree kernel module now [11:57] ok good === railsraider_ is now known as railsraider [12:14] rbasak_, do you have the report generator that looks at server related package merges? === rbasak_ is now known as rbasak [12:14] jamespage: http://bazaar.launchpad.net/~ubuntu-reports-dev/ubuntu-reports/trunk/view/head:/server/merges.py [12:15] rbasak, ah - http://reqorts.qa.ubuntu.com/reports/ubuntu-server/merges.html [12:15] got it [12:15] Oh right. You wanted the report, not the generator. Sorry :) [12:15] One day, I want to get round to being able to prioritise the report. [12:22] rbasak, np - that report gives me enough [12:22] looks like we have some merging todo still... [12:22] I'm still working on apache2. php5 and mysql are on my list. === gary_poster|away is now known as gary_poster === phunyguy_ is now known as phunyguy [14:36] hallyn or stgraber: is it expected that "lxc-ls" no longer works as a normal user, now that /var/lib/lxc permissions are locked down? [14:39] rbasak: yes, sadly. [14:39] rbasak: (you can list your own unprivileged containers, if you're on trusty) [14:43] rbasak: as I just said in the other channel, note that lxc-ls in the next LXC milestone won't try to list system containers anyway [14:43] rbasak: current upstream lxc-ls when run as non-root will list unprivileged containers present in ~/.local/share/lxc/ [14:44] well, unless you force it to look somewhere else with "lxc-ls -P /var/lib/lxc" of course [14:48] stgraber: that's interesting, thanks. Looking forward to your "Unprivileged containers" post :) [14:49] rbasak: just waiting for slangasek to upload my PAM fix so that sshd works in them, then I'll publish it :) [14:49] (well, maybe by that time we'll also have a 3.13 kernel in the archive which would also make unpriv containers slightly nicer) [14:50] Locking down /var/lib/lxc broke adt-virt-lxc, btw, which assumes that it can see inside the guest rootfs as an unprivileged user in order to detect when a container has actually booted (by looking for cloud-init's boot-finished flag). [14:50] I can probably fix that with sudo, but annoyingly that means that I can't just call os.path.exists, etc. [14:51] An extension to lxc-wait to detect container boot-finished status would be awesome ;-) [14:56] hi. i've added moin.conf to /etc/init/ but "start moin" give "unknown job" ? [14:57] have to go [14:57] damn [15:03] rbasak, are your mongodb cross arch changes good for cherry picking? I see they are pending merge upstream now [15:06] anyone used the ML350 G5 or ML150 G6 , I am considering between both for a small office. main cafeterias: SAS RAD 10, ilo, low noise, low size === Guest6432 is now known as bitnumus [15:19] jamespage: I was waiting for the patches to actually be merged upstream in case they were changed at all, but it looks like that's unlikely now, so if you need it then go ahead. [15:20] jamespage: (as we're synced with Debian I was going to send Debian cherry-picks and let Ubuntu sync it) [15:21] rbasak, find - I'm hacking around on the juju-mongodb package at the moment - I've disabled scripting altogether so it might just build on arm64 with your patches :-) [15:21] jamespage: also, I don't think you need the third commit. Might as well leave amd64/i386 alone until upstream take it. [15:34] zul, is savanna actually in 14.04 yet? [15:42] jamespage: not yet i have to upload it === railsraider_ is now known as railsraider === micahg_ is now known as micahg [17:29] how do you go about protecting the data in case of theft ? [17:31] vipconsult: the machine physically getting taken? [17:31] yes [17:32] https://help.ubuntu.com/community/FullDiskEncryptionHowto ..but, if someone can touch the machine, i dont trust anything [17:32] Doesn't that require you to enter a password at boot? [17:33] i think it would make running a headless system more challenging.. if not impossible from a cold reboot without intervention === ryan`c is now known as ryan-c === njbair_ is now known as njbair === balloons_ is now known as balloons [17:38] jamespage: ping [17:41] semiosis, hello [17:42] hi, marcoceppi suggested i get in touch with you about the glusterfs package in trusty, i have some updates [17:43] basically, due to upstart, glusterfs is merged rather than synced. there's a new release of glusterfs, 3.4.2, and also improved upstart jobs, which I'd like to get merged into trusty [17:43] thoughts? advice? [17:44] i take care of upstream ubuntu & debian packaging for the glusterfs project btw === stgraber_ is now known as stgraber === daker_ is now known as daker === Tribaal_ is now known as Tribaal === balloons is now known as Guest99854 === hachre_ is now known as hachre === seiflotfy__ is now known as seiflotfy_ === gazoombo_ is now known as gazoombo === petey is now known as petey_ === balloons_ is now known as balloons [19:09] Hmphhh, I seem to be running into https://bugs.launchpad.net/ubuntu/+source/nfs-utils/+bug/976632 but am a bit scared to get too invasive chaning settings that otherwise work....sigh. [19:13] Wellllll, I gave in and set Domain = localhost on both the client and server /etc/idmapd.conf, which seems to have fixed it. Time for the fun part of eventually discovering unintended consequences and ramifications ;) [19:22] hey, trying to make sense of something, in 12.04.3, the default kernel is 3.8, but under the cloud images, it seems like it defaults to 3.2 instead... is there a reason for that or a place to get the "official" ami with 3.8? [19:26] addisonj: I don't know the exact answer you'r looking for but I hope this is helpful :) http://cloud-images.ubuntu.com/releases/12.04.2/release/ [19:26] hunh, wonder why the url says 12.04.2 but the content all says 12.04.3. [19:27] sarnold: yeah, I was looking at that, if you scroll down the list of content and look at kernel info, it shows 3.2... [19:29] addisonj: how odd === BrianH is now known as Guest65168 [19:56] jamespage: https://github.com/mongodb/mongo/commit/df3d84e and https://github.com/mongodb/mongo/commit/c9edb7f just landed upstream. [19:56] I'll file a Debian bug tomorrow. [19:58] A server was hacked and it runs a fake program named /usr/sbin/nginx, but that file doesn't really exists [19:58] Inspecting /proc/PID_NUMBER I found that the excutable is a perl script [19:58] HI using ubuntu 12.04 LTS update 3, Missing files/directories: I have observed on afew of my systems, sometimes after reboot some of files or directories are missing,any idea why? thx [19:59] And its current executing directory is /tmp [19:59] How can I find the original program being run as "/usr/sbin/nginx" if it doesnt exists? [20:02] jvargas: check /proc/pid/fd/ [20:03] jvargas: you may be able to vi some of those; I can't recall if it still works if the file's been deleted, but I think it may.. [20:03] sarnold: will check. brb [20:04] rostam: check your various /lost+found directories ? if the system lost power before umounting, perhaps the files were disconnected.. [20:09] `which nginx` [20:09] or `lsof | grep nginx` [20:09] to see what it is talking to [20:10] sarnold: just did it, but no files open, just sockets and pipes. I found that its /proc/PID/fd/51 points to /proc/OTHER_PID/auxv [20:10] But that OTHER_PID doesn't exist anymore [20:11] jvargas: crazy [20:14] jvargas: possible remote code execution, program code may have arrived over the network & only reside in memory. [20:16] ouch [20:20] yeah perl does make it easy to deliver executable code in a varieyt of ways.. stdin, -e, load a file and execute, etc.. [20:20] semiosis: maybe. since it is a perl program, is it possible to inspect the source code at execution time? [20:21] check /tmp and /var/tmp just incase [20:21] also any possible memory filesystems like gnome has [20:22] I just did, nothing there. [20:22] is this a web server? runing php perhaps? i've seen plenty of RFI exploits do stuff like this [20:23] (remote file include) [20:24] semiosis: yes, a web server running php. Actually, the process runs as one of the web users [20:24] jvargas: you could gdb attach and have it generate a core dump (without actually killing the process) [20:24] hehe, called it [20:25] the command is "gcore", fwiw [20:26] semiosis: yes. What I am looking for is for the source of the program to eliminate it completely. [20:26] kees: oh! I didn't know that was an option. nice. /proc/self/mem isn't being friendly to e.g. strings -a :) hehe [20:26] sarnold: right, /proc/$pid/mem needs the reader to be ptrace attached [20:26] jvargas: most likely you have a php file on the server which is vulnerable to an rfi attack, and someone (many actually) are scanning teh whole internet looking for such vulnerable systems [20:26] these vulerabilities exist in lots of popular php apps [20:27] kees: oh! cool. thanks. :) [20:27] there's very likely no file on your server being executed [20:27] in the generated core file, you may have a dump of the original source, if you're lucky [20:27] semiosis: I know. I've removed some of these before, but this one is the most curious one ever. [20:27] the /proc/pid/auxv is an odd touch, no doubt. [20:28] fd/51 ? wow. I wonder what it was doing [20:28] (like, that's a fair number of open files) [20:29] kees: yes, I read that auxv file is "contains the contents of the ELF interpreter information passed to the process at exec time" [20:30] So, why does a process would have an open fd pointing to the AUXV file of another process? [20:30] jvargas: there's a lot of weird stuff in the auxv file. the most sensitive is the ELF start location (which can be an ASLR bypass) and the random number seeds for glibc protections (ssp, ptr_mangle) [20:30] Copying the full binary image of its parent process? [20:32] jvargas: you might want to look into running mod_security - http://www.modsecurity.org/ [20:32] jvargas: my arbitrary guess was that it was trying to attack that process by examining its ASLR offset. but that's just a total guess. [20:32] jvargas: but getting a gcore via gdb will tell you the most at this point. [20:32] Hi all [20:32] kees: will run gdb now. [20:32] can apt-mirror not use mirror:// [20:32] ? [20:32] gdb, attach $pid, gcore, quit [20:32] well, quit if you want it to keep running. otherwise it'll stay paused while you have gdb open [20:34] hi. i am having problem with mailman on my ubuntu saucy server. installed and configured mailman with any problem but having trouble with permission and access of the new created mailing list. getting allways this error message here [20:34] AH00670: Options FollowSymLinks and SymLinksIfOwnerMatch are both off, so the RewriteRule directive is also forbidden due to its similar ability to circumvent directory restrictions : /usr/lib/cgi-bin/mailman/listinfo [20:34] have checked the apache conf and enabled Options +FollowSymlinks but it does not help. anybody who can suggest what need to be done? [20:35] kees: got the core dump, now what can I do with that binary file? [20:36] try "strings" on it, see if anything exciting appears :) [20:36] on a first shot, try strings -a on it [20:37] kees: I used vim and found strings like "/tmp/bad" and "/usr/sbin/nginx" too near each other. [20:37] no perl code snippets, eh? [20:38] yes, some code snippets. [20:39] also found several URLs, joomla vulnerabilities info and search engine queries. [20:40] nice! sounds like it might scanning for more vulnerabilities. [20:40] it's most likely a botnet agent, able to propagate, spam, and ddos [20:40] that's my guess [20:40] jvargas: if you want, gzip and email the gcore to me, I can look too. kees@ubuntu.com [20:41] okay could solve the problem. was a apache config problem with the order of the directory rules. [20:42] some of these botnet agenst propagate by searching google for paths that indicate possibly vulnerable hosts [20:43] C&C is often handled over http, where the server address is generated based on a time index & some seed. check what dns queries that host is generating [20:43] jvargas: ^ [20:46] semiosis, kees: gimme a minute [20:50] kees, semiosis: I used strings and checked some target URLs, for example this one: http://www.istanbuldenizotobusu.com/sodd.txt [20:50] i'd recommend against opening that in a browser! [20:50] curl is your friend [20:50] it's a perl script [20:51] indeed [20:51] seems that code is retrieved remotely [20:51] wow, using irc for C&C, thats old school [20:51] Portuguese. [20:52] $proc var is the process names it takes [20:53] "BaDGuyS" is the admin control nick, fwiw (the base64 decode) [20:53] a ddos agent [20:54] curiously it also performs checks against speedtest.net for bandwidth performance [20:55] neat [20:57] so, it scans and also performs ddos === BrianH is now known as Guest12982 [20:58] yeah. you can probably find its outbound network connection in "netstat -anp" as root and find its pid [20:59] It seems that it used a JCE vulenrability on Joomla websites: http://www.istanbuldenizotobusu.com/update.php [20:59] cool, you found its entry point? [21:01] i hope so. ill keep reading. [21:03] jvargas: if you're feeling adventurous, you could set up a honeypot & point that exploit tool at it, then capture what it does... that would allow you to see the whole payload [21:04] oh, hah. sodd == DDoS (backwards) [21:04] honeypot could be as simple as a php script that writes the whole request, with headers & post data, to a file [21:08] yeah, this thing in memory seems to be the attack-finder. [21:08] openflashchart ? another vuln? [21:16] wow a 12.04 mirror is huge === BrianH is now known as Guest92448 [21:29] hi [21:31] I just installed the dovecot-postfix package [21:31] asd configured it as this said: https://help.ubuntu.com/10.04/serverguide/postfix.html [21:32] semiosis, kees: problem solved now :-) thanks for ur help [21:32] but when I connect with netcat and test, it doesn't work as I suppose it should [21:33] I don't see the lines "250-AUTH LOGIN PLAIN" and "250-AUTH=LOGIN PLAIN" among others [21:33] jvargas: yw. what did you do to prevent further attacks? [21:35] semiosis, hey - sorry - i missed your response [21:35] re glusterfs [21:35] semiosis: updated jce extension on that website and for further protection disabled direct php execution that doesnt passes throgh joomla cms [21:35] jvargas: cool [21:35] jamespage: yeah no worries :) [21:35] hi does somebody has mailman running on his ubuntu server? i have some strange permission problems. get allways this error message when i try to confirm my subscription => AH00037: Symbolic link not allowed or link target not accessible: /var/lib/mailman/archives/public/ [21:35] jamespage: what do you think about what I was saying? [21:36] just looking [21:36] thx [21:40] jamespage: fyi, my upstream packages (blessed as the official upstream packages for ubuntu) are here: https://launchpad.net/~semiosis/+archive/ubuntu-glusterfs-3.4/+packages [21:40] semiosis, what does the diff look like between your packags and the distro packages? [21:41] the glusterfs package in trusty has my old upstart job from 2 years ago [21:42] semiosis, that sucks a bit [21:42] in summary, moved the mount block job, mounting-glusterfs.conf, from the -server to the -client package, and changed it to wait for static-network-up instead of started glusterfs-server [21:42] root issue this addresses is mounting glusterfs vols at boot time [21:42] ok [21:42] used to only be a problem when mounting vol from localhost [21:44] damn why are mirrors that huge [21:46] but more recently (last year or so) many people have had issues with the mount being tried before network is up at all [21:46] idk if that was caused by a change in ubuntu since precise, or just more people using/testing revealed an issue [21:46] lots of people use my ppa packages and blocking until static-network-up seems to work for everyone [21:46] only remaining issue i'm aware of with this config is that if you have multiple glusterfs mounts in fstab, the blocker only holds the first one :( [21:46] idk how to resolve that [21:46] or even how to approach it [21:47] semiosis, sounds like there are some good improvements to incorporate [21:47] YamakasY_: why do you need a mirror? maybe just a caching proxy will work for you? [21:48] semiosis: to be sure :) [21:48] semiosis, any chance I can persuade you to raise a merge proposal against lp:ubuntu/glusterfs with relevant changes to incorporate into the Ubuntu package? [21:48] that can include the new point release as well [21:49] jamespage: sure, i did that for precise, about time I do another one :) [21:49] semiosis, thanks - ping me when you have it ready - i'll review and sponsor :-) [21:49] any thoughts on that issue of blocking multiple mounts with a single blocker job? [21:50] semiosis: too much traffic over such proxy [21:50] to too large cache [21:50] not as large as a mirror :) [21:51] well, not larger than [21:51] semiosis, tricky [21:51] yep, i gave up on a general solution. for my own servers I create an extra blocker for each mount with puppet === markthomas_ is now known as markthomas [21:53] semiosis, it might be possible to use the instance stanza in some way [21:53] semiosis: yeah might be [21:53] semiosis: but I wonder how good that works [21:54] jamespage: i'll take another swing at that [21:54] semiosis, I'll think about it as well - and ping jodh for an opinion [21:54] jodh? === Ursinha-afk is now known as Ursinha === Ursinha is now known as Guest50378 === gary_poster is now known as gary_poster|away === Ursinha-afk is now known as Ursinha === BrianH is now known as Guest36871 === manjo` is now known as manjo === Ursinha is now known as Ursinha-afk === SJrX is now known as SJr [23:27] Hmmm, why does virt-manager keep asking me for my passwords for my SSH connections, rather than just using the SSH keys as configured in .ssh/config? === BrianH is now known as Guest71540 === Ursinha-afk is now known as Ursinha