| sarnold | I skipped all the bits about serial cables, but the rest looked decent :) | 00:05 |
|---|---|---|
| Mead | sarnold: thanks, yeah. I know it serial cabling is old and busted, but I'm stoked about having console access. It means I can really fuzz with it in my lab without worrying about lossing network connectivty breaking my SSH access. | 00:28 |
| sarnold | Mead: yeah, and it's often the only way to get a handle on some specific kernel problems | 00:29 |
| sarnold | Mead: quite often server gear will have BMCs on board that can do serial over network and save you the hassle of the serial cable itself :) | 00:29 |
| * Mead googles BMC serial over network | 00:34 | |
| Mead | looks like a potential security problem | 00:37 |
| sarnold | Mead: yeah, the usual implementation of BMC devices is poor enough that they are almost always given their own networks | 01:02 |
| mwhudson | also because ipmi sol is based on udb so is really fun to use on a congested network :) | 01:18 |
| mwhudson | *udp | 01:18 |
| Ham62 | what kinds of things would I need to worry about breaking if I upgrade a system I've been running for 5 years on 14.04 to 16? | 04:56 |
| lotuspsychje | Ham62: i would say a clean apt without issues, and no ppa's enabled | 04:57 |
| lotuspsychje | Ham62: for services you running, best to ask specificly so volunteers can think along | 04:57 |
| lotuspsychje | a backup is also a good idea | 04:57 |
| Ham62 | well the most important things I have running right now are nginx, apache for some CGI stuff, and a gopher server | 04:58 |
| Ham62 | the CGI stuff was mostly done with FreeBASIC and x86 assembly | 04:59 |
| Ham62 | nasm | 04:59 |
| Ham62 | I'm mostly worried a bunch of packages won't support my CPU properly | 04:59 |
| Ham62 | it's running on an Athlon XP | 04:59 |
| Ham62 | and I have a couple services I have are started using the rc.local file | 05:01 |
| lotuspsychje | what about init Ham62 | 05:01 |
| Ham62 | are those going to break? | 05:01 |
| lotuspsychje | from 15.04 and higher its systemd now yeah | 05:01 |
| Ham62 | oh darn | 05:01 |
| Ham62 | yeah the gopher server is launched with socat at boot and I have a custom remote compiler server which is started through there on one of the user accounts | 05:02 |
| lotuspsychje | !systemd | 05:02 |
| ubottu | systemd is the default init system for Ubuntu 15.04 onwards. For information on transitioning from upstart to systemd, see https://wiki.ubuntu.com/SystemdForUpstartUsers For a guide to basic service management with systemd, see https://www.digitalocean.com/community/tutorials/how-to-use-systemctl-to-manage-systemd-services-and-units | 05:02 |
| Ham62 | neither of those are real services though | 05:03 |
| Ham62 | they're just processes I have running as a user in the background constantly | 05:03 |
| lotuspsychje | Ham62: you might also wanna read up https://wiki.ubuntu.com/XenialXerus/ReleaseNotes | 05:05 |
| lotuspsychje | Ham62: anything allright there with the upgrade plan? | 09:58 |
| Skyrider | Greetings everyone | 10:15 |
| Skyrider | In a pickle and need some help ^_^. I hope this is the right channel to ask, seeing it's nginx/www-data user related, but also linux related. What is the best way (with vsftp) to give access to a specific /var/www directory, with it running under www-data user/group? As I assume nginx requires both user/group to be www-data. | 10:15 |
| blackflow | Skyrider: so as I was saying, I'd make the dirs owned by the v/s/ftp user, and then put nginx in that user's group | 10:15 |
| Skyrider | That I saw, wanted to reply to that :D | 10:16 |
| === DerRaiden is now known as DerRaiden`afk | ||
| blackflow | by default dirs and fils are group readable so nginx's www-data user, being in that separate user's group, will have read access. | 10:16 |
| Skyrider | Wouldn't that mess up with file/directory permissions though seeing the owner/group gets changed? | 10:17 |
| Skyrider | Ah | 10:17 |
| Skyrider | I assume the owner/group differs then though? | 10:17 |
| blackflow | as long as you keep the dirs and files readable to their group, and www-data is in that group, all is fine | 10:17 |
| Skyrider | owner by whomever creates the files, while the group remains www-data. | 10:17 |
| blackflow | no, you don't change file/dir ownership to www-data | 10:18 |
| blackflow | example: chown myuser:myuser /var/www/some-website-dir ; chmod -R g+r /var/www/some-website-dir ; usermod -a -G myuser www-data ; | 10:19 |
| Skyrider | I know the last part adds the user to the group, what does g+r do. | 10:19 |
| blackflow | the chmod is just for example here, g+r is default | 10:19 |
| blackflow | makes the files and dirs readable to the group they belong to (in this case, myuser's, if that user is used to place the files via v/s/ftp) | 10:19 |
| Skyrider | Should I use mount though? Currently their FTP is set to the home directory. | 10:20 |
| blackflow | so as you upload files they will be owned by myuser:myuser assuming that's the user you logged into v/s/ftp with | 10:20 |
| blackflow | no need to mounts, you can symlink the website dir under myuser's home somewhere | 10:21 |
| blackflow | no need *for | 10:21 |
| Skyrider | Better to symlink, or directly set their directory to the specific /var/www/xxx directory. | 10:21 |
| blackflow | I'm assuming this is for the v/s/ftp acccess? those daemons running on the same machine, and this is not some nfs export | 10:21 |
| Skyrider | Just FTP access to give specific access to a specific site/sub-domain, ya. | 10:22 |
| Skyrider | **specific user | 10:22 |
| blackflow | Skyrider: I'm old school, and I'd symlink under myuser's ~/public_html/somesite.com | 10:22 |
| blackflow | though really... uh depending on what this is excactly, you can omit /var/www completely and use only the home dirs? | 10:23 |
| blackflow | I mean, if it's some packaged web app that installs under /var/www/ then yeah. if it's not, then just keep it all under ~/ | 10:23 |
| Skyrider | oki, that's set... now to symlink it :D | 10:24 |
| blackflow | or whatever you want. point is, if I understood your problem correctly, you want v/s/ftp uploadable files to be readable to nginx? | 10:24 |
| Skyrider | Merely creating an ftp so users can edit, add, etc web files through the ftp. | 10:25 |
| Skyrider | Seeing I have multiple domains/sub-domains, need to create multiple FTP users for that. | 10:25 |
| blackflow | then you don't want users to log in as www-data. you want this instead, nginx in those users' groups so it can read their files. separation of concerns and least privilege principle. and this covers only static sites. | 10:26 |
| Skyrider | They don't need a /home/ directory though | 10:26 |
| Skyrider | as the current ftp server requires that ( 500 OOPS: cannot change directory:/home/xxxx ) | 10:26 |
| blackflow | they DO need A "home" directory. why not /home/<username> | 10:26 |
| blackflow | that doesn't sound right | 10:26 |
| Skyrider | The directory does not exist, hence the error :p | 10:26 |
| blackflow | ah so you created users without -m ? | 10:27 |
| Skyrider | Indeed. I like being organized. | 10:27 |
| Skyrider | If I have users like test-web, test-forums, test-dev .. like weird in my eyes :p | 10:27 |
| blackflow | well having users have a home is wise. you contain them there. you can also configure sftp with chroot to their homes, instead of insecure (vs)ftp | 10:27 |
| Skyrider | For having a home directory that is. | 10:27 |
| Skyrider | I tried SFTP before, bit weird. They kept having access to the / directory, even though it isn't for writing, they had read. | 10:28 |
| blackflow | yeah because you need to explicitely chroot them for sftp access | 10:28 |
| Skyrider | That bad though to use FTP over SFTP? Even though not secure, can make it more secure by altering the port / whitelist. | 10:29 |
| blackflow | google up "chrooting sftp users". in essence, you have, say, /home/<username>/ as their home and thus chroot. that dir needs to be owned by root and not group writable. dirs _under_ it (like, say, ~/public_html) can be normal user owned dirs they write into. | 10:29 |
| blackflow | Skyrider: problem with FTP is that the data channel is never encrypted. while the control channel is (where log-in happens), data isn't. | 10:30 |
| Skyrider | What about FTP TSL. | 10:30 |
| Skyrider | I never really had to create multiple users over (S)ftp before, hence I'm asking all this :D | 10:31 |
| blackflow | anyway, in addition to root owned chroot, you need to set up sshd for them to force the chroot and internal-sftp command only so they can't ssh in | 10:31 |
| blackflow | FTPS (FTP over TLS/SSL) is exactly what I was talking about. control channel encrypted, data channel isn't. | 10:31 |
| blackflow | with pure FTP, not even control channel is, so you get plaintext passwords over the wire | 10:31 |
| blackflow | FTPS = FTP over TLS/SSl; SFTP = FTP over SSH. | 10:32 |
| blackflow | sftp is really superior. it's well supported by programs like filezilla if your users are windowsites. you can force keys instead of passwords, and everything is nicely encrypted. | 10:34 |
| Skyrider | Is it a hassle to properly set up sftp with proper rights / access only to specific /var/www/ directories with no ssh access over ftp? :P | 10:35 |
| blackflow | no. three lines of confg in sshd_config, and a chown+chmod when you create their home dirs. | 10:36 |
| blackflow | but then you make the /var/www/ dirs as their homes. personally I'd just go with /home/ . /var/www/ is primarily for packaged web applications. | 10:36 |
| Skyrider | and at the end symlinks, got it. | 10:37 |
| Skyrider | Guess I'll remove this ftp package and go with sftp | 10:37 |
| blackflow | no, I'd go with /home/ period. no symlinks | 10:37 |
| blackflow | /home/<username>/public_html/website-dir.com | 10:37 |
| Skyrider | I thought you said you were oldskool | 10:38 |
| blackflow | yes, and this is it with public_html | 10:38 |
| Skyrider | .. /var/www/xxx -> symlink /home/user | 10:38 |
| blackflow | the old school part was about public_html :) | 10:38 |
| Skyrider | ah ^^ | 10:38 |
| Skyrider | I do prefer having things in /var/www though for all web related stuff. | 10:38 |
| blackflow | well you can do in reverse. symlink from /var/www/ to their /home/user/public_html/site.com dirs :) look, that part is really whatever you feel most comfortable with. the important thing is that the dirs/files are _user_ owned and that nginx is in their groups for read access. | 10:39 |
| blackflow | but again, that's for static sites. with php it becomes a bit more tricky if you want it properly secured. | 10:40 |
| Skyrider | All php stuff :p | 10:41 |
| blackflow | the rabbit hole deepens then :) | 10:41 |
| blackflow | what I do in this case is run a php-fpm pool per user, as that user, with apparmor policy that prevents _writing_ except in specified dirs. that I can do becuse we control the application and know exctly what those dirs are. | 10:42 |
| blackflow | if that's not an option for you, then php-fpm pool per user, as that user, is the best you can do, but then php can change its own code and you're vulnerable. | 10:43 |
| Skyrider | Interesting. | 10:45 |
| blackflow | at any rate, you _will_ want to chmod o-rwx their homedirs -- forbid listing and read access to other users, otherise they can upload PHP code that will scan and sniff other users files | 10:45 |
| Skyrider | Good to know, thanks. I'll look into that. As for your last line, I do trust this user :) | 10:45 |
| Skyrider | I have to go for now, but I'll stick in this channel until I set up my irc bouncer again. I appreciate your time in helping me out, gotta pick up my wife. I'll be back soon :), again, thanks! | 10:46 |
| blackflow | so combined with sftp chroots, you have chown root:myuser /home/user-home-dir ; chmod 750 /home/user-home-dir/ and precreate a "public_html" or whatever the name, dir in which they will have write access in their home (As their home roots aren't writable to them) | 10:46 |
| blackflow | "I do trust this user" -- famous last words, aka. "pics taken 5 seconds before disaster" :) | 10:47 |
| oskie | when is "apt-get update" run automatically? | 11:28 |
| tomreyn | "depends" | 11:33 |
| tomreyn | on several factors, such as your ubuntu version | 11:34 |
| oskie | bionic | 11:40 |
| oskie | no..wait.. xenial | 11:41 |
| tomreyn | oskie: systemctl list-timers apt-daily* | 11:50 |
| oskie | tomreyn: awesome, that's what i've been looking for for a while | 11:52 |
| tomreyn | it's been there all the time, whispering your name! | 11:58 |
| ahasenack | good morning | 12:11 |
| AvidWolf43 | sarnold: which logs? I believe the error has to do with azure really long dns names maxing out. https://feedback.azure.com/forums/216843-virtual-machines/suggestions/10197480-the-azure-vm-internal-dns-domain-names-are-too-lon | 13:48 |
| === Eickmeyer is now known as Eickmeyer___ | ||
| Skyrider | blackflow: Back | 14:37 |
| Skyrider | Wow this is confusing. | 14:41 |
| Skyrider | No matter what I try to add in sshd config, I always get "Network error: Software caused connection abort" when trying to connect with the user. | 14:42 |
| rbasak | Skyrider: check auth.log | 14:43 |
| rbasak | Skyrider: another deeper approach is to run sshd manually in debugging mode on a high port. | 14:43 |
| Skyrider | " bad ownership or modes for chroot directory component "/var/www/"" that explains it. | 14:45 |
| blackflow | well, I did mention a few times the chroot dir needs to be owned by root :) | 14:49 |
| blackflow | and mustn't be group/other writable | 14:49 |
| Skyrider | That's the odd thing, it is owned by root. | 14:51 |
| Skyrider | And I believe the chmod is set to .. 755? Somewhere according to the internet. | 14:51 |
| Skyrider | Do you have a tutorial I can follow by any chance? | 14:51 |
| blackflow | you'd have to paste the full sshd_config in question | 14:51 |
| Ussat | that statement scares me --> Somewhere according to the internet | 14:51 |
| blackflow | indeed. | 14:51 |
| Skyrider | I'm running the cmds in a test directory Ussat | 14:52 |
| Ussat | still | 14:52 |
| Skyrider | And I know what the cmds do :p, plus.. I have a backup ready just in cae. | 14:52 |
| Skyrider | ***case | 14:52 |
| Ussat | fair nuff, still | 14:52 |
| blackflow | I'd like to stay and help, but there's a mtb trail with my name on it. bbl. | 14:52 |
| Skyrider | For example, I ran the tutorial: https://45squared.com/setting-sftp-ubuntu-16-04/ - Yet doesn't work properly. | 14:52 |
| Skyrider | No worries bf :) | 14:53 |
| Skyrider | There is something I noticed. | 15:20 |
| Skyrider | Whenever a directory's role is not www-data, I get "errors" / warnings like "The $cfg['TempDir'] (./tmp/) is not accessible." | 15:21 |
| Skyrider | I could of course alter the permissions of the directory to fix that, but how would user/group stay as www-data, regardless the user adding/altering files? | 15:22 |
| teward | anyone ever run into a case where even if you have installed everything and disabled cloud.cfg's preserve_hostname module by setting it to false, the system still resets its hostname every time? | 15:38 |
| teward | 18.04.2 server from the SUbiquity installer | 15:38 |
| Skyrider | I give up >_> | 15:41 |
| teward | Skyrider: you wouldn't be able to set the user of the file | 15:42 |
| teward | but you COULD set the group with stickybit on the directories | 15:42 |
| teward | anything created in a directory with the group stickybit would get group www-data | 15:42 |
| teward | but that's just a 'hack' | 15:42 |
| Skyrider | Yea, I'm familiar with the group cmd. Though how would I best fix this issue? The only thing I want is to create a sftp user under a specific /var/www/ directory, but if the owner www-user is changed to someone else, file permissions on the web application will start to appear unless file permissions is changed. | 15:43 |
| Skyrider | Apparently the www-data has the "proper" rights to auto solve that issue right away. | 15:43 |
| cryptodan | Skyrider: what are you trying to do with sftp in /var/www? | 15:44 |
| tomreyn | teward: during the past two or three days there were two people around on irc who reported that the system hostname they configured got reset. i don't think they knew about "cloud.cfg's preserve_hostname" (i don't, or didn't), though. one of them determined cloud-init to be the source of this issue. | 15:45 |
| tomreyn | also i think there's a related open bug report | 15:45 |
| teward | tomreyn: yeah i had preserve_hostname set to false though | 15:45 |
| teward | tomreyn: and it STILL reset it | 15:45 |
| teward | i just got angry at it and yoinked cloud-init out of the equation | 15:45 |
| teward | apt-get remove'd it and it worked | 15:45 |
| teward | tomreyn: got a link to the bug per chance? | 15:45 |
| tomreyn | that#s what the other user did, too | 15:45 |
| tomreyn | i was afraid you'd ask this | 15:46 |
| teward | tomreyn: i also think it's intermittent | 15:46 |
| teward | because two servers were both deployed with the same ISO | 15:46 |
| teward | one had this happen | 15:46 |
| teward | the other didn't | 15:46 |
| teward | only difference was a really short hostname for the one | 15:46 |
| teward | and that's the one where cloud-init was being derp | 15:46 |
| Skyrider | cryptodan: Setting up 3 different SFTP users to access 3 sub-domains. | 15:46 |
| tomreyn | teward: ugly. :-/ ok, i'll look for this bug report, but no promises | 15:49 |
| teward | tomreyn: never expect any promises :P | 15:50 |
| Skyrider | I saw somewhere last week on the internet that there's a package that checks a directory at all times and changes the user/group if it has changed. | 15:51 |
| Skyrider | Any idea what it might be called? | 15:52 |
| tomreyn | teward: bug 1780867 | 16:07 |
| ubottu | bug 1780867 in subiquity "hostname unchangeable / some daemon changes and resets /etc/hostname" [Critical,Fix committed] https://launchpad.net/bugs/1780867 | 16:07 |
| tomreyn | also bug 1770451 might be related (but that' sjust a random find while searching for the other) | 16:08 |
| ubottu | bug 1770451 in cloud-init (Ubuntu) "hostname not set: Failed to create bus connection: No such file or directory" [Undecided,Incomplete] https://launchpad.net/bugs/1770451 | 16:08 |
| === setuid_ is now known as setuid | ||
| tomreyn | hmm 1780867 isn't really new though (nor its dupe), nor was it updated during the past 3 days. but i think this is what i had in mind. | 16:16 |
| blackflow | Skyrider: nginx won't create such dirs, so that must be some PHP app. of course, the dir must be writable the to the user php-fpm is running as. for such random dirs you can't know in advance, you should run php-fpm as the user owning the dir, not as www-data. | 18:04 |
| teward | blackflow: though... in a default setup, php-fpm *is* running as www-data | 18:05 |
| blackflow | yes but the dirs must be owned by the sftp user in order to freely upload php apps | 18:05 |
| blackflow | that's the use case Skyrider has | 18:05 |
| blackflow | so, nginx as www-data, in supplement group of the sftp user. php-fpm running as that user in full. | 18:06 |
| blackflow | (if there's need for both PHP and sftp user to write files) | 18:06 |
| Skyrider | sftp chroot is weird. | 18:18 |
| Skyrider | The main directory has to be root, I get that. but in that main directory, no one can create directories or files, because that specific directory is owned by root. | 18:19 |
| Skyrider | All the other sub-directories inside the root directory can be altered. | 18:19 |
| blackflow | Skyrider: it's not once you understand why the (ch)root must be root owned. one way to escape chroot is to double-chroot with symlinks, so openssh enforces no-write, no-ownership of the (ch)root | 18:19 |
| Skyrider | So instead of /var/www/testwebsite/subdomain I need to have /var/www/testwebsite/domain/subdomain | 18:20 |
| Skyrider | Because root is messing up the main directory's permissions | 18:20 |
| blackflow | Skyrider: well see, that's why I recommended you to use /home/user/ as (ch)root, and then have ~/public_html/ for all their sites. | 18:20 |
| Skyrider | I'll consider it ^^ | 18:21 |
| blackflow | but you wanted your way, so... :) you'll just have to do the same. a "base" chroot dir/home for the sftp user, and a dir they can upload their sites to | 18:21 |
| Skyrider | the base is the var/ww :D | 18:21 |
| blackflow | just for one user? | 18:22 |
| Skyrider | Each domain/subdomain its own user. | 18:22 |
| Skyrider | Fake the sake that not a single user has access to all. | 18:22 |
| blackflow | well I really recommend you to use /home/ . /var/www was never meant to be used by random sftp user accounts. it's default place to put packaged web applications, root owned, www-data accessible, and not via sftp. | 18:23 |
| blackflow | standard for decades has been ~/public_html, from early apache years, carried over by shared hosting industry, all the commercial and non-commercial hosting panels, etc... /home/<username>/ as home dir, sftp chroot, and then public_html aka htdocs on some platforms, as "docroot" for apache | 18:24 |
| Skyrider | I'm actually using icron :D | 18:26 |
| blackflow | and if you _do_ insist on /var/www/ you will _still_ need to replicate the structure. /var/www/<user>/sites/www.somesite.com/ | 18:27 |
| blackflow | name the "sites" subdir as you wish. | 18:27 |
| Skyrider | replicate the structure? | 18:28 |
| blackflow | you need one extra user-owned dir under chroot | 18:28 |
| blackflow | IF you want to allow them to create subdirs for sites. I don't know how you intend to configure nginx to run with that, you'd still need root to add a server {} stanza for each domain | 18:29 |
| blackflow | do yourself a favor and don't reinvent the wheel, do what the industry has been doing for many years now. /home/<username>/public_html-or-sites-or-htdocs-orwhatever/{somesite.com,anothersite.com,foobarbaz.com} | 18:31 |
| adac | I have locked my docker-ce packages. But now I want to do a release upgrade to 18.04 | 18:46 |
| adac | but I do get: Please install all available updates for your release before upgrading | 18:47 |
| adac | But I do not want to upgrade this package | 18:47 |
| adac | it should stay on the same version | 18:47 |
| Skyrider | blackflow: : To make things simple.. dont really want to bother with it much anymore, I've made a single user to access a single domain with all its subdomains. | 18:49 |
| Skyrider | I got it to work, though all sub directories appear to be empty when I log into the user. | 18:49 |
| Skyrider | setfacl appears to be the cause | 18:51 |
| Skyrider | I give up -_- | 18:53 |
| Skyrider | Not sure why it's such a bothersome to simply add a sftp user access to a specific directory, having the ability to read/write and maintain the original user/owner.. | 18:53 |
| Skyrider | I'm not even sure why it's displaying 0 directories/files right now. | 18:55 |
| teward | adac: chances are you will have to upgrade the docker packages anyways because of new libraries/dependencies for build and runtime that Docker has to build against (outdated ones won't work) | 18:58 |
| adac | teward, I have running this docker version also already with bionic (I think need to check really) | 18:59 |
| teward | maybe the same *version* of the Docker codebase but it still has to build against *newer* libraries in Binoic vs. Xenial. | 19:00 |
| teward | so it's *different* at the binary level | 19:00 |
| adac | teward, thing is the newest docker version wll not work with my kubernetes version | 19:00 |
| teward | but not the code level | 19:00 |
| adac | kk I see | 19:00 |
| teward | for do-release-upgrade it'll still complain, yes. You might have to disable your Docker repository you're using to get it and do a full `apt-get update && apt-get dist-upgrade` afterwards then install the newer docker package version for the newer Ubuntu. However I can't guarantee this'll work | 19:02 |
| teward | Kubernetes is a little tricky. | 19:02 |
| teward | adac: the other thing is, what are you on now, 16.04? | 19:02 |
| teward | why upgrade to 18.04 if things're just working? | 19:02 |
| adac | teward, I'm re-setup my whole infrastrucutre and I have this one single host that is part of this new infrastrcuture already but still has 16.04 | 19:03 |
| adac | and I would really like to have the same versions everywhere | 19:03 |
| teward | i'm assuming you don't want to upgrade kubernetes then :P | 19:04 |
| teward | which you'd probably end up having to do | 19:04 |
| adac | teward, actually I'm using Rancher. Rancher supports 1.13.5 which at most supports docker 18.06.3 | 19:06 |
| adac | yes at some point I will upgrade kubernetes anyway that is true | 19:06 |
| adac | :) | 19:06 |
| adac | teward, I'm trying now out what you have suggested | 19:09 |
| adac | this host can be down a bit no problem if somethign is not working | 19:09 |
| teward | backup first | 19:09 |
| teward | just in case :P | 19:09 |
| adac | :) | 19:09 |
| adac | teward, backup is running | 19:16 |
| blackflow | Skyrider: I'm sorry but it's not bothersome at all. it's very, very simple. User owns files. nginx's www-data is in user's group (supplemantal!). php-fpm runs as user (one pool per user). User's home dir is root owned and there's a subdir (or more) where the user can upload stuff. Very, very simple. | 19:39 |
| blackflow | Skyrider: no idea why you invoked ACLs, that will just unnecessarily complicate matters to no end. | 19:40 |
| sarnold | it's amazing the flexibility you can have with the simplicity of unix acls | 19:40 |
| blackflow | yes but it's rather hard to maintain, the ACLs are not immediately obvious, not visible in ls, you have to know they're there. personally I prefer to put a nice, auditable apparmor profile, instead of fiddle with ACLs | 19:41 |
| blackflow | MAC > DAC and ACLs are DAC on steroid. | 19:41 |
| blackflow | *steroids. still DAC tho. | 19:41 |
| sdeziel | blackflow: I've yet to try php-fpm Apparmor hat support, do you have some experience with it? | 19:42 |
| blackflow | Skyrider: btw, just so we're on the same page, the approach I'm preaching here, been doing that for many years and currently I have tha very setup for hundreds of clients and their websites. | 19:42 |
| blackflow | sdeziel: nope. My setup is simple enough where "owner" is the only variation among pools. for more complex stuff I intend to have custom named profiles via systemd units, and one pool master per user, per unit, per profile. | 19:44 |
| blackflow | in other words, I wouldn't go with one process changing hats, but statically fix processes to profiles. | 19:44 |
| sdeziel | blackflow: OK. The hat thing is a per-pool thing | 19:45 |
| blackflow | yeah. I disliked hats even with apache and selinux, years ago. I prefered MLS instead | 19:45 |
| sdeziel | oh I see, you want multiple masters | 19:46 |
| blackflow | uhuh. | 19:46 |
| blackflow | the masters themselves don't really add any overhead and I can individually restart pools, unlike when with one master | 19:46 |
| sdeziel | interesting idea and side effect. Too bad I was looking for a reason to try Apparmor hats | 19:48 |
| Skyrider | blackflow: Can I undo the acls? | 19:51 |
| blackflow | Skyrider: sure, -b flag to setfacl | 19:52 |
| Skyrider | Thanks :) | 19:53 |
| Skyrider | Just curious.. You say to set it to the users home directory instead. | 19:53 |
| Skyrider | How is the user/group with this exactly with the web files? | 19:54 |
| Skyrider | web files owned by the user, and as you mentioned, www-data in their group? | 19:54 |
| blackflow | Skyrider: web files must be owned by user so that the default ownership (755 dirs, 644 files) allows them to write. with PHP in the game, you need to drop access to "others", so 750 and 640. In that case nginx (www-data) loses access, so you need to add www-data to users groups, so nginx can read the files. | 19:58 |
| blackflow | that's also the least privilege principle in action. and of course, php-fpm process must run as the user, in order to have exclusive access to user/site files, and in order to write them (uploads). | 20:00 |
| === keithzg_ is now known as keithzg | ||
| keithzg | Hmmm, so postfix rejects mail if the domain is in the virtual_alias_domains but the TO address isn't listed in the virtual alias map, even if there's CC's on that email that are? That is surprising to me! | 20:37 |
| blackflow | postfix doesn't care about CC. and the alias map is really the authoritative here, just the domain won't work | 20:39 |
| dlloyd | envelope to or message to? | 20:39 |
| blackflow | has to be envelope, postfix doesn't care about To header either | 20:40 |
| dlloyd | yeah, thats where i was leading to | 20:40 |
| keithzg | Hmm I wonder how it was working *before* I set up the virual_alias stuff, that certainly seems to have been when these emails started getting rejected rather than passed on. | 20:40 |
| paulatx | anybody happen to know how long the Ubuntu 16.04 EC2 images will support new hardware? The first graphic on this page: https://www.ubuntu.com/about/release-cycle makes it look like hardware support has already stopped for 16.04 but I'm not sure if that also applies to the AWS specific kernels which are also based off of the 4.4.0 GA kernel, not the 4.15 HWE kernel | 20:41 |
| * keithzg is having a hard time digging through it all and figuring out what's going on since the verbosity for amavis is set so high, heh, still failing to understand why it dies from time to time | 20:41 | |
| blackflow | keithzg: postfix has its own rahter verbose logs, you shouldn't consult amavis at all. theres #postfix here on freenode if you need more help with it. | 20:42 |
| keithzg | blackflow: Well it's /var/log/mail.log I'm looking at, and `journalctl -u postfix` doesn't have anything | 20:43 |
| blackflow | keithzg: wrong unit, postfix.service. you need postfix@ for the instance running iirc? | 20:44 |
| blackflow | neway, can you pastebin the problem entries? though really, I recommend #postfix for this particular issue, probably isn't specific to ubuntu defaults | 20:45 |
| tomreyn | paulatx: so did you read this? https://wiki.ubuntu.com/Kernel/LTSEnablementStack#Ubuntu_16.04_LTS_-_Xenial_Xerus | 20:46 |
| tomreyn | paulatx: oh yes, according to what you wrote you probably did. | 20:46 |
| tomreyn | so the aws images can't be used with the HWE kernel? | 20:47 |
| tomreyn | i mean, you can't just install it like on regular ubuntu? | 20:47 |
| keithzg | blackflow: Literally none of the postfix@ instances show anything, went through them one by one, but bizarrely postfix@* works, so apparently tab completion for journalctl leaves out whatever one I actually need to use? | 20:49 |
| blackflow | keithzg: postfix@-.service | 20:52 |
| paulatx | tomreyn: well the AWS images obtained from https://cloud-images.ubuntu.com/locator/ec2/ have the AWS tuned kernel enabled by default as detailed here: https://blog.ubuntu.com/2017/04/05/ubuntu-on-aws-gets-serious-performance-boost-with-aws-tuned-kernel. I'm trying to figure out when the support for new hardware will stop on those AWS tuned kernels | 20:52 |
| blackflow | keithzg: that's the default template instance | 20:52 |
| blackflow | unless of course you have something else set up, this should be the default | 20:52 |
| keithzg | blackflow: Huh. Yeah, that works. Just weirdly isn't one of the many things listed when I try and tab-autocomplete `journalctl -u postfix@` | 20:53 |
| blackflow | keithzg: WorksForMe(tm) :) | 20:54 |
| keithzg | blackflow: Hah! | 20:54 |
| keithzg | I'm pleasantly surprised the wildcard approach worked, too; that's a bit more user-friendliness and standard unsurprising handling than I normally expect from the systemd gang | 20:55 |
| tomreyn | paulatx: hmm, sorry, that's indeed beyond my horizon. i suggest you ask the same question here again tomorrow during UK business hours | 20:55 |
| keithzg | blackflow: Funny enough, on another server it *does* work fine! | 20:55 |
| blackflow | keithzg: patience and they will find a way to disappoint ;) | 20:56 |
| keithzg | blackflow: haha, true taht | 20:56 |
| blackflow | keithzg: so, can you pastebin the error, reason for NOQUEUE? | 20:56 |
| paulatx | tomreyn: ok will do, thanks | 20:57 |
| keithzg | blackflow: https://paste.ubuntu.com/p/2mSSGcyNfY/ is the paste | 20:59 |
| * keithzg is trying now heeding that warning and changing the relay_domains to be more strict | 21:02 | |
| blackflow | keithzg: also address that warning in lines 2 and 3 | 21:03 |
| keithzg | blackflow: Yeah that's the warning I'm talking about, heh | 21:03 |
| blackflow | ah k | 21:03 |
| keithzg | There was an overlap, with the emails coming from phabricator.gmcl.internal, and the relay_domains being gmcl.com and gmcl.internal | 21:03 |
| blackflow | 'sfine :) | 21:04 |
| keithzg | (The central issue here is, some emails, but not all emails, from our Phabricator instance aren't making it to users) | 21:04 |
| blackflow | well you'll have to investigate on a per-case basis. in this case, if you have a domain in virtual_alias_domains, you need to have the address in the virtual_alias_maps too. the postfix virtual readme has full explanation with examples. | 21:08 |
| blackflow | keithzg: http://www.postfix.org/VIRTUAL_README.html | 21:08 |
| keithzg | blackflow: Yeah, the weird thing is, in theory the emails are being sent to multiple users, and *one* of them is noreply@phabricator.gmcl.internal. But others are normal users, and they receive email from Phabricator fine in most circumstances. It's just this one subclass of email that's being rejected this way. | 21:09 |
| blackflow | normal users as in they're "mailboxes" (virtual_mailbox_*) ? | 21:10 |
| blackflow | aka destined for the virtual transport | 21:10 |
| keithzg | Well, as in they're someguy@gmcl.com | 21:11 |
| keithzg | (which this mailserver is the endpoint for, and they have "local" accounts (actually LDAP, but valid as real users on the system)) | 21:11 |
| keithzg | Hmm. Everything's still being rejected. | 21:32 |
| keithzg | Time to specify an alias for noreply@phabricator.gmcl.internal and get some of these emails, see what they're actually trying to do | 21:33 |
| adac | guys when rebooting my ubuntu server 16.06 I get: | 21:36 |
| adac | *16.04 | 21:36 |
| blackflow | I thought we established that first. if you want to treat this envelope recipient as alias, you need it in the map. alias = forwarder, btw, so it has to forward to (alias for) a valid address too, which can be a virtual mailbx, or an external relay'd transport | 21:36 |
| adac | device not accepting address 36 71 | 21:36 |
| adac | -71 actually | 21:37 |
| adac | any ideas what that problem might be and how to solve it? | 21:37 |
| adac | actually there should be no USB dongle on that server. is a hosted server | 21:38 |
| keithzg | blackflow: Yeah but the thing is, noreply@phabricator.gmcl.internal is in theory only *one* of the recipients; the others are all valid. And merely receiving emails for noreply wouldn't be terribly helpful, since that wouldn't then get the emails to the actual users. | 21:38 |
| blackflow | keithzg: there's always just one recipient. if your sending MUA had CC, then it ran a RCPT TO for each of them. CC has no meaning for postfix. | 21:39 |
| blackflow | one recipient as in one RCPT TO envelope recipient. | 21:39 |
| keithzg | blackflow: Well exactly, which is why I'm wondering if Phabricator is doing something terribly silly in this case. | 21:39 |
| blackflow | int _this_ case, you simply don't have the address in virtual alias maps, as the error is stating | 21:39 |
| blackflow | and it's not checking anything else it seems which means you do have the domain, hence the expectation to consult the map | 21:40 |
| keithzg | Particularly because of the very suspicious nature of there theoretically being three recipients (noreply, and two cc's) and the postfix log shows three copies being sent to noreply | 21:40 |
| keithzg | Receiving noreply's emails wouldn't actually solve anything per se | 21:40 |
| blackflow | irrelevant. your postfix has no idea where noreply@phabricator.gmcl.internal is, and how to deliver to it. | 21:40 |
| blackflow | (according to the log you pastebin'd) | 21:41 |
| keithzg | blackflow: Sure? But if as you say Postfix has no idea of "TO", then it shouldn't be seeing three copies sent to noreply; and of course any emails to noreply go nowhere, that's actually desired. | 21:43 |
| keithzg | Hence I'm thinking maybe Phabricator is doing something wrong. | 21:43 |
| blackflow | keithzg: I have no idea what your setup is. I'd really recommend you to pop into #postfix. read the /topic and prepare the logs and configs as specified by the !getting_help factoid. | 21:44 |
| blackflow | but on the face value, from that log entry, it's very simple. postfix has no idea how to deliver to that address. it's not defined in the virtual_alias_maps (to have an alias'd destination), but the domain is, hence postfix looking for it there. | 21:45 |
| blackflow | keithzg: where do you want mail RCPT TO that address, be sent instead? | 21:46 |
| keithzg | blackflow: Nowhere! | 21:47 |
| keithzg | The emails shouldn't be going to noreply anyways, and emails to noreply should indeed be rejected. It should be seeing emails to actual users, and most of the time that's how Phabricator sends email, but for some reason here it's sending all three copies to a single TO, which is noreply, instead of the actual Phabricator users with their valid @gmcl.com addresses | 21:47 |
| blackflow | well it IS going nowhere. postfix will either accept and deliver, or respond with NOQUEUE like it is now | 21:47 |
| keithzg | Sure, exactly. | 21:47 |
| keithzg | And hence why I'm thinking the problem at very least involves the Phabricator side of things too, since it shouldn't be just sending to noreply | 21:48 |
| blackflow | uhm, so why are we chasing hte postfix red herring then :) you should check the MUA that's apparently trying to send to that address | 21:48 |
| blackflow | "23:32 < keithzg> Hmm. Everything's still being rejected." <-- implies you don't want it rejected.... you should really get your story straight and start at the beginning, but in #postfix :) | 21:48 |
| keithzg | Well that's why I'm trying to receive the emails, so I can be sure of their exact actual headers :) | 21:48 |
| blackflow | yeah, no, sorry. please pop into #postfix and prepare all the details as explained by the !getting_help factoid there. thnks :) | 21:49 |
| keithzg | I mean, but as you say it's looking like Postfix is probably a red herring | 21:49 |
| blackflow | (you'll get better postfix support there, and this isn't ubuntu issue per se ;) | 21:49 |
| blackflow | keithzg: well it's rejecting which is apparently what you do want it to do. | 21:50 |
| keithzg | blackflow: Yeah exactly, *postfix* seems to be acting according to design and intention, it's just somewhere beforehand where something's going wrong. | 21:50 |
| keithzg | (Probably Phabricator, maybe nullmailer) | 21:50 |
| blackflow | keithzg: "trying to receive emails" -- then just create the alias entry for a local or any other address. | 21:52 |
| blackflow | receive it, see what's in it. | 21:52 |
| keithzg | blackflow: Yup, that is *precisely* what I've done | 21:53 |
| * keithzg now waits on the automated stuff that finds some files and commits a record of them via git-annex, which is then noticed by Phabricator's Diffusion, and then Herald rules send out the emails . . . it's all very Rub Goldbergian ;) | 21:54 | |
| keithzg | s/Rub/Rube | 21:55 |
| blackflow | held by Canonical's duct tape :) | 21:55 |
| keithzg | One of my favourite brands of duct tape :D | 21:56 |
| keithzg | Things seem to be working well enough now and I have enough info to dive in and try and unpick the specifics myself; many thanks, blackflow :) And apologies for broadly ignoring your entreaties to bring my problem over to #postfix instead :D | 22:16 |
| blackflow | keithzg: np, it's just it looked all the way as if you wanted the alias to actually work :) | 22:42 |
| keithzg | blackflow: Yeah naw, as usual for me it's a subtlely weirder problem with more moving parts involved, haha | 22:45 |
| gislaved | anyone still using local mirrors these days ? | 23:24 |
| sarnold | yeah I've got on | 23:25 |
| sarnold | one | 23:25 |
| gislaved | so much install traffic ? | 23:27 |
| gislaved | will be prettu big I guess ? | 23:27 |
| gislaved | &pretty | 23:27 |
| gislaved | * | 23:27 |
| sarnold | gislaved: ~1.5 terabytes would probably do; mine's at 1.27 TB used at the moment: http://paste.ubuntu.com/p/KdKfBDMbts/ | 23:29 |
| gislaved | sarnold heh, share storage I believe ? | 23:29 |
| gislaved | *shared | 23:29 |
| sarnold | ? | 23:29 |
| gislaved | or single disk ? | 23:30 |
| gislaved | or VM disk ? | 23:30 |
| sarnold | ah that zfs pool is on a nine-disk array: three vdevs of triple-mirror spinning metal drives | 23:30 |
| gislaved | :) | 23:31 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!