| === arraybolt3_ is now known as arraybolt3 | ||
| === arraybolt3_ is now known as arraybolt3 | ||
| === keithzg_ is now known as keithzg | ||
| guesswhat | is some amd gpu driver required for 6800U laptop cpu, or is it already included in kernel? | 08:27 |
|---|---|---|
| odc | guesswhat: it's in the kernel, but I'm not sure if 5.19 is recent enough | 09:11 |
| guesswhat | odc got spikes 100% in google meeting | 09:11 |
| guesswhat | chrome with hw accel enabled | 09:11 |
| odc | Ouch | 09:12 |
| guesswhat | `lspci -k | grep -A 3 -E "(VGA|3D)"` using amdgpu module | 09:12 |
| guesswhat | https://www.phoronix.com/news/AMD-Radeon-680M-Linux-Mesa | 09:14 |
| guesswhat | shoud be fine | 09:14 |
| === jgee118 is now known as jgee11 | ||
| foo | Hmm, nginx-logs throwing this: live/live.sock failed (13: Permission denied) while connecting to upstream, client ... live.sock is 777 which is what I have on another system which works. Ubuntu 22.04, new system. Perms seem ok. Python + gunicorn + nginx. Any sugggestions? | 18:22 |
| foo | Spent hours on this already, running out of ideas | 18:22 |
| adac | Hi there! it is strange my htop and iotop command on the server are loadning really really slow. What can cause that actually? | 18:29 |
| adac | I was checking the read/write and disk io, but all seems normal in this regard | 18:30 |
| adac | while the server has some decent load, it still has a lot of free resources | 18:30 |
| adac | How can I properly debug this? | 18:30 |
| effendy[m] | adac: what else is slow? | 18:40 |
| effendy[m] | maybe they're trying to read off a remote disk/share of some sorts also? | 18:41 |
| sdeziel | foo: `namei -l /path/to/live/live.sock` would help so would seeing your nginx config | 18:42 |
| adac | effendy[m], actually "ls" and all this normal commands, "pwd" all this is working very fast | 18:42 |
| adac | effendy[m], actually I run a bare metal kubernetes cluster on this server (3 servers actually). Within this cluster I do some smb mounts with this guy: https://github.com/kubernetes-csi/csi-driver-smb but other then that, and this never made any issues so far on other clusters I have, I do not read from remote disks or so | 18:47 |
| effendy[m] | It might be related to that though, but this is just a supposition. | 18:49 |
| effendy[m] | If there was any way you could disable it temporarily, it would interesting to see if it reacts in the same way. | 18:49 |
| adac | effendy[m], I will try and uninstall it bit later. Just have to leave now for some time. Will inform you a bit later! Thanks for the hint! | 18:53 |
| effendy[m] | :) | 18:53 |
| adac | effendy[m], inform only if you like to know if it worked of course :-) | 18:55 |
| effendy[m] | adac: I do, of course! | 18:56 |
| adac | perfect! :-) | 18:56 |
| foo | sdeziel: thank you. https://bpa.st/54FQ - it's all stock | 19:15 |
| sdeziel | foo: please use `namei -l` as it report on the ownership/perms | 19:15 |
| sdeziel | foo: please also paste the relevant nginx/conf.d and nginx/sites-enabled files | 19:16 |
| foo | sdeziel: err, sorry, I was going to say... that didn't seem too helpful. pebkac | 19:24 |
| foo | sdeziel: thank you! https://bpa.st/R77Q | 19:24 |
| foo | sdeziel: note, groups www-data returns www-data : www-data dev | 19:24 |
| === blackro0t is now known as blackroot | ||
| foo | sdeziel: I'm really curious, I've done this set up many times and not sure what I'm missing. | 19:25 |
| sdeziel | foo: you likely need to edit nginx's systemd unit to add `SupplementaryGroups=dev` to it. `sudo systemctl edit nginx` | 19:25 |
| sdeziel | foo: cause I don't think that nginx will load the supplementaty groups when it switch UID/GID from root:root to www-data:www-data | 19:26 |
| foo | sdeziel: thanks, testing now. I think I want to uncomment out the service from sudo systemctl edit nginx and then add SupplementaryGroups=dev to it, *googles | 19:29 |
| sdeziel | foo: no, you just need `[Service]\nSupplementaryGroups=dev\n` | 19:30 |
| foo | sdeziel: ahh, thank you. It looks like the other test is: chown -R dev:www-data /home/dev/myapp | 19:31 |
| sdeziel | foo: hmm, I tested my SupplementalGroups= theory and it's wrong apparently, you don't need this for nginx to pick up the supplementary groups. Restarting nginx was all that was needed, no need for edits | 19:40 |
| foo | sdeziel: huh, thank you for testing. I appreciate your thoroughness | 19:40 |
| sdeziel | foo: I whipped up a quick PoC trying to mimic your setup and I don't have the permission denied problem you have :/ | 20:06 |
| foo | sdeziel: huh. | 20:19 |
| foo | sdeziel: FWIW, this was a brand new Ubuntu 22.04 set up. Stock everything. I pasted the service conf, nginx conf... I can't think of what else is here that's nuanced. /home/dev is a dev user, but I doubt that matters. | 20:19 |
| sdeziel | foo: I setup a fresh 22.04 container, put nginx-light and created a tiny python3 service using your myapp-api.service unit... | 20:24 |
| foo | sdeziel: I really really really appreciate you taking the time to do that. I wonder what else I can have on here that could be causing that | 20:26 |
| sdeziel | foo: are you using ACLs? | 20:39 |
| foo | sdeziel: negative | 20:42 |
| sdeziel | foo: maybe try enabling debug error logging for your `location /`? `error_log /var/log/nginx/myapp.err debug;` | 20:44 |
| foo | Oh, not a bad idea - can give that a shot. | 20:48 |
| teward | sdeziel: fwiw debug logging is for the whole log for a server block not just a location | 21:33 |
| teward | it's also noisy as sin (that looks like nginx) | 21:33 |
| sdeziel | teward: https://nginx.org/en/docs/ngx_core_module.html#error_log mentions `location` as a valid context | 21:34 |
| sdeziel | but yeah, it's quite noisy | 21:34 |
| teward | sdeziel: agreed, it's VERY noisy, i'm a little behind thoug hand am still doing scrollback to figure out what exactly is going on | 21:34 |
| sdeziel | and no matter how I break the access to the Unix socket, the only error I can get is the permission denied... | 21:34 |
| sdeziel | it won't tell if it cannot connect due to not having "execute right" on something or the path... | 21:35 |
| teward | welcome to how C relays access perms or not | 21:35 |
| teward | you either 'have permission' or 'don't | 21:35 |
| teward | that's not nginx that's just how C works : | 21:36 |
| teward | C / filesystem notices | 21:36 |
| teward | *yawns* i need more coffee | 21:36 |
| teward | let me play some catchup | 21:36 |
| teward | foo: i would NOT have things live in your home directory | 21:40 |
| teward | nginx runs as www-data by default in Ubuntu. per your namei, your socket is at /home/dev/myapp/live/live.sock. All home directories default to 750 which PREVENTS non-owner and non-group access to the directory at the /home/dev/ level. | 21:41 |
| teward | www-data does not have the ability to read or traverse into /home/dev/ | 21:42 |
| teward | you have two choices: | 21:42 |
| teward | (1) give 'other users' traverse/read permissions to the base directory: (UNSAFE) - sudo chmod o+rx /home/dev | 21:42 |
| teward | (2) place your socket somewhere else such as `/var/run` or `/run` or in `/tmp` with proper 666 permissions (read/write). | 21:43 |
| teward | third choice: ACLs | 21:43 |
| teward | (3) explicitly give ACL permissions to the www-data user for your home directory (also still unsafe): sudo apt install acl && sudo setfacl -m u:www-data:rx /home/dev/ | 21:44 |
| teward | (you still have to install ACL to make that work) | 21:44 |
| teward | i don't think the issue lies in your *python* app but instead actually your intermediate directories between / and the actual full location of your socket. | 21:45 |
| teward | sdeziel: ^^ | 21:45 |
| teward | debugging logs won't help it'll just say permission denied | 21:45 |
| teward | but such things should be taking after programs / applications that *make* sockets readable/writable by other people/users (a-la PHP, etc.) rather than run everything in home directory | 21:46 |
| sdeziel | teward: that's the thing, `www-data` is a member of the `dev` group | 22:22 |
| sdeziel | teward: and yeah, I am pretty sure the problem isn't the Python app | 22:22 |
| sdeziel | foo: I agree with teward's suggestion of putting the socket under `/run`. You can even tell systemd to create `/run/myapp-api` (`RuntimeDirectory=`) to contain the socket file | 22:25 |
| === arraybolt3_ is now known as arraybolt3 | ||
| sdeziel | foo: strace'ing the nginx child handling the request (the one getting a perm denied) might be useful | 22:31 |
| sdeziel | probably more useful than that debug log | 22:31 |
| teward | sdeziel: i would not trust www-data being a member of the grou pas far as i can throw it | 22:36 |
| teward | whether the system says so or not | 22:36 |
| teward | it's the *least* reliable mechanism to validate access | 22:36 |
| teward | (runtimedirectory=... is the better approach by *far*) | 22:36 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!