| lordievader | Good morning | 08:49 |
|---|---|---|
| foo | Does this look acceptable? I had a postgres server randomly restart in production: https://paste.ofcode.org/pTsNXxfXPBuCFZ3fTiLmQr | 19:26 |
| blackflow | foo: there's /var/log/postgresql/... log, what does that say for those time frames? | 19:29 |
| foo | blackflow: I have nothing there in postgresql-10-main.log. hmm | 20:09 |
| blackflow | foo: checked the rotated logs? | 20:10 |
| blackflow | Jun 21st was two-ish days ago, the logs rotate daily | 20:10 |
| blackflow | foo: however.... "Starting Daily apt upgrade and clean activities" ... do you have unattended upgrades or something? did the package upgrade? that would restart the service. | 20:11 |
| foo | blackflow: yes, I did - I checked both /var/log/postgresql/postgresql-10-main.log and /var/log/postgresql/postgresql-10-main.log.1 ... .1 starts at 2019-06-16 13:01:33. I would expect to see something there too. | 20:11 |
| foo | Not that I know of. And I don't think I have any automatic apt updates or anything in place | 20:12 |
| foo | This is the first time I've seen this behavior in 6 months | 20:12 |
| blackflow | foo: check /var/log/apt/history.log for that time frame | 20:13 |
| foo | blackflow: thank you, didn't realize apt kept a log. Great. https://paste.ofcode.org/XPJPcfdPTYi94CAnsv9j2i | 20:23 |
| foo | blackflow: Upgrade: postgresql-10:amd64 (10.8-0ubuntu0.18.04.1, 10.9-0ubuntu0.18.04.1) | 20:23 |
| foo | blackflow: I wonder if that's a needed security update? This is on digital ocean. I didn't realize this was happening but maybe it's a critical postgres security issue or such | 20:24 |
| foo | ... if it's even DO. | 20:24 |
| blackflow | I don't do unattended upgrades for this very reason. Always review and test updates manually. | 20:24 |
| blackflow | restarting postgres DB is a bad thing. many apps don't re-connect and hang in broken state with no DB connection, so restarting postgres needs to be done carefully and in concert with those apps. I so much hate .deb world and its idiotic maintainer scripts that restart services willy nilly | 20:25 |
| foo | blackflow: ... agreed. I've never had this happen before... in about 10 years. We migrated to 18.04 early this year. I wonder if this was a new thing? | 20:26 |
| blackflow | it's not, maybe you just didn't pay attention | 20:27 |
| blackflow | or actually.. I think unattended upgrades are now default on -server? dunno. I install from debootstrap, because the default image is chock full of bloatware like snapd, cloud-init, netplan.... | 20:28 |
| blackflow | (now = as of Bionic; but I'm not sure, I never install it, and debootstrapped ubuntu-minimal doesn't either) | 20:28 |
| foo | blackflow: aha, thank you. Ok. | 20:59 |
| foo | blackflow: I'm going to be moving postgres to digital ocean's managed database service soon anyway. But this is still good to be mindful of. I'll likely disable. | 21:00 |
| foo | ... although posgres is the most critical thing | 21:00 |
| blackflow | foo: this managed db, is it postgresql? | 21:07 |
| foo | blackflow: yes | 21:10 |
| foo | blackflow: similar to amazon rds | 21:10 |
| blackflow | but is it actually postgres, or some in-house concoction like amazon's? | 21:10 |
| foo | blackflow: postgres. Ain't nobody got time for amazon's concotions! (And I'm actually on digital ocean, using something similar to rds, with postgres) | 21:13 |
| blackflow | I see. | 21:15 |
| foo | blackflow: Do you have opinions on such systems? | 21:15 |
| blackflow | I do. I don't trust them and prefer my own installations. In fac, since january 2018, we no longer touch public clouds with a ten feet pole. | 21:17 |
| foo | blackflow: good to know. I've been doing my own installations for over 20 years. I thought I'd give it a try... alternative option is to spin up another digital ocean instance and devote it to postgres. | 21:17 |
| blackflow | I don't know how anyone can use public clouds in the post-meltdown-spectre era :) | 21:18 |
| blackflow | they were problematic even before (eg. rowhammer attacks at cheap VPS providers that never used ECC, there's an PoC in one of CCC conferences from few years go) | 21:18 |
| blackflow | *ago | 21:18 |
| foo | blackflow: thank you! | 21:35 |
| foo | I like the idea of not thinking about the database performance or optimizing and thought I'd give it a shot... but I could re-evaluate | 21:35 |
| blackflow | you know what they say. If you want something done right.... do it youself :) | 21:36 |
| foo | haha | 21:37 |
| foo | true. | 21:37 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!