[08:49] Good morning [19:26] Does this look acceptable? I had a postgres server randomly restart in production: https://paste.ofcode.org/pTsNXxfXPBuCFZ3fTiLmQr [19:29] foo: there's /var/log/postgresql/... log, what does that say for those time frames? [20:09] blackflow: I have nothing there in postgresql-10-main.log. hmm [20:10] foo: checked the rotated logs? [20:10] Jun 21st was two-ish days ago, the logs rotate daily [20:11] foo: however.... "Starting Daily apt upgrade and clean activities" ... do you have unattended upgrades or something? did the package upgrade? that would restart the service. [20:11] blackflow: yes, I did - I checked both /var/log/postgresql/postgresql-10-main.log and /var/log/postgresql/postgresql-10-main.log.1 ... .1 starts at 2019-06-16 13:01:33. I would expect to see something there too. [20:12] Not that I know of. And I don't think I have any automatic apt updates or anything in place [20:12] This is the first time I've seen this behavior in 6 months [20:13] foo: check /var/log/apt/history.log for that time frame [20:23] blackflow: thank you, didn't realize apt kept a log. Great. https://paste.ofcode.org/XPJPcfdPTYi94CAnsv9j2i [20:23] blackflow: Upgrade: postgresql-10:amd64 (10.8-0ubuntu0.18.04.1, 10.9-0ubuntu0.18.04.1) [20:24] blackflow: I wonder if that's a needed security update? This is on digital ocean. I didn't realize this was happening but maybe it's a critical postgres security issue or such [20:24] ... if it's even DO. [20:24] I don't do unattended upgrades for this very reason. Always review and test updates manually. [20:25] restarting postgres DB is a bad thing. many apps don't re-connect and hang in broken state with no DB connection, so restarting postgres needs to be done carefully and in concert with those apps. I so much hate .deb world and its idiotic maintainer scripts that restart services willy nilly [20:26] blackflow: ... agreed. I've never had this happen before... in about 10 years. We migrated to 18.04 early this year. I wonder if this was a new thing? [20:27] it's not, maybe you just didn't pay attention [20:28] or actually.. I think unattended upgrades are now default on -server? dunno. I install from debootstrap, because the default image is chock full of bloatware like snapd, cloud-init, netplan.... [20:28] (now = as of Bionic; but I'm not sure, I never install it, and debootstrapped ubuntu-minimal doesn't either) [20:59] blackflow: aha, thank you. Ok. [21:00] blackflow: I'm going to be moving postgres to digital ocean's managed database service soon anyway. But this is still good to be mindful of. I'll likely disable. [21:00] ... although posgres is the most critical thing [21:07] foo: this managed db, is it postgresql? [21:10] blackflow: yes [21:10] blackflow: similar to amazon rds [21:10] but is it actually postgres, or some in-house concoction like amazon's? [21:13] blackflow: postgres. Ain't nobody got time for amazon's concotions! (And I'm actually on digital ocean, using something similar to rds, with postgres) [21:15] I see. [21:15] blackflow: Do you have opinions on such systems? [21:17] I do. I don't trust them and prefer my own installations. In fac, since january 2018, we no longer touch public clouds with a ten feet pole. [21:17] blackflow: good to know. I've been doing my own installations for over 20 years. I thought I'd give it a try... alternative option is to spin up another digital ocean instance and devote it to postgres. [21:18] I don't know how anyone can use public clouds in the post-meltdown-spectre era :) [21:18] they were problematic even before (eg. rowhammer attacks at cheap VPS providers that never used ECC, there's an PoC in one of CCC conferences from few years go) [21:18] *ago [21:35] blackflow: thank you! [21:35] I like the idea of not thinking about the database performance or optimizing and thought I'd give it a shot... but I could re-evaluate [21:36] you know what they say. If you want something done right.... do it youself :) [21:37] haha [21:37] true.