=== miceiken9 is now known as miceiken
=== chris14_ is now known as chris14
=== alucardromero4 is now known as alucardromero
[10:46] <miceiken> I have certain services that time out, typically at night, are there any easy ways to figure out what's happening?
[11:16] <tomreyn> miceiken: services don't time out, network connections to a service might, is that what you're seeing then?
[11:16] <miceiken> yes
[11:17] <tomreyn> does it happen when you have a local (to the server) client making the same / similar requests?
[11:17] <tomreyn> is there a high load on the server around the time it happens?
[11:18] <tomreyn> one of the most common causes of things getting slow at night would be backups
[11:18] <tomreyn> try to find out what the cottleneck is, get better system and service monitoring, more verbose logs (only) if needed
[11:19] <tomreyn> review existing logs first
[11:19] <tomreyn> *bottleneck
[11:23] <tomreyn> (only) IF you have proper monitoring but it does not seem to catch the event that causes connections to time out, you may need to decrease monitoring interval. or, if that's not possible due to causing generally high load, just use a simple solution such as sar to just cover the base facts with a low interval (with very low resource overhead).
[11:29] <miceiken> No scheduled processes running, during nighttime there is pretty much nothing happening. The only place I notice it is actually on irc, where I'll occasionally ping timeouts from libera chat. Funny thing is that other IRC networks don't drop. I can't think of anything that would cause it. If it had been the other irc servers then I would have
[11:29] <miceiken> expected it to be my ISP dropping long-running connections, but it doesn't sound as likely when it's just that one.
[11:38] <tomreyn> i'm not sure i understand how the irc situation relates to this system providing network services to which some (or all, at different locations, using different routes?) run into timeouts. it's also unclear whether the timouts are reported on the system you manage or on the client(s).
[11:41] <tomreyn> if there is no high system load on that server, there could still be high load on the client, or just on (one of the) the network link(s) between them, cuasing connections to time out. could be a noisy neighbour (on either end) squatting upstream bandwidth. so inspect / improve network link monitoring, too.
[12:04] <miceiken> > so inspect / improve network link monitoring, too. -- this is what I want to do, but I don't know what would be a good fit for this
[12:07] <CosmicDJ> miceiken: wild guess, the unattended-upgrades.service runs at night and restart one of network services after ugrading packages
[12:08] <miceiken> Thanks, I'll look into that.
[15:22] <MTecknology> I need to finally get a package mirror set up so that I can enable auto updates across my fleet ...
=== cpaelzer_ is now known as cpaelzer
[18:58] <termina> This morning, an autoinstall user-data config that was previously working is throwing 'subiquity/Refresh/check_for_update: FAIL: cancelled' and bringing me to the interactive install menu. Nothing in /var/crash. Curious what might be causing this, or if there's a way to turn up debugging to see what 'check_for_update' is failing on.
[19:00] <termina> fwiw was hoping I could bypass Refresh entirely by setting 'update: no' under 'refresh-installer' but no go
[20:13] <cpete> termina: my gut says subiquity is not getting the autoinstall config, especially if the install is now interactive
[20:14] <cpete> termina: if you suspect a bug, contents of /var/log/installer  are potentially interesting
[21:01] <termina> Thanks cpete, you were right on. Not sure how this ever worked, but yamllint pointed me in the right direction. ;^.^  Appreciate the response!
[21:16] <cpete> termina: no prob!
=== miceiken7 is now known as miceiken