wallyworldagprado: you dropped out?00:01
axwthumper: I don't know if you already saw, but wgrant is still seeing performance issues with 2.2.9 - still seing saslStart01:11
axwsee backscroll01:11
axwthumper: also, I couldn't repro that intermittent test failure. tried it a couple hundred times01:42
wallyworldaxw: if you had 5, just wanted a quick chat RE: the status PR. easier than typing into a PR01:46
axwwallyworld: sure01:47
wallyworldaxw: thumper has a theory that api rate limiting is broken - that could explain the SASl overhead02:11
wallyworldas the server / mongo is getting hammered02:11
axwokey dokey02:11
wallyworldhe says prodstack retstart showed a thundering herd02:11
wallyworldall agents reconnected within a few seconds02:12
wallyworldmaybe if we address rate limiting issue that will help alleviate the SASL symptoms02:12
axwthumper: do you have some logs to share?02:42
axwor were you looking at a screenshare02:42
thumperaxw: no02:42
thumperI was watching the prodstack grafana instance02:42
thumperjust noticed that the connection count for the controllers jumped back to full very quickly02:43
axwthumper: looks to me like the connection count is incremented before rate-limiting kicks in02:45
axwrate limiting happens in the Login handler, which is after we update teh conn count (which is what feeds the prometheus metric)02:46
* thumper nods02:55
=== frankban|afk is now known as frankban
tasdomasballoons, ping?10:46
tasdomashow do I trigger a test build on a juju PR?11:43
anastasiamactasdomas: !!build!!11:54
tasdomasanastasiamac, thanks11:55
anastasiamacnws :)11:55
tasdomasanastasiamac, could I get a review of https://github.com/juju/juju/pull/8262 ?11:58
anastasiamactasdomas: it's past my eod but tomorrow for sure ;)12:20
tasdomasanastasiamac, ah, sorry - enjoy your evening12:20
=== salmankhan1 is now known as salmankhan
tasdomasballoons, ping?14:45
balloonsgood day tasdomas14:45
tasdomasballoons, are juju acceptance tests actually being run?14:45
balloonstasdomas, not during the PR. They run post-commit14:46
tasdomasballoons, why is it looking into .juju/environments.yaml ( https://github.com/juju/juju/blob/develop/acceptancetests/jujupy/configuration.py#L44 ) ?14:47
balloonstasdomas, it looks for JUJU_HOME first as you can see14:48
balloonstasdomas, that should be JUJU_DATA, but JUJU_HOME is a juju 1ish that is still present14:49
tasdomasballoons, still, on l67 it looks for a file called environments.yaml14:51
balloonstasdomas, yes that file defines the cloud credentials14:52
balloonstasdomas, are you wanting to run your test?14:53
balloonstasdomas, typically we export JUJU_HOME which has the environments.yaml with our CI creds in it, and the tests go from there14:54
balloonsthat means we don't have to juju add-credentials everytime14:54
tasdomasballoons, so that is not the old-style environments.yaml that juju used to use14:54
balloonstasdomas, no.. it's a weird combination of juju1 and juju214:55
balloonstasdomas, we have a plan to eliminate all the juju 1ism's from the test framework, but the work is only about halfway done14:55
balloonsthe use of JUJU_HOME and environments.yaml is some of the last holdouts14:56
balloonstasdomas, if you like I can run your test changes through jenkins manually to ensure it passes15:44
tasdomasballoons, is there a way for me to do that myself?15:44
=== frankban is now known as frankban|afk
=== hml_ is now known as hml
axwthumper: with the "too many open files" thing, forget what I said about state objects being open during upgrade. that's still a problem for upgrades from 2.1, but 2.2.8 to 2.2.9 wouldn't have run any upgrade steps23:22
=== agprado is now known as agprado|afk
=== agprado|afk is now known as agprado

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!