=== rumble is now known as grumble
=== disposable3 is now known as disposable2
gaurangtwhat's the procedure to delete a network space in juju?17:39
gaurangtor can we move the subnet from one space to other?17:39
xarses_is there a useful way to get a log of the actions a unit may have been performing at a specific time from the juju cli?17:41
rick_hxarses_: not from the CLI. I guess you could look for action results that occurred around the timeline? but you'd need to check the logs to get specific18:21
rick_hxarses_: turning on the audit log would make it cleaner/easier a bit but it's still in a log18:22
xarses_urgh, too many signals to just sift in every damned log18:22
xarses_on a related subject, is there a way to disable the local haproxy in the openstack api charms?18:23
xarses_something is raising 503's on me randomly, and that guy has like no log18:25
xarses_and I have a balancer elsewhere18:25
rick_hxarses_: do you know what action you're looking for? I'm curious what the log | grep action might be or the name of the action18:26
xarses_nope, no clue what so ever18:26
xarses_we run (openstack) rally against the cloud to detect failures, and have noticed recently that we are getting random failures on endpoints that are nearly always 503's18:27
rick_hxarses_: so not sure what's actually erroring in the stack?18:27
xarses_but calls prior and after, that in many even include said api pass18:27
xarses_yep, no clue what's causing the errors18:31
xarses_oh, the one I'm looking at currently isn't 503, its 502 (incomplete or invalid response)18:34
xarses_bad gateway18:34
xarses_argh, found it18:50
xarses_this thing, it ruffles my feathers soo much18:50
xarses_something like this happens https://gist.github.com/xarses/5a7beb4cc856588c3f91ce0b094b4f2818:53
xarses_and then anything to look at is SIGTERM and restarted18:54
xarses_apache, npre, the python wsgi's18:54
xarses_ugrh, so the systemd thing is probably just annoying20:19
xarses_there is a session into the host before that, and then a bunch of things related to the charm reload, I'd guess that its the culprit as I originally suspected20:19
xarses_so I'm back to, something in the charm allows for concurrent modification of members and ends up causing an outage20:20
xarses_so `juju debug-log` doesn't seem to work21:41
xarses_among it not having any logs for the units I'm looking for21:44
xarses_--lines isn't respected21:44
xarses_> juju debug-log --no-tail --include unit-percona-cluster-0 --lines=1 --replay | wc -l21:45
xarses_> 448921:45
xarses_and the example in --help has a `-T` flag which doesn't exist21:45
skayI just ran in to a VersionConflict with requests when trying to create a new charm using charmtools. https://paste.ubuntu.com/25486709/22:43
thumperxarses: hey23:23
thumperxarses: which juju version?23:23
thumperxarses: also, thanks for pointing out the -T problem, we'll get that fixed23:25
thumperxarses: are you able to file a bug? if not we could add one23:25
xarsesprobably 2.2.223:28
xarseswe where updating some of the controllers not sure which that was on23:28
xarsesfor which -T or the lines not being respected, or nothing from the unit im looking for?23:29
thumperxarses: we did update the include option so you should just be able to specify the unit like normal, so 'percona-cluster/0'23:29
thumper-T doesn't appear to be an option in the code23:29
thumperI'm kinda surprised it isn't complaining23:29
xarsesoh, it complains of the invalid option23:29
thumperoh, good23:30
xarsesits just in --help long description of examples23:30
thumperyeah, that needs to be fixed23:30
thumperalso the examples should just specify the machine ids and unit ids rather than the unit- and machine- strings23:30
xarsesI was looking initially for unit-neutron-0, which has one entry in the uncompressed log file, it never gave that back23:30
xarsesalso tried neutron/023:31
thumperit is possible that there are lines in the local file that never make it to the server23:31
xarsesonly switched to percona-cluster because it was spewing warnings and not respecting --lines23:31
thumperfor debug-log to pick up23:31
thumperthe local log files are written as things happen23:31
xarsesno there is one line in the /var/lib/juju thingy on the controller23:31
thumperbut debug-log only works on the logs that ended up in the controller23:32
xarsesthat grep found23:32
thumperif it made it into logsink.log then it should be in the db23:32
thumperunless it was removed due to pruning23:32
xarsesya that one23:32
xarsesso long story with debug-log aside23:32
xarsesshouldn't it have logged actions like applying charm updates and restarting services?23:33
xarsesthey wheren't logged on either the controller or the unit23:33
xarsesthey don't emit a syslog23:33
xarsesand are driving my troubleshooting crazy23:33
thumperyou mean the charm actions?23:34
thumperas in the charm is restarting an openstack service23:34
thumperor the restarting of juju agents?23:34
xarseswell as in freaking out and causing an outage on all of my neutron (api) servers at the same time during a charm upgrade23:35
xarsesI can only prove my issue is due to the charm upgrade, because less than a minute prior memcached is started for the first time ever23:35
xarsesthere are no other logs that I can find that show that juju intended to or was actively taking actions on the unit23:35
thumperwhat is the logging config set to?23:36
xarseswho's config?23:36
thumperjuju model-config logging-config23:36
thumperfor the model in question23:36
xarseslooks like a default of warning23:39
thumperin which case, you will only get told of things not working23:39
thumperjuju model-config logging-config=juju=info23:40
xarses_$ juju model-config logging-config23:40
thumperjuju model-config "logging-config=juju=info;unit=warning"23:40
thumperxarses: that would then tell you more about what juju is doing for that model23:41
xarsesle sigh, google didn't give me any information like this about levels23:41
thumperwe are trying to make sure that INFO level logging is useful and not too verbose23:41
xarses_all I could ever find was https://jujucharms.com/docs/2.2/troubleshooting-logs23:41
thumperxarses: I'll make sure that we get the docs updated to talk about the log levels23:42
thumperxarses: I'm going to relocate back home, should be back online in ~20 minutes23:43
xarsesthanks this was insightful23:44

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!