blahdeblahaxw: Welcome back!  How do you feel about advising on how to handle some juju2 controller unresponsive type of bugs? :-)00:31
axwblahdeblah: thanks. absolutely miserable ;p  just kidding, what can I help with?00:31
blahdeblahSo, I've got a juju2 controller not responding, meaning juju status doesn't work on either the controller model, or the single model it's supporting.00:33
axwblahdeblah: ok, got logs for the controller?00:34
blahdeblahYep, but there's been nothing in machine-0.log since 2017-01-20 10:24:1000:34
blahdeblahOh, looks like logsink.log is the new hotness00:35
blahdeblahNothing in that for over half an hour, though00:36
axwblahdeblah: is CPU pegged? memory? disk?00:36
blahdeblahaxw: None of the above? :-)00:37
blahdeblahaxw: Our normal process for this is just to restart jujud-machine-0, and it usually works, but I'm keen to get it narrowed down to which bug this is, and get you folks something useful to make progress on.00:37
blahdeblahOh, just got some log activity00:38
blahdeblahLots of API terminations00:38
axwblahdeblah: what ports is jujud listening on?00:38
blahdeblahtcp6       0      0 :::17070                :::*                    LISTEN      53852/jujud00:39
blahdeblahafter that flurry of log messages, juju status is now responsive00:39
axwblahdeblah: huh :/00:40
axwblahdeblah: can you share the logs? maybe there's something buried that'll jump out00:40
blahdeblahaxw: So it turns out someone else started working on this too, and just restarted the agents00:42
axwblahdeblah: doh00:42
blahdeblahSo I guess this will have to wait until next time00:43
blahdeblahIt's been happening reasonably regularly, though, so hopefully it will be something we can gather again soon.00:44
menn0axw, wallyworld or thumper: https://github.com/juju/juju/pull/685901:09
wallyworldsays it's merged01:10
axwmenn0: wrong PR?01:10
menn0axw: right you are: https://github.com/juju/juju/pull/686601:10
axwok, looking01:11
wallyworldmenn0: should we also be changing the cleanup job to now remove dead local charm docs, since we don't need them for revnum checking anymore01:15
menn0wallyworld: yeah, I almost did that but left it out for now to save time01:16
menn0wallyworld: i'll do that tomorrow01:16
thumperaxw: do you remember if we create an ubuntu user for centos?01:29
thumperaxw: and whether centos has sudo?01:29
axwthumper: cloud-init creates the ubuntu user01:30
axwthumper: and yes it does01:30
thumperaxw: can lxd provider on ubuntu deploy a centos container?01:31
axwmenn0: LGTM01:31
axwoh, I see wallyworld has some comments tho01:31
menn0axw: thank you01:31
wallyworlda few yeah01:31
axwthumper: not at the moment. there's probably not *too* much to do to get it to work01:32
thumperhmm... ok01:33
axwthumper: I did try using manual with the centos LXD image at one point, IIRC it didn't work because they didn't have sshd running by default?01:33
axwthumper: or something else was missing, I forget01:33
* thumper nods01:33
menn0thumper: lxc image list lcorg: | grep -i centos01:37
menn0thumper, axw: if sshd isn't running you can always lxc exec in and sort that out01:37
* redir is eod01:55
wallyworldmenn0: if there are existing tests for removing dead charm docs and checking new rev numbers, how did they pass without sequence support?01:58
menn0wallyworld: previously charm revs were calculated by keeping all charm docs around, including dead ones01:59
axwmenn0: yeah, it was just that it wasn't working OOTB. I think there may have been other things too01:59
wallyworldalso, with the addition of "-1" to the charm urls in MakeCharm(), was that an across the board change or just for those metrics ones?01:59
axwtoo long ago, can't remember now01:59
menn0axw: yeah, the images don't even have sshd installed, they are very bare bones02:00
menn0wallyworld: it should never have been valid to add charms with no revision, but State.AddCharm was allowing it and those (buggy) tests were taking advantage of it02:00
menn0wallyworld: with my changes that's no longer possible, which caused the tests to fail02:01
wallyworldso in apiserver/uniter/uniter_test02:04
wallyworlds.meteredCharm = s.Factory.MakeCharm(c, &jujuFactory.CharmParams{02:04
wallyworldName: "metered",02:04
wallyworldURL:  "cs:quantal/metered",02:04
wallyworldisn't that a case where the revision is missing?02:04
wallyworldmy other comment about testing the url rev number incrementing - unless we remove dead charm docs manually in the unit tests, they will still be around right? which means we are not 100% sure we are testing that the seqence behaviour is hooked in02:06
menn0wallyworld: ah ha, the prep code which requires a revision is specified only applies to local charms and that's a store charm02:08
menn0wallyworld: that's why the test still passed02:08
wallyworlddoh! missed that02:08
menn0wallyworld: it's technically not correct though right?02:08
wallyworldprobably not, but outside the scope of this change02:08
menn0wallyworld: I'll do the removal of charm docs tomorrow (and will make it part of that PR)02:09
wallyworldok, ta. do you agree with my point about thAT?02:09
* wallyworld hates caps lock being next to the A key02:09
menn0wallyworld: I see your point (although you can obviously see that the code which queries the collection to find the next rev number has been replaced)02:10
thumperoh FFS02:10
wallyworldi can. but...... uness we remove the old artifacts that made it possbile for the old behaviour to work, we can't be sure02:11
wallyworldi relaise it's pedantry02:11
wallyworldTDD, write failing test, fix code, make tests pass02:11
wallyworldthere's no failing tests here02:12
wallyworldsince the dead docs still exist02:12
wallyworldwell, i guess if the code is removed they would have fsiled02:12
redirmenn0: that dutch skit was great thanks02:17
menn0redir: my wife showed it to me last night. i had tears rolling down my face02:19
thumperwell, fuck me, that worked02:22
thumperI'm mildly surprised...02:22
babbageclunkaxw: azure pagination - if I don't pass anything for the top-n parameter to a list, will it return all the results at once, or do I still need to handle next-result getting?02:36
axwbabbageclunk: I believe you have to handle next result still02:36
axwbabbageclunk: (we're not doing that anywhere yet, I know)02:36
babbageclunkaxw: yeah, thought that might be the case - thanks02:36
axwbabbageclunk: limits on VMs, vnets, etc. preclude there being too many of a single resource AFAIK, but when you're listing all resources it might be needed02:37
babbageclunkaxw: ok - it's not too fiddly, just thought I'd make sure.02:38
babbageclunkaxw: gah - I can't work out what resourceProviderNamespace and parentResourcePath should be in the CreateOrUpdate call. The docs only tell me helpful things like "The namespace of the resource provider."02:49
babbageclunkaxw: sorry to keep bugging02:50
axwbabbageclunk: no worries02:50
axwbabbageclunk: un moment02:50
axwbabbageclunk: the namespace is something like "Microsoft.Compute", and parent resource path is the URL path proceeding the namespace, preceding the resource type. e.g. for security rules, the parent resource path will be something like "networkSecurityGroups/{network-security-group-name}" (I think)02:54
axwbabbageclunk: compare https://docs.microsoft.com/en-us/rest/api/resources/resources to the path template in https://msdn.microsoft.com/en-us/library/azure/mt163645.aspx02:54
axwbabbageclunk: Microsoft.Network is the namespace, "securityRules" is the resource type, {security-rule-name} is the resource name02:55
axwbabbageclunk: everything between the namespace and the resource type is the parent resource path02:55
babbageclunkaxw: thanks, that's awesome! There turns out to be a lot of azure to understand.02:56
menn0thumper: are we using the tech-board HO?02:59
babbageclunkaxw: can you take a look at this? https://github.com/juju/juju/pull/686804:58
axwbabbageclunk: okey dokey04:59
babbageclunkaxw: WIP at the moment - I'm doing tests now.04:59
axwbabbageclunk: just eating atm, will look soon04:59
babbageclunkaxw: thanks, no rush!04:59
axwbabbageclunk: you should be able to get the azure creds that CI uses05:19
axwbabbageclunk: and test with them05:19
axwbabbageclunk: just looking for them, will pm05:21
axwwallyworld: can you please review https://github.com/juju/juju/pull/6869 when you're free?05:35
wallyworldsure, give me 505:35
wallyworldaxw: looks good, a nice refactoring06:01
axwwallyworld: thanks06:01
axwwallyworld: got time for a HO? I need to brainstorm on a problem I've encountered with LXD creds07:20
wallyworldaxw: otp with uros, give me 10?07:21
axwwallyworld: no rush, let me know when07:21
wallyworldaxw: free now07:42
axwwallyworld: I think I figured out what I need to do. I'll ping you later if I'm wrong :)07:43
wallyworldaxw: ok, no worries. i may be afk for dinner or whatever but i'll be around07:47
axwwallyworld: np. it can wait if not07:47
axwwallyworld: enjoy your long weekend07:47
wallyworldwill do :-)07:47
wallyworldi still need to land a pr first07:48
=== frankban|afk is now known as frankban
=== rvba` is now known as rvba
jammorning perrito66612:09
jamanyone want to review my attempt to make "juju ssh" better when the remote machine has multiple addresses?12:09
perrito666Hey I'll be unresponsive for the next couple of hours my wife is at the hospital with a stomach issue mail me If you need me12:18
jamperrito666: I hope everything is ok. hope she feels better soon12:21
jamaxw: babbageclunk: so I noticed both of you guys are making changes to how we're doing clouds between 2.1 and 2.2. I'd like to make sure your changes are resolved relative to each other.13:37
jamspecifically babbageclunk worked on playing with the names, while it looks like axw got rid of the clouds.BuiltinClouds map13:37
jamI'm trying to make sure other 2.1 landed features get forward ported to 2.213:37
jamcan I get some hints as to how this should be resolved?13:37
perrito666Another bless what a week14:46
rick_hlol congrats jujuteam14:46
jamballoons: ping14:56
balloonshey jam14:59
jamhi balloons. I just sent you an email. Is it possible for us to carve out some time to go over CI and 2.1-dynamic-briges?14:59
balloonsjam, sure14:59
jamI need to go spend time with my family before they disown me, but I can come back in a few hours.14:59
balloonsjam, see to that and we'll talk later15:00
jamballoons: I sent a tentative invite15:01
jamhave a good one15:01
frobwarejam: AFAICT, trusty seems ok for the testing I have done.15:37
frobwarejam: https://github.com/juju/juju/pull/6870 <<- bond-reconfigure-delay16:39
mbruzekcmars: Is there a way to remove a term from a charm? One of the IBM charms still has lorem-ipsum17:18
cmarsmbruzek, push a new version of the charm to the charmstore without that term in the metadata17:18
=== natefinch is now known as natefinch-afk
jamfrobware: I reviewed it, I feel like it would be better to be having the config of 'bond-reconfigure-delay' but have the API down to the script itself just think in terms of "reconfigure-delay"18:33
jamso the API server notices this is a bond, and sends a different value18:33
jamrather than having the "is this a bond" in the script18:33
jamis that too invasive?18:33
frobwarejam: yep, we could do that. i think...18:34
frobwarejam: can we discuss this in the morning?18:35
jamfrobware: certainly18:35
frobwarejam: it would be ideal to shore up these changes _this_ _week_18:35
* frobware sighs.18:36
jamfrobware: I think its ready to land with that change, as I'm happy with the rest18:36
frobwarejam: there was one additional test I thought of in config.go; we test for a custom value, but I don't think for the default value.18:37
frobwarejam: when you say "just think in terms of "reconfigure-delay"" - is that just naming or a deeper semantic change?18:37
jamfrobware: the script should not have the "is this a bond", but the api server should be making that decision and returning a value for the bridges it is creating18:40
frobwarejam: ok, so if you pass reconfigure-delay=<some-value-gt-0> it will do the sleep regardless of type. yes?18:40
frobwarejam: we're just talking about the origin of truth, correct?18:41
jamfrobware: if the script gets that value yes18:41
frobwarejam: makes sense.18:41
jamthe thing deciding how long to sleep is the API server18:41
frobwarejam: and it should know the device type/kind/category.18:41
frobwarejam: caveat emptor: if you run the script manually (as I do a lot) don't forget to pass a value. Just sayin...18:42
jamfrobware: have a default value of 1-3 ?18:43
jamor a default value of 10-20 ?18:43
frobwarejam: 3 was a winner for so long. I vote for 4.2.18:43
jamfrobware: then if you forget it still fails saif18:43
frobwarejam: so we're saying cli arguments win, but if not supplied we'll default to 4.2 (or 3). correct?18:44
jamfrobware: yeah, you had a default of 30 in the script already if I read it correctly18:44
frobwarejam: ah, crap. Hmm.18:45
jamfrobware: add_argument(..., default=30) I thought I read18:45
frobwarejam: this from my hacking.... probably....18:45
frobwarejam: I had meant to leave it to whatever it was previously knowing full well that it would be driven from model-config. but perhaps 30 is nice and safe if you run it manually.18:46
jamfrobware: seems a little too safe18:47
jamfrobware: especially if it is the one that SSH is talking on and you wonder if everything is dead18:47
frobwarejam: a interface is better up than down. :)18:47
frobwarejam: so the only time you'll get 30 is if you specify 0 from juju for bond-reconfigure-delay.18:48
jamfrobware: we should just always pass the aggregate value18:48
jameven if it is 0 so it overrides the default18:48
* frobware needs to go and look at this to make sure. But tomorrow now...18:48
frobwarejam: I think this comes back to: who drives the value and why do we need a default. if juju passes 0, you get 0. if it passes 1234, you get ... and so on.18:49
frobwarejam: should the script carry a default?18:49
jamfrobware: the reason for a default is if you run the script by hand (IMO)18:50
frobwarejam: right. but I would counter, it has no documentation, is wrapped up by juju so in the true spirit of *nix you have rope to hang yourself with.18:51
jamsounds like you're just trying to be spiteful now :)18:52
frobwarejam: you might be interested in: https://github.com/frobware/juju/tree/go-debinterfaces18:52
frobwarejam: I started a rewrite of the script in Go. The parsing side is done, there's 100% test coverage. I didn't do the bridging side as...18:53
frobwarejam: code is in juju/network/debinterfaces/...18:54
=== redir is now known as redir_exercise
=== redir_exercise is now known as redir
=== tvansteenburgh1 is now known as tvansteenburgh
=== menn0_ is now known as menn0
=== frankban is now known as frankban|afk
alexisb_thumper, ping23:50

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!