[03:53] <manadart> humbolt: Canonical devs were at an engineering sprint last week. Most will have had Monday off.
[10:31] <stickupkid> CR anyone https://github.com/juju/juju/pull/10651
[10:32] <stickupkid> this was left over from last weeks sprint duties...
[10:32] <achilleasa> stickupkid: looking
[10:33] <stickupkid> achilleasa, tbh thumper did 99% of this, i just moved it to develop code
[10:56] <achilleasa> stickupkid: done
[10:57] <stickupkid> achilleasa, ty
[10:58] <stickupkid> achilleasa, agree with comment, I'll add that next time.
[15:10] <stickupkid> manadart, we can't use mocks folder in state, as it causes a import cycle :(
[15:10] <stickupkid> manadart, any idea where we can put them, other that in the root of state
[15:10] <stickupkid> manadart, maybe migration-mocks?
[15:10] <manadart> stickupkid: Yeah, you can only use a mocks directory if the tests are in an external package.
[15:11] <stickupkid> manadart, yeah, it's annoying
[15:11] <manadart> For internal, just create a ..._mock_test.go file.
[15:11] <stickupkid> yeah
[17:12] <nammn_de> stickupkid achilleasa or manadart any of guys still have in mind what "costs" means under "environs/instances/instancetype.go", are those the costs to order this machine?
[17:14] <rick_h> nammn_de:  yea, in order of preference
[17:14] <rick_h> nammn_de:  e.g. find the lowest cost instance that meets the users constraint requirements
[17:15] <rick_h> "--constraints mem=32g" - so what's the cheapest machine that has 32gb of ram
[17:15] <nammn_de> rick_h: great, i would use that as the 2. constraint for sorting. E.g. sort by type (a1,a2...) and (cost)
[17:15] <rick_h> nammn_de:  I don't think so because that'll lead to inter mixing?
[17:16] <rick_h> nammn_de:  e.g. a t3 and a c3 might be closer to cost? or do you mean just they should sort that way from large, xl, 2xl?
[17:16] <rick_h> nammn_de:  hmm, now that I say that out loud you're probably right
[17:16] <rick_h> ignore me...
[18:47] <bdxbdx> happy tuesday
[18:47] <bdxbdx> trying to help my guys get to the bottom of an issue
[18:49] <bdxbdx> we are trying to deploy machines in our maas, everything goes smooth until the target machine tries to download the agent binaries from the controller
[18:50] <bdxbdx> when machine with ip 10.30.62.1 tries to download the agent binaries during the final post boot initialization we see https://paste.ubuntu.com/p/WXcpQwjVkd/ in the controller logs
[18:51] <bdxbdx> on the ipmi viewer/console to the maas box we see the machine trying to download the tools from the controller and failing https://imgur.com/a/M58xHAq
[18:52] <bdxbdx> anyone see this before?
[18:53] <bdxbdx> our thoughts are that it may be a time skew between the controller and the maas node
[18:53] <bdxbdx> going to do some digging
[19:11] <rick_h> bdxbdx:  hmm, looking
[19:11] <rick_h> bdxbdx:  yea, make sure that the clocks are good or you won't be able to do secure coms
[19:12] <rick_h> bdxbdx:  hmm, though this "ctrmg6 already saved" doesn't look right
[20:24] <bdxbdx> I feel like a newb today ... so I have another interesting scenario
[20:26] <bdxbdx> this morning, I was added to a "manual" juju controller that has the multi-cloud feature flag enabled
[20:26] <bdxbdx> the manual controller has a maas cloud defined
[20:27] <bdxbdx> I register my user and have access to the models that the admin have granted me access to
[20:27] <bdxbdx> 2 problems arise here that I'm unsure how to solve
[20:30] <bdxbdx> well maybe 1 main issue - I can't add maas credentials to the controller
[20:33] <thumper> bdxbdx: what happens when you try to add a maas credential?
[20:33] <bdxbdx> so thats the second issue
[20:34] <bdxbdx> thumper: I can't add the maas cloud because juju doesn't see one from my local client
[20:34] <bdxbdx> errr
[20:34] <bdxbdx> geh
[20:34] <bdxbdx> sorry, let me see if I can phrase this correctly
[20:35] <thumper> this is an area that is getting some love for 2.7
[20:35] <thumper> it is a bit weird right now
[20:35] <thumper> you need to add the maas cloud and credential locally before you can add to the controller I think
[20:36] <bdxbdx> the only way my local client can see the maas cloud is like so https://paste.ubuntu.com/p/DyMtvJ5BJZ/
[20:36] <bdxbdx> with the --controller flag
[20:36] <rick_h> bdxbdx:  yea, so things "upload" from the client to the controller so best to get the client setup first
[20:38] <bdxbdx> rick_h: do I need to add the pdl-maas-cloud shown in `juju clouds --controller dc--00` to my local config first ?
[20:38] <rick_h> bdxbdx:  you'll need that cloud locally so you can add a credential for it :(
[20:38] <bdxbdx> totally
[20:38] <rick_h> bdxbdx:  otherwise you can't add the credential
[20:39] <bdxbdx> totally, I think thats what I'm hitting
[20:42] <bdxbdx> adding the cloud to my local client config allowed me to add a credential, which allowed me to create a model
[20:43] <bdxbdx> I don't know why that was a trying experience ... seems straight forward and logical saying it out loud
[20:44] <rick_h> bdxbdx:  well it's not ideal as thumper mentions. If the cloud lives on the controller you shoulnd't need to worry about the local setup as well
[20:45] <rick_h> but it's not "that" crazy  currently :P
[20:45] <bdxbdx> totally, thanks all
[20:45] <rick_h> np, did the clock sync help and get you past the earlier issue?
[20:45] <bdxbdx> no
[20:45] <rick_h> :(
[20:46] <bdxbdx> I can curl the tools from the controller from any other box ... just not a maas deployed node
[20:46] <rick_h> oic
[20:46] <rick_h> networking in/out of the maas network?
[20:46] <rick_h> is there a bridge/router on the edge there?
[20:46] <bdxbdx> networking is all legit
[20:46] <bdxbdx> yeah we have verified the network details in and out of and around the nodes
[20:47] <rick_h> :/
[20:47] <bdxbdx> since the juju agent isn't making it
[20:47]  * rick_h has to run the boy to horseback but if you find something let me know or file a bug if you think it's borked
[20:47] <bdxbdx> I think my only way into one of the boxes is to get in by way of my maas user ssh key
[20:48] <rick_h> yea, you can setup ssh keys in maas that get put onto the machines for you when you provision
[20:48] <bdxbdx> just about to deploy a node that should get my ssh key via my user now that I have created a model with my own maas user creds
[20:48] <bdxbdx> totally
[21:01] <bdxbdx> the juju controller and the maas node that is failing to curl the tools are in sync
[21:02] <thumper> bdxbdx: the juju controller can't get the tools?
[21:02] <thumper> or the node brought up can't get them from the controller?
[21:03] <bdxbdx> thumper: the latter
[21:03] <thumper> bdxbdx: can you ssh to the node?
[21:03] <bdxbdx> yeah, I'm in it now
[21:03] <bdxbdx> https://paste.ubuntu.com/p/yC3KJNJWGd/
[21:03] <bdxbdx> ^ tried curling them manually and it seems to timeout
[21:04] <thumper> firewall somewhere?
[21:05] <bdxbdx> but I can curl them from another node right next to the node that can't curl them, same ip range, same switch
[21:05] <thumper> that is weird
[21:05] <bdxbdx> and from my local box over vpn, everywhere else I try to curl them from it works
[21:05] <bdxbdx> yeah ...
[21:06] <thumper> I'm sorry, but I have no idea
[21:06] <thumper> mismatched MTU?
[21:06] <thumper> that is one of the few things I know that can screw up some networking
[21:09] <bdxbdx> the bridge on the host has 1500 mtu ... the bridge the juju controller container is behind
[21:10] <bdxbdx> I bet thats it
[21:10] <bdxbdx> everything else has 9000
[21:10] <bdxbdx> sec
[21:11] <thumper> wow, my one bit of networking knowledge seems helpful
[21:34] <bdxbdx> thumper: that was it - just awesome! thanks!
[22:01] <thumper> bdxbdx: yay
[22:55] <ec0> it's always MTUs
[22:55] <ec0> everything is always MTUs