jtv1bigjools: still not seeing any decent way to test my virsh power script, or to make it quite trivial.05:21
=== jtv1 is now known as jtv
jtvOne thing I could do is allow the caller to override the virsh executable it uses, so that a test can inject “echo” instead.  But that leaves baggage in normal execution paths.05:22
jtvI could make it a power parameter, I guess.05:23
=== Ursinha` is now known as Ursinha
=== Ursinha is now known as Guest90314
bigjoolsjtv: sorry was caught up with things.  I don't think you can test the scripts.06:51
bigjoolsthe approach I took with the wol stuff was to test the templating and to test that the script returns with a 0 code.06:51
jtvThanks — I've got some things I can do now.06:57
bigjoolsjtv: since the scripts are intended to be customised, I think unit testing them is not useful07:01
bigjoolsthey should be QAed instead07:01
jtvI'd like to know that at least it makes some kind of syntactic sense to the shell.07:03
jtvWhich I can, actually, test to some extent.07:03
bigjoolsjtv: exit code!07:04
jtvAlas, no, not that easy.07:04
bigjoolsnot for all, no07:04
bigjoolsand we really don't want to start up VMs07:04
bigjoolsso like I said, I'd really leave it to QA07:05
bigjoolsthis is why I made this level of separation07:05
jtvWell there's one thing I can do in tests that makes it not start up VMs and yet exercises most of the script.07:06
jtvOne simple thing, that is.  Complicated things can do more, I'm sure.  :)07:06
jtvAnd there.  Just found a bug thanks to my test!07:08
jtvbigjools: I was just saying that we could stay on and chat about the anonymous-metadata requirement08:52
bigjoolsjtv: ah ok I have a call with gavin first, will call you right after08:52
rvbaallenap: I see you've got a call now, so just ping me when you will be available to talk about this "migration" problem.08:54
allenaprvba: Cool, ta.08:58
bigjoolsjtv: ok, wanna call?09:34
jtvbigjools: otp09:35
jtvw/someone else09:36
* bigjools is hurt09:37
bigjoolsyou said you'd only ever otp me09:37
bigjoolsjtv: I need to head off - maybe you want to talk to Daviey about the requirements for this?09:40
jtvYes, I think I will.09:40
jtvDaviey: Hi.  It's about the anonymous metadata access… I wanted to be clear about the purpose.  Is it _just_ for debugging?  Because several of the things that were said sort of suggested that it might be needed for commissioning.09:49
jtvWhich I hope is not the case, really.09:49
jtvBut if it's for debugging, we might be able to require run-of-the-mill UI admin authentication.09:50
Davieyjtv: so..09:51
Davieyjtv: We have some hardware that can't boot as maas can expect to..09:51
jtvI see.  This is indeed entirely different from the use-case we had before.09:51
DavieyTherefore, for 'demo / testing' if we can avoid having to push data to the node, we can get a working setup09:52
Davieyjtv: yeah, seems to fit the same model tho?09:52
jtvWait — I'd like to comment on those two lines separately.09:52
jtvAbout “seems to fit the same model,” I think admin auth makes more sense _for debugging_ than opening up the whole thing — whereas for difficult hardware it may not be very user-friendly.09:54
Davieyjtv: well, injecting a known user/pass is MUCH easier than a generated on demand oauth key09:54
jtvThe other thing is: what do you mean by “avoid having to push data to the node” exactly, so that I understand correctly?  Do you mean that we don't want to push the credentials for the metadata service to the node through its boot params?09:55
DavieySo.. using admin auth does help.09:55
jtvWe could probably have a separate key for this.09:55
Davieyjtv: we don't have access to boot params at deploy time.09:55
jtvBut it's a security hazard if it's open in production.09:55
DavieyCrappy hw09:55
jtvBut then how does the node even know where to get its metadata?  We push that in the same way.09:55
Davieyjtv: yeah, this is for debug / demo (closed network) workflow09:55
Davieyjtv: Yeah, knowing where the metadata service is = known constant09:56
DavieyIt's really injecting runtime generated auth is the problem09:56
Daviey(although, could be really smart and use avahi for auto detecting where the metadata service is... but really.. overkill.)09:57
jtvI don't suppose we could have a single, shared key and differentiate by MAC address?  Bit of a hack, and only good for a limited class of networks.09:57
jtvThen again, if you're running in the demo config…09:59
Davieythe mac address isn't normally exposed in the http request.09:59
jtvArgh!  How does the node even know how to identify itself?  It doesn't receive its system_id either!09:59
DavieySo.. the MAAS server would need to arp09:59
jtvMaybe the node could do it.09:59
DavieyX-FORWARDED_FOR: $(hostname) .. seems sane?09:59
jtvHow does it know its hostname at that stage?10:00
DavieyX-Forwarded-For: rather.10:00
Davieyjtv: well.. when using the model of dhcp allocated hostname.. MAAS is informed of the hostname, and the node knows it10:00
jtvDHCP-allocated hostname?  Isn't that the node telling the DHCP server what hostname it wants?  If so, it'd have to know first — quod non.10:02
Davieyjtv: hmm.. there are two models.. MAAS controls dhcp, and an existing dhcp10:02
Davieythis fits existing dhcp quite well.10:02
jtv Ouch ouch ouch you want to make it rely on a given DHCP setup as well?  That's just making things worse & worse.10:02
Davieyit's not all that bad IMO10:03
Davieyremember, this is a non-prodcution setup10:03
Davieyjtv: to contrast, this is how openstack does it.. https://github.com/openstack/nova/blob/master/nova/api/ec2/__init__.py#L25310:05
jtvSo you're talking about setting up a DHCP server with prepared leases with hostnames, for demo purposes?  It feels a lot like designing a whole feature specifically for just one demo.10:06
Davieyjtv: no, this also fits the debug model quite well.10:06
jtvOnly if you have the hostnames.10:07
Davieyi can do an out-of-band install of ubuntu server, apt-get install cloud-init, providing i installed with the hostname MAAS expects, we are GOLD10:07
Davieyjtv: the other solution is to do a reverse dns lookup based on the ip address, and post that.. That caters for MAAS controlling the dhcp10:08
Davieyclient side ^10:08
jtvBut why not have the client look up the mac address it uses for the metadata access?10:12
Davieyalso valid10:12
jtvIn fact it could just loop over its interfaces until it found a hit.10:13
Davieyjtv: technically, i'd say mac address lookup is more insecure :)10:13
Davieybut security isn't an issue here10:13
Davieyjtv: yeah, mac address seems reasonable :)10:13
* jtv refreshes his memory on that topic10:13
Davieyarp -na10:15
jtvNo I mean, ISTR MAC addresses were significant somehow to the EC2 metadata service.  Let me see if it matters to us at all.10:15
jtv(As you say, this is identification, not authentication — and yes, we'll still need to worry about how to fit this in with actual authentication)10:16
Davieyjtv: nah, mac address isn't interesting to the meta data service afaik.10:16
jtvYou're right.  We ditched that.10:17
jtvStill, it's one of the few things the node does know when it starts commissioning.10:18
jtvDaviey: how would we do enlistment on this hardware?10:45
jtv(So I get a picture of what process we'd be working towards)10:45
Davieyjtv: using the cmd line tool, maas-enlist10:49
jtvOK so MAAS knows about its MAC addresses anyway.10:50
jtvWill it need to have a custom(ized) metadata client?10:50
jtvI guess it will — hardcoded base metadata service URL10:50
jtvAnd it can embed its MAC address in the URL as well.10:50
Davieyjtv: sounds good to me.10:51
Davieyi think it will be a wrapper around cloud-init TBH10:51
Davieyrunning cloud-init post-boot isn't as straight forward as calling a command, sadly10:52
jtvDaviey: I think we'd have to make it a whole separate http tree.  Maybe a hidden /metadata/<version>/node/<mac>11:03
Davieyjtv: hmm, mac can't just be passed as a parameter through urls.py to the same function which defaults to None, and cecks django settings if METADATA_MATCH_MAC = True ?11:05
Davieyahh.. unauth'd url11:05
jtvYeah.  Want separate URL anyway, I think.11:05
Davieywould be nice to be able to reuse the same code..11:06
Davieybut i have every confidence you'd do the right thing :)11:06
jtvWell you have to get the URL from a different source anyway, right?11:06
jtvSo that might as well include a mac address.11:06
jtvAnd I take it the enlistment program sends all of the node's MAC addresses, in which case we don't even need to worry about which one we use.11:07
Davieymaas-enlist defaults to pushing all mac's11:09
jtvSo then whatever code picks up the metadata URL from your static setup will append one of its mac addresses as a path component, and presto.11:10
Davieysounds like a plan11:11
jtv(And it'll skip the oauth bit, but that much was obvious)11:11
jtvI can add a setting for just the dev/demo configs.11:11
Davieyjtv: you rock my world.11:12
jtvWell let's wait until it works.  :)11:12
jtvDaviey: I'll work it out in more detail tomorrow.  Until then I remain, your loyal servant &c.  :)11:27
Davieyjtv: thanks.. nn sir11:30
=== Guest90314 is now known as Ursinha
cheez0rhey folks, this isn't a dev question, but I'm struggling with MaaS/dnsmasq right now and I was hoping you might be able to point me in the right direction.17:02
cheez0rI'm trying to stand up a new MaaS, I add 11 nodes to the MaaS, but when I run juju bootstrap, I get an ssh error. When I run verbose, it is trying to connect to the hostname of the node, which fails to resolve.17:02
cheez0rShouldn't MaaS be adding the hostname to cobbler to add to dnsmasq automatically when the node is finished commissioning?17:03
cheez0rThe strangeness comes in where one of my 11 blades seems to resolve but the other 10 do not.17:03
roaksoaxcheez0r: can you ping the bootstrap's node hostname from where you are running juju?17:05
cheez0rroaksoax: no. that's part of the problem. It should have automatically been added via cobbler when I commissioned the node.17:07
cheez0rI'm trying to understand both why the dns hostname adds aren't working and where I could manually add them17:07
roaksoaxcheez0r: are you running an external dns server or are you using maas-dhcp?17:09
cheez0rusing maas-dhcp17:09
cheez0rbecause of that I was under the impression that MaaS would add hostname entries for each of the nodes as they were commissioned.17:09
roaksoaxcheez0r: is the machine where you are running juju, using the maas server as DNS server?17:09
cheez0ryes, should be17:09
cheez0rright- I'm following the howto at https://wiki.ubuntu.com/ServerTeam/MAAS/Juju#MAAS:_getting_started_with_Juju17:10
roaksoaxcheez0r: make sure that the machine were you are running juju from is using the maas server as DNS server (you probably have to do it manually)17:10
cheez0rhow can I do that- the node is set as 'ready' in MaaS, it's fresh out of commissioning. I'm running juju bootstrap from the MaaS node.17:12
cheez0rIt looks like a resolution from the MaaS node issue, not from the node itself17:12
cheez0rand none of the nodes resolve from the MaaS node except for one, for whatever reason.17:12
cheez0rThey were all added identically.17:12
roaksoaxcheez0r: I think I've hit the issue before but can't recall how to fix it but just editing /etc/resolve.conf and nameserver W.X.Y.Z17:17
cheez0rright, except that I don't know the IP addresses MaaS has handed out to my servers to manually configure their hostnames17:18
cheez0rit's resolving via dnsmasq; I just need to add the hostnames to the dnsmasq configuration17:18
roaksoaxcheez0r: that's added automatically, unless there's a bug in MAAS where it is not updating cobbler correctly17:18
roaksoaxcheez0r: try: sudo cobbler sync17:19
cheez0rright, which I think is the problem17:19
cheez0rI've done cobbler sync about a hundred times so far17:19
roaksoaxuhmmm i'll try to reproduce as soon as I can to further troubleshoot this17:19
cheez0rDo you know where the cobbler or dnsmasq configuration is located for the hostnames?17:19
cheez0rI can try manually adding them and see if it gets me past this point17:20
roaksoaxcheez0r: /var/lib/cobbler/cobbler_hosts17:20
roaksoaxcheez0r: what does that file show?17:20
roaksoaxcheez0r: does it show hostname/ip combination?17:20
cheez0rit's empty.17:21
roaksoaxcheez0r: so if you do: sudo cobbler system dumpvars --name node-XYZ(of whatever node) | grep dns17:22
cheez0rno output returned17:22
cheez0rsystem not found <hostname>17:22
roaksoaxcheez0r: so maybe MAAS is not setting the hostname17:23
cheez0rdoesn't matter if I use the hostnames I specified or the default17:23
roaksoaxcheez0r: you could try: sudo cobbler system edit --name node-XYZ --dns-name node-hostname17:23
roaksoaxand then sudo cobbler sync17:23
cheez0rok let me try that17:23
roaksoaxand try to ping it by hostname17:23
cheez0rhow can I get a list of nodes cobbler knows about?17:24
roaksoaxcheez0r: sudo cobbler system list17:24
cheez0rthat's really odd, they all seem to have the same mac address17:24
cheez0rnames are node-<stuff>-d485645878c817:25
cheez0rthe systems in that list seem to reflect the correct dns_name in the config it outputs17:26
cheez0rokay Im guessing this is a bug related to my specifying hostnames when adding nodes to MaaS17:27
roaksoaxyes apparently so17:27
cheez0rlet me delete and readd a node with no specified hostname and see what it does17:27
roaksoaxi'd have to setup a physical cluster to troubleshoot17:27
cheez0rnot asking you to do that, but thanks for the thought ;)17:27
roaksoaxcheez0r: i know :) I just need to make sure it works well :)17:27
cheez0rwell, I'm specifying amd64 arch and was specifiying a hostname in the format city_name-dc_name-enclosure#-blade#17:28
cheez0rmight be throwing errors on all of the hyphens or some such, but I dunno17:29
roaksoaxfor hostnames?17:29
roaksoaxcheez0r: hostnames don't accept underscores, but does accept hypens17:29
cheez0rno underscores in the actual names17:29
cheez0rlike paris-champselysee-enclosure1-blade1217:30
roaksoaxalright, yeah I think it might be realated to not updating cobbler correctly17:31
cheez0rwell I've got one node re-commissioning with the default hostname node-<MAC> so we'll see if that fixes it17:31
cheez0rno change- the newly recommissioned node with the default hostname is still not resolving.17:43
cheez0rcobbler_hosts is still empty17:43
cheez0rthe newly recommissioned node still shows up with a funky name with cobbler system list17:46
cheez0rre-commissioning with architecture set to i386 to see if that fixes anything17:53
cheez0rno change, still doesn't resolve17:58
SoekrisHello I've run juju bootstrap. This was good. I have seen apt-get running. But juju status doens work. I get ERROR Invalid SSH key. ssh ubuntu@hostname works18:02
SoekrisWhere goes it wrong at my setup ?18:02
cheez0rSoekris: the hostname resolves?18:02
cheez0ror are you ssh ubuntu@IP?18:02
Soekrischeez0r: it resolves18:05
Soekrisubuntu@s1-cl1-maas works18:06
Soekrisssh ubuntu@s1-cl1-maas works18:06
cheez0rhrm interesting18:06
cheez0rI have the same SSH issue but mine is because cobbler isn't adding the DNS names for some rason18:06
SoekrisI have configued dns and dhcp on a other server18:07
SoekrisIt's strange that something like so easy can be so difficult :D18:09
Soekrischeez0r: is it for production or for test purposel ?18:13
cheez0rkind of both18:13
Soekrisyou can quick make the entries in the /etc/hosts18:14
cheez0ryeah I think that'll be my workaround right now18:14
SoekrisStrange one of the two servers are showing in the juju status. But how it's a MAAS mistery19:48

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!