#maas 2012-06-04
<cheez0r> hey maas gurus, I could use some help. Anyone around who could spare a few minutes?
<cheez0r> that many people, eh?
<cheez0r> ;)
<ahasenack> well, you scare them with such an entrance :)
<cheez0r> something like that. I'm struggling with out-of-the-box MaaS integration for some reason, have been for a few days now.
<cheez0r> I'm doing a re-installation of the MaaS Server node right now to start from scratch again
#maas 2012-06-05
<bigjools> lesson learned: don't expect the development environment to work properly when the Ubuntu package is installed :(
<bigjools> jtv, so does the model for nodegroup look ok?
<jtv> Looking at it now.
<bigjools> thanking you
<jtv> (I don't think you need to define a manager if it's an empty class; you get a close equivalent as the default)
<bigjools> jtv: the template has it
 * bigjools shrugs
<bigjools> will need one later
<jtv> On which note, kudos for putting this into its own module from the start.
<bigjools> :)
<jtv> How to the ip range and the netmask fit together?
<bigjools> the netmask is what the DHCP server tells the client
<bigjools> the range is the IP addresses it can assign
<bigjools> typically they fit inside the mask
<jtv> I guess the distinction isn't terribly useful yet, but adding them later would only cause pain.
<bigjools> they will be needed quite soon actually
<jtv> OK
<bigjools> since they are values needed to write DHCP server config
<jtv> One other thing that jumps out at me is the unique constraints on api_token and api_key.  Isn't that something we want to leave completely up to the implementation?
<bigjools> actually can you check the architecture doc after this and tell me if my progressive enhancement plan looks ok?
<bigjools> ultimately we want auto-enlistment of new workers
<jtv> That sounds dangerous.
<bigjools> how?
<bigjools> actually don't answer
<bigjools> I'm not worrying about it yet
<jtv> On your head be it.  :)
<jtv> By the way, there's a piece of nastiness in Django where a text field can be allowed to be null, but it won't actually give you any sensible behaviour.  Instead, the operative question is whether the field will be allowed to be empty.
<jtv> I think you express that with a constructor parameter âblank.â
<bigjools> yeah, None equiv to empty or something?
<jtv> I'm not sure.  Once people go down that route, there's no telling how far they'll go.
<bigjools> the only field that I'm allowing none or blank is "name"
<jtv> What I meant is that you're passing null=False for api_key, which to django doesn't seem to mean all that much in this case.
<jtv> I don't know what the default is, but if it's True then this will happily work around and against your wishes like a sailing boat tacking against the wind, and if it's False then the default for ânameâ shouldn't work.
<jtv> But that's all just little details.  I don't see any real problems off the top of my head.  Where's the plan you wanted me to look at as well?
<bigjools> https://docs.google.com/a/canonical.com/document/d/1PtFKcizW3bmP3QBv771tSGMO4VaCiLkWlr-oDzn-FqU/edit?pli=1#
<bigjools> see page 3
<bigjools> basically I want to have a module that generates DHCPD config
<bigjools> and we can re-use it in a standalone excecutable for the single worker case
<bigjools> or on the worker as a Task when required
<jtv> Reuse ftw
<bigjools> ok
<bigjools> now Firefox, why won;t you change tabs when I click on one?
<jtv> Vell, ze first sing ist zat zey must _vant_ zu change.
<bigjools> zey clearly don't
<jtv> Interesting.  Vy don't you start by telling me about your mozzer?
<bigjools> my mozzer bites?
<jtv> Bites?  I hope not!
<jtv> If your mozzer vere like a dog, zen ssientifikally you must be like a dog also, nein?
<bigjools> ah FF had decided to put a "do you like this cookie?" dialog *underneath* everything else
<bigjools> sigh
<jtv> On a note unrelated to your mother, we're getting rather a lot of the mention-an-imported-identifier-so-it's-not-considered-unused pattern.  Maybe we should just define an empty function with a name like never_mind()?
<bigjools> mozzer bites, from mozzer fliez
<jtv> ?
<bigjools> zey bite zu and zuck ze blud
<bigjools> jtv: ideally I'd like flake to stfu
<bigjools> there's a directive to make it ignore those iirc
<jtv> That's the idea: you call a function whose name tells the reader âthis is unused but that's fine,â and flake shuts up because you're doing something with the identifier.
<jtv> def not_actually_used(*args):
<jtv>     """Arguments are deliberately left unused.  Pass them to this function to suppress lint warnings about it."""
<jtv> Now you can suppress the warning for foo by calling not_actually_used(foo)
<bigjools> ugly hack though :()
<jtv> I don't think so â it's just giving an explicit name to existing practice.
<jtv> Call it suppress_warning_if_unused if you prefer.
<jtv> You don't need to care how it's implemented when you call it.  And the implementation as such will be literally as clean as any function's!
<bigjools> flake needs to be fixed really
<bigjools> if it's in the __all__ then don't complain
<bigjools> jtv: please bless https://code.launchpad.net/~julian-edwards/maas/node_group_model/+merge/108678
<bigjools> well, shall I do that hack first
 * jtv casts his sacerdotal eye over https://code.launchpad.net/~julian-edwards/maas/node_group_model/+merge/108678
<bigjools> it's not changed since you last looked, I just added a commit msg :)
<jtv> And you added NodeGroupManager to __all__, which seems redundant.
<jtv> Or maybe you did that before.
<bigjools> it was there before
<jtv> It probably should not have been.  We have one canonical way of referring to the manager.
<jtv> Hi rvba
<bigjools> jtv: ok
 * jtv growls at django docs
<bigjools> hello roaksoax
<bigjools> errr hello rvba
<jtv> specifically this part: Â«When using the Oracle database backend, theÂ null=TrueÂ option will be coerced for string-based fields that have the empty string as a possible value, and the valueÂ NULLÂ will be stored to denote the empty string.Â»
<jtv> So the ânullâ parameter defaults to False, except on one database which doesn't really do nulls anyway.
<jtv> Heyyy, there's rvba again :)
<rvba> Hello bigjools.  Hi jtv.
<jtv> bigjools: you have my blessing.  Go forth and sin some more.
<bigjools> jtv: thank you my child
<jtv> Or at least, that's what it always sounded like to me.  I have may have gotten some of the details wrong.
<bigjools> or is that, thank you father
 * jtv is reminded of Blackadder and his father in the archbishop episode
<jtv> rvba: I already made the same suggestion about leaving out the default manager class, but I was hoping to use it against Julian next time he thinks my branches are too big.  :)
<rvba> jtv: haha
 * jtv searches for coffee, which he hasn't had all day
<jtv> Hitting the office before the Ozzies wears one out.
<jtv> bigjools: do we have data that supports the single-DNS design?  I thought we didn't.
<bigjools> jtv: it's largely irrelevant actually
<bigjools> because it has a scaling solution of its own
<jtv> Ah
<jtv> rvba: would it be bad in any way to import django settings in a urls.py?
<jtv> Argh.  Did I break the build? I get UserProfile test failures now!
<rvba> jtv: In an application's url.py it's ok.  It's already done in src/maasserver/urls.py.
<jtv> OK
<jtv> Hmm Jenkins seems to blame bigjools for the breakage.
<rvba> Yep.
 * jtv stumbles towards coffee to shake off his dÃ©jÃ  vu.
<jtv> Hmmâ¦ I wonderâ¦
 * jtv tries something
<rvba> Hum, all tests pass locallyâ¦
<rvba> Well, I only tried the faulty tests.
<rvba> But they pass.
<jtv> Wrong TestCase class, I suspect.
<jtv> (I'm already running tests to confirm)
 * rvba runs the whole test suiteâ¦
<jtv> Don't bother.  I've already got a branch that fixes this & a bunch of other small things.
<rvba> ok, cool, thanks jtv.
<bigjools> I swear I ran everything
<bigjools> WTF
<jtv> Would anyone care to review?  Urgent urgent urgent test breakage urgent: https://code.launchpad.net/~jtv/maas/test-before-landing-bitch/+merge/108697
<bigjools> looking
<rvba> jtv: I'll take it.
<jtv> Thanks.
 * bigjools wins
<jtv> Work it out amongst yourselves.  :)
<bigjools> jtv: also, blow me!
<jtv> >-O
<jtv> !?  The diff doesn't show all my changes.
<ubot5> jtv: I am only a bot, please don't think I'm intelligent :)
<jtv> Did I commit wrongly?
<jtv> ubot5: stay out of this
<ubot5> jtv: I am only a bot, please don't think I'm intelligent :)
<jtv> Ah no, this is the whole thing.
<jtv> Well here we are then.
<jtv> And I think I brought that same question up in pre-review.  :)
<rvba> bigjools: skip_check is exposed to the user because it's used in the API.
<bigjools> ah
<rvba> bigjools: that's the whole point of 'skip_check', it has to be usable by the user to bypass all the checks if he wants to.
<jtv> Maybe it should even be off by default then... so that nobody assumes that the check always happens.
<bigjools> rvba: ok it makes more sense now
<rvba> bigjools: Cool.
<rvba> jtv: well, I don't think that would be wise, the check is also a guid for the user to provide the right parameters.
<jtv> I could imagine the validity of a particular power_parameters object changing without that object itself changing.
<rvba> guide*
<rvba> jtv: not sure I followâ¦
<jtv> Damn.  I had something in mind back then and didn't write it down.
<jtv> There may be scenarios (not in the code now perhaps, but at some point) where you set a node's power parameters, and they are valid; and then you make some other change, and now that node's power parameters are no longer valid.
<rvba> That can already happen if you change the default parameter and power_type is not set on the node.
<rvba> the default power_type that is.
<jtv> Yes, but would you go through the check in that case?
<jtv> Ah, default power type.
<rvba> Yes, the check would be done.
<jtv> What I'm driving at is that, in cases like that, the check provides no guarantee.  Which is fine, but it may not always be obvious.
<jtv> Also, I've seen cases in the past where the logic around such a parameter shifted and grew and its effective meaning changed subtly, and it led to no-go zones in the code.
<jtv> By which I mean pieces of code you can't maintain effectively, not dead code.
<jtv> Not saying that any of us would make that mistake, but I'm reasoning from the AK-47 / SA-80 perspective here.
<rvba> Well, in this instance I think it would be quite easy to maintain that code if the requirements change for this.  Mainly because all the logic is fairly well encapsulated in a field.  It's really decoupled from the usage of the field.
<rvba> What I mean is that we could easily create another field with another logic and replace the old field by the new one.
<rvba> As long as the storage is compatible (JSON blob).
<jtv> Actually, where does the skip_check parameter go?
<rvba> It's used by the field to decide whether or not validation should be performed.
<rvba> It's not stored.
<jtv> Okay, but where is it used?  I see it tested, and I see it documented, but where does its value get read?
<jtv> I don't see it in the current trunk, nor do I see it being read anywhere in the diff.  Prerequisite branch?
<rvba> In a pre-req branch (where the custom field/widget are defined): https://code.launchpad.net/~rvb/maas/multifield/+merge/108188
<jtv> Ah.  And its mentioned quite a lot there I see!
<jtv> Then messing with it would be too much work for this branch anyway, so I'll stop worrying about it.
<jtv> I think I'll worry about people setting hostnames like âlocalhostâ instead.
<rvba> bigjools: did you change this branch's status manually?
<rvba> The 'multifield' branch.
<bigjools> rvba: I put an email directive in to set my vote to approved
<bigjools> ah bollocks
<rvba> bigjools: not sure what happened, it's now mergeâ¦ even before I had time to commit my changes
<rvba> merged*
<bigjools> I set the MP status rather than my vote
<bigjools> sorry
<bigjools> I confused "status" and "merge"
<rvba> bigjools: ok, no worries, I'll fix that in the follow-up branch.
<bigjools> ok, sorry for the trouble
<rvba> Well, actually I think the problem will fix itself since I pumped the new revisions into the followup branch already.
<rvba> hum, no, it doesn't seem to be workingâ¦
<jtv> Good feeling: putting my branch up for review at 18:03.  I think I'll be off.
<rvba> jtv: I'll review it later this afternoon.
<jtv> Thanks!
<rvba> nn jtv.
<jtv> nn!
<roaksoax> smoser: ok, so I've hit that issue, and I've seen people hitting that issue
<smoser> roaksoax, so.. the easiest way to recover there is to hav the installer set a password for the ubuntu user
<smoser> so the person can get into the instance to look around
<roaksoax> smoser: I can't recall how I managed to "fix" it but I'm guessin it has something to do with DNS being different from what is given, to what it is stored in maas
<roaksoax> smoser: right, but wasn't that the idea of having the ability to add anSSH key to MAAS?
<cheez0r> I'm having this issue right now, where I've bootstrapped juju, but get "Invalid SSH key" from juju status, and I have no way (no l/p, no ssh key) to access the system it's running on to troubleshoot/fix the issue
<smoser> roaksoax, well, yes.
<roaksoax> smoser: ^^
<smoser> but that ssh key is pulled in by cloud-init
<smoser> and that is failing
<smoser> which is why you're not getting in
<roaksoax> smoser: isn't the cloud-init failing due to the timezone being different from maas client/server?
<smoser> so setting a password is an easy way to fix "and I have no way (no l/p, no ssh key) to access the system its running on to trubleshoot/fix the issue"
<smoser> roaksoax, well,t hat will fail first in commissioning state
<smoser> but yes, it is a potential point of failure, and admittedly it could also have such a bad clock that time was lost between commissioning and deployment
<smoser> (but unlikely)
<smoser> cheez0r, are you willing to debug a bit?
<cheez0r> smoser: gladly.
<cheez0r> I've been stuck on this for a while.
<roaksoax> pmatulis: in your case, are you guys sstill using the wrt router as DNS/DhCP server?
<pmatulis> roaksoax: no, all is on the maas server
<smoser> cheez0r, in /var/lib/cobbler/kickstarts you will have a file named 'maas.preseed' (or somethign to that effect)
<smoser> (sorry, just going from memory)
<smoser> then...
<smoser> you want to set a password in there
<smoser>  d-i   passwd/user-password-crypted  password $6$.1eHH0iY$ArGzKX2YeQ3G6U.mlOO3A.NaL22Ewgz8Fi4qqz.Ns7EMKjEJRIW2Pm/TikDptZpuu7I92frytmk5YeL.9fRY4.
<smoser> tha tis 'ubuntu'
<roaksoax> pmatulis: i see. smoser i think I know what the issue is
<smoser> clearly you dont want this for production, but after a re-installation, you should be able to log in as 'ubuntu:ubuntu'
<roaksoax> smoser: both pmatulis and cheez0r are using cobbler's dnsmasq and I think maas is not correctly updating what it has to update
<roaksoax> smoser: err maas-dhcp
<cheez0r> s'ok with you if I grab my own crypted password?
<roaksoax> cheez0r: yeah go for it
<cheez0r> ok done
<cheez0r> then re-commission the nodes?
<pmatulis> roaksoax: i also noticed the server's resolv.conf was populated with 127.0.0.1 for some reason.  dns was broken and i had to hand-edit the file to get anywhere
<smoser> cheez0r, re provision
<smoser> cheez0r, or, were you failing in commission stage?
<roaksoax> pmatulis: that's foundationschanges to how resolvconf is being handle now
<roaksoax> pmatulis: so aaaall dns clients will have 127.0.0.1
<smoser> pmatulis, resolv.conf should probably have 127.0.0.1, as you'll be running dnsmasq
<cheez0r> I'd succeeded with commissioning, the nodes were in "Ready", but when I run juju bootstrap then juju status, I get the Invalid SSH Key error.
<cheez0r> so I guess just juju destroy-environment and juju bootstrap and see what happens?
<smoser> cheez0r, yeah
<smoser> and at the point when it re-createds
<smoser> then you can try to ssh in as 'ubuntu:ubuntu'
<smoser> and we can poke around a bit.
<cheez0r> Also, my nodes are all configured for WOL but for some reason start node in the MAAS GUI isn't doing anything, but we can skip that for now
<smoser> yeah.
<cheez0r> I figured out that poweron sidesteps that
<smoser> WOL is often buggy anyway
<smoser> but that is a separate issue.
<smoser> use sneaker power now.
<cheez0r> No doubt.
<smoser> :)
<cheez0r> ILO for the win :p
<pmatulis> roaksoax, smoser: i'm aware of the resolvconf stuff but somehow i need to configure dnsmasq to talk to my actual nameserver
<smoser> pmatulis, well, that should happen.
<smoser> it does potentially point to an issue with dhcp not giving it a dns server
<pmatulis> smoser: well, my sever has a static address
<smoser> pmatulis, i don tknow how that would be supported in maas.
<smoser> but maybe i'm missing something.
<smoser> oh.
<smoser> i see.
<smoser> a non-maas system.
<pmatulis> smoser: i'm talking about my maas server
<roaksoax> pmatulis: you should just be able to replace /etc/resolv.conf with a file rather than being a symlink
<roaksoax> pmatulis: http://www.stgraber.org/2012/02/24/dns-in-ubuntu-12-04/
<roaksoax> that would help then ^^
<pmatulis> roaksoax: that's true, but is that the proper way?
<roaksoax> pmatulis: if it is a static addres, then you can specify the dns servers on /etc/network/interfaces
<pmatulis> roaksoax: i did that already
<smoser> pmatulis, dns-nameservers 12.34.56.78 12.34.56.79
<smoser> http://askubuntu.com/questions/32875/network-manager-discards-changes-to-etc-resolv-conf-on-boot
<smoser> that is the correct way to configure /etc/resolv.conf with static networking in the presense of resolvconf.
<roaksoax> yeah
<smoser> sadly 'dns-nameservers' is not even mentioned in 'man intefaces' :-(
 * smoser files a bug
<roaksoax> pmatulis: stgraber's blog has more info about it too
<roaksoax> smoser: it is mentioned on man resolvconf though
<cheez0r> I think you have to man resolvconf to find it
<pmatulis> roaksoax: well, dns-nameservers is already done
<roaksoax> yep
<cheez0r> hey smoser/roaksoax, that worked. I'm in the juju system now.
<roaksoax> cheez0r: are there any juju logs ? (/var/log/juju/*)
<cheez0r> didn't even have to re-provision, just reboot the machine
<cheez0r> yes
<roaksoax> cheez0r: so you can now juju status?
<cheez0r> client environment stuff, zookeeper_init, juju.agents.machine Machine agent started
<roaksoax> cheez0r: wait wait, juju status now works after a reboot?
<cheez0r> well, I had to fix some ssh issues, but yes
<cheez0r> the ssh fingerprint changed for some reason
<cheez0r> now it shows machines: 0: agent-state: running
<smoser> cheez0r, well, the ssh fingerprint *should* change
<smoser> you re-installed the system.
<smoser> oh.
<smoser> i just saw above.
<roaksoax> smoser: he didn't
<smoser> but that is still kind of expected.
<roaksoax> yeah
<smoser> cloud-init re-generated your ssh keys i suspect
<cheez0r> yeah I noticed that in the boot scroll
<smoser> as it found the metadata service this time.
<cheez0r> if this cold boots again, this happens again?
<cheez0r> kind of makes it sketchy for ssh key mgmt
<smoser> cheez0r, no. only once per "instance" (install)
<cheez0r> ah, ok
<roaksoax> cheez0r: you only have 1 network interface in the maas server right?
<cheez0r> no, two, but I've got routes set up properly and dpkg-reconfigure maas done, as well as maas-dhcp configured for the correct parameters
<roaksoax> uhmm
<smoser> cheez0r, do you hchok... so cloud-init on reboot found the maas server.
<smoser> so we have ot fiturer out why it did not on the first boot
<smoser> hoepfully that information is in /var/loog/maas (or maybe it goes to /var/log/apache/...)
<smoser> but we should look at logs there around the timestamps
<smoser> (or just try to reproduce)
<smoser> i have no idea what 'do you hchok' meant.
<cheez0r> me either, so I'll say no, I don't.
<smoser> :)
<cheez0r> ok so I was able to successfully deploy mysql and wordpress charms, which is progress.
<cheez0r> smoser: /var/log/maas on the maas node or the juju system?
<smoser> maas system
<cheez0r> lots of data in /var/log/maas/maas.log on the maas node.
<smoser> want to pastebin it?
<roaksoax> we would also need the juju logs for the bootstrap
<smoser> also /var/log/apache/error.log
<roaksoax> s/for/of
<cheez0r> hrm, to the end, lots of "No matching node is available"
<cheez0r> towards the beginning, 'no registered public keys'
<roaksoax> cheez0r: pastebins would be very helpful :)
<cheez0r> http://pastebin.ubuntu.com/1025043
<smoser> cheez0r, did you look at apache error.log ?
<smoser> that has been where there were hints before
<cheez0r> no errors other than favicon.ico
<smoser> :-(
<roaksoax> cheez0r: can you please pastebin the juju logs from the bootstrap node?
<roaksoax> rvba: ping
<roaksoax> rvba: rvba NodesNotAvailable: No matching node is available. --> how do we check for matching nodes, by MAC address?
<cheez0r> apache access.log: http://pastebin.ubuntu.com/1025046
<cheez0r> juju logs from bootstrapped machine: http://pastebin.ubuntu.com/1025049
<roaksoax> cheez0r:
<roaksoax> cheez0r: there
<roaksoax> 's ponly 1 log for juju?
<roaksoax> ah nevermind
<cheez0r> sorry, should have delimited that better
<roaksoax> so while there doesn't seem to be an issue with juju for the bootstrap it does seem to be an issue when deploying wordpress and mysql
<cheez0r> well, only one of the two nodes I have is successfully resolving via DNS.
<cheez0r> I'm using dnsmasq on the MAAS node to do resolution but for some reason cobbler isn't successfully populating the DNS entries.
<roaksoax> yeah i need to setup a test environment
<roaksoax> cause I have no idea what might be wrong
<cheez0r> hrm
<cheez0r> second node won't let me log in.
<cheez0r> let me cold boot it and see if it fixes the same way the other did.
<cheez0r> I've got to break away for a bit here in a few minutes but let me know how else I can help
<cheez0r> My ultimate goal is to get openstack running on this system.
<roaksoax> ok cool
<roaksoax> cheez0r: i'll try to reproduce locally and see what I find
<cheez0r> thanks brother
<cheez0r> the only strangeness about my config is that my MAAS Node has a public IP and a private IP, and that it's got external, upstream DNS servers to resolve against.
<cheez0r> all of the DNS for within the MAAS is supposed to come from the MAAS Node.
<roaksoax> exactly
<roaksoax> cheez0r: so DEFAULT_MAAS_URL = "http://PRIVATE-IP/" so that should be also set in /etc/maas/maas_local_settings.py when you dpkg-reconfigure maas
<pmatulis> roaksoax: fyi, i redid another node and 'juju status' now gives 'ERROR Invalid host for SSH forwarding: ssh: Could not resolve hostname node5: Name or service not known'.  where to troubleshoot name resolution in this context?
<roaksoax> pmatulis: well the maas server should be the one who knows how node5 is
<roaksoax> s/how/who
<roaksoax> pmatulis: are you running juju from the maas server?
<pmatulis> roaksoax: doesn't know about it.  note that i changed the node name within the maas UI
<pmatulis> roaksoax: yes
<roaksoax> pmatulis: so 1 would be to check whether the hostname has been updated in cobbler
<roaksoax> pmatulis: 2, would be to make sure the maas server can resolve the hostnames for the nodes
<roaksoax> locally
<roaksoax> 3. that should make juju be able to resolve the hostnames
<pmatulis> roaksoax: sure, the question is how
<roaksoax> pmatulis: /etc/resolv.conf add maas server IP as dns server
<pmatulis> roaksoax: it's already pointing at localhost
<roaksoax> pmatulis: 127.0.0.1?
<roaksoax> or 127.0.1.1?
<pmatulis> roaksoax: the latter
<roaksoax> pmatulis: what if you make it to be the IP of the maas server
<pmatulis> roaksoax: sorry, the former
<pmatulis> 127.0.0.1
<roaksoax> pmatulis: make it the MAAS IP address
<pmatulis> roaksoax: so do not use the resolvconf stuff, use a real file (not symlink)?
<roaksoax> pmatulis: eitherway, thething is that juju should e able to resolve dns provided *by* the maas server
<roaksoax> so resolconf provides its own dns with dnsmasq, while maas also provides a dns with dnsmasq
<roaksoax> so you can setup resolvconf to change the dns server
<roaksoax> so you should be able to use the same IP addres (for dns) as the MAAS server
<cheez0r> roaksoax: it is set properly in /etc/maas/maas_local_settings.py
<roaksoax> smoser: pmatulis cheez0r eok so I think I know what's wrong
<cheez0r> the suspense is killing me here
<cheez0r> :p
<roaksoax> basically, it seems that we might have to specify the IP address for the system because onbviosly, in order to be a dns entry we need to know the IP address for that DNS entry
<roaksoax> so since dnsmasq has a dns entry but no IP address for a cobbler system, then it doesn't create the entry
<cheez0r> isn't dnsmasq serving the dhcp IP?
<cheez0r> or does maas-dhcp install dnsmasq and some dhcpd?
<roaksoax> cheez0r: right, so here's the deal
<roaksoax> If dnsmasq has a hostname for Node0, but it doesn't know what IP address to assign yet (as there's being no requests yet), but it knows about Node0 mac address
<roaksoax> should it resolve Node0 once dnsmasq assigns that IP address?
<roaksoax> that's my question
<cheez0r> it should, but what mechanism would update the dnsmasq entry with the IP Address?
<cheez0r> Perhaps we should be statically assigning dhcp entries so that the DNS can be mapped?
<roaksoax> yeah that's the common fix
<cheez0r> Add a static assignment for the MAC into the dhcpd configuration, then populate that assignment into the dns
<roaksoax> cause, either way, whne you deploy a DNS server you *have* toassign an IP address for that dns name, right?
<cheez0r> for what DNS name? That of the node?
<roaksoax> in general terms
<roaksoax> when you have a DNS solution, in order for you to assign nodeX a hostname being handled by DNS, you need to know the hostname's IP address, correct?
<cheez0r> right, so a node is commissioned. The MAC is entered into MAAS. The MAC is statically assigned a DHCP IP and the DHCP server is configured as such. The IP and nodename are then pushed into the DNS server.
<cheez0r> correct
<cheez0r> that's the only workaround I see unless there's some mechanism called after the node boots and pulls a DHCP IP to then update the dns entry.
<roaksoax> so yeah the flow in maas should be, once it PXE boots, update MAAS (cobbler really) with the obtained IP address to match dns/ip
<roaksoax> smoser: ^^
<cheez0r> And/or a package designed to watch the DHCP allocation and match up MAC vs. MAC from cobbler and then map the nodename to DNS entry
<smoser> sort of reading.. wasn't following.
<smoser> essentially the nodes did not have dns (or reverse dns) entries?
<roaksoax> yeah
<roaksoax> because cobbler doesn't know what IP address was assigned for that node, so consequently, dnsmasq doesn't know what hostname to match with what IP address
<roaksoax> smoser: I have a couple ideas of how to fix this. 1. is using something like # Always set the name of the host with hardware address
<roaksoax> # 11:22:33:44:55:66 to be "fred"
<roaksoax> #dhcp-host=11:22:33:44:55:66,fred
<pmatulis> isn't this what ddns is for?
#maas 2012-06-06
<cheez0r> roaksoax: smoser: would any of the other node commissioning processes (booting the nodes from a CD and enrolling them in the MAAS for instance) sidestep the issue I'm experiencing with DNS name additions for commissioned nodes?
<cheez0r> I need a working process so that I can move forward- one node has been successfully added to DNS but the others I add do not get added no matter what I've tried.
<smoser> cheez0r, i have to defer to roaksoax
<cheez0r> smoser: I hear ya, I'm just stuck and need to be moving forward.
<cheez0r> My main issue is that I don't know how to manually add DNS entries to dnsmasq- I can't find where it successfully populated the one node that works so that I can manually add the ten others I've got in my MAAS.
<smoser> understanable
<cheez0r> Any pointers there?
<smoser> really, i have to defer to roaksoax sorry.
<cheez0r> k thx
<pmatulis> cheez0r: one feature of dnsmasq is the ability to use the host's hosts file.  not sure if that can help here
<cheez0r> hrm, ok. I'll play with that, thanks.
<roaksoax> cheez0r: yeah, you could do that or maybe in cobbler_hosts specify host/ip
<pmatulis> roaksoax: as a summary, name resolution of the nodes is not working?
<roaksoax> pmatulis: it is, but you need to specify an IP address for the particular system
<cheez0r> I am doing the following: static IP assignments in /etc/dnsmasq.conf and static hostname mappings in /var/lib/cobbler/cobbler_hosts
<roaksoax> cheez0r: what I would do is to edit all the systmes and add an ip-address in cobbler
<cheez0r> can I just edit the .json files directly?
<pmatulis> roaksoax: meaning the resolution configuration is done manually, right?
<roaksoax> cheez0r: pmatulis you'd have to do it manually yes
<roaksoax> cheez0r: you could do something like this: http://pastebin.ubuntu.com/1027131/
<roaksoax> cheez0r: and run sudo cobbler sync after that
<cheez0r> schweet, you're the man
<cheez0r> hrm. I still can't run cobbler system dumpvars "nodename" successfully
<roaksoax> cheez0r: sudo cobbler system dumpvars --name "node-XYZ"
<cheez0r> that works, thanks
<adam_g> is support for deploying ubuntu releases other than precise on the roadmap?
<roaksoax> adam_g: it should be
<roaksoax> adam_g: we were talking about it this morning with Daviey and smoser
<adam_g> ah, okay
<adam_g> im currently just assigning the 'precise-x86_64' profile to the desired distro i want
<roaksoax> adam_g: yeah I guess for now we should be doing that
<roaksoax> for now until we have it upstream
<Daviey> adam_g: Yeah, i didn't realise it wasn't currently possible.. :(
<adam_g> Daviey: i'm not sure what the limitation or blockers are on the maas side, but its currently hard-coded in the juju maas provider to only support precise
<Daviey> adam_g: yeah, when it was outlined to me.. i thought 'oh yeah!'..
#maas 2012-06-07
<pmatulis> ping me please if anyone knows why i cannot get past this.  ssh key added via web ui.
<pmatulis> $ juju -v status
<pmatulis> 2012-06-06 22:49:17,153 DEBUG Initializing juju status runtime
<pmatulis> 2012-06-06 22:49:17,162 INFO Connecting to environment...
<pmatulis> 2012-06-06 22:49:17,227 DEBUG Connecting to environment using node-e4115b138176.local...
<pmatulis> 2012-06-06 22:49:17,227 DEBUG Spawning SSH process with remote_user="ubuntu" remote_host="node-e4115b138176.local" remote_port="2181" local_port="37322".
<pmatulis> 2012-06-06 22:49:17,312 ERROR Invalid SSH key
<pmatulis> .
<pmatulis> disregard.  i think i know what's wrong
<bigjools> pmatulis: do tell, for future reference
<pmatulis> bigjools: well, i've been having a lot of WOL trouble on my nodes.  i re-read the docs (wiki) and it does say that after 'juju bootstrap' WOL is supposed to kick in so ubuntu gets installed, *then* i do 'juju status'
<bigjools> yeah, it takes a while to install
<pmatulis> but i don't even get that far.  WOL doesn't work for me
<bigjools> that's a bizarre error from juju status then
<pmatulis> unless i'm not reading the docs right, does WOL kick in after 'juju bootstrap' or does the OS get installed right away?
<pmatulis> and then WOL
<bigjools> bootstrap will power up a node and then install ZK on it
<pmatulis> so by power up you mean WOL?
<pmatulis> and what is ZK?
<bigjools> WoL if you are using it yes.  ZK is zookeeper
<pmatulis> k
<bigjools> well, the OS is installed first
<bigjools> can take an hour or so :(
<bigjools> I got it down to 10m with an SSD
<pmatulis> an hour??
<pmatulis> you have SSD on your nodes?  what h/w is that?
<bigjools> the dev environment on my laptop :)
<pmatulis> ah
<bigjools> I use VMs to test
<pmatulis> i didn't realize you could do that
<bigjools> yeah it needs some trickery to set up
<pmatulis> feel like sharing?
<bigjools> there's a vdenv directory in the source tree and it contains a readme
<bigjools> it pokes stuff directly into cobbler
<bigjools> you can use that or use it as inspiration
<pmatulis> thanks.  i'll consider that
<pmatulis> so the server team wiki says 'juju bootstrap' will install ubuntu on a node yet after following the doc very closely all my nodes in 'Ready' state (prior to bootstrap) already have ubuntu installed
<pmatulis> going ahead anyway, 'juju bootstrap' does not 'boot the nodes' (why plural?) and says if WoL is not working i should reset the power manually, which i did but to no avail.  nothing seems to happen and 'juju status' still give errors:
<pmatulis> 2012-06-07 08:40:55,867 DEBUG Initializing juju status runtime
<pmatulis> 2012-06-07 08:40:55,875 INFO Connecting to environment...
<pmatulis> 2012-06-07 08:40:55,992 DEBUG Connecting to environment using node-e4115b138176.local...
<pmatulis> 2012-06-07 08:40:55,993 DEBUG Spawning SSH process with remote_user="ubuntu" remote_host="node-e4115b138176.local" remote_port="2181" local_port="47339".
<pmatulis> 2012-06-07 08:40:56,690 ERROR Invalid SSH key
<roaksoax> pmatulis: is this the same cluster alexmoldovan was using yesterday?
<pmatulis> roaksoax: yup
<roaksoax> pmatulis: so I believe he got them installed without using juju
<roaksoax> to try to gfix the wol issue
<pmatulis> roaksoax: yes, like i said above, ubuntu gets installed on all nodes
<roaksoax> pmatulis: right, but juju bootstrap only selects 1 node and sends a message to only 1 node and that's it
<roaksoax> pmatulis: it doesn't WoL all the nodes
<pmatulis> roaksoax: that's fine, it still doesn't work
<roaksoax> pmatulis: is cloud init correctly importing the keys on the node?
<pmatulis> roaksoax: no idea
<roaksoax> pmatulis: well that might be the problme then
<pmatulis> roaksoax: how to check?
<roaksoax> pmatulis: ssh into the node and check cloud-init logs in /var/log
<pmatulis> roaksoax: credentials?
<roaksoax> pmatulis: did you guys set up an SSH key inMAAS?
<pmatulis> roaksoax: yes
<roaksoax> smoser: MAAS still has the bug for which it doesn't update cloud-init meta-server right?
<roaksoax> pmatulis: so if you added a public ssh key in MAAS, then you should be able to ssh using that (from the node the private key is of course)
<smoser> roaksoax, i dont know. we saw that in the ephemeral (commissioning) environment
<smoser> but surely it does it on deployment
<smoser> as because up until the deployment request, the user to do that wasn't yet known
<roaksoax> smoser: right, but remember that during the demo, if we added the ssh public key in maas *after* enlisting a node, once it got deployed, it didn't obtain the ssh key
<pmatulis> roaksoax: after grepping syslog for dhcp assignment (ip address), dns is broke, and trying ssh i am prompted for a p/w.  is the node user 'ubuntu'?
<roaksoax> so we had to add the public key to maas before enlistment
<smoser> roaksoax, that was the commissioning environment i think.
<roaksoax> pmatulis: yes
<smoser> otherwise,t hats completely broken
<roaksoax> smoser: nope, that was maas not updating the meta server
<pmatulis> roaksoax: add the ssh key before enlistment?
<smoser> well, roaksoax yes, but the meta-data for the commissioning environment. not for the install.
<smoser> otherwise it is 100% completely and stupidly broken
<smoser> as how could it possibly guess which user is going to deploy?
<smoser> or i'm just missing something
<roaksoax> smoser: ubuntu?
<smoser> no. the MAAS user
<smoser> a maas user has ssh keys
<smoser> ssh keys are in the metadata provided to the instance after it iinstalls
<smoser> so maas has to present the ssh keys of the correct maas user
<roaksoax> smoser: right, so that's why each node is assigned to a maas user right?
<smoser> roaksoax, right.
<smoser> but when does that assignment happen?
<smoser> and how does it happen?
<pmatulis> alexmoldovan: hi
<roaksoax> smoser: idk if this was removed, but you can manually do that in the WebUI, and if no maas user was selected, defaults to the admin user
<roaksoax> pmatulis: so we found an isue on which the meta-data server was not being updated correctly, I just can't recall this affect the ssh keys, but we had to made changes in the meta-data server first, and then enlist so that these get to the machines when commissioned
<alexmoldovan> pmatulis: hi
<smoser> pmatulis, if you have the ability, you could re-install setting the password as i suggested a couple days ago
<smoser> so it is 'ubuntu' and then ssh in that way
<roaksoax> pmatulis: yeah you can also do that
<smoser> pmatulis, i'm suggesting that not as a fix.
<smoser> but as a way to figure out what is going wrong.
<smoser> because i thikn this is a serious issue and want it to be fixed.
<pmatulis> smoser: acknowledged
<cheez0r> question: where does the ssh key that is embedded into the node originate? I am getting 'Invalid SSH Key' on juju status even though I have ssh keys generated; when I get into the node I see no entries in ubuntu user's ~/.ssh/authorized_keys file.
<cheez0r> hello
<m_3> hey.. look at that!
<hazmat> cheez0r, hi
<hazmat> cheez0r, if you have a moment, i'd like to debug that with you
<cheez0r> hazmat: I do.
<hazmat> cheez0r, if your on the machine, do you see any  files in /var/log/cloud
<hazmat> cheez0r, cloudinit is what actually puts the keys into place
<cheez0r> What I'm doing now is that I've populated an SSH key in the MAAS gui for my user, and I'm about to go and re-commission all of my nodes in an attempt to make it embed that key in them.
<hazmat> cheez0r, it would be nice to look at those files to understand why its not able to put the files into place
<hazmat> cheez0r, if you could pastebin those files that would helpful
<cheez0r> on the MAAS node, there is no /var/log/cloud
<cheez0r> the bootstrap node is rebooting.
<hazmat> sudo apt-get install pastebin && pastebin /var/log/cloud-init.log
<hazmat> and pastebin /var/log/cloud-init-output.log
<hazmat> sorry its not /var/log/cloud .. there just files in /var/log
<cheez0r> neither of those files exist on the MAAS node, I'll look on the bootstrap node when it's done rebooting
<hazmat> yeah.. it won't be on the maas node, but the nodes maas boots as a result of juju commands
<hazmat> cheez0r, thanks
<cheez0r> no, thank you ;)
<m_3> hazmat's da man
<hazmat> i have to step out for 15m, my dog desperately needs a walk, back soon
<cheez0r> have fun with that ;)
<roaksoax> smoser: the commissioning image now cleanly shutdowns the servers right?
<roaksoax> smoser: or was that a hack?
<cheez0r> http://pastebin.ubuntu.com/1028750 is the cloud-init.log
<smoser> roaksoax, it does a shutdown.
<roaksoax> pmatulis: so after commissioning, it should WoL just fine ^^
<roaksoax> pmatulis: since it does a shutdown
<roaksoax> as stated above
<smoser> hazmat, well, its not getting the key.
<smoser> and we can't see that (unfortunately) in this output
<smoser> because i suspect its just not there, and cloud-init is allowing that.
<cheez0r> where does cloud-init pull the key from?
<smoser> now.... juju would insert the keys itself, so that would work around the issue.
<smoser> cheez0r, it pulls it from the metadata service.
<cheez0r> smoser: juju bootstrap is not inserting my keys at all
<cheez0r> smoser: how do they get populated there?
<smoser> and you can get that data from somewhere other than the node, but you ahve to get the nodes ccredentials
<smoser> which rae in the db
<smoser> its less than simple
<cheez0r> yeah I can dig that
<smoser> ok.
<cheez0r> I'm just trying to figure out in the standard maas build-up process where that ssh key originates from
<smoser> well, the node has some Oauth creds associated with it.
<smoser> those creds are given to it, and then it uses it to get userdata and metadata (including ssh keys)
<cheez0r> the step-by-step just says run ssh-keygen and then juju bootstrap and that's the sum total of work to embed the ssh key
<smoser> well, if you used juju, then it doesn't need ssh keys from maas
<cheez0r> how does juju embed them, then?
<smoser> it shoves them into user-data
<cheez0r> my authorized_keys file on the target node remains empty
<smoser> in which case, if the node you showed cloud-init log of was delpoyed by juju i would have thought it would get your ssh keys.
<cheez0r> that's what the pastebin is- a juju bootstrapped node
<smoser> cheez0r, /home/ubuntu/.ssh/authorized_keys is empty?
<cheez0r> yes
<smoser> can you pastebin the /var/log/cloud-init-output.log ?
<cheez0r> there isn't one.
<cheez0r> only cloud-init.log exists
<smoser> if there isnt one, then it did not get user-data from maas.
<smoser> so the debugging path here then is to get the oauth creds fro that node from the maas server
<smoser> and then use cloud-init's MAAS datasource as a client to see what it is seeing
<cheez0r> ok so I just rebooted the bootstrapped node
<smoser> and it seems to me that it is not seeing anything :)
<cheez0r> and cloud-init-output.log exists
<smoser> roaksoax, ^
<smoser> was it you, cheez0r that aw this "reboot fixes it" issue?
<smoser> before?
<smoser> someone did. maybe it was pmatulis
<cheez0r> the total content of the file, however, is "cloud-init boot finished at Thu, 07 Jun 2012 15:06:34 +0000. Up 15.03 seconds
<cheez0r> "
<cheez0r> yeah it was me, but it only did once
<cheez0r> authorized_keys remains empty
<roaksoax> cheez0r: pastebin apache error and access log
<cheez0r> from maas node?
<roaksoax> yep
<cheez0r> working
<cheez0r> access.log is 45k lines
<cheez0r> want all of that?
<hazmat> back
<hazmat> so we should look at the output of the metadata service then
<hazmat> to verify what cloud-init is seeing
<hazmat> or poke at the instance user-data on disk
<hazmat> smoser, does the maas data source cache that to /var/lib/cloud/instance/user-data?
<smoser> hazmat, it shoudl, but that will be empty.
<smoser> oh, cheez0r i forgot that you can get to the instance (duh, you were pastebining from it)
<hazmat> yeah.. its at http://192.168.1.1/MAAS/metadata//2012-03-01/meta-data/instance-id
<cheez0r> we still after the apache2 logs then?
<smoser> well, its not really at that.
<smoser> cheez0r, can you.
<hazmat> smoser, well there's a user-data and public-keys at the same address
<hazmat> smoser, that's how cloudinit gets the data from my read of maasdatasource
<smoser> hazmat, they're oauth protected
<smoser>  * open up the file in /etc/cloud/cloud.cfg.d/91*.cfg
<smoser>  * there is a 'url' setting there
<cheez0r> http://paste.ubuntu.com/1028772 and http://paste.ubuntu.com/1028774 for the apache2 access.log and error.log
<smoser>  * run: python /usr/share/pyshared/cloudinit/DataSourceMAAS.py --config /etc/cloud/cloud.cfg.d/91*.cfg crawl <htat-url>
<smoser> or even just "get" on that url (it should show metadata and userdata)
<smoser> and then you can 'get' the same url with those appended
<smoser> but i'm pretty sure you're not getting anything there.
<cheez0r> so from the bootstrapped node, I open that file, then run that python command against the URL?
<cheez0r> there is no 91*.cfg in that directory
<smoser> what files are in that directory?
<smoser> this is wierd, btw :)
<cheez0r> 05_logging.cfg; 90_dpkg.cfg; 90_dpkg_local_cloud_config.cfg; 90_dpkg_maas.cfg
<cheez0r> README
<cheez0r> that's it
<smoser> one of the other 90_. probably _dpkg-maas
<smoser> sorry
<cheez0r> yes, I agree, since I'm following the most basic process possible to build up this config
<smoser> i was just going from memory
<smoser> look at those, one of them will have yaml with oauth creds
<cheez0r> there's a consumer_key metadata_url token_key and token_secret in the file
<cheez0r> so I ran "python /usr/share/pyshared/cloudinit/DataSourceMAAS.py --config /etc/cloud/cloud.cfg.d/90_dpkg_maas.cfg crawl http://192.168.1.1/MAAS/metadata/
<cheez0r> "
<cheez0r> response was "== http://192.168.1.1/MAAS/metadata/2012-03-01 ==
<cheez0r> 2012-03-01
<cheez0r> latest
<cheez0r>  
<cheez0r> == http//192.168.1.1/MAAS/metadata/latest ==
<cheez0r> 2012-03-01
<cheez0r> latest
<cheez0r>  
<cheez0r> "
<hazmat> that's a serious bug in maas then
<hazmat> smoser, what's the difference between crawl and check-seed?
<cheez0r> should the ssh key be in the cobbler system variables?
<cheez0r> I don't see anything regarding SSH in it.
<hazmat> cheez0r, could you run the same command with check-seed instead of crawl
<hazmat> ie. python /usr/share/pyshared/cloudinit/DataSourceMAAS.py --config /etc/cloud/cloud.cfg.d/90_dpkg_maas.cfg check-seed http://192.168.1.1/MAAS/metadata/
<cheez0r> k
<cheez0r> let me pastebin output, more significant
<smoser> crawl would potentiall do more.
<smoser> check-seed would only verify that the data it wants is there.
<hazmat> smoser, well thats sort of what i want to see
<smoser> right. i just hadn'tremembered the correct usage of that tool
<cheez0r> I see the public-keys in the output
<hazmat> cheez0r, can you pastebin the output of that command
<smoser> hm..
<hazmat> the check-seed one
<cheez0r> http://pastebin.ubuntu.com/1028800
<cheez0r> yeah, that's it
<cheez0r> that's my correct ssh key at the bottom
<hazmat> hmm
<hazmat> okay.. so the data is there
<hazmat> that's good
<cheez0r> so if the data is there, what would make it not be utilized by cloud-init?
<hazmat> cheez0r, a bug in cloud-init
<cheez0r> hrm, ok
<hazmat> smoser, can we get anymore verbose out of cloud-init for this
<smoser> well...
<smoser> kindo f no
<smoser> because i'm suspecting right now
<smoser> that if he reboots
<smoser> if cheez0r does:
<smoser>  * rm -Rf /var/lib/cloud
<smoser>  * sudo reboot
<hazmat> smoser, but before he reboots can we determine what went wrong
<smoser> then it will "just work"
<hazmat> smoser, sure, that will make it work..
<smoser> what i think is wrong is that nothing was there.
<smoser> and now it is.
<hazmat> smoser, but how do we figure out what's wrong
<hazmat> smoser, yeah.. that would do it
<hazmat> smoser, it would be nice if cloud init captured that data that its using to disk
<hazmat> smoser, i thought it did that into /var/lib/cloud/instance
<smoser> well, it does
<smoser> :)
<hazmat> smoser, doh
<smoser> and it captured a nice empty cloud-config, right?
<hazmat> smoser, we haven't verifeid tyat
<smoser> cheez0r, do you have a /var/lib/cloud/instance dir ?
<hazmat> cheez0r, could you pastebin  /var/lib/cloud/instance/user-data.txt
<hazmat> you'll probably need a sudo on that
<smoser> actually. thi sis really wierd.
<smoser> just for fun, can you also tell me what is in 'date -R' on the node?
<hazmat> cheez0r, ^
<smoser> (and compared against that on the maas server)
<cheez0r> give me a sec
<cheez0r> shows current date in EDT
<smoser> the issue i'm thinking of (bug 978127) doesn't usually expose itself here.
<ubot5> Launchpad bug 978127 in MAAS "incorrect time on node causes failed oauth" [High,Triaged] https://launchpad.net/bugs/978127
<smoser> but rather in commissioning.
<hazmat> ChanServ, could you pastebin user-data.txt
<smoser> cheez0r, i'm interested in seeing if they're off
<smoser> so 'date -R' on one and 'date -R' on the other.
<hazmat> smoser, let's see the data first
<smoser> to see if they're reasonably in sync
<hazmat> smoser, date out of sync would have hit us running the datasource by hand as well
<cheez0r> yes, the bootstrapped node is in EDT and the MAAS node is in EDT
<cheez0r> err.
<cheez0r> CDT.
<cheez0r> bootstrapped node == EDT; MAAS node == CDT
<smoser> nah. i'm not worried about timezone
<smoser> thats why i said date -R
<hazmat> cheez0r, please pastebin  /var/lib/cloud/instance/user-data.txt
<hazmat> smoser, it got some of the data, it picked up hostname for example and set that
<hazmat> smoser, per the cloud-init output
<cheez0r> that is output of date -R
<smoser> $ date -R
<smoser> Thu, 07 Jun 2012 11:42:53 -0400
<smoser> thats what my date -R shows.
<cheez0r> there is one hour difference, MAAS node is accurate to CDT, bootstrapped node is accurate to EDT
<smoser> but yes, plesae go ahead and pastebin what hazmat is asking for.
<cheez0r> MAAS node == -0500; bootstrapped node == -0400
<smoser> cheez0r, ok. thats good enough then, that they're reasonably accurate. (they need to be within 5 minutes i think)
<cheez0r> they're an hour different but the same UTC
<smoser> ok. then, yeah, get what hazmat asked for.
<cheez0r> date -u shows same on both
<smoser> thakns.
<cheez0r> accurate to within 10 sec
<cheez0r> http://pastebin.ubuntu.com/1028816
<hazmat> yeah.. instead of playing smoke and mirrors games theorizing about time, we already know the dates are in sync because we can fetch the data in the first place. so please see what its actually running first..
<hazmat> so its a different problem
<smoser> hazmat, well, not really.
<hazmat> smoser, so that data is correct
<smoser> something could have fixed the time since
<hazmat> smoser, not if the user-data.txt is in place
<smoser> correct.
<smoser> then i'm really confused.
<hazmat> smoser, did running the datasource overwrite the instance cache?
<smoser> no. its just an oauth client
<cheez0r> well, the preseed does set an NTP server
<smoser> but, sure, cheez0r can you verify the timestamps on that file?
<cheez0r> and it is accessible from the bootstrapped node
<smoser> actually..
<cheez0r> today at 11:06am
<cheez0r> that's 41 minutes ago
<hazmat> cheez0r, that's roughly when you commissioned it?
<smoser> could you :
<smoser>  sudo ls -lR /var/lib/cloud/ | pastebinit
<cheez0r> yeah, roughly
<smoser> cheez0r, but http://pastebin.ubuntu.com/1028750/
<smoser> says that stuff happened on Jun 6
<smoser> (i htink that was your pastebin of /var/log/cloud-init.log
<cheez0r> http://pastebin.ubuntu.com/1028826
<hazmat> smoser, ah..
<smoser> the reboot already happend.
<hazmat> its a recomissioned node maybe?
<smoser> -rw-r--r-- 1 root root 13 Jun  7 11:06 config-scripts-per-boot.always
<smoser> -rw-r--r-- 1 root root 14 Jun  6 16:09 config-scripts-per-once.once
<cheez0r> I can destroy-environment and rebootstrap if you like
<hazmat> and it never cleared out the old data when it recomissions?
<hazmat> i thought maas would do a fresh install on a recommissioned node
<cheez0r> it was never recommissioned.
<cheez0r> it was bootstrapped and then rebooted.
<cheez0r> I think, at least.
<hazmat> ic
<smoser> ok
<smoser> so here is my hypothesis:
<smoser>  * on initial first boot (yesterday) instance got no user-data from maas server, so it came up happily and did nothing.
<smoser>  * in order to remove the dependency on the maas MD server on every boot, maas configures cloud-init to only ever look on the first boot
<smoser> hm..
<smoser> nah
<smoser> my hypothesis is flawed.
<smoser> i'm confused.
<hazmat> smoser, it seems to be me that the issue is basically what your surmise.. effectively cloud-init is being rerun with existing contents of /var/lib/cloud and only parts are executing because of the pre-existing state
<hazmat> but specifically parts like putting keys into place and user scripts are not being run
<smoser> oh.
<smoser> crap
<smoser> yeah.
<smoser> you're right.
<smoser> so the first time it came up
<smoser> it got its instance ID
<smoser> because that is constant for the node (it is not per "instance")
<smoser> then, second boot, it came up and got different user data
<smoser> but it had already run that stuff once per the given instance-id
<smoser> so it did not run again
<smoser> i'm still kind of confused as to why we dont see evidence of cloud-init running *at all* on second boot in http://pastebin.ubuntu.com/1028750/
<hazmat> smoser, that is curious
<hazmat> it should still have out there just running through the handlers
<hazmat> smoser, hmm.. well
<hazmat> smoser, what if it never actually rebooted the machine
<smoser> and actually, we have evidence that it *did*
<smoser> in the timestamps
<smoser> http://pastebin.ubuntu.com/1028826/
<hazmat> hmm.. yeah
<smoser> see '.always' timestamps
<smoser> so i'm jsut really confused
<smoser> cheez0r, one more thing...
<smoser> acutally never mind.
<hazmat> cheez0r, is there additional content at bottom of /var/log/cloud-init.log
<smoser> i'm almost certain maas gave cloud-init empty user data on first boot
<hazmat> that didn't get to the pastebin?
<hazmat> smoser, so what's the maas usage here.. does it clean out the disk/reinstall when we destroy/bootstrap again?
<hazmat> destroy-environment that is
<smoser> hazmat, yes, its a full install.
<hazmat> but it doesn't do that when doing the first boot dance
<smoser> its slow as molasses
<hazmat> so the first time you setup, the nodes have pre-existing state
<hazmat> and never fully initialize with cloud-init
<hazmat> but subsequent to destroy-environment, bootstrap it gets a clean disk
<hazmat> adam_g, ping
<smoser> no. there is never any pre-existing state.
<hazmat> smoser, so when/why is it doing a reboot?
<smoser> that was human
<smoser> actually, nothing in maas has the ability to reboot :)
<hazmat> ah, ic
<hazmat> smoser, so our working hypothesis atm then is that when it first comes up it doesn't get any init metadata
<hazmat> but subsequently it does
<hazmat> but its too late, since cloud-init already ran with a mostly empty metadata
<hazmat> cheez0r, smoser okay.. i think we've exhausted the debug info we can get from that system
<hazmat> cheez0r, you should be able to destroy-environment/bootstrap and have things work
<smoser> hazmat, yeah.
<smoser> thats my hypothesis.
<smoser> cloud-init isn't very friendly to maas MD either.
<smoser> it doesn't even retry.
<smoser> but it sure seems that it must have gotten a 200 response with empty data to me
<hazmat> smoser, for juju's purposes, it would be fine for it to re run the user data
<hazmat> scripts
<smoser> well, thats neither here nor there.
<smoser> runcmd by design is run only once per instance.
<hazmat> we'd have to make one minor adjustment to the initialize stuff, but then it would be idempotent
<cheez0r> sorry guys I had to step away
<cheez0r> will read scroll and see what's doing in a bit
<smoser> hazmat, even if it was idempotent, you wouldn't want to do it.
<smoser> you dont want to apt-get update && apt-get install on every boot
<smoser> thats just a silly waste of time
<hazmat> smoser, i sent out a reply
<hazmat> smoser, i'm pretty sure that the root of this is a timing issue getting metadata to the api within maas
<hazmat> and i wonder if it works subsequently because its returning old metadta
<hazmat> i'd have to dig into the code of maas to verify though
<smoser> ah. you're right
<smoser> its race
<hazmat> smoser, going through maas code its not clear why
<hazmat> it updates the db before it starts the nodes
<hazmat> so the userdata is in place
<hazmat> are there any maas developers up?
<smoser>  "before it starts the nodes"
<hazmat> allenap, ping
<smoser> flacoste
<hazmat> roaksoax, ping
<hazmat> the more the merrier
<roaksoax> hazmat: pong
<hazmat> egads i hate brighttalk for videos.. why should folks have to signup
<hazmat> roaksoax, trying to understand why there might be a delay for userdata showing up for a node
<hazmat> roaksoax, we're trying to debug the invalid ssh key thing
<hazmat> roaksoax, i just sent email to the maas-dev list which outlines our hypothesis
<roaksoax> ok
<hazmat> roaksoax, looking at the code i'm not sure how that could be the case
<hazmat> it looks like the api will update the userdata before launching the node
<hazmat> is there some celery task that will try to overwrite that? or i'm misunderstanding this bit..
<roaksoax> hazmat: in precise maas we are not using celery
<hazmat> roaksoax, gotcha..
<hazmat> roaksoax, but does delay availability of metadata sound familiar or possible?
<adam_g> hazmat: pong
<roaksoax> hazmat: sounds familiar. I've been in a situation on which a node is enlisted, and then a ssh public key gets added and the node should have had the meta-data update so than on deployment it would have imported that ssh key. however, it didn't happen, so I had to enlist the nodes after the key was added.
<hazmat> adam_g, there's a question up on stack overflow/askubuntu.. which i think you might be the only who can respond effectively..
<hazmat> adam_g, http://askubuntu.com/questions/141552/creating-volume-group-in-nova-volume-juju-charm
<roaksoax> hazmat: while it is not the same issue (and this is a MAAS SSH key), it might be related
<adam_g> hazmat: ill take a look, thanks
<Daviey> hazmat: metadata issues?  check the server apache logs.
<hazmat> Daviey, does it print out the full response?
<Daviey> The common hunch that has hit most of us so far, is bad localtime offset.. meaning that the oauth token was expired
<Daviey> hazmat: you can ask apache nicely to do that.
<smoser> Daviey, that might be useful to help prove this.
<smoser> adam_g, i'm interested in seeing your answer to that question
<hazmat> smoser, Daviey so how does the ntp get setup?
<smoser> it does not.
<smoser> nothing sets up ntp
<smoser> this is not a time related issue.
<smoser> i'm virtually certain of that.
<smoser> virtually
<adam_g> smoser: yeah, me too. :) i need to setup a local provider to test this. you added the loopback stuff to that charm, wasn't there some namespacing / insmod'ing that needed to be done on the host before LVM would work?
<smoser> adam_g, you can't do it.
<smoser> there is no device name space
<smoser> so you have to grant the container access to loopback devices
<smoser> its a rathole
<adam_g> smoser: right, the loopback as the PV worked IIRC, but i thought it required something to be done on the host too
<smoser> http://bazaar.launchpad.net/~smoser/+junk/jstack/view/head:/jstack.txt
<adam_g> thanks
<smoser> you have to allow the lxc  container to write to /dev/loop-control
<smoser> and thats fine... maybe that even makes nova-volume charm work
<smoser> but the issue just keeps popping its head up.
<smoser> you have to basically turn lxc into chroot if you want it to work. which defeats most of the purpose of lxc
<adam_g> right
<pmatulis> i just juju-deployed the mysql/wordpress stuff and pulling up wordpress site yields an error.  it's looking for file /etc/wordpress/config-localhost.php but i see that i have /etc/wordpress/config-node-e4115b137b36.local.php .
<cheez0r> smoser/hazmat/roaksoax: I'm re-commissioning my nodes- I went ahead and populated a SSH public key for my user in MAAS and I want to see if maybe that's what I'm missing.
<adam_g> smoser: jeez, i can't even get loop-control allowed into the container anymore
<smoser> you mgiht be fighting app armour now?
<adam_g> probably
#maas 2013-06-03
<Bicyus> Hello! Hello Hello! i don't know why you say goodbye i say Hello! ;-)
<jtv> roaksoax_: hi there â thanks for noticing that I forgot to change the merge target on my packaging MP.  (It defaults to "merge into trunk" and happily produces a diff even if your branch wasn't related to trunk at all).  Could you have a look at the re-proposal?
<roaksoax_> jtv yes! will take a look in a bit :) thanks
#maas 2013-06-04
<smoser> bigjools_, "do you know a way of sending some PXE config that will make the machine power off again?"
<bigjools_> hey smoser
<smoser> ipmipower -h IPMI_ADDR -u USER -p PASSWORD --off
<smoser> ^ is that what you were wanting bigjools_ ?
<bigjools_> smoser: not exactly
<bigjools_> it's easy to recognise the node is in the wrong state and execute a power command
<bigjools_> but I just wondered if there was a pxe way
<bigjools_> either way it would be nice to have something on the node's console so users are not surprised
<smoser> oh. pxe.
<smoser> duh.
<smoser> you saidthat, i read ipmi
<smoser> we have somthing in plae that does that.
<smoser> i think.
<smoser> right?
<smoser> maybe i'm confused as to what youe're wanting.
<smoser> while possibly more heavy, one solution is to just boot it into commissioning environment with user-data that says:
<smoser> ==== this node is not supposed to be powered on ===
<smoser> ==== it will power off in 120 seconds ====
<smoser> ==== good-bye ====
<bigjools_> ah that would work
<bigjools> roaksoax: I think we should move maas-enlist into maas trunk and build a binary package from it there
<roaksoax> bigjools: yeah already on my todo for this week :)
<bigjools> \o/
<smoser> gray--, so... you're sure that nothing has changed in routing since you ran that ?
<smoser> ie, since you saw this issue:
<smoser> http://pastebin.com/Wkguv4Fg
<gray--> there hasâ¦ i trashed the machine and rebuilt it
<smoser> ah.
<smoser> :)
<gray--> if it's not reproducable, it's probably my error, not maas's error :)
<smoser> ok. so thre is definitely a bug in that script, that woudl result in you seeing what you saw
<smoser> (the stack trace there in that pastebin)
<smoser> if you had no default route
<smoser> where "that script" = maas-region-controller.postinst
<gray--> i did have a default route, was one i checked
<gray--> actually, wait, i take that back, i don't think i checked
<gray--> balls, oh well
<smoser> so your isse now, what is that?
<gray--> i actually have the maas console in my browser, how about that? :)
<gray--> so nowâ¦ http://192.168.10.3/MAAS/settings/, that page seems to time out
<gray--> i also have this in the apache error.log:
<gray--> [Tue Jun 04 14:22:06 2013] [alert] (2)No such file or directory: mod_wsgi (pid=1267): Unable to change working directory to '/home/maas'.
<gray--> after the actual install, i saw this: [Tue Jun 04 14:12:30 2013] [error] [client 192.168.10.1] DBusException: org.freedesktop.DBus.Error.FileNotFound: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
<gray--> but a reboot (eugh) fixed that
<gray--> so, right now, the settings page is timing out
<gray--> hrm, doesn't like being proxied to it seems
<gray--> proxypassreverse could be my friend here, rather than an ssh -L to the server in question
<gray--> i think this is probably turning away from a maas issue, towards an apache issue
<gray--> smoser: thanks for your assistance thus far, i'll look at the apache thing for now, cheers
<smoser> gray--, hm.. i've used ssh -L many times.
<smoser> to remote maas.
<gray--> remote server address is 192.168.0.3, i hit up localhost in the browser, and hitting the banner that says 'Some cluster controllers are missing boot images.', it tries to send me to 192.168.0.3/MAAS/settings/
<gray--> http://localhost/MAAS/settings works fine
<smoser> hm..
<mwhudson> is https://bugs.launchpad.net/maas/+bug/1177932 going to be fixed any time soon?
<ubot5> Launchpad bug 1177932 in MAAS "Unable to select which pxe files to download by both series and architecture." [High,Triaged]
<mwhudson> i could even spend an hour or two on it myself if it's going to be easy i suppose
<bigjools> mwhudson: not soon, but scheduled for a few weeks away
<bigjools> if you want to fix it, I'd not complain :)
<mwhudson> bigjools: maybe we can have a quick skype/similar about how hard it would be this afternoon?
<bigjools> mwhudson: sure - I need to be suitably caffeinated first and try to think how I'd fix it
#maas 2013-06-05
<bigjools> mwhudson: we never had that chat.  Remind me in the morning!
<mwhudson> bigjools: will try
<bigjools> mwhudson: in other news, I will be at the Reds vs Lions game on Saturday \o/
<mwhudson> bigjools: nice
<mwhudson> bigjools: going to watch state of origin tonight?
<bigjools> mwhudson: not if I can help it
<mwhudson> heh
<mwhudson> it you're going to watch one league game, it should be origin i think
<bigjools> that's one too many?
<mwhudson> heh
<bbcmicrocomputer> how do you add a file to file storage via maas-cli?
<bbcmicrocomputer> I keep doing 'maas-cli maas files add filename=x file=y' and getting 'File not supplied'
<bbcmicrocomputer> looking at http post request, it sends multipart request with filename and file components (not file data)
<jtv> bbcmicrocomputer: I don't know off the top of my head what maas-cli does about file uploads...  they're a very different thing from how we construct the rest of the requests.
<jtv> bbcmicrocomputer: ISTR it can do it, but don't remember how.  I've asked allenap, who may be able to help.
<bbcmicrocomputer> jtv: ok, thanks
<allenap> bbcmicrocomputer: I can't remember off the top of my head. Istr that it's a little ugly right now. I'm looking it up.
<jtv> allenap: I'm not finding anything...  maybe it's not something we ever got around to building after all.  :(
<bbcmicrocomputer> allenap: k, thanks
<allenap> bbcmicrocomputer: It's not possible with maas-cli, which sucks. I'm filing a bug about that. We never got round to it during the last phase of work on MAAS, but we will this time. Sorry about that. I'm finding a workaround for you.
<bbcmicrocomputer> allenap: ok, thanks!
<allenap> bbcmicrocomputer: Here's a script that'll work for now: http://paste.ubuntu.com/5736203/
<allenap> bbcmicrocomputer: Use it like: upload.py http://example.com/MAAS/api/1.0/ my:api:key: my_filename < my_file
<bbcmicrocomputer> allenap: ah, awesome.. thanks!
<allenap> bbcmicrocomputer: No problem. I'm really sorry you didn't Just Work before.
<allenap> s/you/ :)
<bbcmicrocomputer> allenap: :)
<bbcmicrocomputer> allenap: works perfectly, thanks
<bbcmicrocomputer> allenap: although oddly, maas-cli get and get-by-key always add an extra line break on stdout of the file
<allenap> bbcmicrocomputer: I'll file a but about that.
<bbcmicrocomputer> allenap: cool, thanks
<allenap> bbcmicrocomputer: https://bugs.launchpad.net/maas/+bug/1187851
<ubot5> Launchpad bug 1187851 in MAAS "Newline added to end of files obtained with maas-cli" [High,Triaged]
<bbcmicrocomputer> allenap: cool, thanks for all your help
<allenap> bbcmicrocomputer: You're very welcome.
<hazmat> does maas work in lxc?
<hazmat> i run into a dbus/avahi issue w/ packages ootb on raring.. trying to access maas ui.. http://pastebin.ubuntu.com/5736459/
<roaksoax> hazmat: yeah we saw that yesterday too, is dbus running?
<hazmat> roaksoax, i just disabled it from starutp and moved on.. avahi-daemon is installed..
<hazmat> roaksoax, yes dbus is running
<roaksoax> hazmat: try restarting dbus and/or avahi-daemon
<roaksoax> that seems to be the issue
<hazmat> k.. i'lll try that..  context switched already.. but i assume its just for advertising itself for the server installer
<roaksoax> yeah
<hazmat>  roaksoax running into an odd issue with maas
<hazmat> its got 30 machines registered, but bootstrap says.. no matching node available
<hazmat> roaksoax, Campbell is running into this
<roaksoax> hazmat: i'd need to see logs (maas.log for now)
<hazmat> Campbell, could you send over the maas.log .. email or pastebin'
<Campbell> ok
<roaksoax> pastebin much easier :)
<Campbell> OK
<Campbell> http://pastebin.com/FfJ63G3K
<Campbell> there you go
<roaksoax> Campbell: where you able to bootstrap?
<roaksoax> Campbell: what are the nodes specs?
<Campbell> Nodes are Poweredge M620s
<Campbell> no bootstrap
<Campbell> openstack@openstack-maas-test:~$ juju bootstrap 2013-06-05 12:25:15,813 INFO Bootstrapping environment 'maas' (origin: ppa type: maas)... 2013-06-05 12:25:16,350 ERROR No matching node is available. openstack@openstack-maas-test:~$
<roaksoax> Campbell: go to the webui and see if the nodes are in Ready status and tell me what does it say there about CPU, RAM, etc
<Campbell> I'm maybe doing something wrong
<roaksoax> CPU count and Memory
<Campbell> the nodes are all in declared state
<roaksoax> Campbell: ah that's the reason the
<Campbell> Do I need to commission them
<Campbell> :)
<roaksoax> Campbell: you need to "Accept & COmmission" all the nodes
<Campbell> I figured juju would do that, my bad
<Campbell> OK
<Campbell> and lay them down with precise?
<roaksoax> Campbell: once the nodes are in 'Ready' state they will be availablefor juju
<roaksoax> Campbell: once you do juju bootstra, it will install precise and start settings things wup
<Campbell> Mint. I'll get on it
<roaksoax> Campbell: this overview will help you understand a bit btter how maas works: http://www.roaksoax.com/2013/05/getting-started-with-maas-and-juju
<Campbell> Thanks
<hazmat> roaksoax, he's not able to bootstrap in a juju sense hence the error, the nodes are beefy blades
<roaksoax> hazmat: yeah the nodes were in 'Declared' state, which is why there were no available nodes
<Campbell> I'm good and bootstrapped
<Campbell> All the nodes are available and I've bootstrapped them
<Campbell> Question: can juju handle the .local FQDNs? I'm getting an error wrt SSH when I try and do a juju status
<Campbell> Hmm, maybe need to leave it a while before this will work, just rememebred that the bootstrap can take quite a while :)
<Campbell> put the IPs in the hosts. Working. Really should have configured DNS :)
<roaksoax> Campbell: yeah maas manages DNS/DHCP
<Campbell> we arleady had a DHCP server in the environment so wanted to use that
<Campbell> i'm still in the bootstrap. taking a while but I guess that can happen
#maas 2013-06-06
<bigjools> mwhudson: now would be a good time to chat...
<mwhudson> bigjools: hey
<mwhudson> bigjools: still good?  HO?
<bigjools> mwhudson: yes - give me a couple of mins
<mwhudson> bigjools: i need to caffeinate so np
<bigjools> understandable
<mwhudson> bigjools: let me know when you're ready
<bigjools> mwhudson: ready!
<mwhudson> ok
<bigjools> call meh
<mwhudson> bigjools: can you drive g+ reliably?
<bigjools> yeah...
 * mwhudson clicks some things
<mwhudson> "Julian isn't on Hangouts right now. He'll see your messages later
<mwhudson> "
<bigjools> hahahaha
<bigjools> calling you
<mwhudson> ah
<mwhudson> i think i'm calling you
<bigjools> and failed
<bigjools> first time I had this with anyone
<lifeless> could be hangouts vs chat :)
<lifeless> cause youknow, standards, meh.
<bigjools> the online presence detection seems a little flaky
 * bigjools lunches
<roaksoax> rvba: around?
<roaksoax> jtv: aound?
<jtv> Hi roaksoax
<roaksoax> jtv: howdy! So I'm wondering something quick. If I add a template snippet for commissioning user data, I should do this right? http://paste.ubuntu.com/5739010/
<roaksoax> jtv: in other words, if I add a snippet, i need to do this in the template: {{maas_ipmi_autodetect_py}}
<jtv> roaksoax: it's been a while, to be honest â best thing for this one is to cargo-cult it from how the other snippets work.
<roaksoax> jtv: ack! thanks
<jtv> If you're moving monolithic code into snippets, that's good news â means we can test them etc.
<roaksoax> jtv: yeah.. though I don't see tests for maas_signal for example
<jtv> Be careful of dependencies between snippets, and on snippets etc. though.
<jtv> I think maas_signal was somewhat out of our control.
<roaksoax> yeah well I guess the same applies for maas-ipmi-autodetect
<roaksoax> jtv: ok so now I need to inject the same file into the enlistment preseed
<roaksoax> jtv: I'm guessing {{maas_ipmi_autodetect_py}} is not being made available elsewhere?
<jtv> Right â the snippets aren't available except as part of the commissioning preseeds.
<roaksoax> jtv: ok, so since enlistment "is" commissioning, I think I could easily make it available. Will give it a try, thanks!
<jtv> What do you mean by "is"?
<jtv> (I sound like Bill Clinton, don't I?)
<roaksoax> jtv: heh, so enlistment is basically the same as commissioning, the only basic difference is that it uses a different preseed file
<jtv> From which perspective exactly?  There are different ways of looking at it.  For the MAAS code, the two are separate *but* there doesn't have to be a human interaction between them.
<roaksoax> jtv: from the maas point of view, IIRC, they both have the same 'purpose'
<jtv> But as for the preseeds, maybe if you can colocate, you can reuse the snippets.  I just hope the two won't "contaminate" each other!
<jtv> Ah, the boot-image purpose!  Yes they're both using the same one there.
<roaksoax> yep
<roaksoax> jtv: where is all thede code in charge of rendering the commissioning userdata?
<jtv> roaksoax: the lowest-level code is in src/provisioningserver/commissioning/snippets.py IIRC
<jtv> Ahem.  src/metadataserver/commissioning, that is.
<roaksoax> cool thanks :)
<jtv> And the user_data.py module works at the level just above that.
<roaksoax> jtv: ok so what I want to do is to make  the snippets available for the enlistment
<roaksoax>  processes
<roaksoax> how do you best suggest to do that?
<jtv> Yeah, that's going to take a bit of design.
<roaksoax> should I simply do something similar to what generate_user_data does?
<roaksoax> jewell yeah, I already can list the snippets, so importing that won't be that hard, the only thing is to made them available to the enlistment context
<jtv> It may be a matter of parameterizing that code, so that you can apply it using different snippets directories.
<roaksoax> jtv: right, but i want the same snippets
<roaksoax> to not have the same code in two different places
<roaksoax> so consuming what it's there makes more sense
<jtv> I don't remember if the main template needs to include those snippets explicitly or not...  If it does then yes, you could just combine them into a single directory, like a kind of standard library.
<roaksoax> jtv: http://paste.ubuntu.com/5739174/ -> this is what I waas thinking
<jtv> My machine is becoming unusable...  will reboot soon
<jtv> roaksoax: is generate_user_data too different from what you need?
<roaksoax> jtv: doesn't seem to be other than the location of files are different
<jtv> If all it takes is some different templates (and the same snippets directory), I'd approach it like this:
<jtv> 1. Split out an implementation function that basically does all the work, except for locating those templates at the top.
<jtv> 2. See if the call sites can be conveniently changed to look up the templates themselves, and just call that new function instead (so that the remaining version of generate_user_data can just go away).
<jtv> 3. Have the enlistment version call the same function, but with different templates.
<jtv> 4. See if the code needs moving.
<jtv> Mind the "If" at the beginning though.  :)
<roaksoax> jtv: ok cool
<roaksoax> i'll look into that
<jtv> Great.  I'm off now, to be back Tuesday.
<jtv> So feel free to ask the others.  :)
<roaksoax> alright have a good long weekend
<jtv> Thanks!
<Campbell> hey, has anyone seen issues where the TFTP server stops responding on MAAS 1.4.5? I understand this is now a twisted based python server rather than a standalone. it worked before to get all my nodes PXE booted and enlisted but not it's stopped working and I am at a loss as to why
<hazmat> roaksoax, ^
<hazmat> Campbell, haven't seen it before.. is there anything in /var/log/maas/pserv.log
<Campbell> Lots of healthy logs from yesterday when I brought the servers in but today very little since I've been trying to bootstrap juju http://pastebin.com/vtFRbneU
<Campbell> Is there a specific process I should be looking for to see if it's alive?
<Campbell> netstat shows that nothing is listening on port 69
<roaksoax> Campbell: what happens when you try to PXE boot? it simply won't?
<Campbell> yeah, it sits there trying to connect to the TFTP server and cant
<roaksoax> Campbell: is your maas server using DHCP?
<Campbell> No, I have a separate DHCP server which is serving this network
<Campbell> it seems to be working fine, the machines are getting an IP address as teh PXE boot and we've got the ip helper enabled to forward UDP traffic
<Campbell> I am unable to telnet onto port 69 on the MAAS server locally either. Looks like TFTP is dead.
<hazmat> Campbell, the maas pserv instance
<hazmat> is the tftp server
<hazmat> Campbell,  sudo service maas-pserv status
<Campbell> maas-pserv start/running, process 1420
<Campbell> Looks like it's there
<Campbell> actually its not when I do ps -d
<roaksoax> Campbell: sudo service maas-pserv restart
<roaksoax> and see what happens
<roaksoax> pserv.log should give an idea of what's happening
<Campbell> nothing from what I pasted earlier
<Campbell> I'll try and force a restart on pserv and see what that does
<Campbell> Just get this in the log 2013-06-06 11:04:40-0700 [-] Received SIGTERM, shutting down.
<roaksoax> Campbell: can you do this in the maas server?:
<roaksoax> ubuntu@cluster1:~$ tftp localhost
<roaksoax> tftp> get pxelinux.0
<roaksoax> Received 27154 bytes in 0.0 seconds
<Campbell> permmission denied
<Campbell> ahnot I got it
<Campbell> so its probably network related
<Campbell> I was running the get from a directory I didn't have write permissions in
<roaksoax> Campbell: yeah, so to me, given that you have a external DHCP< it is either giving a wrong ip to the MAAS server (different from next-server)
<roaksoax> that's the most likely cause of the issue
<Campbell> OK. I'll do some more investigation. Thanks for the pointers :)
<Campbell> I've got the PXE boot happening again. I think recycling the maas-pserv did the trick. That's the only thing I've done that I think might have been material.
#maas 2013-06-07
<roaksoax> bigjools: are you around today?
<bigjools> roaksoax: barely
<roaksoax> bigjools: no worries then :)
<bigjools> roaksoax: as in, I am here in body but not necessarily all in mind
<roaksoax> bigjools: oh ok, so Idk whether you started reviewing branches... but I have proposed one I'm sure you'd like:
<roaksoax> https://code.launchpad.net/~andreserl/maas/consolidate_maas_ipmi_autodetect/+merge/167806
<bigjools> roaksoax: \o/
<bigjools> I'll try and get to it later, I'm not feeling great right now
<roaksoax> bigjools: no worries :) take your time
<Konrad> Hi. I updated MAAS to 1.2+bzr1373+dfsg-0ubuntu1~12.04.1 and got the following error: django.db.utils.DatabaseError: relation "maasserver_node" already exists
<Konrad> Does anyone have an idea what went wrong?
<roaksoax> rvba: ping
<roaksoax> rvba: http://pastebin.com/ckuQZkQF
<roaksoax> rvba: kind of critical priority
<roaksoax> Konrad: that's weird, I just did and upgrade and it worked just fine
<Campbell> I force started a node in maas from the web UI. Is there an easy way to force it to stop and recover it to ready state?
<Konrad> roaksoax: the complete update log is here: http://pastebin.com/ckuQZkQF
<hazmat> Campbell, if you've logged into the maas-cli ... $ maas-cli maas node-list .. $ maas-cli maas node stop --debug <nodeid>
<Campbell> thanks hazmat
#maas 2014-06-02
<gmb> jtv: Where do we stand with the bootresources.yaml killage? I see a bunch of cards on the board but I assume that theyâre just going to wither and die now.
<jtv> gmb: the killage in itself is done.
<gmb> Wewt
<gmb> jtv: So weâre left with documentation and then UI stuffâ¦ Whatâs the word on the latter?
<jtv> gmb: didn't we halt work on the latter?  The documentation is really just for making people aware of the change â the new way of doing things is already documented.
<gmb> jtv: Yeah, we did, but I havenât pored through all my email yet, so I wanted to check that weâre still halted :)
<bigjools> jtv, gmb: ui is deeeed. long live the ui
<gmb> bigjools: Thatâs *precisely* the conversation that Iâm thinking could have happened somewhere :)
<gmb> Iâve removed the UI cards anywayâ¦ we can always put âem back.
<bigjools> gmb: yeah, well...
<gmb> Ruh roh
<bigjools> rvba: halp. django.
<bigjools> self.filter(ip__gte=range_low, ip__lte=range_high)
<bigjools> results in:
<bigjools> TypeError: argument of type 'IPAddress' is not iterable
<bigjools> dafuq
<rvba> humâ¦
<bigjools> ip is a GenericIPAddressField so I guess I need to do something
<rvba> Weird that you're getting the error "is not iterable".
<rvba> Nothing looks like it needs to be iterable in your query.
<rvba> Do you have the full stacktrace?
<bigjools> I am
<bigjools> err I do
<rvba> You're a stacktrace?
<rvba> Please dial 911!
<bigjools> http://paste.ubuntu.com/7571266/
<bigjools> no such number
<bigjools> :)
<rvba> bigjools: looks like the operators for the GenericIPAddressField expect strings (not IPAddresses).
<bigjools> !
<bigjools> sigh
<bigjools> thanks rvba
<rvba> bigjools: They use "':' in value" to check if the value you give them is IPv6 or not.
<bigjools> yeah
<bigjools> rvba: I also have a dilemma
<bigjools> rvba: my new model is called IPAddress which clashes nicely with the one in netaddr :(
<rvba> Ah, right.
<bigjools> gonna have to rename it I think, otherwise confusion will likely ensue
<rvba> +1
<bigjools> unless we prefix it everywhere
<bigjools> suggestions for a new name?
 * rvba thinksâ¦
<rvba> NodeIPAddress?
<rvba> (Or is this going to be used to store non-node IPs as well?)
<bigjools> yes
<bigjools> I need to fix the column name called "type"... wasn't thinking too hard about that one
<rvba> DeviceIPAddress then?
<bigjools> rvba: StaticIPAddress
<bigjools> since that's what it is
<bigjools> and no more
<bigjools> :)
<rvba> Yeah, sounds good.
<bigjools> they will be used for non-devices as well
<bigjools> VIPs
 * bigjools hax-rs
<jtv> ManagedIPAddress?
<bigjools> they're not all managed
<Mosibi> Hi all.
<Mosibi> Is this the right channel to ask for a MAAS problem? :: "No matching node is available" with juju deploy nova-compute
<jtv> Mosibi: yes, this is the right channel â at least for the MAAS part.  It's possible that the origin of the problem is somewhere else, but let's have a look.
<jtv> The error message means that a node was requested, but there was no node in the Available state that matched all of the constraints given in the request.
<Mosibi> jtv: in tried several times, with and without setting constraints.
<Mosibi> Could it be that there are constraints that i do not see?
<jtv> Yes, that's quite possible.
<Mosibi> okay...
<Mosibi> Is there a possibility to show/extract that constraints?
<Mosibi> Setting MAAS in debug mode did not show me anyhting more i allready knew.
<rbasak> Mosibi: I've seen constraints embedded in the HTTP GET URI inside one of the log files in /var/log/maas/
<rbasak> I'm not sure if that's still the case but it maybe that will help?
<rvba> bigjools: happy to have a pre-imp about the DHCP work now if you're available.
<bigjools> rvba: bien sur
<jtv> Mosibi, rbasak: Yes â some clients will pass those parameters in that way â I think those would go in /var/log/apache2/access.log though.
<bigjools> rvba: I'll head back to same hangout
<rvba> okay
<rvba> bigjools: s/sur/sÃ»r/ (sur ~= dessus, sÃ»r = certain)
<jtv> Weird place for a vestigial âsâ...
<gmb> jtv, rvba, bigjools, allenap: A review, if you please: https://code.launchpad.net/~gmb/maas/update-changelog-bootresources.yaml/+merge/221678
<jtv> Coming.
<gmb> Had to invent the 1.5.2 release for the sake of the updateâ¦
<Mosibi> rbasak and jtv: thx will have a look at the httpd logs
<Mosibi> Sadly i could not find any contraints in the httpd log files.
<Mosibi> I noticed when i force a tag on one off my 'ready' machines in MAAS and after that, do a juju deploy with --contraints tag=mytag, the deployment starts
<rbasak> Mosibi: did you set constraints when you bootstrapped the environment?
<rbasak> IIRC, that sets default constraints. There is a command to change the default.
<Mosibi> rbasak: nope, i did not set any constraints when i bootstrapped the environment.
<Mosibi> let me explain what i am trying/doing :)
<Mosibi> i am building a demo openstack environment with juju and allready deployed the openstack components with juju
<Mosibi> So i have, keystone, cloud-controller, quantum-gateway etc...
<Mosibi> Thus i have a working MAAS/Juju env.
<Mosibi> Now i want to deploy a nova-compute host and i have 4 (virtual :: KVM) compute nodes available in MAAS.
<Mosibi> And that's te troubling part..
<Mosibi> +h somewhere :)
<Mosibi> To me it looks like the nova-compute charm should have set some constraints (i need x mem and minimaal x CPU's), but i can not find any of that in the charm itself.
<jtv> Maybe even a constraint to ensure that you get a physical machine.  I guess that might explain what's happening â if we can find it.
<rbasak> Mosibi: you would do that in your deployment yourself, or via your bundle I think. It sounds to me that MAAS is doing exactly the right thing for you, and that you have a juju question if anything. Try #juju for more expertise maybe?
<Mosibi> Since MAAS is returning the 'No matching node is available', i would love to see the list of contraint MAAS is working with.
<jtv> No reason to show those in the error message, really.  I'll file a bug about that.
<jtv> No reason *not* to, I mean!
<Mosibi> rbasak: indeed, it looks like a juju problem, but i would like a more 'verbose' notice from MAAS
<Mosibi> jtv: :)
<Mosibi> jtv: thx!
<Mosibi> rbasak: also thx for your insights
<Mosibi> I'll switch to #juju :)
<jtv> Ah, we already have bug 1274085 â but the counter-argument is that MAAS returns a raw API error, and maybe it's up to Juju to show what it was trying to do (at a higher level of abstraction).
<ubot5> bug 1274085 in MAAS "error when maas can't meet juju constraints is confusing and not helpful" [Wishlist,Triaged] https://launchpad.net/bugs/1274085
<Mosibi> jtv: that's a 'he, but we aren
<Mosibi> 't doing anything wrong'
<Mosibi> Since MAAS is receiving the call, more verbosility would be welcome and more important: it would lower the support calls in #maas :)
<jtv> Yes, I agree our message could be clearer.  I can probably just fix the message.
<Mosibi> jtv++;
<Mosibi> :)
<jtv> It may be that Juju can do a better job, but that's not in itself an argument against not doing one ourselves.
<Mosibi> idd
<gmb> allenap, jtv, rvba: anyone free for a preimp about https://bugs.launchpad.net/maas/+bug/1312844?
<ubot5> Ubuntu bug 1312844 in MAAS "MAAS commissioning page shows distributions that are not available" [High,In progress]
<jtv> gmb: will be in a minute
<gmb> jtv: Okay, call me when youâre ready.
<jtv> Will do.
<jtv> gmb: rinnggg
<gmb> Total lack of notifications
<jtv> Maybe I invited your wrong persona.
<bigjools> there's JS lint... how did that appear I wonder
<jtv> There seems to be some weirdness where sometimes lint just doesn't get reported.
<bigjools> rvba: https://code.launchpad.net/~julian-edwards/maas/allocate-static-ip/+merge/221702
<bigjools> and now I shall EOD
<bigjools> jtv: do you have an IRC notification on the word "lint" ? :)
<jtv> Hey, the NSA's got to be good for _something_...
<jtv> It may be version skew, or something at a low level limiting itself to revision-controlled files, or something.
<jtv> allenap: session-level locks... but the locking code opens and closes a dedicated cursor for each locking command.  Do we know that those cursors are on the same session..?
<allenap> jtv: Thatâs up to Django ;)
<allenap> jtv: Maybe it should keep that cursor around?
<jtv> Then that would explain the problem...
<allenap> jtv: If the session is going away, then thatâs Djangoâs responsibility I think, though it might just be going away because of garbage collection, in which case holding the cursor should prevent that.
<allenap> jtv: Do you think itâs worth trying to validate that, or should we just go straight to keeping a reference to the cursor around?
<jtv> allenap: it really ought to be the same session that's performing the locked operation.
<rvba> jtv: Although Julian's solution is probably good as a first step, I seem to remember you had a better idea, to allocate new IP addresses, than select one in the allowed range different from the pool of existing allocated IP addresses.  This solution is a bit racy, obviously.
<jtv> rvba: not sure I fully undestand you, but that sounds like how we thought our existing scheme would work â which might work with a different dhcpd.
<rvba> jtv: this is about the code Julian mentioned this morning.
<rvba> The code to allocate a new IP address.
<jtv> Oh, for picking a free address.
<rvba> Yeah.
<jtv> Right.  The idea there was to cache a pool of available addresses.
<rvba> cache?
<rvba> This means like a weird way to second-guess the DBâ¦
<jtv> That's pretty much what caches are.  The database is the source of truth, but a cache can help find efficient "guesses."
<rvba> Right.  Well, I guess that's probably something we can do as a second step.
<rvba> Looks like an optimization.
<jtv> Exactly.  It's optimisation.
<gmb> allenap: Is there âniceâ way to untangle where a circular import is creeping in? I see errors that are utterly unrelated to where Iâm working at the moment and itâs highly frustrating to have to comment stuff out, especially in a big file.
<gmb> rvba, jtv: That question was for you too; I jsut wanted to engage the Gavinatorâ¦
<rvba> gmb: the only solution I know is the local import.
<gmb> rvba: Yeah, I know the solution, Iâm trying to find a nice way to identify the actual problem imports.
<rvba> Oh, I see.
<gmb> So Iâm working on maasserver.forms and I see things like:
<gmb>   File "/home/graham/workspace/maas-work/maas/src/maasserver/views/clusters.py", line 45, in <module>
<gmb>     from maasserver.forms import (
<gmb> ImportError: cannot import name NodeGroupEdit
<gmb> I know that maasserver.forms has the problem but it has 80 lines of imports :)
<gmb> And I donât know which one to kill with fire.
<rvba> In Django, it's usually the models that are the problem.
<gmb> Well, that helps a bitâ¦ It at least gives me a place to start, which is better than my current scattergun approach.
<allenap> jtv: I have discovered that Django closes the connection when it pleases, so DatabaseLock.__enter__() and .__exit__() can run in entirely different sessions.
<allenap> jtv: It explicitly closes it too, so hanging onto a reference does no good.
<allenap> I feel that Django owes it to us to remove itself from the MAAS codebase.
<jtv> allenap: it can't reasonably close a connection while it's in a transaction though, so if we can get a hold of our "regular" DB session, we'll be alright.
<allenap> jtv: Ensuring that the _session_ lock is taken within a _transaction_ seems to satisfy Django.
<jtv> allenap: satisfy in what way?
<allenap> jtv: It keeps the connection open.
<allenap> And thus the session.
<jtv> Is that a promise?  Or could there be some pool with its own eviction policies?
<allenap> jtv: Iâve added a check to DatabaseLock.__exit__() to ensure that itâs actually released a lock that it held. That ought to keep us on our toes.
<jtv> Thanks.  At least it'll give us a more precise failure.
<allenap> jtv: If youâre around, would you mind having a look at https://code.launchpad.net/~allenap/maas/database-locks-revisited/+merge/221762?
<jtv> Not at all.
<allenap> Thanks :)
<jtv> allenap: done
<allenap> Tip top, thanks jtv. I hope this will plug the problem.
<jtv> As we say in Dutch, I'll help you hope.
<allenap> Heh, I like that.
<blake_r> allenap: could you do a review of this review!
<blake_r> allenap: https://code.launchpad.net/~blake-rouse/maas/powernv-support/+merge/220840
<allenap> blake_r: Sure. OTP right now, Iâll do it later. Is that okay?
<blake_r> allenap: i think roaksoax was wanting it really soon, for sru
<blake_r> allenap: anyone else around maybe?
<allenap> jtv, gmb: Either of you free for urgent review? ^
<jtv> I can take it.
<blake_r> jtv: thanks
<jtv> Is it Blake's?
<jtv> That answers that.  :)
<blake_r> jtv: yeah
<blake_r> jtv: so don't be to harsh!
<blake_r> jtv: also jhobbs already did a review on it
<jtv> I'll be harsh if it's huge!
<blake_r> jtv: <800
<blake_r> jtv: like requested!
<jtv> Took me a while that wasn't an emoticon... "obese KKK member" or something.
<jtv> *to figure out
<blake_r> Haha!
<jtv> blake_r: Jason is right about the factory convention.  We try not to be fanatical about it, but it does help answer the question "where from?"
<jtv> Ahem.  That's a bit non-ab-ovo.  I mean: we do usually prefer "make_foo" factory functions over "get_foo" ones.
<jtv> Now all you need to do is read my messages in the wrong order.  :)
<blake_r> jtv: no make_mac_address in pserv
<blake_r> jtv: did want to add to the line count, :)
<blake_r> jtv: didn't*
<jtv> If it's for testing, there's one in maastesting.factory.  But it follows an aberrant convention: getRandomMACAddress.  :(
<blake_r> jtv: yep, the one i used!
<jtv> On another arbitrary note, compiling regexes probably isn't useful in many situations.  IIRC there's transparent caching for those million-in-a-row cases.
<jtv> Not that I'm against it, but if it gets in the way, don't let your conscience stop you.
<rvba> allenap: looks like your recent modification broke the build.
<rvba> allenap: d-jenkins.ubuntu-ci:8080/view/MAAS/job/utopic-adt-maas/54/
<rvba> allenap: paste.ubuntu.com/7574376/
<allenap> rvba: Hey, thatâs great news! Really :)
<allenap> rvba: Iâll investigate this evening.
<allenap> rvba: Is that CI of trunk?
#maas 2014-06-03
<rvba> allenap: Both trunk and 1.5 CI are failing now.
<bigjools> also, the lander is down
<bigjools> THE SKY IS FALLING
<rvba> The day is off to a good start.
<bigjools> well mine was good :)
<bigjools> the end, however, is pants
<bigjools> rvba: so I thought of a big problem today
<bigjools> I don't think we have a way to work out on which nodegroupinterface a mac belongs
<bigjools> and we need that to do the allocation
<bigjools> since django turned my brain to cottage cheese this afternoon, I am hoping your freshly awoken one will have some ideas
<rvba> Okay, I need some contextâ¦ hangout?
<lifeless> mmm cottage cheese
<rvba> heh
 * bigjools spies a lurking lifeless
<rvba> Hi lifeless
<lifeless> lurking 4 eva :)
<lifeless> rvba: o/
<jtv> Hi there lifeless!
<bigjools> rvba: if you make no progress with the lander machine we can kill it and deploy a new one
<rvba> bigjools: okay.
<bigjools> rvba: I added cards on the board from our discussion
<rvba> k, ta
<bigjools> rvba: just to check, when django deletes ipaddress, it will remove the link table entry automatically, right?
<rvba> bigjools: yeah, I think so.
<bigjools> shall we just call them stips? :)
<rvba> allenap: Actually, your change seems to have fixed the immediate problem.  But the tests failed later on: the import of the images failed.
<allenap> rvba: Do you have a paste of that?
<rvba> allenap: all the logs are here: http://d-jenkins.ubuntu-ci:8080/view/MAAS/job/utopic-adt-maas-manual/11/artifact/results/
<rvba> allenap: it's the reporting of the images that is failing with http://paste.ubuntu.com/7579121/
<allenap> rvba: Do you have the Apache logs that correspond to that? I still canât get this damned VPN to work.
<jtv> jhobbs, blake_r: always run "make lint" before submitting.  Ideally, run it with every commit.
<jtv> Two lint problems right now, one cosmetic, the other serious:
<jtv> 1. A bunch of JS lint.  This sometimes gets masked somehow.
<jtv> You know how picky browsers are â lint could mean that you've got something that'll break on somebody else's browser.
<jtv> 2. We no longer have call_capture_and_check.  I said so on the review!  The tests don't notice because they patch out the call.
<jtv> The linter notices though, so it's not just there for the cosmetics.
<blake_r> jtv: I was going to fix that in a later branch, as that was getting back ported into 1.5
<rvba> allenap: When I tried configuring the VPN with a script, it was a bit of a nightmare, but using the UI it was pretty simple.  Maybe you should try that.
<jtv> I see.  It's OK to land incomplete work, but having code actually break is a bit much...  Sometimes these things can hit spots where we can't easily diagnose the failure.
<blake_r> jtv: also i run "make lint" on everything before I submit
<jtv> Ah good.
<blake_r> jtv: I don't remeber it erroring
<allenap> rvba: The UI didnât work, and I donât know why. From the command-line it starts okay, but thereâs no indication that it sets up a difference/additional DNS server.
<jtv> Hmm...
<blake_r> jtv: as for the js errors, how do you get them to show?
<jtv> blake_r: maybe the problem then is that you ran the lint check before merging a fresh trunk.
<rvba> allenap: Maybe it's related to running Ubuntu inside a VM.
<blake_r> jtv: speaking of js look at this, maybe I am just using it incorrectly https://bugs.launchpad.net/maas/+bug/1325927
<ubot5> Ubuntu bug 1325927 in MAAS "YUI.Array.each not working as expected" [High,Confirmed]
<blake_r> jtv: i bet that was it
 * jtv looks
<rvba> gmb: any idea on how to get rid of an unreachable dying instance? (I know you've done things on the landers before.)
<gmb> rvba: Unreachable in what way?
<rvba> gmb: "It's stone dead."
<rvba> As in, it rebooted and lost its network access.
<rvba> It's as good as dead.
<jtv> blake_r: that's my main dislike about JS... so easy to make a mistake that it doesn't complain about.  Cruel to be kind, etc.
<gmb> Ooh, nasty.
<gmb> rvba: So you canât do destroy-service or remove-unit or anything like that?
<blake_r> jtv: am I using YUI.Array.each correctly?
<blake_r> jtv: as the same method, works in another piece of that file
<jtv> I don't get it either.
<blake_r> jtv: oh okay
<rvba> gmb: that's what I did.  Now the instance is 'dying'.
<rvba> Has been 'dying' for 30 minutes now.
<blake_r> jtv: i will just use options.each
<jtv> Yeah.
<rvba> I'm afraid it's a zombie now.
<gmb> rvba: I think youâre hosed then.
<gmb> Yeah.
<gmb> Juju is waiting for the unit to report back that itâs powering off
<gmb> But it can't
<rvba> gmb: so, what do you recommend?  Getting rid of the entire environment and recreating from scratch?
<gmb> rvba: Thatâs about your only option.
<rvba> It's a bit crazy that there isn't a better way out of this.
<jtv> blake_r: to run just the JS check, "make lint-js".
<blake_r> jtv: did you fix the call_capture_and_check?
<jtv> No, I only just noticed.
<gmb> rvba: Well, there may be, but I donât know enough juju gris-gris to know about it.
<blake_r> jtv: okay
<blake_r> jtv: "make lint-js" on trunk shows no isues?
<jtv> blake_r: oh dear, I've noticed that in the past, and thought it was version skew in the linter...
<jtv> It _could_ be an upstream revision that never made it into the package.
<jtv> I've got python-pocket-lint installed...  Have you?  If not, the Makefile will download the upstream tarball.
<blake_r> jtv: Installed: 0.5.31-0ubuntu1
<rvba> allenap: I tried that.  `nova list` now says its 'DELETED' but Juju still thinks it's there.
<jtv> blake_r: same here...
<blake_r> jtv: did "bzr branch lp:maas" and then "make lint-js" no issues
<jtv> Puzzled.
<blake_r> jtv: https://code.launchpad.net/~blake-rouse/maas/fix-find_mac_via_arp/+merge/221864
<jtv> Thanks.  Will review.
<blake_r> jtv: np
<jtv> Not even getting the warnings on a clean branch...
<jtv> I mean, I *am* getting them on a clean branch.
<jtv> As well as a built branch.
<jtv> blake_r: reviewed.  Thanks again.
<blake_r> jtv: np
<rvba> gmb: This is the output of `juju status`: http://paste.ubuntu.com/7579736/; do you know what are the two services are the bottom?
<blake_r> jtv: http://paste.ubuntu.com/7579740/
<rvba> tarmac-maas14 / tarmac-maas15
<gmb> rvba: The landers for 1.5 and 1.4
<rvba> Ah, right.   Silly me.
<jtv> blake_r: I did the same thing, and got the errors.  Could you maybe run the "find" command from the Makefile (in the lint-js target) manually, see if we get different results?
<rvba> gmb: allenap: Unless you guys have a better idea, I'm going to destroy the environment and re-create it.
<gmb> rvba: Go for it. Burn it.
<gmb> It canât work worse than it is right now.
<rvba> Well, some of the landers work.
<jtv> blake_r: in my case, I get a whole bunch of JS filenames vomited into my terminal when I run "find src/maasserver/static/js -type f -print0" â as expected.  Same for you?
<blake_r> jtv: yep
<blake_r> jtv: i get them all
<rvba> gmb: isn't possible to deploy the same service twice?
<rvba> Like under a different name or something.
<blake_r> jtv: http://paste.ubuntu.com/7579761/
<gmb> rvba: Yes, you could do that.
<gmb> I think itâs just an extra positional argument on the end of the deploy command.
<blake_r> jtv: i removed the print0
<blake_r> jtv: just for a cleaner output
<jtv> I also find that I get the list if I keep the -print0, and pipe it through "xargs -r0 ls"
<jtv> And I get the errors when I run just "pocketlint src/maasserver/static/js/os_distro_select.js"
<jtv> (On an up-to-date trunk)
<blake_r> jtv: i get nothing
<jtv> "which -a pocketlint" for me prints /usr/bin/pocketlint â must be the same for you I guess...
<blake_r> jtv: yep
<jtv> Gah.
<jtv> Oh, I haven't looked into the "available" magic.  Because it's, well, magic to me.
<jtv> I'll try removing the '@' sign from that command line in the Makefile, to see what command it actually runs.
<rvba> gmb: that's what the doc says, but it doesn't seem to work.  I'm using: paste.ubuntu.com/7579793/
<gmb> rvba: Do you get an error?
<rvba> error: unrecognized args: ["tarmac-maas-trunk-fixed"]
<blake_r> jtv: just did "apt-get install --reinstall python-pocket-lint"
<blake_r> jtv: still getting no errors
<jtv> blake_r: surprisingly, removing the @ isn't enough.  I had to remove the "sources = src/..." line right above, _and_ remove the @.  Then I got:
<jtv> find src/maasserver/static/js -type f -print0 | xargs -r0   pocketlint
<jtv> (I had to spell out the $(sources) value in the command line, of course)
<blake_r> jtv: no errors for that command as well
<jtv> But "make lint-js" printed that same command line?
<jtv> I'm mainly curious if it might be substituting something for the "pocketlint" command.
<blake_r> jtv: yep got  the same command
<jtv> Even tried different locale settings... no change.  :(
<rvba> gmb: I got that charm startedâ¦ let's see if it worksâ¦
<gmb> Fingers crossed
<rvba> Machine is still pendingâ¦ doesn't look good.
<jtv> blake_r: I also tried looking for latent pocketlint configuration on my system, but no dice.  Maybe there's some linking going on that makes it ignore files?  Are you using lightweight checkouts or anything like that?
<blake_r> jtv: i use buildout
<jtv> In this case you're getting an installed version of pocketlint though, so I wouldn't expect it to make a difference in itself.
<blake_r> jtv: i even added "which $(pocketlint)" to the makefile and got /usr/bin/pocketlint
<blake_r> jtv: so its using the installed pocketlint and the newest version as i just did a reinstall of it
<jtv> Yeah.
<jtv> I wonder if something somewhere installs config for it.
<jtv> Maybe an old version of the package, where if you had the older version, some warnings get suppressed or something...
<jtv> ...even if you'd since upgraded or reinstalled.
<rvba> gmb: It didn't work.  The instance never came up.
<gmb> rvba: Then weâre back to option one: burn it.
<rvba> gmb: I'm wondering why the existing (dead) instance seems to claim it was running Trusty and in Julian's instructions, it says to deploy on Saucy.
<rvba> gmb: I'm afraid we're not going to be able to bring up the other landers :/
<gmb> rvba: Oh, thatâs very weird indeed.
<rvba> When I try deploying the same charm/config on Trusty I get an error about the instance image (specified in the configâI guess) not being found.
<gmb> rvba: Is this for the trunk lander?
<rvba> Yeah
<gmb> rvba: I think that the saucy instruction is a holdover; ca
<gmb> *can you update the config and try running it again?
<rvba> Also, I'm using juju 1.19 and the bootstrap node is using 1.18.
<gmb> rvba: juju upgrade-juju should fix that, I think.
<rvba> gmb: or kill the whole env for good :)
<gmb> rvba: Well, thereâs that option :D
<rvba> gmb: looking into it there is nothing in the config that says 'Saucy'.
<gmb> Ahahahaohgod.
<rvba> Maybe we're running out of instances.  I know there is a limit.
<gmb> Hmm, that could be it.
<gmb> rvba: Or else Canonistack is falling over.
<rvba> I'll kill the 1.4 lander and try againâ¦
<gmb> ok
<rvba> tarmac-maas14 is now 'dying'.
 * rvba hopes for the bestâ¦
<rvba> I'm afraid I created another zombie machine. :/
<jtv> Brainzzz
<jtv> rvba, can you think why passing an "arch" string to the node constraints filter form might be fundamentally different from passing a "name"?
<jtv> I'm sure we've seen this before, but I keep getting the arch as a string representation of a list containing my one value.
<rvba> jtv: maybe the form expects a list of values.  And thus it would use data.request.getlist('arch').
<jtv> rvba: yes, but the field types for those two fields are identical...  I wonder if the cleaning does something weird to it.
<rvba> jtv: the only difference I can see is the form's clean_arch() method.
<jtv> rvba: ahhhh, clean_arch does "return [value]"
<jtv> Because... what?
<jtv> Changing that breaks a different test, but it's not clear to me why.
<jtv> Ah, we filter on architecture__in=arch
<rvba> Well, the filtering code assume it's a list.
<rvba> assumes*
<rvba> Right.
<jtv> And yet it's a single-value field.
<jtv> I'll try making it a consistently single-value vield.
<jtv> Ahhhh, this is for the wildcards.
<jtv> Grrr.  So we have a single-item field becoming a multi-value field during cleaning.
<jtv> blake_r: pocketlint will call external linters...  Maybe I've got some other linter installed which then gets called.
<jtv> blake_r: I'm giving up on the lint mystery for now...  a web search suggests I'm seeing messages from jslint, which I don't have installed, but which pocketlint seems to call anyway!
<jtv> rvba: for the record, I didn't find any constraints-rendering code... and it's surprisingly hard to get right, what with the type changes happening in cleaning!
<rvba> allenap: gmb: jtv:  Making progress here: the trunk lander is up and running.  Now, it tried to land allenap's branch and failed, maybe there is a config problem.
<jtv> New failure or old failure?
<jtv> The ssh one?
<rvba> https://code.launchpad.net/~allenap/maas/database-locks-revisited-at-start-up/+merge/221799
<jtv> Oh goodie, more locking
<allenap> rvba: I suspect it needs an apt-get update.
<allenap> rvba: Ah, itâs running on saucy!
<rvba> > E: Unable to locate package python-crochet
<rvba> > E: Unable to locate package python-seamicroclient
<rvba> > make: *** [install-dependencies] Error 100
<allenap> Trunk needs Trusty.
<rvba> Right, saucy.  Silly me, I followed the instructions from the google doc.
<rvba> allenap: same charm is deploying on Trustyâ¦ fingers crossed.
<allenap> rvba: Iâll mark that branch for merging again.
<rvba> allenap: already done
<allenap> I was trop tard.
<rvba> De peu.
<rvba> allenap: gosh, another failure.
<allenap> rvba: Guess what, our old friend Mr Rabbit.
<rvba> allenap: I did a manual 'make install-dependencies' on the machine and it went fine.
<rvba> allenap: lp:~allenap/maas/database-locks-revisited-at-start-up is merged!
<rvba> \o/
<rvba> Victory!
<rvba> allenap: it needs backporting to 1.5 btw.
<allenap> La victoire!
<rvba> Guys, the lander for trunk is (finally) back up.
<rvba> Anyone up for a tiny review? https://code.launchpad.net/~rvb/maas/pkg-stop-dhcp/+merge/221851
<jtv> rvba: I can probably do it in a few.
<rvba> Thanks.
<jtv> blake_r, maybe I've found the problem: pocketlint has a tendency to crash if it finds files that it doesn't recognise â and the Makefile rule doesn't filter the filenames at all!
<jtv> rvba: reviewed.
<rvba> Ta
<jtv> Got a branch up to produce more helpful error messages when no nodes can be allocated.  Anyone want to review?  https://code.launchpad.net/~jtv/maas/bug-1274085/+merge/221896
<jtv> blake_r: lint branch is up for review.  I restricted the Makefile rule.  Probably not going to solve our output difference, but it may prevent some crashes.
<rvba> jtv: I'll have a look now
<rvba> allenap: your change (revision 2400) fixed the build.
<allenap> rvba: l'aigle a atterri
<allenap> rvba: Youâre a good guy today for fixing the landers, even if you do want to kill most of your colleagues pets.
<allenap> EAPOSTROPE
<rvba> heh
<allenap> ESPELLING
 * allenap sense he is low on thinking fuel, goes to get tea
<rvba> allenap: if you backport your fix to 1.5 I'll leave your dog alone.  At least this week.
<jtv> Thanks for the review rvba.
<rvba> Welcome.
<allenap> rvba: Agreed.
<allenap> rvba: We missed an opportunity to fix the landerâs name; itâs still âMaaS Landerâ (capitalisation).
<rvba> allenap: I don't see that in the configâ¦ must be in the charmâ¦
<designated> can anyone tell me how to suppress netcfg in a preseed?  I've tried commenting all of the d-i netcfg commands but it continues to overwrite /etc/network/interfaces.
<newell> interesting discussing going on about MAAS here: https://news.ycombinator.com/item?id=7839943
<designated> blake_r, I'm getting an OAUTH error during commissioning with a difference of 6 hours.  during commisioning, the node is using UTC for some reason, but my maas server is set to local time.  I resolved the issue in the preseed by setting an NTP server but commissioning is failing now.  Any idea how to resolve this?
<roaksoax> designated: is the maas server in a different timezone from the nodes you are deployin?
<roaksoax> designated: that's the issue, OAUTH issues with different timezones
<designated> roaksoax, no, they are all in the same timezone
<roaksoax> that;s weird then
<roaksoax> the only reason why that would happen is because of a mismatch between timezones
<roaksoax> or maybe due to them not be in UTC?
<designated> roaksoax, i found /etc/init/hwclock.conf which is setting the node to UTC during commissioning
<roaksoax> designated: and is that being done by cloud init maybe?
<designated> i ran into the same problem during OS load because maas was set to local time, I resolved that issue by setting an ntp server in the preseed
<designated> roaksoax, yes but I don't know where.  I found an old bug that explains this problem but it was supposedly fixed.  they also reference modifying /etc/init/cloud-init, which doesn't exists anymore
<roaksoax> designated: well smoser is not around and he would be the one to help you with this issue
<designated> roaksoax, i found the following files /etc/init/hwclock.conf and /etc/init/hwclock-save.conf which explain it sets hwclock to UTC on boot then back to localtime on shutdown but it's not documented well enough to permit changing it.
<roaksoax> right
<roaksoax> designated: again, i do not know much about that issue, smoser would be the person you need
<designated> roaksoax, thanks
<smoser> roaksoax, o/
<roaksoax> smoser: o/
<roaksoax> smoser: designated was having some issue with changing timezones
<roaksoax> smoser: on maas
<roaksoax> designated: ^^
<smoser> cloud-init should work around any difference with clocks.
<designated> smoser, I'm getting oauth failure during commissioning of a node with a 6 hour time difference
<designated> smoser, my maas server is set to local time and i resolved the issue during OS load by adding an NTP server to the preseed but now commissioning is failing with the same problem.
<jpds> Have you considered fixing your clock?
<designated> jpds, which clock?
<jpds> Your hardware clock.
<designated> jpds, why would i want to go through the bios of hundreds of servers to set the time?  Why can't it just synch with my NTP server?
<smoser> designated, i see a cloud-init.log ? or a console log ?
<smoser> that really should be fixed.
<designated> smoser, i don't have access to the node being commissioned to get the cloud-init.log because it doesn't complete.  I do have the following on maas under /var/log/maas/maas.log: OAuthUnauthorized: 'Expired timestamp: given 1401801931 and now 1401823500 has a greater difference than threshold 300'
<smoser> designated, do you have console logs ?
<smoser> and what version / release is this that is doing commissioning ?
<designated> smoser, 1.5.1+bzr2269-0ubuntu0.1
<smoser> designated, what ubuntu release is running on commissioning.
<smoser> http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/cloudinit/sources/DataSourceMAAS.py is the code (_except_cb) that fudges the oauth headers to match whatever the server sent.
<smoser> interstingly your ntp fix causes issues also for cloud-init. which make it such that we need the monotonic timer.
<designated> smoser, 14.04
<smoser> on console output of the system , you should see things like 'Setting oath clockskew'
<designated> smoser, i have limited console output because I'm using a java application through BMC
<designated> only thing it shows right now is "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
<smoser> yeah, that sucks.
<smoser> disables what message ?
<designated> task systemd-udev:577 blocked for more than 120 seconds
<smoser> hm.. not sure thats relevant.
<designated> it probably isn't
<designated> I was just sharing the limited console output i see
<designated> the point is, the time difference is 6 hours, which indicates the time is being set to UTC on the node being commissioned, whereas the time set on my maas box is MDT
<smoser> designated, but that should not be a problem.
<smoser> we ran into "bad clocks" quite a while ago and fixed that.
<smoser> cloud-init just reads the time from the server in the 403 header
<smoser> and says "ok, i'll just ues that timestamp".
<designated> smoser, i read that bug
<smoser> but if ntp is in the mix, then you can hit the timeout and never try more than once.
<designated> let me try rebooting the node and see if i can capture console output
<designated> smoser, the only thing that was changed to resolve the oauth issue i had during OS load was the addition of the following in the preseed.
<designated> d-i     clock-setup/utc boolean false
<designated> d-i     clock-setup/ntp boolean true
<designated> d-i     clock-setup/ntp-server  string 192.168.168.1
<designated> d-i     time/zone string US/Mountain
<smoser> designated, you can also "backdoor" the image so you can get in.
<smoser> the d-i has no affect to commissioning.
<smoser> designated, the other change you can make tha tmight get you through commissioning, but i'm not sure why is maasserver/compose_preseed.py
<designated> smoser, I followed that backdoor you wrote and I couldn't get it to work.
<designated> from here: https://maas.ubuntu.com/docs/troubleshooting.html#debugging-ephemeral-image
<smoser> http://paste.ubuntu.com/7582798/
<smoser> well, the 'sudo' line should show output, if it does, then you may have to restart tgtd, but i'm not sure you should have to.
<smoser> then it should be good.
<smoser> designated, ^ try that modification in the paste above.
<smoser> you can also add 'timeout' there like this
<smoser> http://paste.ubuntu.com/7582817/
<designated> ok I added 'max_wait': 10 * 365 * 24 * 60 * 60, to /usr/lib/python2.7/dist-packages/maasserver/compose_preseed.py
<smoser> right.
<smoser> and you may likely have to restart maas-pserv to make that take affect
<designated> smoser, what is the preferred method of restarting maas-pserv?
<designated> smoser, is service maas-pserv restart acceptable?
<smoser> yeah
<smoser> shoudl be good.
<smoser> i think its pserv.
<smoser> but i might just restart everything in /etc/init/maas*
<smoser> just because i dont know and odnt want to deal with that not being it
<designated> so what does this change do besides increase the wait time to a large number?
<smoser> thats really it.
<smoser> you probably could have changed it to 60*60*6+30
<smoser> (6 hours and 30 seconds)
<designated> smoser, okay
<smoser> the problem is how cloud-init waits.
<smoser> it reads clock, tries, reads clock, and determines how much time has passed.
<smoser> if the first attempt fails because of bad clock
<smoser> and then clock gets fixed (and jumps ahead 6 hours)
<smoser> then it will think the amount of time it was supposed to wait total has passed.
<smoser> designated, you really should see *something* some kind of warning on the console. or if you get in, and collect /var/log/cloud-init* then you'll see a WARN that will direct us appropriately i think.
<designated> smoser, right now it's just sitting at initramfs
<designated> not doing anything
<smoser> thats iscsi.
<smoser> its jsut trying to get into its root.
<designated> smoser, now it's just sitting at "(initramfs) [ 29.209404] random: nonblocking pool is initialized"
<smoser> would seem unrelated.
<smoser> it shouldnt sit that long. unless tgtd didnt come back up or something.
<designated> i can try rebooting the maas server
<blake_r> designated: you will need to restart "service apache2 restart" for that preseed change
<smoser> bah. sorry.
<designated> okay, I restarted apache, rebooting node now
<designated> still hanging at the same point.  is there a way to cancel the commissioning task for this server and restart it?
<blake_r> designated: it would just restart the server
<blake_r> designated: looks like a iscsi issue then
<blake_r> designated: sudo service tgt restart
<designated> i restarted the maas server, and then booted node marked for provisioning, it got past that part but right back to oauth failure.
<designated> ERROR 2014-06-03 15:36:37,245 maasserver ################################ Exception: 'Expired timestamp: given 1401809828 and now 1401831397 has a greater difference than threshold 300' ################################
<blake_r> smoser: ^
<designated> i followed the instructions provided here: https://maas.ubuntu.com/docs/troubleshooting.html#debugging-ephemeral-image to try and add the backdoor account that Scott Moser wrote but it still does not allow me to log into the node directly so I cannot access the logs on the commissioned node.
<designated> smoser, finally got into the node.  this is the WARN message from /var/log/cloud-init.log
<designated> 2014-06-03 15:37:08,873 - DataSourceMAAS.py[WARNING]: Setting oauth clockskew to 21568
<designated> under /var/lib/maas/boot-resources/current/amd64/generic/trusty/release/ I have a root-image and a root-image.dist, both 1.4GB.  Can anyone tell me the difference between the two and which one gets used to commission nodes?
<designated> smoser, when I log into the node that is supposed to be commissioning, the "date" command shows it's using UTC instead of local time.
#maas 2014-06-04
<rvba> bigjools: I can do this: https://code.launchpad.net/~rvb/maas/ipaddresses3/+merge/221988.  But I'm not sure how to improve the readability of the condition.  It's not easy to split because each check depends on the previous check being true.
<bigjools> looking
<bigjools> rvba: I think a nested for might be better here
<bigjools> the nested list comp is uuuugly
<rvba> A nested for would be much slower though.
<bigjools> why?
<rvba> Using list comprehension means the iteration is done at a lower level.
<bigjools> also why not use itertools.chain?
<rvba> I can do that.
<bigjools> are you just using sum to expand the query set?
<rvba> It's basically the same.
<rvba> Yes.
<bigjools> right
<bigjools> sum obfuscates it a bit
<bigjools> it's a bit like me getting a MatchError earlier today because "[] != []"
<bigjools> the joy of repr :)
<bigjools> rvba: https://code.launchpad.net/~julian-edwards/maas/deallocate-static-ip/+merge/221825 if you fancy it
<rvba> bigjools: sure
<bigjools> rvba: and https://code.launchpad.net/~julian-edwards/maas/macaddress_to_ngi/+merge/221984 is easy karma
<rvba> bigjools: reviews done.
<rvba> I'm landing jtv's lint branch.  It looks pretty safe but I'll deal with any fallout.
<xperia> Hi All. i was able to successful install Maas on my Ubuntu Server and to Commision my first Node. Maas Show now that the Status of my Node is Ready. When i Try however to PXE Boot my new Node Nothing Happens if i do however try to PXE Boot the Node over my Router however it works!
<xperia> What could be the Problem and how can i fix it? I would love to be able to Start Up the Nodes with MaaS itself instead over the Router.
<smoser> designated, the .dist was created by that shell snippet.
<smoser> hm..
<smoser> interresting.
<smoser> i wonder if htat is busted.
<smoser> nah. we are using hard links there, which might make it ocnfusing, but the script does a cp to the .dist, and then modifies the original so that should be ok.
<gmb> rvba, allenap, jtv: Can I get a review for https://code.launchpad.net/~gmb/maas/fix-commissioning-page-distro-list-bug-1312844/+merge/222010? Iâm losing the ability to tell if it passes the smell test, so Iâm perfectly happy with informed rejection :)
<allenap> gmb: Sure.
<gmb> Ta
<rvba> Code smell detector activated.
<allenap> rvba: Are you taking a look at that branch too then?
<rvba> allenap: no, since you're reviewing it.
<allenap> rvba: Would you mind casting your eye over it for the forms stuff?
<rvba> allenap: sure
<designated> smoser, I'm not sure what happened but I'm going to save off all of our custom preseeds and blow maas away.  I'm going to do a fresh install.  One of the interesting things I was seeing on nodes being commissioned, the hwclock was being adjusted to the correct local time but the timezone was still being reported as UTC.
<smoser> designated, time zones dont really matter.
<smoser> the only thing that really matters is the bios clock.
<designated> is there a way to get a list of all ipmi addresses of all nodes in maas from the CLI?  'maas <profile> nodes list' doesn't list the ipmi ip address.
<smoser> once you're up, a reboot will have ubuntu set the hardware clock to be utc.
<smoser> designated, no. there is not. you can probalby get it fairly easily with python
<smoser> https://bugs.launchpad.net/maas/+bug/1233158
<ubot5> Ubuntu bug 1233158 in MAAS "no way to get power parameters in api" [Medium,Triaged]
<designated> word
<designated> I'll just dump them from the arp table on one of my routers and write a quick shell script to power cycle each one using ipmitools
<designated> smoser, should the DNS zone name under cluster controller be a FQDN?
<smoser> i'd tihnk so. like example.com
<smoser> i think
<designated> smoser, thanks
<designated> smoser, so I have a fresh load of maas, finished importing boot images, added backdoor login and tried enlisting a node.  the node boots and sits at login screen, it never finishes and powers off, and is not showing up in maas.
<designated> smoser, i do see some failure messages in cloud-init.log such as: 2014-06-04 14:24:42,994 - importer.py[DEBUG]: Failed at attempted import of 'ubuntu' due to: No module named ubuntu
<smoser> was this always the problem? that it was failing to enlist ?
<designated> smoser, no, it was working previously
<designated> smoser, running 1.5.1+bzr2269-0ubuntu0.1
<designated> smoser, like i said this is a fresh build.
<smoser> there is some output going somewhere probably. not sure whats happening, but if it turns off then some code ran. that output would have been to the vga console i think. so you should have seen it.
<smoser> it will sit doing nothing for a while
<designated> smoser, it doesn't power off.  i can try powering it off via ipmi
<designated> smoser, i powered it off and it showed up in maas.  it must have failed to detect ipmi.
<smoser> designated, oh. if it doesnt power off, then you should be albe to ssh in and look around.
<smoser> (since you back-doored it)
<smoser> and apt-get install pastebinit and then pastebinit /var/log/cloud-init*
<designated> smoser, i manually set ipmi settings, trying to commission it now to see if that will work.
<designated> smoser, and right back to where i was yesterday during commissioning.
<designated> OAuthUnauthorized: 'Expired timestamp: given 1401893824 and now 1401915424 has a greater difference than threshold 300'
<magicrobotmonkey> is there a way to read the power settings from the maas cli tool?
<magicrobotmonkey> on the gui i can see the ip, user, and pass
<magicrobotmonkey> but i cant get it from the cli
<magicrobotmonkey> i have power_type in read
<magicrobotmonkey> but not the other information
<designated> magicrobotmonkey, 'maas <profile> nodes list' will show you the power type but there is no CLI command to show ipmi, ip address, username or password.
<magicrobotmonkey> ok thanks
<designated> blake_r, smoser either one of you around?
<designated> during commissioning, it looks like it never finishes because it cannot apt-get update.  I see a process running: root      1413  0.0  0.0  29008  2376 ?        S    17:13   0:00 apt-get --assume-yes -q update
<designated> if i try to update it manually, i get the following:
<designated> $ sudo apt-get update
<designated> sudo: unable to resolve host vn-mgmt-0
<designated> E: Could not get lock /var/lib/apt/lists/lock - open (11: Resource temporarily unavailable)
<designated> E: Unable to lock directory /var/lib/apt/lists/
#maas 2014-06-05
<smoser> designated, your error is probably true.
<smoser> it couldn't get a lock because the process running (that never completeded) had it locke
<smoser> you could strace that process and probably get some more info
<smoser> maybe you have outbound internet access blocked ?
<designated> smoser, it's not blocked, maas is not recognizing the domain as being managed even though it's configured correctly and forwarding the dns request to the configured dns forwarder
<smoser> the sudo error is not relevant.
<smoser> it happens any time dns resolution of 'hostname' fails.
<smoser> which is fine.
<designated> smoser, correct but that's what I'm trying to explain.
<smoser> thats fine though.
<smoser> if you're apt-get update hung, then thats the problem.
<designated> apt-get update is hanging because of a dns issue and never completing enlistment.
<smoser> no. i don't think so.
<smoser> dns resolution of `hostname` is fialing
<designated> it is because if i kill the process it finishes enlistment immediately
<smoser> can you verify it works for anything else ?
<smoser> and resolution would *fail* not hang.
<smoser> you're on the system now?
<designated> yes
<smoser> do 'ping archive.ubuntu.com'
<smoser> i think you'll get dns resolution
<smoser> or if you dont, then yes, dns is the issue.
<designated> it will resolve that
<designated> apt-get update doesn't succeed because it cannot resolve it's own hostname
<smoser> doesn't care.
<smoser> thats not why its hanging. i'm certain of that.
<smoser> you can replicate that anywhere like this:
<designated> this process is running: root      1384  0.0  0.0  31064  2380 ?        S    20:13   0:00 /usr/bin/apt-get --option=Dpkg::Options::=--force-confold --option=Dpkg::options::=--force-unsafe-io --assume-yes --quiet update
<designated> if i kill it, enlistment will finish
<smoser> strace it
<smoser> what is it doing.
<smoser> or even just tail /var/log/cloud-init-output.log
<smoser> sudo strace -p 2380
<designated> $ sudo strace -p 1384
<designated> sudo: unable to resolve host maas-enlisting-node
<designated> Process 1384 attached
<designated> select(8, [6 7], [], NULL, {0, 23729})  = 0 (Timeout)
<designated> select(8, [6 7], [], NULL, {0, 500000}) = 0 (Timeout)
<designated> select(8, [6 7], [], NULL, {0, 500000}) = 0 (Timeout)
<designated> select(8, [6 7], [], NULL, {0, 500000}) = 0 (Timeout)
<designated> Ign http://archive.ubuntu.com trusty-updates InRelease
<designated> Err http://security.ubuntu.com trusty-security Release.gpg
<designated>   Connection failed
<designated> smoser, mackrel is working with me on this issue
<designated> just letting you know so he can ask questions about the same issue
<designated> smoser, could it be an issue with the proxy server on maas?
<smoser> designated, see, its timing out on a network connection.
<smoser> you set a proxy in maas ?
<designated> smoser, no but i thought by default the apt-get requests got proxied through maas
<smoser> you can see if that was correctly written into /etc/apt (grep -r Proxy /etc/apt)
<designated> under maas gui there is an option to configure proxy server, it says if you leave it blank "This will also be passed onto provisioned nodes instead of the default proxy (the region controller proxy)."
<smoser> it could be an issue then on the squid proxy on the maas region controller
<smoser> try:
<designated> /etc/apt/apt.conf.d/95cloud-init-proxy:Acquire::HTTP::Proxy "http://192.168.168.7:8000/";
<smoser>  http_proxy=http://your.maas.ip.addr:3128 wget http://security.ubuntu.com
<smoser> i suspect that will hang similarly.
<smoser> er... s/3128/8000/
<designated> Resolving your.maas.ip.addr (your.maas.ip.addr)... failed: Name or service not known.
<designated> wget: unable to resolve host address Ã¢your.maas.ip.addrÃ¢
<designated> shit...just a sec
<smoser> :)
<mackrel> smoser, designated, yes it is stalling.
<smoser> yeah, thats not going to work :)
<designated> noobing it up tonight
<designated> just hangs
<designated> Connecting to 192.168.168.7:8000... connected.
<designated> Proxy request sent, awaiting response...
<designated> i wonder how squid proxy got jacked up
<mackrel> sudo service squid-deb-proxy status squid-deb-proxy start/running, process 1005
<designated> i show squid-deb-proxy as well as squid3 installed.  are both of these needed?
<designated> squid3 is listening on TCP/8000
<designated> proxy     1005  0.0  0.1 127620 30736 ?        Ssl  18:30   0:01 /usr/sbin/squid3 -N -f /etc/squid-deb-proxy/squid-deb-proxy.conf
<designated> proxy     1233  0.0  0.1 113348 20260 ?        Ss   18:32   0:00 /usr/sbin/squid3 -N -YC -f /etc/squid3/squid.conf
<smoser> is squid proxy simply blocked on outbound network connections ?
<smoser> ie, from maas sytem can you hit archive.ubuntu.com ?
<designated> smoser, it's not blocked
<designated> smoser, yes i can
<smoser> can you try the wget above on the maas region controller ?
<smoser> maybe just try restarting squid. that doesn't give you warm fuzzies, but id ont know.
<designated> smoser, the wget from the controller succeeds but not when i proxy the request through itself.
<designated> mackrel, didn't we have a similar problem in the lab?
<smoser> right.
<smoser> so yeah, squid is messed up. did you try restarting it ?
<designated> i did
<designated> no difference
<smoser> i know thats a hack.
<smoser> but i dont know why it would be hung.
<smoser> you can look at its logs for some info.
<mackrel> yes.  we experienced this issue before but this was working and has deteriated to this
<designated> mackrel, right, no changes were made to squid.  everything was working, then it stopped.  i rebuilt maas from scratch today and we're still seeing this issue.
<smoser> so see if there is anything in squid error or access logs that gives you any hiints
<smoser> the only guess i have a this point is that squid's dns resolution is borked. suspecting something to do with maas taking over dns on that system.
<smoser> but i dont have a lot of faith in that theory
<designated> 1401936515.205   4723 192.168.168.7 TCP_MISS_ABORTED/000 0 GET http://security.ubuntu.com/ - HIER_DIRECT/2001:67c:1562::15 -
<designated> i don't understand why there are two squid processes running
<designated> proxy     8372  0.0  0.1 115640 22480 ?        Ss   20:45   0:00 /usr/sbin/squid3 -N -f /etc/squid-deb-proxy/squid-deb-proxy.conf
<designated> proxy     8464  0.0  0.1 113216 19956 ?        Ss   20:48   0:00 /usr/sbin/squid3 -N -YC -f /etc/squid3/squid.conf
<smoser> well, yeah, that is kind of silly. :)
<smoser> but one of them is squid deb proxy and one is just squid.
<designated> maas uses squid-deb-proxy...no?
<smoser> squid deb proxy actually i think probably runs on 3128
<smoser> err.
<smoser> i might be rwong
<smoser> i am wrong
<smoser> 8000 is squid deb proxy
<mackrel> sudo netstat -tanpo | grep 3128
<mackrel> tcp6       0      0 :::3128                 :::*                    LISTEN      8464/squid3      off (0.00/0/0)
<designated> tcp6 not 4
<designated> both of them say tcp6
<mackrel>  sudo netstat -tanpo | grep 8000
<mackrel> tcp6       0      0 :::8000                 :::*                    LISTEN      8372/squid3      off (0.00/0/0)
<mackrel> one process is listening 3128 and another on 8000
<smoser> you're saying squid just isn't listening on ipv4 ?
<smoser> telnet localhost 8000 ?
<designated> smoser, it connects
<designated> does squid get installed as a depend of maas-dns?
<mackrel> localhost resolves to ::1 in /etc/hosts, so I imagine when squid starts it binds itself to localhost and resolve ipv6.  we can probably comment that out, restart squid and it would list on ipv4
<smoser> probably not maas-dns. prbobaly maas region controller
<designated> mackrel, any ideas?
<mackrel> designated, not really.  dns resolution was first step but now it is proxy... seems pretty weird we got three dozen nodes to register before and now kaput
<designated> smoser, thanks for your help.  we're going to have to figure out wth squid is doing
<smoser> designated, sorry couldn't get you past that issue.
<rvba> gmb: I'm marking your fix-commissioning-page-distro-list-bug-1312844 branch "needs fixing".  The problem I describe could have gone unnoticed because the testing coverage is not complete in this area.  Happy to help you with this when you're back from your mini sprint.
<gmb> rvba: Merci. Yeah, youâre rightâ¦ I kind of knew I was on a bit of a wing and a prayer tests-wise :)
<gmb> anyway
 * gmb -> sprinting
<rvba> bigjools: why do you think it's best to set it in start_nodes()?
<bigjools> rvba: lol
<bigjools> rvba: because I figured chaining the jobs would be better, but as you just pointed out we need host entries for other types too
<bigjools> the other job being the power_on
<bigjools> hmmm
<bigjools> I'll add it tomorrow.
<bigjools> in claim_static_ip() I mean
<rvba> Sounds good.
<bigjools> we have to hope celery gets to it before the power_on :)
<rvba> This is a bit of a gamble.
<bigjools> quite
<bigjools> hence my question
<bigjools> it's not so cut and dry
<rvba> bigjools: we can't reasonably take that risk.
<bigjools> rvba: yeah, and I think in terms of abstraction it makes sense to leave it out
<rvba> I would have preferred to have the change in the DB and the setting of the host entry in one place.  Because they are the two sides of the same coin (one part is internal the other is external).
<JayJ> I need help debugging MaaS. The node (VM) fails to Commission
<Jay_> I need help to debug MaaS. Anyone?
<Jay_> I posted the question in askubuntu: http://askubuntu.com/questions/477028/maas-fails-to-commission-nodes
<rvba> Hi Jay_; thanks for the question.  I'll have a look at the logs you provided in a short while.  I'll get back to you when I do (probably on askubuntu.com).
<Jay_> rvba: Thank you very much
<Jay_> rvba: Please let me know if you need any more logs from the box.
<rvba> Jay_: I suggest you have a look at the machine's logs (syslog & co) when it fails to get its IP address.  Maybe you'll find a hint as to why it failed commissioning.
<Jay_> rvba: Do these logs make any sense?
<Jay_> Jun  4 20:03:56 maas dhcpd: DHCPNAK on 50.50.50.13 to 08:00:27:b9:e6:16 via eth0
<Jay_> Jun  4 20:04:12 maas dhcpd: DHCPDISCOVER from 08:00:27:b9:e6:16 via eth0
<Jay_> Jun  4 20:04:12 maas dhcpd: DHCPOFFER on 50.50.50.58 to 08:00:27:b9:e6:16 via eth0
<Jay_> Jun  4 20:04:12 maas dhcpd: DHCPREQUEST for 50.50.50.13 (50.50.50.3) from 08:00:27:b9:e6:16 via eth0: lease 50.50.50.13 unavailable.
<rvba> Jay_: the "lease <ip> unavailable" doesn't look too good.
<Jay_> rvba: That's where I think MaaS messing up something. Don't know how to proceed as I know little about MaaS
<rvba> Jay_: can you share the content of the lease file? /var/lib/maas/dhcp/dhcpd.leases
<rvba> leases* even
<rvba> Jay_: The DHCPNAK message also seems to indicate something is wrong with the network config/the DHCP config.
<Jay_> rvba: Copied here: https://www.dropbox.com/s/fwu0db3orphz2jk/dhcp-leases
<Jay_> rvba: Appears that MaaS assigns an IP and binds MAC during enlistment. Then it is getting confused during Commissioning!
<rvba> Jay_: yes, the assignment is made the first time MAAS sees a node.  It should be used throughout a node's lifecycle so that the IP doesn't change.
<Jay_> makes sense. I thought so too. However, the VM PXE boots again during Commissioning, whcih is when it is getting confised
<magicrobotmonkey> which tool do you use to change the volume quotas?
<magicrobotmonkey> cinder!
<designated> blake_r, do you know of a quick way to restart all maas services?
<blake_r> designated: i just normally restart them one at a time
<designated> blake_r, so just restart everything in /etc/init/maas-* one at a time?
<blake_r> yes
<blake_r> and apache2
<blake_r> and tgt
<designated> blake_r, thank you.
<designated> blake_r, do you know of a way to disable maas forcing the nodes to use the maas controller as a proxy?  during enlistment, squid-deb-proxy doesn't seem to be functioning correctly, I've been troubleshooting it for a couple of days now with no success.  I keep getting:
<designated> Err http://security.ubuntu.com trusty-security Release.gpg
<designated>   Connection failed
<designated> all of my nodes have direct internet access.
<designated> smoser, who is responsible for working on the squid-deb-proxy portion of maas and can assist in troubleshooting this issue?
<smoser> well squid-deb-proxy is just an ubuntu package. maas depends on it. you can file a bug against squid-deb-proxy using 'ubuntu-bug squid-deb-proxy'.
<smoser> and you can probably turn up debug info in squid
<smoser> http://www.squid-cache.org/Doc/config/debug_options/
<designated> smoser, do you know a way to prevent maas from forcing the nodes to proxy the apt requests?
<smoser> designated, grep through /etc/maas -r for http_proxy or just proxy and see if you see anything
<smoser> i think it should show up there.
<smoser> and i think you should be even able to set the proxy in the maas web ui
<designated> smoser, i don't want a proxy
<smoser> right. i suspec tyou'll see it set to some value
<smoser> and you can unset it
<designated> smoser, I'll try that.  thank you
<designated> smoser, the file /etc/maas/preseeds/commissioning only contains {{preseed_data}}.  Does that get pulled in from 'generic' and 'preseed_master'?
<smoser> probably rendered in maas internal.
<smoser> i'd hvae to look at it.
<smoser> i dont really know., anyone know how to globally disable the squid proxy ?
<designated> i successfully disabled the proxy server during enlistment and it enlisted perfectly.  now trying to go the same for commissioning
<designated> smoser, I think I found it here: /etc/maas/templates/commissioning-user-data/user_data_config.template
#maas 2014-06-06
<Mosibi> I have 2 dhcp and dns managed networks connected to maas. Some hosts (compute nodes) have connections in both network (management and neutron/network). I now see that the short hostname sometimes is CNAME'd to the address on the second interface.
<Mosibi> I would be nice to set a domainname per interface. Then we never have CNAME collisions...
<Mosibi> Is this possible?
<jtv> No, I don't think it is.
<Mosibi> That's not the answer i hoped for ;)
<Mosibi> But... of course it is possible, it's just a bit harder to code :)
<jtv> And configure, I guess â I don't think anyone can predict on which network you want to address the node.
<Mosibi> jtv: but setting the CNAME could be bound to one interface.
<jtv> You can ask for a node's addresses, but all the DNS server can really do is give you one, or give you all.
<jtv> We could bind the cname to one interface on the node, yes...  You'd then have to tell it which node though.
<jtv> Which still ends up being extra work for you.  :(
<Mosibi> On the second net i would just have the generated names 1-1-1-1 A short_name on the 'not primary' interfaces.
<Mosibi> Only on the primary interface the CNAMEs
<Mosibi> Maybe it's an idea to put all the addresses in the sql database and from there deploy DNS and DHCP. MAAS then can be extended with more control over the adresses by giving the user a GUI for DNS/DHCP management if he/she wants
<Mosibi> I do not care if i have to change something to my wishes. But now if i change it on disk and MAAS is regenerating the zone files, my changes are gone.
<allenap> Mosibi, jtv: BIND views could be used to serve up the correct address depending on which network youâre asking via, but MAAS doesnât support that. We donât have any plans to support it either, but if enough people want it we could.
<Mosibi> allenap: that's also a possibility indeed...
<Mosibi> Next week I will discus this within our team and probably we will come with a proposition.
#maas 2014-06-08
<dchem> Hello everyone
<dchem> I have a question. I have a dell cloud computing server (F1DH), and when I commission it with one 1 TB sata HDD in, it commissioning works as intended and I see it as having 1TB of storage. However, when I put two or more disks in, the installation either 1) recognizes only one drive 2) only shows 1406 mb of space after commissioning
<dchem> is this an issue with preseed?
#maas 2015-06-01
<mup> Bug #1460515 was opened: package maas-region-controller 1.7.3+bzr3363-0ubuntu2 failed to install/upgrade: sub-processo script post-installation instalado retornou estado de saÃ­da de erro 1 <apport-package> <i386> <vivid> <maas (Ubuntu):New> <https://launchpad.net/bugs/1460515>
<mup> Bug #1460515 changed: package maas-region-controller 1.7.3+bzr3363-0ubuntu2 failed to install/upgrade: sub-processo script post-installation instalado retornou estado de saÃ­da de erro 1 <apport-package> <i386> <vivid> <maas (Ubuntu):New> <https://launchpad.net/bugs/1460515>
<mup> Bug #1460515 was opened: package maas-region-controller 1.7.3+bzr3363-0ubuntu2 failed to install/upgrade: sub-processo script post-installation instalado retornou estado de saÃ­da de erro 1 <apport-package> <i386> <vivid> <maas (Ubuntu):New> <https://launchpad.net/bugs/1460515>
<mup> Bug #1460614 was opened: FQDN column heading underline space <MAAS:New> <https://launchpad.net/bugs/1460614>
<voidspace> so I have a failed deployment - maas.log says:
<voidspace> Marking node failed: Installation failed (refer to the installation log for more information).
<voidspace> Which installation log is that - on the node?  (and where specifically)
<mup> Bug #1460643 was opened: [1.8] PhysicalBlockDevice endpoint is incorrect for 1.9 <api> <MAAS:In Progress by blake-rouse> <https://launchpad.net/bugs/1460643>
<mup> Bug #1460643 changed: [1.8] PhysicalBlockDevice endpoint is incorrect for 1.9 <api> <MAAS:In Progress by blake-rouse> <https://launchpad.net/bugs/1460643>
<mup> Bug #1460643 was opened: [1.8] PhysicalBlockDevice endpoint is incorrect for 1.9 <api> <MAAS:In Progress by blake-rouse> <https://launchpad.net/bugs/1460643>
<voidspace> so I have a failed deployment - maas.log says:
<voidspace> Marking node failed: Installation failed (refer to the installation log for more information).
<voidspace> Which installation log is that - on the node?  (and where specifically)
<voidspace> allenap: know anyone who can help me with this? ^^^
<allenap> voidspace: I'll find someone who knows what they're doing :)
<rvba> voidspace: on the node details page.
<rvba> voidspace: or using the API http://maas.ubuntu.com/docs/api.html#commissioning-results
<voidspace> rvba: thanks the machine details page doesn't give any more information
<voidspace> in fact it gives less information
<rvba> voidspace: which version of MAAS is this?
<mup> Bug #1460678 was opened: MAAS writes logs to maas.log.1 <MAAS:New> <https://launchpad.net/bugs/1460678>
<voidspace> rvba: 1.8 from dailybuilds
<voidspace> rvba: this is from the machine details page
<voidspace> http://pastebin.ubuntu.com/11498415/
<voidspace> ah wait
<voidspace> I didn't scroll down far enough it seems...
<roaksoax> voidspace: go sall the way to the bottom
<voidspace> modprobe: FATAL: Module hpvsa not found.
<voidspace> Installation failed with exception: Unexpected error while running command.
<voidspace> Command: ['sh', '-c', 'depmod && modprobe hpvsa']
<rvba> voidspace: at the bottom of the page you should have the install log
<voidspace> thanks
<voidspace> this is the full log: http://pastebin.ubuntu.com/11498446/
<voidspace> so I will disable proprietary drivers and try again
<voidspace> rvba: with propietary drivers disabled it's working
<voidspace> rvba: want me to file an issue?
<mup> Bug #1460199 changed: NoConnectionsAvailable: Unable to connect to cluster 3e375000-9ece-453c-9b5e-f817ab3a51b2; no connections available. <MAAS:Incomplete> <https://launchpad.net/bugs/1460199>
<rvba> voidspace: please
<mup> Bug #1460199 was opened: NoConnectionsAvailable: Unable to connect to cluster 3e375000-9ece-453c-9b5e-f817ab3a51b2; no connections available. <MAAS:Incomplete> <https://launchpad.net/bugs/1460199>
<mup> Bug #1460199 changed: NoConnectionsAvailable: Unable to connect to cluster 3e375000-9ece-453c-9b5e-f817ab3a51b2; no connections available. <MAAS:Incomplete> <https://launchpad.net/bugs/1460199>
<garibaldi> Hello - I set up MAAS 1.6 on Ubuntu 14.04 and it works well, but I cannot figure out how to run a script on the node after commissioning
<garibaldi> how can this be done?
<mup> Bug #1460128 changed: whitespace above header keeps growing on click for chromium <ui> <MAAS:Won't Fix by ricgard> <https://launchpad.net/bugs/1460128>
<rbasak> garibaldi: I'm pretty sure you can add custom commissioning scripts. Would that do?
<garibaldi> that would work... all I need to do is modify the contents of a file after the server has been provisioned
<garibaldi> how can I add a custom commissioning script
<garibaldi> ?
<mup> Bug #1460714 was opened: WebUI does not support soft power off <MAAS:New> <https://launchpad.net/bugs/1460714>
<rbasak> Well, commissioning is before a server is provisioned.
<kiko> rbasak, shouldn't he use cloud-init for that?
<kiko> garibaldi, can you tell us a bit more about what you are trying to do?
<mup> Bug #1460714 changed: WebUI does not support soft power off <MAAS:New> <https://launchpad.net/bugs/1460714>
<mup> Bug #1460714 was opened: WebUI does not support soft power off <MAAS:New> <https://launchpad.net/bugs/1460714>
<mup> Bug #1317601 changed: [SRU] SRU New upstream bugfix release of MAAS <verification-done> <maas (Ubuntu):Fix Released> <maas (Ubuntu Trusty):Fix Released> <https://launchpad.net/bugs/1317601>
<mup> Bug #1329267 was opened: CLI does not tell users to issue a "refresh" when the API gets out of date <cli> <upgrade> <MAAS:Fix Committed by allenap> <MAAS 1.7:Fix Committed by allenap> <maas (Ubuntu):New> <https://launchpad.net/bugs/1329267>
<mup> Bug #1387859 was opened:  When MAAS has too many leases, and lease parsing fails, MAAS fails to auto-map NIC with network <MAAS:Fix Committed by julian-edwards> <MAAS 1.7:New> <maas (Ubuntu):New> <https://launchpad.net/bugs/1387859>
<mup> Bug #1456969 was opened: MAAS cli/API: missing option set use-fast-installer / use-debian-installer <api> <canonical-bootstack> <MAAS:Fix Committed by rvb> <MAAS 1.7:In Progress by rvb> <maas (Ubuntu):New> <https://launchpad.net/bugs/1456969>
<roaksoax> Good signature on ../maas_1.7.5+bzr3369-0ubuntu1~14.04.1.dsc.
<garibaldi> kiko: I want to be able to ssh to the node as root, but the default /root/.ssh/authorized_keys file contains a command that echos out a string and then closes the connection
<garibaldi> kiko: so I want to modify the /root/.ssh/authorized_keys file to allow root ssh access
<garibaldi> kiko: I see also that there is "disable_root" in /etc/cloud/cloud.cfg, but I don't know how to set this from MAAS so that it gets propagated to the node
<kiko> smoser, do you have advice for this? or jhobbs?
<kiko> oh, jhobbs ain't here
<smoser> garibaldi, provide user-data when you deploy nodes.
<smoser> #cloud-config
<smoser> disable_root: false
<smoser> or just learn how to use sudo :)
<smoser> maas's cli for 'node start' takes a user_data key
<smoser> that is a base64 endoded file. that becomes cloud-init's user-data
<garibaldi> smoser: thanks for the help; I am using Ansible to deploy my nodes
<garibaldi> smoser: and my Ansible setup connects to the nodes as root, so I need to be able to directly connect to root with an SSH key, not using sudo
<kiko> I see
<smoser> you'll need to provide user-data.that user-data will function for you anywhere (ec2, azure ... where you see an ubuntu cloud image)
<smoser> i suspect that if ansible has support for launching ubuntu images elsewhere, that this is not a new problem
<kiko> garibaldi, sounds easy!
<garibaldi> smoser, kiko: how do I provide user data?
<kiko> <smoser> maas's cli for 'node start' takes a user_data key
<kiko> garibaldi, ^^
<garibaldi> how do I instruct MAAS to pass it to the nodes?
<kiko> use the maas cli as smoser suggests above
<smoser> garibaldi, cli/api . you pass 'user_data' key in.
<smoser> maas $MYMAASNAME node start system_id user_data=$(base64 my-user-data) distro_series=trusty
<smoser> http://bazaar.launchpad.net/~virtual-maasers/+junk/maas-libvirt-utils/view/head:/maas-deploy-node is a wrapper i use lots of places "maas-deploy-node".
<kiko> smoser, this is missing from the cli docs on m.u.c it seems?
<garibaldi> smoser: the user_data string could contain a bash command, or the contents of a bash script?
<smoser> garibaldi, https://help.ubuntu.com/community/CloudInit covers user-data
<smoser> kiko, i dont know if its documented, i dont know how i found it :). i know it does work.
<garibaldi> smoser: e.g maas $MYMAASNAME node start system_id user_data=$(base64 $(cat /tmp/script.sh)) distro_series=trusty
<smoser> garibaldi, mine didnt work ?
<smoser> i dont think you want or need 'cat' there
<smoser> ah. sorry.. .here.
<garibaldi> or is it --user-data-file
<smoser> for maas it is:
<smoser>  maas MAASNAME node start SYSTEM_ID distro_series=RELEASE user_data=$(base64 --wrap=0 YOUR_USERDATA_FILE)
<smoser> MAASNAME: as seen in
<smoser> MAASNAME: as seen in maas list
<smoser> SYSTEM_ID: the system_id from maas MAASNAME nodes list
<smoser> RELEASE: trusty, vivid, wily ...
<smoser> YOUR_USERDATA_FILE: whatever you want as described in https://help.ubuntu.com/community/CloudInit
<garibaldi> can I pass a userdata file via the Web UI as well?
<garibaldi> when commissioning nodes via the Web UI?
<garibaldi> smoser: I have 4 nodes commissioned in the Web UI, however after running "maas login local http://<my-mass-hostname>/MAAS/api/1.0 <key>", I only see the key data when I do "maas list"
<garibaldi> smoser: I do not see any nodes listed, nor MAASNAME or SYSTEM_ID items
<mup> Bug #1460816 was opened: MAAS doesn't power cycle systems after not seeing a PXE request <oil> <MAAS:Triaged> <https://launchpad.net/bugs/1460816>
<kiko> garibaldi, I don't know whether you can pass userdata via the web UI, I think not
<kiko> garibaldi, sounds like you don't have the cli working, oddly
<kiko> garibaldi, before you annoy yourself further, I wonder, could you upgrade to 1.7 as per the topic?
#maas 2015-06-02
<pepe233> Hi
<CyborgCygnus> Are maas & juju two completely different things, are they competitors or do you use them together/same time?
<rbasak> CyborgCygnus: they perform at two different layers in the stack.
<rbasak> CyborgCygnus: Juju works at "I need an instance" level. It can use EC2, Openstack, Azure and others to fulfil that need. Or it can use MAAS.
<rbasak> CyborgCygnus: MAAS is a "cloud provider" in the same sense as EC2's "just give me a working instance" by API model. Except that it uses bare metal to get those instances.
<rbasak> CyborgCygnus: so, MAAS to provision bare metal, and Juju to make use of that in a smart way. Or just use MAAS for bare metal and have something else call the MAAS API directly. Or just use Juju with a different provider.
<CyborgCygnus> rbasak, okay still confusing but good information, I've been eyeing off all of these ubuntu server, openstack cloud & other affiliated those tools.
<rbasak> CyborgCygnus: still confusing> maybe I can try again? MAAS is relatively simple to explain. Get freshly installed bare metal machines from a pool of bare metal machines by API. MAAS takes care of managing the machines and provides the API. Does that make sense?
<CyborgCygnus> rbasak, yeah get that pretty much, does juju use maas to do its thing? I was confused as to whether maas is the essential thing to have an juju just an accessory for web browser based controller for ease
<rbasak> CyborgCygnus: let me try and explain Juju first. Juju lets you supply a model of the deployment you want in terms of a set of services and how those services are connected to each other. Then it turns that model into reality by deploying instances are required.
<rbasak> by deploying instances as required.
<rbasak> Does that make sense?
<rbasak> That means that Juju needs to somehow deploy instances.
<rbasak> To do that Juju uses a pluggable architecture of "providers".
<rbasak> There is a Juju MAAS provider, which you can use to turn your Juju model into reality on bare metal.
<CyborgCygnus> rbasak, okay yeah a bit more understandable for noob slow me
<rbasak> There is also a Juju EC2 provider, which you can use to turn the same Juju model into reality on EC2 instead.
<CyborgCygnus> rbasak, which one do you prefer, EC2 or MAAS?
<rbasak> CyborgCygnus: depends on where you want your deployment.
<rbasak> CyborgCygnus: want it on your own physical machines? You'll spend money on those machines, electricity and hardware maintenance but nothing after that.
<rbasak> CyborgCygnus: or, instead, want Amazon to take care of capital expenditure, electricity and hardware maintenance for you? Then use EC2 instead.
<CyborgCygnus> rbasak, Oh okay :S , so say at home for some reason I wanted 6 machines in the house to install ubuntu through the one server & have their accounts stored on the server so they can log on from any computer & log into their account
<rbasak> 6 machines to do what, and what do you mean by account exactly?
<CyborgCygnus> rbasak, well I remember way back when I was in high school every student had a login that they can put into any school pc & they get access to their files aswell as the operating system. The 6 machines just ubuntu operating system
<rbasak> CyborgCygnus: so you want 6 machines running Ubuntu desktop with central authentication and a central file server? You can do that with Ubuntu, all the tools are there. But there isn't a turnkey system to provide that - most sysadmins put together a customised deployment manually.
<CyborgCygnus> rbasak, so what would be an example use of the ubuntu server with openstack, maas & juju?
<rbasak> CyborgCygnus: a simple example? A web app.
<rbasak> CyborgCygnus: a complex example is an Openstack deployment itself, though that doesn't generally use Openstack obviously.
<CyborgCygnus> rbasak, alright thanks, So deploying a web app to where? Or where do you deploy openstack to? As in installing it to another server or computer over the network to ?
<rbasak> CyborgCygnus: deploying a web app to anywhere that can provide instances. EC2, Azure, an Openstack cloud, or directly on bare metal with no virtualisation with MAAS (though for a basic web app that'd be quite wasteful).
<rbasak> CyborgCygnus: deploying Openstack is normally to bare metal in a datacentre somewhere in order to run a private cloud - like EC2 but a private one.
<CyborgCygnus> rbasak, thanks for your help
<smoser> garibaldi, sorry, i went afk
<smoser> the 'maas list' shows the accounts that you have set up.
<smoser> so you did: maas login some-name-here some-url-here some-creds-here
<smoser> then, from then on out, MYMAASNAME is 'some-name-here'
<smoser> ie, you have to tell the maas cli which maas end point you're talking to
<garibaldi> smoser: I commissioned a new node and then after was able to successfully start it with "maas local node start <system_id> distro_series=trusty user_data=$(base64 --wrap=0 /tmp/script.sh)"
<garibaldi> smoser: /tmp/script.sh contains the bash shebang followed by "echo "test $(whoami)" > /home/ubuntu/test.txt" on a new line; the node successfully installed the OS, but once it was done /home/ubuntu/test.txt did not exist so it seems it didn't run my user_data script
<garibaldi> smoser: what could be wrong?
<smoser> htat should work
<smoser> can you pastebin /var/log/cloud-init.log and /var/log/cloud-init-output.log ?
<garibaldi> smoser: yes, I'm re-installing the node now but when it finishes I will post them
<smoser> k. thanks.
<mup> Bug #1461135 was opened: Can't assign the same IP address to two different MAC addresses <networking> <MAAS:Triaged> <https://launchpad.net/bugs/1461135>
<drhalan> hey all. is there a way to run a custom script after startup of a node? basically I want to deploy a ssh key so i have root access to the machine. our hardware only has WOL and we need a way to shut them off...
<smoser> drhalan, you can use user-data
<drhalan> smoser: what's that exactly?
<smoser> user-data is the same concept in maas that it is in ec2.
<smoser> cloud-init runs in maas or ec2 instances and consumes user-data that the user fed it.
<smoser> what you can do with user-data is described about:blank
<smoser> err..
<smoser> https://help.ubuntu.com/community/CloudInit
<smoser> and you can deploy a node with user-data by passing user_data key to maas node start
<mup> Bug #1461181 was opened: [1.8rc1] Too many open files, after upgrade to rc1 <MAAS:New> <https://launchpad.net/bugs/1461181>
<kiko> hah!
<garibaldi> smoser: here are my cloud-init.log (http://pastebin.com/vvbbDNDm) and cloud-init-output.log (http://pastebin.com/1h1snXQM) files
<garibaldi> smoser: the command I used to start the VM (after commissioning it) was maas local node start <system_id> distro_series=trusty user_data=$(base64 --wrap=0 /tmp/script.sh)
<garibaldi> smoser: where /tmp/script.sh contains the bash shebang followed by "echo "test $(whoami)" > /home/ubuntu/test.txt" on a new line
<smoser> garibaldi, just for your own sanity, there is a tool called 'pastebinit' that allows use of sane paste bins via:
<smoser> pastebinit /var/log/cloud-init.log
<smoser> or pastebinit < /var/log/cloud-init.log
<drhalan> is there a way to set a comment or something for each machine that for example indicates the location of the machine?
<smoser> garibaldi, it does seem like you didn't get user_data through though
<drhalan> we have a tag on each machine that i would like maas to know about
<smoser> drhalan, you may be able to abuse tags for that.
<mup> Bug #1461226 was opened: Clicking "Nodes" in 1.8 takes several seconds to load - seems much slower than 1.7.5 <MAAS:Incomplete> <https://launchpad.net/bugs/1461226>
<mup> Bug #1461233 was opened: Node power summary less informative in 1.8 <MAAS:New> <https://launchpad.net/bugs/1461233>
<smoser> drhalan, maas admin tags new name=row35
<smoser> maas admin tag update-nodes row35 add=node-260cbb52-094d-11e5-a379-00163ee0d91e
<smoser> its not a perfect fit.
<drhalan> maybe i can just maintain a list outside of maas :)
<drhalan> thanks!
<mup> Bug #1461236 was opened: Clicking Nodes makes three webfont requests that return 404 in 1.8 UI <MAAS:New> <https://launchpad.net/bugs/1461236>
<drhalan> i am using the fast installer and get the following error during install. any idea? http://pastebin.com/m5jmmxdG
<garibaldi> smoser: thanks for the tip about pastebinit; any ideas on why the user_data did not get through for this node?
<smoser> garibaldi, well...
<smoser> could you verify that it didn't ?
<smoser>  can you verify taht /var/lib/cloud/instance/user-data.txt has nothing in it ?
<smoser> and also.. what version of maas are you using ?
<smoser> if user-data were completely busted, then we'd know. as juju uses it when pointed at maas.
<smoser> and that gets tested heavily
<smoser> but if the maas cli was busted, probably would'nt know so fast, but would still seem odd.
<drhalan> can somebody explain me what maas actually does when acquiring a node? it seems to reboot during the process
<drhalan> i could ssh into my machine and my updated curtin script seemed to have deployed the ssh keys and packages I needed. suddenly it rebooted and everything was gone
<garibaldi> smoser: confirmed, the test file I tried to create as part of the script is not there; and /var/lib/cloud/instance/user-data.txt is empty
<garibaldi> is there a debug mode I can enable?
<mup> Bug #1461256 was opened: 1.8rc1: Filter by node broken in Chromium - angular errors in java script console <oil> <MAAS:New> <https://launchpad.net/bugs/1461256>
<mup> Bug #1461258 was opened: 1.8rc1: Deploy/Release in single node view keeps adding blank pages at the top <oil> <MAAS:New> <https://launchpad.net/bugs/1461258>
<smoser> drhalan, yeah, thats sort of a bug
<drhalan> what do you mean ?
<smoser> the ephemeral environment that maas uses to run curtin in also gets your user's keys
<smoser> so you can ssh into the install enviornment as 'ubuntu@'
<drhalan> oh yeah
<smoser> and not realize that you're in iscsi root
<smoser> the sanest fix seems to be to just have the user in the ephemeral root environment be 'ephemeral' or 'admin' or something.
<smoser> then you wouldnt think you were in
<drhalan> that makes sense. but i sitll dont understand which file i have to modify to change the final install. i tried /etc/maas/preseed/curtin and /etc/maas/preseed/curtin_userdata
<smoser> garibaldi, what versoin of maas ?
<drhalan> the first one only seems to affect the install and the second one has no effect for me
<drhalan> there is also curtin_userdata_custom which i am not sure what it is for
<smoser>  /etc/maas/preseed/curtin_userdata is the right file.
<smoser> what i'd suggest is running 'curtin in-target' in a late_command
<smoser> as the easiest way to affect the install
<garibaldi> smoser: MAAS 1.6 (the version available in trusty); I didn't know that I needed to modify curtin_userdata, I thought I just needed to pass the user_data bundle along with the "maas" cli command?
<garibaldi> I tried putting this under late_commands in curtin_userdata but it also did not work: "  enable_test: ['sh', '-c', 'echo "this is a test" > /home/ubuntu/test.txt']"
<smoser> garibaldi, you're right
<smoser> the user_data comment was for drhalan
<garibaldi> smoser: ah, okay
<drhalan> smoser: thanks! is there a document that explains the syntax that curtain expects?
<smoser> not really. drhalan :-(
<drhalan> :/
<drhalan> i just want to define packages to be installed
<smoser> drhalan, well, personally i'd do that in user-data
<drhalan> oh that is something else?
<smoser> when you deploy a node, you can say "use this user-data".
<smoser> which can be anything... and then cloud-init interprets it on boot
<smoser> same way it does on EC2 or on azure
<smoser> except for garibaldi is having trouble getting user-data to work :)
<smoser> https://help.ubuntu.com/community/CloudInit <-- that describes cloud-init user-data
<drhalan> oh but i need it for every node
<smoser> well, you can do it every time you deploy a node.
<smoser> i do understand the desire for central place.
<smoser> i have to run.
<mup> Bug #1461295 was opened: Maas is not picking up new release root-tgz automatically <oil> <MAAS:New> <https://launchpad.net/bugs/1461295>
<mup> Bug #1461298 was opened: maas documentation should document how to configure manual DHCP for UEFI <doc> <MAAS:New> <https://launchpad.net/bugs/1461298>
<catbus1> hi, the maas was working fine, until now it says the only cluster I have is disconnected. I don't see anything suggesting errors from /var/log/maas/maas.log to check, I have dhcp-reconfigured maas-region and cluster-controller, but that didn't help.
<catbus1> any ideas where I can look at to find out why it's disconnected?
<catbus1> both region and cluster are on the same server.
<catbus1> an error keeps repeating: twisted.internet.error.CannotlistenError
<catbus1> on pserv.log
<catbus1> https://bugs.launchpad.net/maas/+bug/1374233
<mup> Bug #1374233 was opened: pserv continually failing: address already in use <packaging> <MAAS:Confirmed> <https://launchpad.net/bugs/1374233>
#maas 2015-06-03
<drhalan> if anyone knows the syntax for /etc/maas/presee/curtain_userdata please ping me
<drhalan> found a good one: http://astokes.org/customizing-fastpath-curtin-installations/
<drhalan> this should be part of the official documentation!
<mup> Bug #1461389 was opened: Parse errors for curtin preseeds aren't reported <1.7> <trusty> <MAAS:New> <https://launchpad.net/bugs/1461389>
<mup> Bug #1461462 was opened: Node action permission not updated as node status changes <ui> <MAAS:Triaged> <https://launchpad.net/bugs/1461462>
<bleepbloop> Quick question for everyone, any suggestions on where to start with debugging dns not working? Whenever I try to ssh in between boxes that were stood up with maas using their dns names I just get no route to host even though I have dns configured
<bleepbloop> Everything else about my maas setup is working
<roaksoax> bleepbloop: are you sure that /etc/resolv.conf is pointing to the MAAS DNS server when you are trying to ssh ?
<bleepbloop> roaksoax, the resolv.conf on one of the instantiated machines has nameserver 172.16.0.1 and search maas in it
<bleepbloop> roaksoax: however trying to ping ping kit-fisto.maas (the maas server) from it gets ping: unknown host kit-fisto.maas
<roaksoax> bleepbloop: where are you trying to ping kit-fisto.maas ?
<roaksoax> bleepbloop: in the same machine where you are trying to ping kit-fisto.maas, does /etc/resolv.conf have nameserver <ip-of-maas-server> as first nameserver entry?
<bleepbloop> roaksoax: from a server that is on the same network as maas and which was provisioned by maas, and yeah the resolv.conf was on the machine provisioned by maas  (which is the one I was trying to ping from)
<roaksoax> bleepbloop: so kit-fisto.maas is the machine deployed by mAAS?
<roaksoax> bleepbloop: or you are trying to ping another machine deployed by MAAS
<bleepbloop> roaksoax: I am kit-fisto.maas is the maas server, plo-koon (the machine I tried to ping kit-fisto from) is the maas provisioned client
<roaksoax> bleepbloop: right, so if kit-fisto.maas is the maas server, that's not in the DNS server that MAAS server controls
<roaksoax> rvba: ^^
<rvba> bleepbloop: so 172.16.0.1 is kit-fisto.maas ?
<bleepbloop> rvba:yes
<bleepbloop> roaksoax: so the maas server doesn't add an entry for itself in dns?
<roaksoax> bleepbloop: nope
<roaksoax> bleepbloop: otherwise it would have to add an entry for the same DNS name in all of the networks MAAS can be present
<roaksoax> i.e. if we have 10 cluster controllers on different networks, there would be 10 dns entries for different IP's
<bleepbloop> roaksoax: hmm okay fair, would it be a bad idea to add it manually?
<mup> Bug #1461295 changed: Maas is not picking up new release root-tgz automatically <oil> <MAAS:Invalid> <https://launchpad.net/bugs/1461295>
<mup> Bug #1461389 changed: Parse errors for curtin preseeds aren't reported <1.7> <trusty> <MAAS:Triaged> <https://launchpad.net/bugs/1461389>
<mup> Bug #1461612 was opened: make static ip ranges required <MAAS:New> <https://launchpad.net/bugs/1461612>
<drhalan> what is the difference between the follwing: "curtin in-target -- sh -c script", "['curtin', 'in-target', '--',  script], ['curtin', 'in-target', '--', 'sh', '-c', script] ?
<drhalan> maybe smoser can help?
<smoser> if the entry in the late_commands is a string, then it will be passed to sh
<smoser> if it is an array, then it will be executed like execve
<smoser> array is much more specific, you dont leave naything up to shell interpretation.
<smoser> ['curtin', 'in-target', '--',  script]
<smoser> thats not going to do anything. unless the string 'script' is an executable.
<smoser> inside the root
<smoser> ...which it actually is :)
<mup> Bug #1461639 was opened: Spurious test failure maasserver.tests.test_forms_nodeaction.TestNodeActionForm.test_save_performs_requested_action <MAAS:Triaged by rvb> <https://launchpad.net/bugs/1461639>
<drhalan> mmh so custom: ["mkdir", "/some/dir"] should do the same as custom: "mkdir /some/dir" ?
<drhalan> none of them seem to have an effect on my machine. :/
<garibaldi> smoser: I upgraded MAAS to the latest stable, 1.7.5, and still have not had any luck getting user_data to run on nodes commissioned with this new version
<kiko> smoser, if we could figure this out I'd like it documented in the maas docs -- but we need to find out how it is to be done. maybe write up a txt file we could use as a starter? :)
<garibaldi> kiko: I'd be happy to test out any techniques for deploying user_data to help with the documentation process
<kiko> thanks
<smoser> drhalan, yes to your answer above.
<smoser> kiko, i'm 98% certain that user_data works
<smoser> so i'm 100% certain that user data works
<smoser> otherwise juju wouldn twork
<smoser> i'm 99% certain that it works via maas cli
<smoser> i've used it before, but dont have anything i can use it with righ tnow
<drhalan> garibaldi: same here.. it does something but not sure what... gave up after about 10 hours of trying different commands
<kiko> smoser, we can get you access to a maas installation easily, would that suffice? you're right now the only person who I think knows how this works
<solsTiCe> hi. So I installed maas while installing ubuntu server. When I connect to maas region controller web interface, I see standard apache web page instead of maas page, why ?
<roaksoax> solsTiCe: http://XYZ/MAAS ?
<solsTiCe> ah ok. forgot /MAAS
<mup> Bug #1461659 was opened: UEFi x64 failing to commission <canonical-bootstack> <MAAS:New> <https://launchpad.net/bugs/1461659>
<smoser> kiko, sure. let me see a maas.
<kiko> smoser, does the garage maas work?
<smoser> checking
<kiko> cool
<smoser> kiko, ok. i just did:
<smoser>  http://paste.ubuntu.com/11549450/
<kiko> smoser, and can you paste the raw userdata?
<smoser> nice
<smoser> well, garage maas is broken
<smoser> its left in bcache-development limbo
<kiko> and that means we can't?
<smoser> so kiko until that is fixed i cant test install something, so ...
<kiko> test
<smoser> http://paste.ubuntu.com/11549482/
<smoser> but that is the user-data
<smoser> ok. just ran this.
<smoser> http://paste.ubuntu.com/11549501/
<smoser> on my local maas (i didn't want to shoot that node, but i did)
<smoser> local maas is
<smoser> # dpkg-query --show maas
<smoser> maas	1.7.3+bzr3363-0ubuntu1~trusty1
<kiko> weird
<kiko> ah
<kiko> I see what you are saying
<kiko> smoser, and what is the raw user data for that?
<smoser> same
<smoser> you can see it.
<kiko> ah yes I for some reason didn't match the strings
<kiko> garibaldi, drhalan: can either of you verify that the above works for you?
<kiko> i.e. using http://paste.ubuntu.com/11549482/ as your user-data
<kiko> which should result in the string that you see on the command-line here: http://paste.ubuntu.com/11549501/
<smoser> kiko, and then...
<smoser> $ cat /run/my.log
<smoser> ========================================
<smoser> ============== Wed Jun 3 20:07:02 UTC 2015 =================
<smoser> ======== Hi World ======================
<smoser> ========================================
<smoser> system came up. had that there.
<kiko> yeah, I was expecting you'd say that 8)
<smoser> and 'sudo cat /var/lib/cloud/instance/user-data.txt'
<smoser> will show you the input user-data
<smoser> so, tats about all i can easily do at the moment. but it does show user-data functioning via maas-cli
<smoser> with maas at 1.7.3
<solsTiCe> I am using VM to test maas. What size must be my hdd to be able to import the boot image ?
<garibaldi> smoser: can you send me the exact maas cli command you used?
<garibaldi> just to make sure I reproduce exactly
<mup> Bug #1461712 was opened: 1.8rc1:  sm15k systems - non-PXE NIC's mac address doesn't get linked back to cluster <oil> <MAAS:New> <https://launchpad.net/bugs/1461712>
<smoser> garibaldi, its in that pastebin
<smoser> http://paste.ubuntu.com/11549450/
<smoser> maas smoser nodes acquire
<smoser> 'smoser' is the maas name (first argument to maas)
<smoser> acquire returned the node with system-id node-86787e7e-cd78-11e4-8df8-001b21006d7f
<smoser> then maas smoser node start node-86787e7e-cd78-11e4-8df8-001b21006d7f user_data=IyEvYmluL3NoCgooCmVjaG8gPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQplY2hvID09PT09PT09PT09PT09ICQoZGF0ZSkgPT09PT09PT09PT09PT09PT0KZWNobyA9PT09PT09PSBIaSBXb3JsZCA9PT09PT09PT09PT09PT09PT09PT09CmVjaG8gPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQopIHwgdGVlIC9ydW4vbXkubG9nCg== distro_series=trusty
#maas 2015-06-04
<vibol> Hello :)
<mup> Bug #1461863 was opened: 1.8rc1: number of different failures in maas.log due to '... [Errno 24] too many open files ...' <oil> <MAAS:New> <https://launchpad.net/bugs/1461863>
<solsTiCe> hi. So I have setup an interface to be dhcp and dns managed by maas clsuter controler. but interface still does not any ip and dhcp is down.
<mup> Bug #1461977 was opened: Unused "Check component compatibility and certification" field should be removed <MAAS:New for mpontillo> <https://launchpad.net/bugs/1461977>
<mup> Bug #1462019 was opened: 1.8rc1: UI hangs after doing block Mark Broken of 3 failed releasing servers <oil> <MAAS:New> <https://launchpad.net/bugs/1462019>
<mup> Bug #1462029 was opened: Filters don't clear if there are no more machines for that filter <MAAS:New> <https://launchpad.net/bugs/1462029>
<mup> Bug #1462030 was opened: Filters don't clear if there are no more machines for that filter <MAAS:New> <https://launchpad.net/bugs/1462030>
<kiko> garibaldi, the exact maas cli command is in the pastebin I pasted you
<mup> Bug #1462078 was opened: Can't add a device and it does not show why <MAAS:Confirmed for blake-rouse> <MAAS 1.8:New> <https://launchpad.net/bugs/1462078>
<mup> Bug #1462079 was opened: Devices can't add a device with a Static IP address outside of dyanmic/static range <MAAS:Confirmed for blake-rouse> <https://launchpad.net/bugs/1462079>
<c4tech_> Hi Team! Does maas have good support for bonding and LACP?
<c4tech_> I searched the documentation and didn't find anything on this/
<c4tech_> The question is mostly in regards to post installation support.
<mup> Bug #1462136 opened: 1.8rc2: Cluster periodically disconnecting and reconnecting <oil> <MAAS:New> <https://launchpad.net/bugs/1462136>
<mup> Bug #1462136 changed: 1.8rc2: Cluster periodically disconnecting and reconnecting <oil> <MAAS:New> <https://launchpad.net/bugs/1462136>
<mup> Bug #1462136 opened: 1.8rc2: Cluster periodically disconnecting and reconnecting <oil> <MAAS:New> <https://launchpad.net/bugs/1462136>
#maas 2015-06-05
<NixNull> Hey there. I'm looking for some help on an issue with custom images for MaaS. Is nyone around that can give a pointer or two?
<mup> Bug #1462155 opened: Rendering artifacts with Chromium 41.0.2272.76 Ubuntu 14.04 (64-bit) <MAAS:New> <https://launchpad.net/bugs/1462155>
<mup> Bug #1462299 opened: MAAS instance has 7 regiond processes running instead of 4 <MAAS:Triaged> <https://launchpad.net/bugs/1462299>
<mup> Bug #1462320 opened: eventloop table is out of date <MAAS:Triaged> <https://launchpad.net/bugs/1462320>
<mup> Bug #1462320 changed: eventloop table is out of date <MAAS:In Progress by allenap> <https://launchpad.net/bugs/1462320>
<mup> Bug #1462320 opened: eventloop table is out of date <MAAS:In Progress by allenap> <https://launchpad.net/bugs/1462320>
<kiko> yeah, I was expecting you'd say that 8)
<mup> Bug #1300266 changed: squid-deb-proxy returns 403 when admin configures a custom APT archive <canonical-is> <MAAS:Fix Released> <https://launchpad.net/bugs/1300266>
<kiko> interesting
<bleepbloop> Are there any known issues with 15.04 and maas not provisioning properly? All my 15.04 machines come up without ssh working and I have to go manually reboot them
<bleepbloop> using 14.04 works fine.
<bleepbloop> nvm looks like a juju issue
<mup> Bug #1430025 changed: maas uninstallable on vivid <amd64> <apport-bug> <ec2-images> <juju-net> <vivid> <MAAS:Fix Released> <maas (Ubuntu):Fix Released> <https://launchpad.net/bugs/1430025>
<mup> Bug #1430025 changed: maas uninstallable on vivid <amd64> <apport-bug> <ec2-images> <juju-net> <vivid> <MAAS:Fix Released> <maas (Ubuntu):Fix Released> <https://launchpad.net/bugs/1430025>
<mup> Bug #1453869 changed: jp.archive.ubuntu.com is a broken mirror <landscape> <curtin:Invalid> <MAAS:Invalid> <https://launchpad.net/bugs/1453869>
<mup> Bug #1462498 opened: install environment should not let user in as 'ubuntu' <MAAS:New> <https://launchpad.net/bugs/1462498>
<garibaldi> smoser: using maas-deploy-node and the steps you outlined worked, I am now able to execute userdata on the maas node!
<garibaldi> smoser and kiko: thanks for the help!
<kiko> garibaldi, cool -- what were you getting wrong in your first attempt?
<smoser> garibaldi, yeah, i wondered that too
<smoser> fwiw, that 'maas-deploy-node' script should be general purpose
<mup> Bug #1462507 opened: BlockDevice API is not under the nodes endpoint <MAAS:In Progress by blake-rouse> <https://launchpad.net/bugs/1462507>
<mup> Bug #1462514 opened: 1.8rc3: Right-click then launch in new tab or new window on a filter property should load filter in new view <oil> <ui> <ux> <MAAS:Triaged> <https://launchpad.net/bugs/1462514>
<mup> Bug #1461863 changed: 1.8rc1: failures including region not available due to '[Errno 24] too many open files' <oil> <openstack> <uosci> <MAAS:Confirmed> <https://launchpad.net/bugs/1461863>
<mup> Bug #1461863 opened: 1.8rc1: failures including region not available due to '[Errno 24] too many open files' <oil> <openstack> <uosci> <MAAS:Confirmed> <https://launchpad.net/bugs/1461863>
<mup> Bug #1461863 changed: 1.8rc1: failures including region not available due to '[Errno 24] too many open files' <oil> <openstack> <uosci> <MAAS:Confirmed> <https://launchpad.net/bugs/1461863>
#maas 2015-06-06
<mup> Bug #1399824 changed: pserv datagram traceback <oil> <MAAS:Invalid> <https://launchpad.net/bugs/1399824>
<mup> Bug #1441606 changed: Error: Page not found The requested URL /MAAS/nodes/ was not found on this server. <MAAS:Invalid> <https://launchpad.net/bugs/1441606>
<mup> Bug #1399824 opened: pserv datagram traceback <oil> <MAAS:Invalid> <https://launchpad.net/bugs/1399824>
<mup> Bug #1441606 opened: Error: Page not found The requested URL /MAAS/nodes/ was not found on this server. <MAAS:Invalid> <https://launchpad.net/bugs/1441606>
<mup> Bug #1399824 changed: pserv datagram traceback <oil> <MAAS:Invalid> <https://launchpad.net/bugs/1399824>
<mup> Bug #1441606 changed: Error: Page not found The requested URL /MAAS/nodes/ was not found on this server. <MAAS:Invalid> <https://launchpad.net/bugs/1441606>
<mup> Bug # changed: 1266893, 1268713, 1268783, 1269046, 1270165, 1312988, 1331909, 1350270, 1358586
<mup> Bug # changed: 972077, 1266893, 1268713, 1268783, 1269046, 1270165, 1312988, 1331909, 1350270, 1358586, 1396474, 1425327, 1442379, 1448901, 1462029
<mup> Bug # changed: 1400417, 1400932, 1401342, 1401841, 1411173, 1416431, 1420425, 1433702, 1434602, 1435403, 1442157, 1442552, 1443631, 1445223, 1445717, 1455722, 1457245, 1457293
<mup> Bug #1305907 changed: boot: section in pserv.yaml is obsolete <MAAS:Confirmed for allenap> <https://launchpad.net/bugs/1305907>
<mup> Bug # changed: 1395375, 1420563, 1420566, 1460097, 1460151
<mup> Bug #1457293 opened: 1.8b6: log entries missing in archived clusterd logs <oil> <MAAS:New> <https://launchpad.net/bugs/1457293>
<mup> Bug #1462634 opened: Error on request (62) node.check_power <MAAS:New> <https://launchpad.net/bugs/1462634>
<mup> Bug #1462635 opened: Error on request (62) node.check_power <MAAS:Triaged> <https://launchpad.net/bugs/1462635>
<mup> Bug #1462637 opened: Failure: twisted.web.error.Error: 503 SERVICE UNAVAILABLE <MAAS:Triaged> <https://launchpad.net/bugs/1462637>
<mup> Bug #1462637 changed: Failure: twisted.web.error.Error: 503 SERVICE UNAVAILABLE <MAAS:Triaged> <https://launchpad.net/bugs/1462637>
<mup> Bug #1462637 opened: Failure: twisted.web.error.Error: 503 SERVICE UNAVAILABLE <MAAS:Triaged> <https://launchpad.net/bugs/1462637>
<mup> Bug #1462637 changed: Failure: twisted.web.error.Error: 503 SERVICE UNAVAILABLE <MAAS:Triaged> <https://launchpad.net/bugs/1462637>
#maas 2015-06-07
<mup> Bug #1440823 changed: Using both MAC address fields in AMT config causes AMT Power to fail <MAAS:Expired> <https://launchpad.net/bugs/1440823>
<mup> Bug #1440823 opened: Using both MAC address fields in AMT config causes AMT Power to fail <MAAS:Expired> <https://launchpad.net/bugs/1440823>
<mup> Bug #1440823 changed: Using both MAC address fields in AMT config causes AMT Power to fail <MAAS:Expired> <https://launchpad.net/bugs/1440823>
#maas 2016-06-06
<mup> Bug #1542326 changed: [1.10] Cannot deploy Ubuntu 14.04/15.10 on EFI system after 1.8 version to 1.9 upgrade <curtin:Invalid> <MAAS:Expired> <https://launchpad.net/bugs/1542326>
<mup> Bug #1589385 opened: leftover eth0.cfg in /etc/network/interfaces.d <4010> <juju-core:New> <MAAS:New> <https://launchpad.net/bugs/1589385>
<mup> Bug #1589385 changed: leftover eth0.cfg in /etc/network/interfaces.d <4010> <juju-core:New> <MAAS:Invalid> <https://launchpad.net/bugs/1589385>
<xodox> Hi. WOL in 2.0.3.beta? Unable to find anything in the official docs. Is it not part of 2.0 yet?
<gimmic> Man, I really wish machine templates were a thing
<mup> Bug #1589560 opened: [2.6b6 UI] Adding a fabric with an optional name ends up with a new fabric with automatically assigned name <MAAS:New> <https://launchpad.net/bugs/1589560>
<mup> Bug #1589562 opened: [2.0b6, UI] When I delete a fabric, it takes me back to the Node listing page <MAAS:New> <https://launchpad.net/bugs/1589562>
<roaksoax> /wi/win 3
<mup> Bug #1589583 opened: [2.0b6, UI] Can't add a VLAN over the WebUI. <MAAS:New> <https://launchpad.net/bugs/1589583>
<mup> Bug #1589587 opened: [2.0b6] Attempting to delete a VLAN that cannot be deleted, shows traceback in regiond.log <MAAS:New> <https://launchpad.net/bugs/1589587>
<mup> Bug #1589595 opened: [2.6b6 UI] Adding a fabric with an optional name ends up with a new space with automatically assigned name <MAAS:New> <https://launchpad.net/bugs/1589595>
<mup> Bug #1589596 opened: [2.0b6, UI] When I delete a space, it takes me back to the Node listing page <MAAS:New> <https://launchpad.net/bugs/1589596>
<mup> Bug #1589606 opened: [2.0b6, UI] Message "No IP ranges have been reserved for this subnet." doesn't go away after adding IP Range <MAAS:New> <https://launchpad.net/bugs/1589606>
<D4RKS1D3> Hi, someone know about add new events in maas code?
<kiko> D4RKS1D3, events for the log?
<D4RKS1D3> I prefer it is possible add to the database
<D4RKS1D3> I want to add new events in the table maasserver_event
<D4RKS1D3> you know how to do it kiko ?
<kiko> D4RKS1D3, not exactly -- but I was asking first, what are you trying to do, add more logging for the node event log?
<D4RKS1D3> Yes
<mup> Bug #1589626 opened: DHCP pool for relay do not work <4010> <MAAS:New> <https://launchpad.net/bugs/1589626>
<mup> Bug #1589626 changed: DHCP conf for relay does not work <4010> <MAAS:Won't Fix> <https://launchpad.net/bugs/1589626>
<mup> Bug #1589640 opened: [2.0b6] MAAS should validate a boot source path actually provides images <MAAS:Triaged> <https://launchpad.net/bugs/1589640>
<roaksoax> /win/win 3
<blake_r_> D4RKS1D3: look at src/provisioningserver/events.py that might point you in the correct direction
<D4RKS1D3> I saw that class yesterday blake_r_ but I do not saw any insert
<blake_r_> D4RKS1D3: depends on where you want to do the inserting
<blake_r_> D4RKS1D3: what are you trying to do
<D4RKS1D3> I want to add new events in the database
<roaksoax> D4RKS1D3: what type of events ?
<D4RKS1D3> I want to log the same information is logged when you comission
<D4RKS1D3> or when you change new to ready
<D4RKS1D3> with the FTP files logger
<D4RKS1D3> but when change "detect" to "new"
<roaksoax> D4RKS1D3: grep for this fynction: add_event_to_node_event_log
<roaksoax> D4RKS1D3: that will give you an example of how events are being added
<D4RKS1D3> Thank you roaksoax
<D4RKS1D3> roaksoax, In which file? I do not received any file
<D4RKS1D3> grep -Eiorn "add_event_to_node_event_log" .
<roaksoax> D4RKS1D3: src/metadataserver/api.py
<D4RKS1D3> thanks roaksoax I searched within the folder provisioningserver
<blake_r_> D4RKS1D3: an event is written when a node status changes so when it goes Commissioning -> Ready it will be logged
<mup> Bug #1589664 opened: commission fails with 500 error for /MAAS/metadata/latest/by-id/ and django.db.utils.IntegrityError <cdo-qa> <cdo-qa-blocker> <MAAS:New> <https://launchpad.net/bugs/1589664>
<mup> Bug #1589664 changed: commission fails with 500 error for /MAAS/metadata/latest/by-id/ and django.db.utils.IntegrityError <cdo-qa> <cdo-qa-blocker> <MAAS:New> <https://launchpad.net/bugs/1589664>
<mup> Bug #1589719 opened: [2.1] MAAS should have 'createadmin' and 'changepassword' under 'maas' binary <MAAS:Confirmed> <https://launchpad.net/bugs/1589719>
<mup> Bug #1589719 changed: [2.1] MAAS should have 'createadmin' and 'changepassword' under 'maas' binary <MAAS:Confirmed> <https://launchpad.net/bugs/1589719>
<mup> Bug #1589719 opened: [2.1] MAAS should have 'createadmin' and 'changepassword' under 'maas' binary <MAAS:Confirmed> <https://launchpad.net/bugs/1589719>
#maas 2016-06-07
<thetrav> is there any way in the maas UI for me to configure the MTU for an interface?
<mup> Bug #1589789 opened: is it impossible to add 'A' type record for a host which is not managed by MAAS? <MAAS:New> <https://launchpad.net/bugs/1589789>
<mup> Bug #1567249 changed: 'missing_packages' missing and causes traceback <MAAS:Expired> <https://launchpad.net/bugs/1567249>
<mup> Bug #1567614 changed: [MAAS 2.0 ] does not add default gw route to nodes. <MAAS:Expired> <https://launchpad.net/bugs/1567614>
<thetrav> ok, so couldn't find anything in the UI, found something in the cli around the vlan, which I assume is something of a default
<thetrav> managed to update it, however restarting instances in that vlan is not getting them a new MTU
<thetrav> so far as the CLI is concerned, my interfaces should have an effective MTU of 2000
<thetrav> so far as my nodes are concerned, their interfaces have an MTU of 1500
<jojden> I am getting following error, "Specify a storage device to be able to deploy this node."
<jojden> how to Specify a storage device
<jojden> Specify a storage device to be able to deploy this node.
<jojden> Mount the root '/' filesystem to be able to deploy this node.
<jojden> hi
<brendand> jojden, i'm not sure that's really an error - it might be a ui bug. possibly a little confusing
<jojden> brendand
<jojden> ok
<jojden> but under File system it showing that
<jojden> No filesystems defined.
<brendand> jojden, has the node been commisioned yet?
<jojden> its in commisioning state
<jojden> @brendand
<jojden> Node commissioning - 'cloudinit' running modules for final	Tue, 07 Jun. 2016 09:45:04
<jojden> Node commissioning - 'cloudinit' config-power-state-change ran successfully	Tue, 07 Jun. 2016 09:45:03
<brendand> once it's finished that should be gone
<jojden> ok
<jojden> usally how much time will take for the commissioning ?
<brendand> should be quick
<jojden> hmmm
<jojden> but its taking more time
<jojden> so something is wrong with that
<jojden> :(
<mup> Bug #1589951 opened: why pxe booted node is stucked in commissioning status <MAAS:New> <https://launchpad.net/bugs/1589951>
<mup> Bug #1589951 changed: why pxe booted node is stucked in commissioning status <MAAS:New> <https://launchpad.net/bugs/1589951>
<mup> Bug #1589951 opened: why pxe booted node is stucked in commissioning status <MAAS:New> <https://launchpad.net/bugs/1589951>
<mup> Bug #1590021 opened: [2.0] Cannot create an IP reservation with a hostname <MAAS:New for lamont> <https://launchpad.net/bugs/1590021>
<gimmic> man this curtin storage bug is really chappin my ass
<gimmic> wonder if I should move over to the 2.0 betas
<kiko> gimmic, did you make any progress on finding out what the issue is?
<kiko> if not, smoser, do you have some time to look at it with gimmic? if not, someone on your team that knows their stuff?
<gimmic> I've got a couple of nodes exhibiting the problem, looking at the logs doesn't tell me much
<kiko> gimmic, they fail to deploy randomly when using flat only, right?
<smoser> gimmic, did you file a bug ?
<LiftedKilt> does maas support ed25519 keys?
<kiko> LiftedKilt, for ssh do you mean?
<LiftedKilt> kiko: yes
<LiftedKilt> when I try to add my key it tells me "Invalid SSH public key."
<kiko> i.e. when setting up ssh accounts for the ubuntu/centos users, allow specifying ed25519 public keys?
<kiko> I think it does not
<kiko> LiftedKilt, could you file a bug?
<LiftedKilt> sure
<LiftedKilt> kiko: while I have you here - is there a way to have maas change hostnames?
<kiko> LiftedKilt, of the deployed nodes? or pre-deployment?
<LiftedKilt> either
<kiko> pre-deployment sure, just edit the node
<LiftedKilt> seriously? haha how did I miss that
<LiftedKilt> thanks
<kiko> LiftedKilt, post-deployment possibly, but even if so it obviously won't be reflected in the deployed node  (because we don't have a running machine agent nor cloud-init reconfiguration)
<LiftedKilt> right - yeah pre-deployment is perfect
<mup> Bug #1590081 opened: when setting up ssh accounts for the key injection, allow specifying ed25519 and ecdsa public keys <MAAS:New> <https://launchpad.net/bugs/1590081>
<kiko> LiftedKilt, thanks for filing a solid bug report
<LiftedKilt> kiko: for sure
<smoser> ltrager, https://code.launchpad.net/~smoser/maas-images/trunk.fix-powerpc-mining/+merge/296705 would be nice if you had a chance.
<gimmic> kiko: it doesn't seem to be limited to flat only.
<kiko> gimmic, okay. so let me restate.
<kiko> gimmic, they fail to deploy randomly when using any layout, right?
<kiko> your stated success rate was 1/3?
<gimmic> or so, yeah. If I knew where in the logs to look to tshoot I would, the layout just isn't very clear (or if there's increased debug options)
<kiko> smoser, where should be look at the logs?
<kiko> smoser, s/be/he
<mup> Bug #1590121 opened: [xenial][maas beta5] [arm64] system tries to enlist when I commission.  <MAAS:Triaged by newell-jensen> <https://launchpad.net/bugs/1590121>
<mup> Bug #1590144 opened: core count not updated during commissioning <v-pil> <MAAS:New> <https://launchpad.net/bugs/1590144>
#maas 2016-06-08
<Ram_> i need some help in MaaS 2.0 installation
<Ram_> I am not able to import the images
<Ram_> Traceback (most recent call last):   File "/usr/lib/python3.5/urllib/request.py", line 1243, in do_open     h.request(req.get_method(), req.selector, req.data, headers)   File "/usr/lib/python3.5/http/client.py", line 1106, in request     self._send_request(method, url, body, headers)   File "/usr/lib/python3.5/http/client.py", line 1151, in _send_request     self.endheaders(body)   File "/usr/lib/python3.5/http/client.py", line 1102, in
<mup> Bug #1590406 opened: Cluster is not getting connected <MAAS:New> <https://launchpad.net/bugs/1590406>
<kiko> smoser, ^^
<kiko> smoser, where should be look at the logs?
<kiko> gar, typod in the recall
<jojden> issue with cluster controller
<jojden> my cluster controller is not getting connected with region controller
<jojden> i have done the configuration and gave the secret key too
<jojden> but still it showing disconnected in the cluster page
<jojden> I am using 1.9.3
<jojden> any idea why
<jojden> https://bugs.launchpad.net/maas/+bug/1590406
<smoser> kiko, cluster to region controller enlistment is not something i have anything to do with
<kiko> smoser, no, I meant the comment from gimmic from yesterday
<smoser> for gimmic, if it is runnign the installer, i believe maas says to put an install log in /tmp/install.log
<smoser> anad also /var/log/cloud-init* is probably useful
<smoser> if you can get intot he install environment and stop the poweroff, then you can sit and re-run the curtin installation manually and see whats going on
<kiko> that's what I thought
<kiko> thanks smoser
<huats> Hi
<huats> I am facing something weird
<kiko> huats, the suspense is killing me
<huats> kiko: hey
<huats> sorry for delay
<huats> :)
<huats> phonecall :)
<huats> I have a weird thing
<kiko> ah
<huats> I have uploaded a centos image on my maas (1.9)
<huats> (built using maas-image-builder)
<huats> and when I try to deploy using it, I keep ending with an ubuntu deployment :(
<kiko> ubuntu beats centos!
<kiko> first question: does deploying [unmodified] centos from our streams work?
<huats> your streams ?
<huats> never tried...
<kiko> huats, that's much easier -- just enable centos
<huats> hum
<huats> I would be interested if you have any url :)
<kiko> roaksoax, blake_r_: how does huats enable the CentOS stream?
<kiko> huats, what version of MAAS are you on?
<huats> 1.9
<roaksoax> kiko: daily streams
<roaksoax> huats: ^^
<kiko> roaksoax, just settings, etc stream, s/release/daily?
<roaksoax> kiko: indeed
<huats> kiko: so there is an another way than https://maas.ubuntu.com/docs/os-support.html
<huats> to get centos ?
<kiko> yes
<kiko> huats, see above
<huats> yeah daily streams ? but I couldn't find where to do that :(
<kiko> <kiko> roaksoax, just settings, etc stream, s/release/daily?
<kiko> huats, go to settings, edit the stream URL to say daily
<kiko> save
<kiko> import images
<kiko> selecting centos
<huats> so you mean the sync URL to point to http://maas.ubuntu.com/images/ephemeral-v2/daily
<huats> ?
<huats> It is the only mention or release I have on that page :(
<huats> Oh and now I have centos images !
<huats> well there are mentionned "not sync"
<huats> I'll wait a bit
<huats> :)
<kiko> you got it<tm>
<huats> thanks !
<huats> kiko: and roaksoax ^
<kiko> huats, happy to help!
<mup> Bug #1590499 opened: 2.0Beta6 | Can edit fabric and subnet on deployed node <ui> <ux> <MAAS:New> <https://launchpad.net/bugs/1590499>
<mup> Bug #1590499 changed: 2.0Beta6 | Can edit fabric and subnet on deployed node <ui> <ux> <MAAS:New> <https://launchpad.net/bugs/1590499>
<mup> Bug #1590499 opened: 2.0Beta6 | Can edit fabric and subnet on deployed node <ui> <ux> <MAAS:New> <https://launchpad.net/bugs/1590499>
<mup> Bug #1590518 opened: Tool to back up and restore MAAS nodes database would be useful <MAAS:New> <https://launchpad.net/bugs/1590518>
<mup> Bug #1590546 opened: No reliable way to detect whether a NIC is 10Gig through MAAS commissioning data <oil> <MAAS:Triaged> <https://launchpad.net/bugs/1590546>
<babbageclunk> blake_r_: ping>
<babbageclunk> ?
<mup> Bug #1590598 opened: ipv6 interfaces on a machine (in maas) are not added to lxc containers deployed to that machine <juju-core:New> <https://launchpad.net/bugs/1590598>
<mup> Bug #1389351 changed: Duplicate eth0 definition <cloud-installer> <landscape> <MAAS:Fix Released> <https://launchpad.net/bugs/1389351>
<mup> Bug #1587946 changed: IPMI connection failure causes failed deployment and releasing <MAAS 1.9:Triaged> <MAAS 2.0:Fix Committed> <https://launchpad.net/bugs/1587946>
<mup> Bug #1590598 changed: ipv6 interfaces configured on a machine (in maas) are not added to lxc containers deployed to that machine <juju-core:New> <MAAS:Invalid> <https://launchpad.net/bugs/1590598>
<mup> Bug #1039701 changed: Wrong RAM memory size <verification-done> <lshw:New> <MAAS:Invalid> <landscape-client (Ubuntu):Invalid> <lshw (Ubuntu):Fix Released by dannf> <landscape-client (Ubuntu Xenial):Confirmed> <lshw (Ubuntu Xenial):Fix Released by dannf> <https://launchpad.net/bugs/1039701>
<mup> Bug #1514648 changed: Got more than one item. - unable to add/modify machines in the UI <MAAS:Won't Fix> <https://launchpad.net/bugs/1514648>
<mup> Bug #1536207 changed: bootstrap times out waiting for deployed state - curtin install > 27 minutes for amd ocp roadrunner system <oil> <curtin:New> <MAAS:Won't Fix> <https://launchpad.net/bugs/1536207>
#maas 2016-06-09
<mup> Bug # changed: 1578395, 1580818, 1585814, 1590144
<mup> Bug #1562013 changed: "Install MAAS region controller" should be available in 16.04  <MAAS:Fix Released> <https://launchpad.net/bugs/1562013>
<mup> Bug #1590144 opened: core count not updated during commissioning <v-pil> <MAAS:New> <https://launchpad.net/bugs/1590144>
<mup> Bug #1590689 opened: MAAS 1.9.3 + Juju 1.25.5 - on the Juju controller node eth0 and juju-br0 interfaces have the same IP address at the same time <juju> <maas> <MAAS:New> <https://launchpad.net/bugs/1590689>
<malina33> hello
<malina33> how can I set same time on all my nodes?
<mup> Bug #1590768 opened: [2.0B6] Action buttons are not displayed when a user clicks edit storage device <ui> <ux> <MAAS:New for ricgard> <https://launchpad.net/bugs/1590768>
<mup> Bug #1590775 opened: Race in t.p.lockfile.FilesystemLock <MAAS:Triaged> <Twisted:Unknown> <https://launchpad.net/bugs/1590775>
<mup> Bug #1590689 changed: MAAS 1.9.3 + Juju 1.25.5 - on the Juju controller node eth0 and juju-br0 interfaces have the same IP address at the same time <cpec> <juju> <maas> <juju-core:Fix Committed> <juju-core 1.25:Triaged> <MAAS:Invalid> <https://launchpad.net/bugs/1590689>
<mup> Bug #1590846 opened: [2.0b6] Installation log of a failed install not sent back to MAAS <MAAS:Triaged> <https://launchpad.net/bugs/1590846>
<mup> Bug #1590846 changed: [2.0b6] Installation log of a failed install not sent back to MAAS <MAAS:Triaged> <https://launchpad.net/bugs/1590846>
<mup> Bug #1590846 opened: [2.0b6] Installation log of a failed install not sent back to MAAS <MAAS:Triaged> <https://launchpad.net/bugs/1590846>
<mup> Bug #1590866 opened: [2.0b6] Cannot deploy Windows Server 2012 R2  <MAAS:Confirmed> <https://launchpad.net/bugs/1590866>
<mup> Bug #1583395 changed: "Enable DHCP" on VLAN does not setup a Reserved Dynamic range for secondary (IPv6) subnets <MAAS:Opinion> <https://launchpad.net/bugs/1583395>
<mup> Bug #1583395 opened: "Enable DHCP" on VLAN does not setup a Reserved Dynamic range for secondary (IPv6) subnets <MAAS:Opinion> <https://launchpad.net/bugs/1583395>
<mup> Bug #1583395 changed: "Enable DHCP" on VLAN does not setup a Reserved Dynamic range for secondary (IPv6) subnets <MAAS:Opinion> <https://launchpad.net/bugs/1583395>
<mup> Bug #1590887 opened: APC PDU support in MAAS 1.9 <MAAS:New> <https://launchpad.net/bugs/1590887>
<bdx> heya whats up guys?
<bdx> does the 'maas-rack' charm exist yet?
<smoser> kiko, fyim, i wrote https://gist.github.com/smoser/2610e9b78b8d7b54319675d9e3986a1b
<smoser> fyi
<smoser> i believe it would be helpful for whoever it was that was seeing (i believe) transient failures to get in and poke around with some things.
<mup> Bug #1590946 opened: Auto detection of running virtual environment during commissioning almost always fails <MAAS:Triaged> <https://launchpad.net/bugs/1590946>
<mup> Bug #1590946 changed: Auto detection of running virtual environment during commissioning almost always fails <MAAS:Triaged> <https://launchpad.net/bugs/1590946>
<mup> Bug #1590946 opened: Auto detection of running virtual environment during commissioning almost always fails <MAAS:Triaged> <https://launchpad.net/bugs/1590946>
<mup> Bug #1588914 changed: [2.0b6] MAAS writes DHCP multiple times while not much is going on <MAAS:Invalid> <https://launchpad.net/bugs/1588914>
<meepmeep22> This channel is understaffed
<mup> Bug #1571233 changed: MAAS Region/Rack Controller does start on Bonds/VLANs <MAAS:Won't Fix> <https://launchpad.net/bugs/1571233>
<mup> Bug #1484551 changed: Failure powering up nodes <MAAS:Invalid by newell-jensen> <https://launchpad.net/bugs/1484551>
<mup> Bug #1590986 opened: Centos Failed deployment, installs to SSD but local boot comes up on different drive <cdo-qa> <MAAS:New> <https://launchpad.net/bugs/1590986>
<mup> Bug #1590991 opened: Cannot allocate a node based on its system_id <MAAS:Triaged by mpontillo> <https://launchpad.net/bugs/1590991>
#maas 2016-06-10
<mup> Bug #1591093 opened: 3rd party HP drivers (archive hostname renamed) - deployment fails <canonical-bootstack> <MAAS:New> <https://launchpad.net/bugs/1591093>
<manolas> hello! how can I set the same date/time on all nodes?
<manolas> anyone? :)
<olavgg> I'm trying MAAS for the first time here, but I have a problem. When I boot a machine with PXE from MAAS. It hang during boot with the following error message "no url to host". It is like the client gets no internet connection. Is there something specific I need to configure for that to work?
<olavgg> sorry, the error message says: "no route to host"
<smoser> olavgg, do you get to a linux boot ?
<smoser> or is this the pxe environment erroring
<mup> Bug #1591250 opened: 1.9.3 changelog information missing from maas.ubuntu.com/docs1.9/changelog.html <cdo-qa> <MAAS:New> <https://launchpad.net/bugs/1591250>
<mup> Bug #1502253 changed: Partition table should be deleted after all partitions have been removed <storage> <MAAS:Fix Released> <https://launchpad.net/bugs/1502253>
<mup> Bug #1590866 changed: [2.0b6] Cannot deploy Windows Server 2012 R2  <jujuqa> <MAAS Image Builder:Triaged> <https://launchpad.net/bugs/1590866>
<jhobbs> I have a bunch of physically identical nodes that I want to have the same storage configuration
<jhobbs> what's the best way to do that? do i need to just configure each one the same way one by one?
<roaksoax> jhobbs: yes
<roaksoax> jhobbs: if it is custom, you can only do it on one by one basis
<jhobbs> ok, thanks roaksoax
<abi_> hi
<mup> Bug #1583349 changed: ESXi VM fails to commission after adding VMware chassis : no image ubuntu/amd64/hwe-x/trusty/no-such-image/boot-kernel <oil> <MAAS:Invalid by newell-jensen> <https://launchpad.net/bugs/1583349>
<mup> Bug #1534942 changed: Failed curtin install: An error occured handling 'sda': ValueError - no disk with serial '<diskserialnumber>' found <landscape> <oil> <curtin:Invalid> <MAAS:Invalid> <https://launchpad.net/bugs/1534942>
<mup> Bug #1591336 opened: MAAS allows formatting /dev/sda as one big ext4 filesystem, but that doesn't work <v-pil> <MAAS:New> <https://launchpad.net/bugs/1591336>
<mup> Bug #1591346 opened: [2.0b7] maas createadmin fails <MAAS:New> <https://launchpad.net/bugs/1591346>
<mup> Bug #1591364 opened: disks not renamed on deployed systems following renaming on maas UI <oil> <curtin:New> <MAAS:New> <https://launchpad.net/bugs/1591364>
<mup> Bug #1246626 changed: WAKE_ON_LAN power method cannot power off a node. <power> <wol> <MAAS:Won't Fix> <https://launchpad.net/bugs/1246626>
<mup> Bug #1591364 changed: Can't use new dname on trusty deployed systems following renaming on maas UI <oil> <curtin:Invalid> <MAAS:New> <https://launchpad.net/bugs/1591364>
<mup> Bug #1590846 changed: [2.0b6] Installation log of a failed install not sent back to MAAS <curtin:New> <MAAS:Invalid> <https://launchpad.net/bugs/1590846>
<mup> Bug #1591395 opened: [arm64] some arm64 systems need ipmi_ssif module instead of ipmi_si <MAAS:New> <https://launchpad.net/bugs/1591395>
#maas 2016-06-11
<mup> Bug #1591412 opened: Random test case failure in maasserver.models.tests.test_node:TestRackControllerManager <MAAS:In Progress by trapnine> <https://launchpad.net/bugs/1591412>
#maas 2016-06-12
<mup> Bug #1581250 changed: [1.9] Device & deployed machine both have same static IP's <oil> <MAAS:Won't Fix> <MAAS 2.0:Fix Committed> <https://launchpad.net/bugs/1581250>
<mup> Bug #1581250 opened: [1.9] Device & deployed machine both have same static IP's <oil> <MAAS:Fix Committed> <MAAS 1.9:Won't Fix> <https://launchpad.net/bugs/1581250>
#maas 2017-06-05
<benlake> any thoughts as to why using a host name in the qemu+ssh:// stanza doesnât function? The MAAS controller can resolve it, but it just doesnât work unless I use an IP address.
<pmatulis> benlake, as the user 'maas' on the rack controller, does 'host <kvm hostname>' work?
<benlake> pmatulis: yessir
<benlake> what the smack, why does makring something broken power it downâ¦ ugh
<mup> Bug #1695868 opened: [2.2. bzr6054-0ubuntu1~16.04.1, Machine details - Interfaces] I tried to create a bond from two bridges and the creation failed but I didn't get any feedback on the UI <ui> <MAAS:New> <https://launchpad.net/bugs/1695868>
<ThiagoCMC> hey guys, is there any way to deploy nodes with root login enabled (via ssh)?
<roaksoax> ThiagoCMC: no, you can customize your deployment though
<roaksoax> ThiagoCMC: e.g. userdata, or via preseeds
<ThiagoCMC> ok, I know... I was just wondering if there was an option on UI to do that...
<ThiagoCMC> No biggied
<mup> Bug #1695937 opened: [2.2] NTP server selection might not be optimal for racks serving multiple VLANs <MAAS:Triaged> <https://launchpad.net/bugs/1695937>
<benlake> roaksoax: you mentioned custom curtin scripts and I expressed interest. Iâll now express interest in your mention of customer preseeds. Please esplain? Just point me at something, I havenât found such info myself.
<roaksoax> benlake: https://insights.ubuntu.com/2017/06/02/customising-maas-installs/
<benlake> roaksoax: hero!
<roaksoax> :)
<roaksoax> benlake: /win 4
<roaksoax> err
<roaksoax> sorry
<benlake> is there a way to configure a VLAN associated subnet on an interface in an untagged manner?
<benlake> thatâs a heavy question. Basically, I know this interface has access to VLAN 123, but I canât tell MAAS to use subnet 456 on the interface unless I also set it up to be tagged, which I donât want.
<ThiagoCMC> benlake, deploy it as tagged, then, edit the /etc/network/interfaces file and remove the tag manually (I'm not an MaaS expert but, should work)... I'm doing similar things...
<benlake> well, I mean I know I can setup what I need manually. Trying to use MAAS native functions as much as possible
<benlake> but doing what you suggest at least letâs the IPAM function
<benlake> was hoping to let MAAS handle the interfaces file solely and not cross streams with deployment tooling
<mup> Bug #1695994 opened: [2.2] Error message when MAAS cannot resolve upstream is not user friendly <MAAS:New> <MAAS 2.2:New> <https://launchpad.net/bugs/1695994>
<benlake> is there a way to change an IP post deployment? that seems like itâd be a pretty reasonable use case. Especially if it was statically assigned.
#maas 2017-06-06
<mup> Bug #1680398 changed: Improper separator used when parsing OAuth Authorization Header <MAAS:Expired> <https://launchpad.net/bugs/1680398>
<mup> Bug #1680567 changed: [2.0] PTR RR is incorrectly placed in parent DNS zone when MAAS controls both. <dns> <MAAS:Expired> <https://launchpad.net/bugs/1680567>
<cshen>  Hi guys, we saw version 2.2 is released. but in our production system, we want to stick to 2.1 version. How can we still get 2.1 from PPA?
<mup> Bug #1696058 opened: [2.2][UI] No way no view bond configuration after deployment in web ui <MAAS:New> <https://launchpad.net/bugs/1696058>
<mup> Bug #1696108 opened: Interface model validates the MAC address twice <MAAS:New> <https://launchpad.net/bugs/1696108>
<mup> Bug #1696058 changed: [2.2][UI] No way no view bond configuration after deployment in web ui <MAAS:New> <https://launchpad.net/bugs/1696058>
<mup> Bug #1696122 opened: [2.2] Failed to get virsh pod storage: cryptic message if no pools are defined <MAAS:New> <https://launchpad.net/bugs/1696122>
<mup> Bug #1696122 changed: [2.2] Failed to get virsh pod storage: cryptic message if no pools are defined <MAAS:New> <https://launchpad.net/bugs/1696122>
<mup> Bug #1696058 opened: [2.2][UI] No way no view bond configuration after deployment in web ui <MAAS:New> <https://launchpad.net/bugs/1696058>
<mup> Bug #1696058 changed: [2.2][UI] No way no view bond configuration after deployment in web ui <MAAS:New> <https://launchpad.net/bugs/1696058>
<mup> Bug #1696122 opened: [2.2] Failed to get virsh pod storage: cryptic message if no pools are defined <MAAS:New> <https://launchpad.net/bugs/1696122>
<mup> Bug #1680398 opened: Improper separator used when parsing OAuth Authorization Header <MAAS:Triaged> <https://launchpad.net/bugs/1680398>
<mup> Bug #1680398 changed: Improper separator used when parsing OAuth Authorization Header <MAAS:Triaged> <https://launchpad.net/bugs/1680398>
<mup> Bug #1680398 opened: Improper separator used when parsing OAuth Authorization Header <MAAS:Triaged> <https://launchpad.net/bugs/1680398>
<benlake> anyone have any pointers regarding debugging post-up entries in interfaces? MAAS installs them, and it seems theyâve stopped firing on reboot.
<fabloub> hi there, i have a maas setup on a server. another server is able to boot (pxe). it gets to the login page but i can't log into the machine (ssh or directly using "ubuntu" for user and pass)
<fabloub> tips? ideas?
<mup> Bug #1696270 opened: Managed Allocation dangerously toggles the semantics for Reserved Ranges  <MAAS:New> <https://launchpad.net/bugs/1696270>
<mup> Bug #1696272 opened: Clicking on info icon toggles option <MAAS:New> <https://launchpad.net/bugs/1696272>
<fabloub> thank you mup i'll look into these
<mup> Bug #1696276 opened: Comissioning scripts should trigger a hardware reinventory <MAAS:New> <https://launchpad.net/bugs/1696276>
#maas 2017-06-07
<mup> Bug #1696276 changed: Comissioning scripts should trigger a hardware reinventory <MAAS:New> <https://launchpad.net/bugs/1696276>
<mup> Bug #1696276 opened: Comissioning scripts should trigger a hardware reinventory <MAAS:Incomplete> <https://launchpad.net/bugs/1696276>
<mup> Bug #1696285 opened: Machines that fail hardware testing need to be marked broken before they can be recommissioned <MAAS:New> <https://launchpad.net/bugs/1696285>
<mup> Bug #1696285 changed: Machines that fail hardware testing need to be marked broken before they can be recommissioned <MAAS:New> <https://launchpad.net/bugs/1696285>
<mup> Bug #1696285 opened: Machines that fail hardware testing need to be marked broken before they can be recommissioned <MAAS:New> <https://launchpad.net/bugs/1696285>
<cabinet> I'm having some problems getting a node to come up over PXE -- cloud-init says it can't find it's data source. I haven't worked with maas or cloud-init before, but I was wondering if someone could point me at either a way to log into the box to try to debug it from there, or a set of troubleshooting steps to try around this problem
<cabinet> I've looked through the documentation I could find, and haven't turned up anything that's helped
<cabinet> What about just pointing me at how I can edit the image that is pushed over and executed? I can't find any 'cloudinit' files on the maas machine
<mup> Bug # changed: 1416417, 1436531, 1438091, 1451580, 1468757, 1469663, 1506516, 1506517, 1506518, 1506519, 1506520, 1506523, 1506524, 1506525, 1506528, 1506529, 1512297, 1513392, 1513507, 1536092, 1536094, 1536097, 1536099, 1536101, 1536147
<mup> Bug # changed: 1536102, 1536104, 1536106, 1536109, 1536122, 1536123, 1536125, 1536132, 1536137, 1536139, 1536142, 1536144, 1536145, 1536146, 1536148, 1536153, 1536157
<mup> Bug #1536131 changed: [Node Networking] Change table widths <design> <ui> <MAAS:Invalid by ricgard> <https://launchpad.net/bugs/1536131>
<roaksoax> cabinet: /etc/maas/rackd.conf
<roaksoax> cabinet: maas_url, ensure it is pointint to an IP the machines can communicate to
<mup> Bug #1696485 opened: MAAS dhcp does not offer up multiple domains to search <MAAS:Triaged> <MAAS 2.2:Triaged> <https://launchpad.net/bugs/1696485>
<mup> Bug #1696485 changed: MAAS dhcp does not offer up multiple domains to search <MAAS:Triaged> <MAAS 2.2:Triaged> <https://launchpad.net/bugs/1696485>
<mup> Bug #1696485 opened: MAAS dhcp does not offer up multiple domains to search <MAAS:Triaged> <MAAS 2.2:Triaged> <https://launchpad.net/bugs/1696485>
<mup> Bug #1696512 opened: [2.2] UI: adding a virsh pod immediately commissions existing kvms <MAAS:New> <https://launchpad.net/bugs/1696512>
<xygnal> roaksoax: something about syntax in curtin_userdata_custom vs _centos?
<xygnal> getting errors after copying my custom over to try a native cos6 image
<ltrager> xygnal: In the root of your CentOS tarball you need to include Curtin hooks which finalize the install
<ltrager> xygnal: The ones we ship are at http://bazaar.launchpad.net/~maas-images-maintainers/maas-images/maas-ephemerals/files/head:/curtin/
<ltrager> xygnal: Those files should be stored in /curtin in the custom CentOS tar.gz
<ltrager> xygnal: We are looking into incorporating those hooks directly into Curtin so that won't be necessary in the future.
<xygnal> oh i see
<xygnal> you dont install curtin
<xygnal> on those
<ltrager> xygnal: Curtin is what MAAS uses to install any OS. The current version only knows how to install Ubuntu naively. The hooks tell Curtin how to do things like setup grub.
<ltrager> xygnal: One other things I forgot to mention you need to have cloud-init installed
<ltrager> xygnal: On CentOS 7 you also need to install https://kojipkgs.fedoraproject.org/packages/python-oauth/1.0.1/10.el7/noarch/python-oauth-1.0.1\
<ltrager> -10.el7.noarch.rpm
<xygnal> ltrager in this case its only a minor exception need for cos6 which is why I am trying to use the included cos6/7 images
<xygnal> we already pre-treat out custom cos7 image, i didnt realize the in-maas option ones were not pre-treated.  I'll look at that next.
<sanjay> hi
#maas 2017-06-08
<mup> Bug #1696661 opened: MAAS should offer multiple DNS servers in HA case <MAAS:New> <https://launchpad.net/bugs/1696661>
<mup> Bug #1696752 opened: [SRU] MAAS 2.2.0 <maas (Ubuntu):New> <https://launchpad.net/bugs/1696752>
<mup> Bug #1696771 opened: [snap] Preseeds are not 'common' that would allow the user to configure them <MAAS:New> <MAAS 2.2:New> <https://launchpad.net/bugs/1696771>
<mup> Bug #1696819 opened: After upgrade from MAAS 2.1 to MAAS 2.2, unable to commission or deploy systems <MAAS:New> <https://launchpad.net/bugs/1696819>
<mup> Bug #1696819 changed: After upgrade from MAAS 2.1 to MAAS 2.2, unable to commission or deploy systems <MAAS:New> <https://launchpad.net/bugs/1696819>
<mup> Bug #1696819 opened: After upgrade from MAAS 2.1 to MAAS 2.2, unable to commission or deploy systems <MAAS:New> <https://launchpad.net/bugs/1696819>
<mup> Bug #1618467 changed: cloud-init does not take into consideration proxy when adding PPA's. <cloud-init:New> <MAAS:Opinion> <https://launchpad.net/bugs/1618467>
<mup> Bug #1618467 opened: cloud-init does not take into consideration proxy when adding PPA's. <cloud-init:New> <MAAS:Opinion> <https://launchpad.net/bugs/1618467>
<mup> Bug #1618467 changed: cloud-init does not take into consideration proxy when adding PPA's. <cloud-init:New> <MAAS:Opinion> <https://launchpad.net/bugs/1618467>
<mup> Bug #1696819 changed: After upgrade from MAAS 2.1 to MAAS 2.2, unable to commission or deploy systems <docteam> <MAAS:Incomplete> <https://launchpad.net/bugs/1696819>
<mup> Bug #1696819 opened: After upgrade from MAAS 2.1 to MAAS 2.2, unable to commission or deploy systems <docteam> <MAAS:Incomplete> <https://launchpad.net/bugs/1696819>
<xygnal> ltrager: can you explain more about the hooks required?  I extracted our curtin tarball into the image as we have been doing with the cos7 image, but the latest build test on that new image shows the same curtin hook empty output
<xygnal> ltrager: I can see the curtin directory, including curtin-hooks.py,  what else could be wrong?
<mup> Bug #1696819 changed: After upgrade from MAAS 2.1 to MAAS 2.2, unable to commission or deploy systems <docteam> <MAAS:Incomplete> <https://launchpad.net/bugs/1696819>
<xygnal> ltrager: ok upon looking closely I can see this has both curtin and cloud-init in the image, so why am i still getting errors about curtin-hook?
<xygnal> paste.ubuntu.com/24810424  please check my evidence.  Why is curtin-hooks.py returning nothing to stdout?
<xygnal> ltrager: roakroax:
<ltrager> xygnal: Is the python-oauth package installed?
<ltrager> xygnal: its not currently in the CentOS repos you have to pull it directly from koji - https://kojipkgs.fedoraproject.org/packages/python-oauth/1.0.1/10.el7/noarch/python-oauth-1.0.1\
<ltrager> -10.el7.noarch.rpm
<ltrager> xygnal: although looking at the stack trace you posted it appears you're on CentOS 6?
<ltrager> xygnal: the command is failing when using grub to figure out what the boot device is
<ltrager> xygnal: make sure grub is installed in the image and try running the commands manually
<xygnal> this isna cos6 image
<xygnal> you said only cos7 needs that
<xygnal> ah yes sorry. reading.
<xygnal> this is the centos6 image from your code
<xygnal> just exported from maas and reimported after adding curtin
#maas 2017-06-09
<mup> Bug #1696892 opened: maas should have a reason./notes field for why a machine is marked broken <MAAS:New> <https://launchpad.net/bugs/1696892>
<mup> Bug #1218997 changed: maas throws unauthorized sometimes for no reason <intermittent-failure> <maas> <juju-core:Fix Released> <MAAS:Invalid> <https://launchpad.net/bugs/1218997>
<mup> Bug #1696892 changed: maas should have a reason./notes field for why a machine is marked broken <MAAS:New> <https://launchpad.net/bugs/1696892>
<mup> Bug #1697108 opened: Nodes created in virsh pod always use 'default' virtual network for NIC <MAAS:New> <https://launchpad.net/bugs/1697108>
<mup> Bug #1697108 changed: Nodes created in virsh pod always use 'default' virtual network for NIC <MAAS:New> <https://launchpad.net/bugs/1697108>
<mup> Bug #1697108 opened: Nodes created in virsh pod always use 'default' virtual network for NIC <MAAS:New> <https://launchpad.net/bugs/1697108>
#maas 2017-06-10
<mup> Bug #1697158 opened: MAAS not attach disk after commissioning <MAAS:New> <https://launchpad.net/bugs/1697158>
<atm_it> Hello, I'm having issues deploying openstack from landscape. I noticed I cannot ssh to some nodes deployed by maas
<atm_it> i get an "SSH Permission denied (publickey)" message. I think this is the reason why the deployment fails
<mup> Bug #1697209 opened: Network testing fail - ntp <MAAS:New> <https://launchpad.net/bugs/1697209>
#maas 2018-06-04
<mup> Bug #1761282 changed: [2.3] Re-installation fails with "Volume already exists" <curtin:Expired> <MAAS:Expired> <https://launchpad.net/bugs/1761282>
<myrat> hello everybody
<Dazul> Hi people.
<Dazul> I have a question about the status of the nodes.
<Dazul> I had a node who was running on production, and I released by accident.
<Dazul> I do not want to redeploy it, but I wish it was on statute deployed.
<Dazul> Is any way to force the deployed status for a node on maas?
<cliu> Hi... I am trying to upload a script to MAAS via GUI Settings-> Commissioning scripts, but  I got error "script: Missing "#" on YAML line". any idea what I did wrong?
<mup> Bug #1775032 opened: Allow clearing a disk during install <cdo-qa> <foundations-engine> <MAAS:New> <https://launchpad.net/bugs/1775032>
<roaksoax> cliu: you are missing a ## in the script e.g. #!/bin sh manybe ?
<cliu> roaksoax: thanks. I found my problem. when I copied the script, the line broke into 2. thanks.
<cliu> roaksoax: the error was a bit misleading... I thought there is a dictionary for YAML field with a value.
<mup> Bug #1775032 changed: [enhancement] Allow clearing a disk during install <cdo-qa> <enhancement> <foundations-engine> <MAAS:Invalid> <https://launchpad.net/bugs/1775032>
<mup> Bug #1775099 opened: MAAS Configuration | Name allows for unescaped quote [UI] <ui> <MAAS:New> <https://launchpad.net/bugs/1775099>
#maas 2018-06-05
<roaksoax> xygnal_: how's 2.4 going for you ?
<Supo> I have asked this question couple of times and got disconnected so my appologies for asking again but does anyone know why i would get a 409 Conflict on a brandnew Maas install with 2.4 when i try to deploy
<kiko> where do you get that error message? never seen it before
<Supo> maas local machine deploy -d hrrkk3 409 Conflict         Content-Type: text/html; charset=utf-8                Date: Tue, 05 Jun 2018 15:10:27 GMT              Server: TwistedWeb/17.9.0              Status: 409   Transfer-Encoding: chunked  <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">   <head> 
<Supo> i can get the machines to commision/test but i can not deploy
<mup> Bug #1775210 opened: Rogue dynamic range address remains after auto-assigned static IP issued upon deployment <MAAS:New> <https://launchpad.net/bugs/1775210>
<bdx> roaksoax: trying to build/release something with the latest fixes to maas/next or something?
#maas 2018-06-06
<mup> Bug #1771885 changed: bionic: static maas missing search domain in systemd-resolve configuration <bionic> <network> <cloud-init:Won't Fix> <juju:Fix Committed by ecjones> <juju 2.3:Fix Released by ecjones> <MAAS:Invalid> <https://launchpad.net/bugs/1771885>
<enrico_> hello, please hope you can help, I have deployed a maas cluster, I am able to manage power control of a node that I added manually on maas
<enrico_> the node is well commissioned
<enrico_> I want do deploy ubuntu on top
<enrico_> it seems stucked at: curtin: Installation started
<enrico_> cloud-init third party drivers not installed or necessary
<enrico_> and nothing more
<enrico_> maybe it does not arrive to use maas images ?
<enrico_> can you help to debug and figure out the problem please?
<mup> Bug #1775424 opened: CentOS 6 deployment fails with AttributeError: 'NoneType' object has no attribute 'groups' <MAAS:New> <https://launchpad.net/bugs/1775424>
<roaksoax> enrico_: can you share the rsyslog for the machine ? /var/log/maas/rsyslog/<machine-name>/<date>/messages
<xygnal_> roaxsoax;
<xygnal_> only tried in dev so far
<enrico_> roaksoax https://pastebin.com/kExGTtS2
<enrico_> these are tail -100 messages
<roaksoax> xygnal_: k, let me know how it goes
<mup> Bug #1775430 opened: Make gw optional for static routes <canonical-bootstack> <networking> <MAAS:New> <https://launchpad.net/bugs/1775430>
<roaksoax> enrico_: uhmm there's no real error surfaced at all
<roaksoax> enrico_: what's the status of the machine while you see this? 'Deploying'?
<enrico_> yes Deploying... and isntallation output still System is booting...
<roaksoax> enrico_: yeah that's because it never failed, and what happens if you leave it ?
<enrico_> if I dont touch it is seems to be in a stuck state forever ( I waited for more than 30min )
<enrico_> If I release the node.. the result is failed deployment
<roaksoax> enrico_: looks to me like curtin gets stuck
<enrico_> trying to redeploy again to generate new log
<enrico_> yes it could be but dont know why
<enrico_> you have any idea?
<roaksoax> enrico_: ssh into the machine while dpeloying
<roaksoax> touch /tmp/block-reboot
<roaksoax> enrico_: and tail /var/log/cloud-init.log
<roaksoax> and see if there are any errors
<enrico_> ok
<enrico_> 2018-06-06 16:49:22,334 - handlers.py[DEBUG]: finish: modules-final/config-scripts-per-instance: SUCCESS: config-scripts-per-instance ran successfully 2018-06-06 16:49:22,335 - url_helper.py[DEBUG]: [0/1] open 'http://55.0.51.31:5240/MAAS/metadata/status/npnec6' with {'method': 'POST', 'allow_redirects': True, 'url': 'http://55.0.51.31:5240/MAAS/metadata/status/npnec6', 'headers': {'Authorization': 'OAuth oauth_nonce="1746154607730300
<enrico_> https://pastebin.com/NhKAtHNz
<roaksoax> seems likes running just fine
<roaksoax> enrico_: dpkg -l | grep cloud-init
<roaksoax> to get the version
<enrico_> ii  cloud-init                       18.2-4-g05926e48-0ubuntu1~16.04.2          all          Init scripts for cloud instances
<enrico_> ii  cloud-initramfs-copymods         0.27ubuntu1.5                              all          copy initramfs modules into root filesystem for later use
<enrico_> ii  cloud-initramfs-dyn-netconf      0.27ubuntu1.5                              all          write a network interface file in /run for BOOTIF
<roaksoax> enrico_: can you try to refresh your maas images and see if it downloads updated images
<roaksoax> seems a new version of cloud-init is already released
<roaksoax> on may 16th, which should be on the latest images
<enrico_> I have installed maas after may 16th
<enrico_> I use play maas over ubuntu 16
<enrico_> cloud-init should be installed on maas rack/region controller right?
<roaksoax> enrico_: no, it comes from the maas images themselve
<roaksoax> enrico_: so go to the images tab and click 'Update selection' on the ubuntu section
<enrico_> when you say: maas images... you refer to ubuntu ISO that includes maas ?
<enrico_> ok
<roaksoax> enrico_: In the 'Images' tab
<roaksoax> enrico_: the images maas downloads
<enrico_> yes updated
<roaksoax> enrico_: ok, so try agian and check the cloud-init version to see if that has changed
<enrico_> yes done.. it is the same
<roaksoax> uhmm strange, so i winder if its a cloud-init issue
<roaksoax> i'll dig and ask around
<enrico_> Thanks you for your time.. I appreciate :)
<enrico_> If you discover something please contact me on enricodhd@gmail.com
<enrico_> anyway I'll ping you in a while here
<xygnal_> roaksoax:  whats the max latency/distance between rack controllers and region controllers, and how might it impact maas if we had a region controller in another country than a rack controller.
<roaksoax> xygnal_: well in most cases should be just fine, depending how many machines you may want to deploy at the same time
<roaksoax> xygnal_: i guess the major bottleneck is tftp because tftp sucks, which could mean if you deploying several machines at the same time they would take a long time to donwload the initrd/kernel
<roaksoax> but other than that everything should be ok, since its http
<xygnal_> we have two rack we want to deploy to India but them to test and play in, but it would not see a great deal of builds itself
<xygnal_> how would it handle the delays in check in?
<xygnal_> this would be literally across the globe
<roaksoax> xygnal_: 2 racks, say 10 machines? 20 machines ?
<xygnal_> about 16 maciens
<xygnal_> machines*
<roaksoax> xygnal_: yeah so the only "bottleneck" i could see if if you deploy the 16 machines at once, whether they would be able to download the initrd/kernel over tftp fast enough
<roaksoax> xygnal_: but technically, shouldn't really be a problem
<xygnal_> tftp would be coming from the region?
<xygnal_> not the rack?
<roaksoax> xygnal_: rack
<xygnal_> rack would be in same place at those 16 blades, its region that would be far away.
<xygnal_> why would tftp be slow?
<roaksoax> xygnal_: you should be fine then
<bdx> roaksoax: could you advise on how I might get out of this debacle with the stuck controller?
<xygnal_> roaksoax: I can't seem to find evidence of a CentOS 7 package for Cloud-init newer then 0.7.x :/  ever done a custom build/install of it?
<xygnal_> roaksoax: 0.7.9-24 is the latest in theirs repos
<xygnal_> cloud-init
<mup> Bug #1775461 opened: manual node inaccessible after commissioning  <MAAS:New> <https://launchpad.net/bugs/1775461>
<mup> Bug #1770201 changed: DNS resolution issues during enlistment with Bionic ephemeral environment. <cdo-qa> <cdo-release-blocker> <foundations-engine> <MAAS:Invalid> <https://launchpad.net/bugs/1770201>
<mup> Bug #1774666 changed: Bond interfaces stuck at 1500 MTU on Bionic <cdo-qa> <foundations-engine> <mtu> <netplan> <cloud-init:New> <MAAS:Invalid> <cloud-init (Ubuntu):Confirmed> <netplan.io (Ubuntu):Confirmed> <https://launchpad.net/bugs/1774666>
#maas 2018-06-07
<mup> Bug #1775490 opened: [2.5, 2.4, UI] Error as a notification in machine actions <ui> <MAAS:New for blr> <https://launchpad.net/bugs/1775490>
<sikander_> hi
<sikander_> Can we add existing installed OS nodes to MAAS?
<mup> Bug #1773117 opened: MAAS temporarily reporting no instance while bringing up a new instance. <enable-ha> <kvm> <maas> <maas-provider> <network> <juju:Triaged> <MAAS:New> <https://launchpad.net/bugs/1773117>
<mup> Bug #1744072 changed: [MIR] Chrony in 18.04 <NTP Charm:In Progress by paulgear> <Ubuntu Server Guide:Fix Released> <ceph (Ubuntu):Fix Released> <chrony (Ubuntu):Fix Released> <cloud-init (Ubuntu):Fix Released> <maas (Ubuntu):Fix Released by andreserl> <https://launchpad.net/bugs/1744072>
<mup> Bug #1775715 opened: Deployment of Bionic fails after curtin errors during install of grub <MAAS:New> <https://launchpad.net/bugs/1775715>
<mup> Bug #1775715 changed: Deployment of Bionic fails after curtin errors during install of grub <cdo-qa> <foundations-engine> <MAAS:Opinion> <https://launchpad.net/bugs/1775715>
<mup> Bug #1775715 opened: Deployment of Bionic fails after curtin errors during install of grub <cdo-qa> <foundations-engine> <MAAS:Opinion> <https://launchpad.net/bugs/1775715>
<mup> Bug #1775715 changed: Investigate if MAAS can automatically tell the machine to boot EFI when power_boot_type is 'auto' without impacting backwards compatibility/behavior on older firmwares. <cdo-qa> <foundations-engine> <MAAS:Opinion> <https://launchpad.net/bugs/1775715>
#maas 2018-06-08
<mup> Bug #1775728 opened: Pod VM fails commissioning on AARCH64 -  Failed to power on VM <MAAS:New> <https://launchpad.net/bugs/1775728>
<mup> Bug #1775728 changed: Pod VM fails commissioning on AARCH64 -  Failed to power on VM <MAAS:New> <https://launchpad.net/bugs/1775728>
<mup> Bug #1775728 opened: Pod VM fails commissioning on AARCH64 -  Failed to power on VM <MAAS:New> <https://launchpad.net/bugs/1775728>
<adham> Hi all, I have MAAS server that keeps recreating a new machine (using VM) with the same name and specs of an old machine that has been deleted, I keep deleting it, but it keeps coming back
<adham> have anyone came across this issue before?
<adham> A help would be really appreciated, as when it recreates that old node that has been previously deleted makes MAAS to consume too much resources
<adham> Can anyone help pleaes?
<mup> Bug #1775860 opened: [2.3.3] node failing to PXE boot with TFTP back-end failed <cpe-onsite> <MAAS:New> <https://launchpad.net/bugs/1775860>
<enrico__> \MOTD
<enrico__> hello, I am trying to add kvm guest as node in MAAS ! Till now I am able to power control the VMs but the commission does not work, can you help to debug please ?
<enrico__> ok solved thanks you aw
<mup> Bug #1775860 changed: [2.3.3] node failing to PXE boot with TFTP back-end failed <cpe-onsite> <MAAS:Invalid> <https://launchpad.net/bugs/1775860>
#maas 2018-06-10
<mup> Bug #1776056 opened: LXD/MAAS Integration failure in DNS (and no link in Subnet) <MAAS:New> <https://launchpad.net/bugs/1776056>
<mup> Bug #1776084 opened: Error deleting a machine <MAAS:New> <https://launchpad.net/bugs/1776084>
#maas 2020-06-01
<mup> Bug #1870042 changed: maas network connectivity check fails <MAAS:Expired> <https://launchpad.net/bugs/1870042>
<mup> Bug #1870042 opened: maas network connectivity check fails <MAAS:Expired> <https://launchpad.net/bugs/1870042>
<mup> Bug #1870042 changed: maas network connectivity check fails <MAAS:Expired> <https://launchpad.net/bugs/1870042>
<mup> Bug #1870042 opened: maas network connectivity check fails <MAAS:Expired> <https://launchpad.net/bugs/1870042>
<mup> Bug #1870042 changed: maas network connectivity check fails <MAAS:Expired> <https://launchpad.net/bugs/1870042>
<mup> Bug #1881547 opened: pxe power on <MAAS:New> <https://launchpad.net/bugs/1881547>
<mup> Bug #1881547 changed: pxe power on <MAAS:New> <https://launchpad.net/bugs/1881547>
<mup> Bug #1881547 opened: pxe power on <MAAS:New> <https://launchpad.net/bugs/1881547>
<mup> Bug #1881555 opened: UI stuck in error state after error loading machine <ui> <MAAS:New> <https://launchpad.net/bugs/1881555>
<mup> Bug #1881203 changed: SnapStore and Snap Blocks Install more Apps  <MAAS:Invalid> <https://launchpad.net/bugs/1881203>
<mup> Bug #1881203 opened: SnapStore and Snap Blocks Install more Apps  <MAAS:Invalid> <https://launchpad.net/bugs/1881203>
<mup> Bug #1881203 changed: SnapStore and Snap Blocks Install more Apps  <MAAS:Invalid> <https://launchpad.net/bugs/1881203>
<mup> Bug #1881203 opened: SnapStore and Snap Blocks Install more Apps  <MAAS:Invalid> <https://launchpad.net/bugs/1881203>
<mup> Bug #1881203 changed: SnapStore and Snap Blocks Install more Apps  <MAAS:Invalid> <https://launchpad.net/bugs/1881203>
<mup> Bug #1881560 opened: Poor UX for adding machine that's network booted into MAAS <ui> <MAAS:New> <https://launchpad.net/bugs/1881560>
<mup> Bug #1881560 changed: Poor UX for adding machine that's network booted into MAAS <ui> <MAAS:New> <https://launchpad.net/bugs/1881560>
<mup> Bug #1881203 opened: SnapStore and Snap Blocks Install more Apps  <MAAS:Invalid> <https://launchpad.net/bugs/1881203>
<mup> Bug #1881203 changed: SnapStore and Snap Blocks Install more Apps  <MAAS:Invalid> <https://launchpad.net/bugs/1881203>
<mup> Bug #1881560 opened: Poor UX for adding machine that's network booted into MAAS <ui> <MAAS:New> <https://launchpad.net/bugs/1881560>
<mup> Bug #1881577 opened: Conflict error when MAAS tries to add user with same email address <MAAS:New> <https://launchpad.net/bugs/1881577>
<mup> Bug #1881595 opened: RBAC denial of machine deployment considered lacklustre <MAAS:New> <https://launchpad.net/bugs/1881595>
<mup> Bug #1881595 changed: RBAC denial of machine deployment considered lacklustre <MAAS:New> <https://launchpad.net/bugs/1881595>
<mup> Bug #1881595 opened: RBAC denial of machine deployment considered lacklustre <MAAS:New> <https://launchpad.net/bugs/1881595>
<mup> Bug #1881595 changed: RBAC denial of machine deployment considered lacklustre <MAAS:New> <https://launchpad.net/bugs/1881595>
<mup> Bug #1881595 opened: RBAC denial of machine deployment considered lacklustre <MAAS:New> <https://launchpad.net/bugs/1881595>
<d0ugal> ltrager: also here :)
<mup> Bug #1878311 changed: Power and Pod drivers need way to accept private keys and certificates <ui> <MAAS:Invalid by ltrager> <https://launchpad.net/bugs/1878311>
<mup> Bug #1878311 opened: Power and Pod drivers need way to accept private keys and certificates <ui> <MAAS:Invalid by ltrager> <https://launchpad.net/bugs/1878311>
<mup> Bug #1878311 changed: Power and Pod drivers need way to accept private keys and certificates <ui> <MAAS:Invalid by ltrager> <https://launchpad.net/bugs/1878311>
<mup> Bug #1881655 opened: Can't deploy focal KVM hosts <MAAS:New> <https://launchpad.net/bugs/1881655>
<mup> Bug #1881663 opened: Unable to have two devices with same names but different domains <MAAS:New> <https://launchpad.net/bugs/1881663>
<mup> Bug #1881663 changed: Unable to have two devices with same names but different domains <MAAS:New> <https://launchpad.net/bugs/1881663>
<mup> Bug #1881663 opened: Unable to have two devices with same names but different domains <MAAS:New> <https://launchpad.net/bugs/1881663>
<mup> Bug #1881663 changed: Unable to have two devices with same names but different domains <MAAS:New> <https://launchpad.net/bugs/1881663>
<mup> Bug #1881663 opened: Unable to have two devices with same names but different domains <MAAS:New> <https://launchpad.net/bugs/1881663>
<mup> Bug #1881664 opened: Support multiple device types <MAAS:New> <https://launchpad.net/bugs/1881664>
<mup> Bug #1881665 opened: Machine-specific token for use to create devices <MAAS:New> <https://launchpad.net/bugs/1881665>
#maas 2020-06-02
<mup> Bug #1639054 changed: syslog flooded with dhcpd messages that seem to be due to the use of actions in MAAS 2.0 <MAAS:Expired> <https://launchpad.net/bugs/1639054>
<mup> Bug #1881737 opened: maas doesn't add nameserver from the second oam network on the deployed machine <MAAS:New> <https://launchpad.net/bugs/1881737>
<mup> Bug #1881737 changed: maas doesn't add nameserver from the second oam network on the deployed machine <MAAS:New> <https://launchpad.net/bugs/1881737>
<mup> Bug #1881737 opened: maas doesn't add nameserver from the second oam network on the deployed machine <MAAS:New> <https://launchpad.net/bugs/1881737>
<mup> Bug #1719473 opened: Add ability to overwrite DNS search list or defined which domains to be in search list <cpe-onsite> <internal> <wishlist> <MAAS:New> <https://launchpad.net/bugs/1719473>
<mup> Bug #1881803 opened: IPv6 leases UI is unreadable <MAAS:New> <https://launchpad.net/bugs/1881803>
<mup> Bug #1881804 opened: Adding aliases on top of bridges result in broken UI <MAAS:New> <https://launchpad.net/bugs/1881804>
<mup> Bug #1881805 opened: non-efi netboot through IPXE broken when IPv6 configured properly <MAAS:New> <https://launchpad.net/bugs/1881805>
<mup> Bug #1881807 opened: Putting IPv6 DNS servers in subnet configuration breaks network <MAAS:New> <https://launchpad.net/bugs/1881807>
<mup> Bug #1881807 changed: Putting IPv6 DNS servers in subnet configuration breaks network <MAAS:New> <https://launchpad.net/bugs/1881807>
<mup> Bug #1881807 opened: Putting IPv6 DNS servers in subnet configuration breaks network <MAAS:New> <https://launchpad.net/bugs/1881807>
<mup> Bug #1881807 changed: Putting IPv6 DNS servers in subnet configuration breaks network <MAAS:New> <https://launchpad.net/bugs/1881807>
<mup> Bug #1881807 opened: Putting IPv6 DNS servers in subnet configuration breaks network <MAAS:New> <https://launchpad.net/bugs/1881807>
<mup> Bug #1881807 changed: Putting IPv6 DNS servers in subnet configuration breaks network <MAAS:New> <https://launchpad.net/bugs/1881807>
<mup> Bug #1881807 opened: Putting IPv6 DNS servers in subnet configuration breaks network <MAAS:New> <https://launchpad.net/bugs/1881807>
<mup> Bug #1881809 opened: Interfaces configured as "Disconnected" not disabled in netplan <MAAS:New> <https://launchpad.net/bugs/1881809>
<mup> Bug #1881821 opened: node inventory isn't being properly reported for Cisco UCS B200 M5 blade <cpe-onsite> <MAAS:New> <https://launchpad.net/bugs/1881821>
#maas 2020-06-03
<mup> Bug #1881892 opened: CPU usage bar not shown in UI for KVMs <MAAS:New> <https://launchpad.net/bugs/1881892>
<mup> Bug #1881901 opened: Functional VLAN tagged bonding interfaces are shown as disconnected   <MAAS:New> <https://launchpad.net/bugs/1881901>
<mup> Bug #1881901 changed: Functional VLAN tagged bonding interfaces are shown as disconnected   <MAAS:New> <https://launchpad.net/bugs/1881901>
<mup> Bug #1881901 opened: Functional VLAN tagged bonding interfaces are shown as disconnected   <MAAS:New> <https://launchpad.net/bugs/1881901>
<mup> Bug #1881916 opened: [feature] Additional commissioning script parameters <MAAS:New> <https://launchpad.net/bugs/1881916>
<mup> Bug #1881919 opened: [feature] Manually run commissioning scripts <MAAS:New> <https://launchpad.net/bugs/1881919>
<mup> Bug #1881922 opened: [feature] Allow API to use AND and OR boolean operators <MAAS:New> <https://launchpad.net/bugs/1881922>
<mup> Bug #1881948 opened: IPv6 address for power control fails <MAAS:Triaged> <https://launchpad.net/bugs/1881948>
<mup> Bug #1881952 opened: Adding KVM host using LXD fails on IPv6 <MAAS:New> <https://launchpad.net/bugs/1881952>
<mup> Bug #1881954 opened: UI uses absolute URLs for machine entries (2.8) <MAAS:New> <https://launchpad.net/bugs/1881954>
<mup> Bug #1881977 opened: Ubuntu 14.04 failing to deploy on UEFI <MAAS:New> <https://launchpad.net/bugs/1881977>
<mup> Bug #1881977 changed: Ubuntu 14.04 failing to deploy on UEFI <MAAS:New> <https://launchpad.net/bugs/1881977>
<mup> Bug #1881977 opened: Ubuntu 14.04 failing to deploy on UEFI <MAAS:New> <https://launchpad.net/bugs/1881977>
#maas 2020-06-04
<mup> Bug #1881977 changed: Ubuntu 14.04 failing to deploy on UEFI <curtin:Confirmed for ltrager> <MAAS:Invalid> <https://launchpad.net/bugs/1881977>
<mup> Bug #1878769 changed: [SRU] updates to the maas deb2snap package <verification-done> <verification-done-focal> <maas (Ubuntu):Fix Released> <maas (Ubuntu Focal):Fix Released> <https://launchpad.net/bugs/1878769>
<mup> Bug #1882045 opened: pool admins can't create subnets <MAAS:New> <https://launchpad.net/bugs/1882045>
<mup> Bug #1882048 opened: [RBAC] normal users can't customize preseeds <MAAS:New> <https://launchpad.net/bugs/1882048>
<mup> Bug #1882053 opened: [RBAC] normal users can't create dhcp snippets <MAAS:New> <https://launchpad.net/bugs/1882053>
<mup> Bug # changed: 1662361, 1826015, 1848170, 1848781, 1860619, 1865866, 1869958, 1870117, 1871356, 1876180, 1876217, 1877126, 1877220, 1878117, 1878591, 1878685,
<mup> 1878923, 1879205, 1879416, 1879772, 1879774, 1879827, 1879978, 1879979, 1880746, 1881116, 1881117, 1881143, 1881201, 1881247, 1881262, 1881555, 1881954
<admcleod> ltrager: hi! i have updated a bug (1879933) i logged a couple of weeks back about a missing default route. do you still want the curtin output?
<mup> Bug #1882122 opened: MAAS DHCPv6 static leases don't work <MAAS:New> <https://launchpad.net/bugs/1882122>
<mup> Bug #1882122 changed: MAAS DHCPv6 static leases don't work <MAAS:New> <https://launchpad.net/bugs/1882122>
<mup> Bug #1882122 opened: MAAS DHCPv6 static leases don't work <MAAS:New> <https://launchpad.net/bugs/1882122>
<mup> Bug #1882133 opened: MAAS 2.8 snap install hangs on "INFO Waiting for restart..." <MAAS:New> <https://launchpad.net/bugs/1882133>
<mup> Bug #1882145 opened: MAAS 2.8 snap test-db install fails when running as (default) root in a container <MAAS:New> <https://launchpad.net/bugs/1882145>
<ltrager> admcleod: I reopen the bug for cloud-init and netplan as MAAS only hands off the config.
<mup> Bug #1882154 opened: rackd errors due to IPv6/IPv4 confusion <MAAS:New> <https://launchpad.net/bugs/1882154>
<mup> Bug #1882154 changed: rackd errors due to IPv6/IPv4 confusion <MAAS:New> <https://launchpad.net/bugs/1882154>
<mup> Bug #1882154 opened: rackd errors due to IPv6/IPv4 confusion <MAAS:New> <https://launchpad.net/bugs/1882154>
<mup> Bug #1882154 changed: rackd errors due to IPv6/IPv4 confusion <MAAS:New> <https://launchpad.net/bugs/1882154>
<mup> Bug #1882154 opened: rackd errors due to IPv6/IPv4 confusion <MAAS:New> <https://launchpad.net/bugs/1882154>
<mup> Bug #1882155 opened: MAAS 2.8 production mode sometimes loses connection when finished downloading an image <MAAS:New> <https://launchpad.net/bugs/1882155>
#maas 2020-06-05
<Drulgaard> Hi, all.  Been using MAAS 2.5.x for a year now, just trying MAAS 2.7.1, and getting frustrated with the fact it's a snap and my MAAS-building script needs a complete overhaul.  I'm now stuck on how to get my MAAS to remote-control a couple of libvirt VMs, as there's no longer a 'maas' user with a home directory to put an SSH key in.  Anyone know
<Drulgaard> how to do SSH (with public key auth) control of libvirt from MAAS 2.7?
<Drulgaard> (Basically, this... https://maas.io/docs/vm-host-networking#heading--set-up-ssh ... is now incorrect, as there is no 'maas' user.)
<Drulgaard> Oh, never mind, I just scrolled down that page a bit. :)
<mup> Bug #1882229 opened: Unable to upload any script <MAAS:New> <https://launchpad.net/bugs/1882229>
<mup> Bug #1882133 changed: MAAS 2.8 snap install hangs on "INFO Waiting for restart..." <MAAS:Invalid> <https://launchpad.net/bugs/1882133>
<mup> Bug #1882295 opened: 4 types of IPMI status in UI for same way configured nodes <MAAS:New> <https://launchpad.net/bugs/1882295>
<cloaked1> Hey all. We have a problem where maas thinks all of the IP addresses in a subnet are saturated, but when you scan the subnet, not even a quarter of the addresses are in use. How can we get maas to recheck the address usage? I've removed dhcpd leases and restarted dhcpd.
<mup> Bug #1882313 opened: Unable to upload commissioning/test script <ui> <MAAS:New> <https://launchpad.net/bugs/1882313>
<mup> Bug #1882313 changed: Unable to upload commissioning/test script <ui> <MAAS:New> <https://launchpad.net/bugs/1882313>
<mup> Bug #1882313 opened: Unable to upload commissioning/test script <ui> <MAAS:New> <https://launchpad.net/bugs/1882313>
<mup> Bug #1882313 changed: Unable to upload commissioning/test script <ui> <MAAS:New> <https://launchpad.net/bugs/1882313>
<mup> Bug #1882313 opened: Unable to upload commissioning/test script <ui> <MAAS:New> <https://launchpad.net/bugs/1882313>
<mup> Bug #1882229 changed: Unable to upload any script <MAAS:New> <https://launchpad.net/bugs/1882229>
#maas 2020-06-06
<mup> Bug #1882229 opened: Unable to upload any script <MAAS:New> <https://launchpad.net/bugs/1882229>
<mup> Bug #1882229 changed: Unable to upload any script <MAAS:New> <https://launchpad.net/bugs/1882229>
<mup> Bug #1882229 opened: Unable to upload any script <MAAS:New> <https://launchpad.net/bugs/1882229>
<mup> Bug #1882229 changed: Unable to upload any script <MAAS:New> <https://launchpad.net/bugs/1882229>
<cloaked1> I think I got it figured out. For some reason this subnet had a reservation for dynamic addresses that was taking up the entire /27 even though only 12 machines were in use.
<mup> Bug #1882371 opened: [2.8/candidate] /snap/maas/current/helpers/maas-database-setup missing <MAAS:New> <https://launchpad.net/bugs/1882371>
