[12:00] <AskUbuntu> juju state server upgrade | http://askubuntu.com/q/609569
[12:02] <jcastro> marcoceppi_, can you pass this around in  nuremberg? ^^
[12:26] <marcoceppi_> jcastro: ack
[12:32] <drbidwell> Is there a way to import/export a MAAS configuration from/to a yaml file?
[12:54] <marcoceppi_> drbidwell: what do yoy mean maas configuration?
[13:05] <drbidwell> marcoceppi: the networks, cluster, and zones information.  The stuff that I need to beat into by hand when I have to rebuild it.
[13:06] <drbidwell> marcoceppi: The Mirantis fuel software has a process for feeding this kind of information into the system from a yaml file.  It was handy to have.
[13:16] <AskUbuntu> How many nodes to use Maas/Juju for Openstack (Juno)? | http://askubuntu.com/q/609589
[13:16] <puffi> Not sure if this is really a juju question, but i just did an openstack install on ubuntu 14.04 using juju. I rebooted the machine after install. It's not clear what needs to be started? It's a single instance
[13:19] <cory_fu> bcsaller: https://insights.ubuntu.com/2015/04/15/using-the-services-framework-to-implement-your-charms-intent/
[13:19] <cory_fu> et al.
[13:20] <bcsaller> cory_fu: tl;dr
[13:20] <bcsaller> jk
[13:20] <cory_fu> :)
[13:22] <bcsaller> cory_fu: that section on rewriting charms with the framework is gold
[13:26] <cory_fu> bcsaller: Agreed
[13:55] <Spads> bcsaller: thanks!
[13:55] <bcsaller> :)
[13:56] <bcsaller> Spads: thank you
[13:59] <lukasa> If a charm relation is changed, can I determine exactly which fields in the relation changed?
[14:03] <bcsaller> lukasa: not without keeping state in the unit. Its common to be able to process the whole state and take what ever actions are possible given the data you have
[14:06] <lukasa> bcsaller: Yeah, sadly I'm working on charm that has a particularly annoying component in it
[14:06] <lukasa> I have a follow-on question, but it works best if you know about the config file infrastructure the openstack charms have
[15:08] <cmars> wwitzel3, i;ve got some pull requests into juju-pyramid for you, when you get a chance
[15:52] <redelmann> Hi, im using bootstraped in remote maas
[15:52] <redelmann> and "juju debug-log"
[15:52] <redelmann> says: ERROR cannot open log file: open /var/log/juju/all-machines.log: no such file or directory
[15:52] <redelmann> strange, why it is reading local file?
[15:54] <redelmann> mh.. ok, i just  realized that juju is telling me that is not all-machine.log in machine0
[15:54] <ctlaugh> lazyPower_, catbus1: I did a manual reset, put the password in that file, and run 'juju resolved mysql/0', but now I am getting all kinds of errors regarding authorization failures from horizon: "Unable to retrieve public images", "Unable to retrieve images for the current project" when I try to launch instances or create volumes.  Is the mysql root password used by the other charms in some way?
[16:14] <lazyPower_> ctlaugh: it may be that horizon is using an administrative credential, which would denote it needs the password update as well
[16:15] <lazyPower_> beisner: Heyo, Can i tap your brain for a second with regard to ctlaugh's issue? ^
[16:19] <ctlaugh> lazyPower_, beisner: I looked in the horizon logs and they are full of the authorization errors from glance.  So, then, I looked in the glance logs, and they are full of this: "WARNING keystoneclient.middleware.auth_token [-] Authorization failed for token".
[16:20] <beisner> o/ hi lazyPower_ ctlaugh
[16:20] <lazyPower_> beisner: the TLDR; on whats going on - ctlaugh joined yesterday and was missing /var/lib/mysql.passwd
[16:20] <ctlaugh> So, I've been looking in the keystone configs and logs trying to see what might be the cause, but haven't found it yet.  Keystone's config looks like it uses a 'keystone' user, which shouldn't have changed.
[16:21] <lazyPower_> to get root access back, i linked to MySQL docs on how to do a single-user-admin reset, this appears to have exploded into dropping auth for the other openstack components (assuming they were consuming the db-admin role)
[16:21] <catbus1> I just want to point out, run juju resolved with --retry so the hook script will be re-run. running it without --retry only sets the status to resolved.
[16:24] <beisner> could this be bug 1423153 ?
[16:24] <mup> Bug #1423153: /var/lib/mysql/mysql.passwd no longer exist <mysql (Juju Charms Collection):Fix Released by hopem> <percona-cluster (Juju Charms Collection):Fix Released by hopem> <https://launchpad.net/bugs/1423153>
[16:25] <beisner> ^ ctlaugh, lazyPower_    helps if I tag ya
[16:28] <ctlaugh> beisner:, lazyPower_ - I saw that bug, but wasn't sure if it was related to my situtation or not.  I deployed this cloud months ago, and it's been working fine since then... until I powered the whole system off and moved it.  That's when I started getting the errors and the file was missing (well, not sure if it was ever there since I had never looked for it, but the charm was complaining that it was missing).
[16:33] <beisner> ctlaugh, lazyPower_ - the only deployment i have up at the moment to poke on uses percona cluster instead of mysql.  on that, there is no /var/lib/mysql.passwd file.   that is with the 15.01 charms i believe.
[16:33] <beisner> ctlaugh, so do the usual api commands fail?    keystone user-list && keystone token-get && nova list && cinder list && glance image-list  ...etc?
[16:34] <beisner> +     keystone catalog  &&  nova service-list
[16:35] <beisner> ctlaugh, lazyPower_ - i'd definitely start by looking at keystone, rabbit and mysql logs.   if those three aren't happy, the rest of the stack will be full of errors.
[16:38] <lazyPower_> beisner: thanks for taking a look
[16:38] <ctlaugh> beisner: lazyPower_ : some work, some don't: http://paste.ubuntu.com/10827803/
[16:40] <beisner> lazyPower_, sure welcome!
[16:42] <lazyPower_> ctlaugh: which revision of the keystone charm are you using?
[16:43] <ctlaugh> lazyPower_:     charm: cs:trusty/keystone-14
[16:43] <ctlaugh>     can-upgrade-to: cs:trusty/keystone-20
[16:45] <ctlaugh> It does appear to be keystone that's having the problem, but I see nothing that looks like an error in /var/log/keystone/keystone.log.
[16:45] <ctlaugh> I take that back...
[16:46] <ctlaugh> Well, I see a few errors related to not being able to connect to mysql, but I think that is from when it was restarting.
[16:47] <beisner> ctlaugh, basically, until    keystone token-get    works, everything else will be problematic.     i'd restart the keystone service, whilst watching its log, to see if it is connecting.
[16:47] <beisner> connecting to mysql that is.
[16:51] <beisner> ctlaugh, gotta step out for ~30, but will return.   let us know re:  keystone:mysql
[16:58] <ctlaugh> beisner: ok, thank you.  I'm trying, but keystone.log is showing NO errors when token-get is failing.  Just lots and lots of debug spam.
[16:58] <ctlaugh> beisner: I'm about to leave for lunch as well, but will be back soon.
[17:07] <beisner> ctlaugh,    keystone --debug token-get
[17:21] <bitchecker> hi
[18:18] <ctlaugh> beisner: http://paste.ubuntu.com/10828233/
[18:20] <beisner> ctlaugh, can you pastebin the keystone log?
[18:37] <ctlaugh> beisner: working on it.  it's choking on the massive file size.  i am going to split it.
[18:44] <ctlaugh> beisner: http://svartalfheim.laughlins.org/share/keystone.txt
[19:06] <beisner> ctlaugh, sorry, pulled in another direction.   things to check:  what openstack environment variables are set?   it looks like you're using a token.  i generally don't, i use something like this, where x.x.x.x is the keystone IP.   you may get more meaningful output from   keystone --debug token-get      and keystone --debug catalog   if you set your environment like so.
[19:06] <beisner> http://paste.ubuntu.com/10828404/
[19:12] <ctlaugh> beisner: I took the token out, made sure I have all the EVs set like in your example (I had them all there already, but also had the token one) and I get this: Expecting a token provided via either --os-token or env[OS_SERVICE_TOKEN]
[19:12] <ctlaugh> (from keystone --debug token-get)
[19:14] <beisner> ctlaugh, what does this show?:     env | grep OS     ... be sure to mask your actual password.
[19:14] <ctlaugh> beisner: http://paste.ubuntu.com/10828440/
[19:15] <beisner> ctlaugh, dpkg-query --show python-keystoneclient   ?
[19:16] <beisner> +:        apt-cache policy python-keystoneclient     ?
[19:17] <ctlaugh> beisner: http://paste.ubuntu.com/10828453/
[19:18] <AskUbuntu> OpenStack Juno and restarting it | http://askubuntu.com/q/609718
[19:19] <beisner> ctlaugh - do ANY keystone cmds work?     keystone catalog      keystone user-list    keystone service-list     keystone endpoint-list
[19:20] <ctlaugh> beisner: not after removing the token EV.  with the token, these work: endpoint-list, user-list, service-list
[19:21] <ctlaugh> beisner: keystone catalog fails with this, even with the token: 'NoneType' object has no attribute 'has_service_catalog'
[19:22] <beisner> ctlaugh,  weird.  i'm not sure what's going on.  i was hoping to see some meaningful keystone errors.    any hints from mysql logs?
[19:22] <beisner> failed auth, errors connecting, etc?
[19:24] <ctlaugh> beisner: nothing that I see
[19:24] <beisner> ctlaugh, are these all deployed with juju?   any changes made to the stack outside of using juju config options?
[19:25] <beisner> rather, charm config options
[19:25] <ctlaugh> beisner: no, all with Juju -- no change to any charm config outside of that
[19:26] <ctlaugh> beisner: I was hoping some kind of solution was going to jump out, but I'm thinking now it might be easier to ust blow it all away and start over.
[19:27] <beisner> ctlaugh, so did the mysql password end up getting manually reset?
[19:28] <ctlaugh> Yes, I manually reset it (the root password), and verified I could log in using it.  I put that password in the mysql.passwd file and run juju resolved --retry.
[19:29] <ctlaugh> beisner: I looked in the config files of nova, keystone, glance, etc. etc and if there is a database URL, they all have separate users -- never root. So, I can't see how a root password change could have affected that.
[19:29] <beisner> ctlaugh, right, i would think the same.
[19:29] <ctlaugh> beisner: Is there something to do to force the keystone charm to re-do whatever it does to see if it can correct itself?
[19:32] <beisner> ctlaugh, there is a debug hooks developer mode that can re-trigger individual hooks, but it's generally for charm development.
[20:26] <arosales> anyone available to kick an automated charm test for me on https://bugs.launchpad.net/charms/+bug/1441622
[20:26] <mup> Bug #1441622: Review queue:  xCAT  <Juju Charms Collection:New> <https://launchpad.net/bugs/1441622>
[20:26] <ctlaugh> beisner: I really appreciate all your time today and all your attempts to help.  I am going to redeploy my openstack setup -- hopefully that'll be quicker to get back up and running.  I didn't have anything critical in it... it was just used for an Openstack CI setup I was creating.
[20:29] <MitchM> Can I use Juju / MaaS to provision barebone servers with any image (not just openstack) ?
[20:35] <beisner> ctlaugh, you're welcome - sorry i couldn't help dig deeper today.
[20:44] <AskUbuntu> How to edit machine hardware detail deployed by Juju? | http://askubuntu.com/q/609741
[22:20] <ctlaugh> beisner: I redeployed from scratch, and now all the keystone commands are working.  The only thing I see that is still failing (both in horizon and command-line) is 'glance image-list'  -- keeps returning an "Authorization failed for token".  Very strange... still looking.