[00:29] blackboxsw: may also consider running with lxc from ppa [00:29] I know smoser and I both run artful, which will have latest and greatest, and base version in xenial probably isn't totally going to have everything [00:30] ppa: https://launchpad.net/~ubuntu-lxc/+archive/ubuntu/stable?field.series_filter=xenial [00:53] powersj, blackboxsw yeah, i figured i might need to address different lcx versions. [01:35] blackboxsw, good evening. Were you able to have a look to the cloud.cfg I posted? [02:03] redcavalier:thanks for posting. hmm ok looks like you have resizefs as you mentioned. only difference I'm seeing from ubuntu related to disks is you are missing a disk_setup in your cloud config modules... but I don't think that alone would preclude you from resizing [02:03] yea, it has always worked before the upgrade, with that cloud.cfg [02:05] Hence why I think there's definitely a bug there and why I assumed it is related to the vendordata issue, but it could also be an error that'S not logged [02:05] I mean, do you think the vendordata issue could prevent growpart from running? [02:07] redcavalier: I talked w/ smoser today about your bug, we're hoping to work out a fix for that in earnest next week. At least the vendordata_raw will be sorted in 17.1+ shortly (and you'd be able to pull our daily builds from copr). [02:08] Ah, then we'll see if that'S the cause or if there's something else [02:08] redcavalier: I don't believe that is really breaking the resize is the thing..... I just don't see that module even running on your attached cloud-init.log [02:08] that's what baffles me. I see other cloudinit modules running [02:09] It's definitely running at least once though. Let me give you a small explanation regarding what we do. [02:09] maybe you could share some of your #cloud-config with us/me in pvt message ? I'll share with scott as we look this over on Monday. my email is chad.smith@canonical.com if there is stuff you want to keep semi-private. (or you could redact some of the content) [02:10] We actually have a VM that's constantly running. To create (and update) our images, we make a snapshot of it's volume, convert it to image and then boot up on a new volume. [02:11] Our latest image is still on the older cloud-init, so the VM does resize on first boot. I do believe that should at least be in the log. It's not though. [02:12] We don't even erase the log from the previous VM when we create an image, so the file is already there when the VM boots. [02:12] I don'T really get it. [02:13] Anyhow, I'll send you a copy of our cloud-init by email, it'll give you a good idea. [02:14] on 17.1 you could run "cloud-init analyze show" on the commandline and it'd give a simple breakdown of what modules ran from your cloud-init.log files and how long it took. I'd also expect to see growpart and resizefs in that report [02:15] thanks for the additional context. it'll help us understand what might be happening [02:15] ah, there might be a thing I forgot to say also [02:15] The cloud-init daemon is crashed [02:15] Like, the daemon doesn'T start [02:16] again, the only error it gives in journalctl is that it can'T retrieve the vendordata, but I think there's something else [02:17] hrm. ok I'll poke at centos7 again tonight/this weekend just to make sure we can run. we test all the time in lxc w/ centos, but I'm not sure how frequent our coverage is on openstack [02:18] Ok. All this makes me wonder if there's something wrong in our user data [02:23] it also might be worth a quick check of the pickled datasource object [02:23] redcavalier: something like this http://pastebin.ubuntu.com/25735734/ [02:24] just find the pkl filename under /var/lib/cloud [02:24] then you can pkl load it and check userdata userdata_raw vendordata vendordata_raw on your instance [02:24] make sue it has the content you expect [02:25] Alright, I'll do that. Not right now because it'S Saturday morning here, but I'll have a look monday [02:26] Just sent the email [02:27] thanks again. we'll be in touch [02:27] yeah, it's Friday night, gotta get some movie time in ;) [02:27] have a good weekend [02:30] You too! see ya next week