[16:46] <blackboxsw> smoser: rharper : does cloud init always go through all cc_* modules  when trying to process a config or does it attempt to filter to only those cc_* modules for which a section key is present.
[16:46] <blackboxsw> err that was a ?
[16:46] <rharper> blackboxsw: no
[16:46] <rharper> there are several controlling factors
[16:46] <rharper> I think we've documented this; lemme find it
[16:47] <rharper> not that I can find easliy
[16:47] <rharper> =(
[16:47] <rharper> blackboxsw: so, in the system_info configuration (defined in /etc/cloud/cloud.cfg) there are lists of modules to run at various stages
[16:47] <blackboxsw> cool yeah I just missed the docs/comments explaining why as i was wondering why cc/handle needs to check for key and log a "skip"
[16:47] <rharper> those are considered the default list of modules that run
[16:51] <blackboxsw> so, for example,  chef is defined in the final module stage, but if there is no chef config defined why would we attempt to run the module's handle()
[16:52] <blackboxsw> do some modules actually perform work if no section key is present?
[16:53] <blackboxsw> I know ntp performs work if both pools and server lists are empty.(creating default pools). But that at least requires an ntp  key in your cloud-config
[16:54] <farcaller> smoser: hi
[16:54] <rharper> blackboxsw: so, typically if you provide top-level config for a module, it will run; some modules (if in the default list) will run whether config is provided or not;
[16:54] <rharper> ntp, for example, is in the default list in /etc/cloud/cloud.cfg
[16:54] <rharper> chef, however, is not
[16:58] <blackboxsw> I misunderstood, I thought cloud_final_modules list containing chef & puppet in /etc/cloud/cloud.cfg means chef will be run by default. Is that cloud_final_modules treated differently than cloud_config_modules in the cloud.cfg?
[16:58] <rharper> sorry, I didn't realize what was in the list
[16:58] <blackboxsw> oh ok. good good
[16:58] <rharper> yes, it will run if in the list;  typically the modules should have a check to see if they have config or not and quickly exit
[16:59] <blackboxsw> yeah I was wondering about shortcutting that last part "e if they have config or not and quickly exit" so that they don't even get called if no config
[17:00] <blackboxsw> so that the handlers get a little simpler (knowing that they aren't called if their schema-defined keys are not present).
[17:00] <blackboxsw> but not something for this branch I'm working currently. Just thoughts bouncing around in my head
[17:01] <blackboxsw> ... as there's a lot of empty space in there (my head
[17:05] <smoser> farcaller, hey.
[17:05] <smoser> whats up, farcaller ?
[17:47] <farcaller> smoser: sorry, got afk for a moment
[17:48] <farcaller> smoser: I got you a simple implementation of NoCloud via dmi: https://bugs.launchpad.net/cloud-init/+bug/1691772
[17:48] <farcaller> I think it's safe enough in that you need to have nocloud magic string for it to work, and it's very simple to pass through via qemu / libvirt / whatever.
[17:56] <smoser> farcaller, that looks nice.
[17:56] <smoser> a few thoughts
[17:56] <smoser> a.) can you give an eexample of qemu cmdline that you use. (i'im guessing its like the kernel command line... you're puttting ds=nocloud;seedfrom=http://)
[17:57] <farcaller> yep, I use exactly that
[17:57] <smoser> b.) not sure if serial number seems right or not here (admittedly we'll be abusing something)
[17:58] <farcaller> and yes, it's definitely quite an abuse, I guess other fields could work better, but I'm not sure. It's up to "cloud" owners to see what they are willing to sacrifice, I guess?
[17:58] <smoser> well, if you're passing 'seed' in that way, your'e not really a "cloud" owner i dont think. right?
[17:58] <farcaller> yep
[17:58] <smoser> ie, if you've *acutally* got a cloud, lets take a similar path to what brightbox did
[17:59] <farcaller> that's what I think too
[17:59] <farcaller> it's more useful for small scale / hobby stuff
[17:59] <smoser> its really good idea. and somethign iv'e wanted to do before. thanks for raising it.
[18:00] <smoser>  (unfortunate that smbios is only x86... arm64 has it, but qemu bugs prevent using)
[18:00] <farcaller> yep
[18:01] <farcaller> it's just that ec2-like md server is still the simplest way to bootstrap
[18:03] <smoser> farcaller, you can also fairly easily modify your smbios to look like ec2
[18:04] <farcaller> not really, I tried that :)
[18:04] <farcaller> cloud-init seems to look into sysfs for ec2-like uuid
[18:04] <smoser> well... thats because it was broken
[18:04] <farcaller> I couldn't find a way to fake that
[18:04] <smoser> its fixed now i'm pretty sure (in trunk)
[18:05] <farcaller> I see. I was experimenting with 16.10 image, I believe
[18:05] <smoser> you ahve to set product serial and uuid to the same value and they ahve to start with ec2 (case insensitive)
[18:05] <farcaller> didn't work for me there
[18:05] <farcaller> yep, I definitely tried doing that
[18:06] <smoser> commit was at 370a04e8d7b530c1ef8280e15eb628ff6880c736
[18:06] <smoser> it went in last week, not in an image yet.
[18:06] <farcaller> ack
[18:07] <smoser> farcaller, do you know how big that can be ?
[18:07] <smoser> how long the fied
[18:07] <smoser> field
[18:08] <farcaller> nope. I didn't check tbh. I verified that it fits seedfrom with ip address, and that's it
[18:37] <larsks> smoser: I'm seeing unit test failures on fedora because DataSourceCloudStack is trying to access /var/lib/NetworkManager, which requires root access.
[18:38] <smoser> selinux=off
[18:38] <smoser> ;)
[18:38] <larsks> smoser: nope.
[18:39] <larsks> drwx------. 2 root root 4096 May 17 10:53 /var/lib/NetworkManager
[18:39] <larsks> Unit tests shouldn't be trying to access anything outside of the current directory in any case.
[18:39] <larsks> Or current directory + any tmpdir for the test.
[18:39] <smoser> agreed.
[18:39] <smoser> (i was just joking about selinux)
[18:39] <larsks> Is there any chance we can get automated ci running on a centos system?
[18:39] <smoser> larsks, there sure is!
[18:40] <larsks> Is there anything I can do to help?
[18:40] <smoser> https://jenkins.ubuntu.com/server/view/Cloud-init/
[18:40] <smoser> the centos-6 and centos-7 are running now (and passing)
[18:40] <smoser> powersj just got that up a few days ago.
[18:41] <larsks> smoser: I am confused as to why they are passing, given the errors I am seeing locally.
[18:41] <smoser> they dont run on MP but on trunk nightly
[18:41] <larsks> My centos7 box has the same permissions on /var/lib/NetworkManager as the  fedora system on which I'm working.
[18:41] <smoser> probably becausae they run in an lxd image that doesn't have networtk manager
[18:41] <powersj> yeah I'm cheating by using lxd, don't have hardware for dedicated centos system yet
[18:42] <larsks> powersj: just installing the NM package into the LXD container should trigger this problem, right?
[18:42] <larsks> But I guess the real solution is to fix the unit tests.
[18:42] <smoser> ii suspect it should trigger it, and yeah, we can fix the unit tests. i'll give it a try here.
[18:43] <smoser> i suspect i can just mkdir -p /var/lib/NetworkManager && chmod 700
[18:43] <rharper> you need to touch a file in there
[18:43] <larsks> smoser: awesome, thanks.
[18:43] <rharper> if os.path.exists(d) and len(os.listdir(d)) > 0:
[18:43] <rharper> then it'll try to parse it
[18:44] <larsks> smoser: I would vote for just installing the package, rather than trying faking it.  Because who knows what other permissions we might be missing?
[18:45] <smoser> hm.. well it wasnt that easy
[18:45] <smoser>  sudo chmod go-rX /var/lib/NetworkManager/
[18:45] <smoser> and
[18:45] <smoser> tox-venv py3 python3 -m nose tests/unittests/test_datasource/test_cloudstack.py
[18:45] <smoser> still works for me.
[18:47] <larsks> Iiiinteresting.  I bet that https://github.com/cloud-init/cloud-init/blob/master/cloudinit/sources/DataSourceCloudStack.py#L179 is finding one of the other directories in that list.
[18:47] <smoser> oh. because i have a /var/lib/dhclient probably
[18:47] <larsks> Yeah.
[18:47] <smoser> yeah
[18:48] <larsks> :)
[18:50] <larsks> What if we just mocked out get_dhclient_d?
[18:52] <smoser> larsks, i think just mock get_vr_address
[18:53] <raspado> hi all, I tried to disable autmount of ephemeral storage to /dev/vdb via cloud.cfg but our instance is still creating ephemeral -> /dev/vdb, here is my example https://pastebin.com/8HkNbbEd , any other method I can take to preventing epehemeral storage from auto mounting to vdb?
[18:53] <larsks> smoser: or that.
[18:53] <smoser> but get_client_d would work to, and then you could even populate it
[18:53] <smoser> with a file that shoudl get correctly parsed.
[18:54] <larsks> Right, that was my thought.
[19:11] <larsks> smoser: https://code.launchpad.net/~larsks/cloud-init/+git/cloud-init/+merge/324277
[19:12] <larsks> that mocks out get_latest_lease (which was simpler than mocking out get_vr_address, since we can just return None)
[19:20] <smoser> that'll do.t hanks.
[19:24] <raspado> hi all, anyway to disable mounting of ephemeral0 -> /dev/vdb?
[19:32] <larsks> raspado: that's not a "mount"; that's just the name of the block device.  I don't think it gets mounted anywhere automatically.
[19:37] <raspado> cloud.cfg doesnt manage that?
[19:38] <larsks> No, the linux kernel automatically creates block devices entries for any devices it discovers.
[19:38] <larsks> That's just normal kernel behavior.
[19:39] <raspado> so if ephemeral storage is available, the kernel will auto discover and allocate it to a device id? hmm is that just cloud-aware os's?
[19:39] <larsks> No, that's just normal Linux.  IF there is a block device attached, THEN you get an entry in /dev.
[19:40] <raspado> hmm so in my case when i build an instance that has ephemeral storage, it automounts to /dev/vdb, I was hoping I can manipulate that via cloud.cfg somehow
[19:40] <raspado> ah well..
[19:43] <raspado> http://cloudinit.readthedocs.io/en/latest/topics/examples.html#adjust-mount-points-mounted
[19:43] <raspado> fs_setup: may be able to disable this?
[22:25] <blackboxsw> powersj: rharper smoser ok here's the start of the schema validation tool for ntp with sample failures (poor formatting from by for loop) http://pastebin.ubuntu.com/24601457/
[22:25] <blackboxsw> from *my* for loop
[22:41]  * rharper looks
[23:20] <rharper> blackboxsw: super interesting ... so, looking at the valid ones; and I'm not sure we can do much about it;  the strings for hosts or ips...
[23:21] <rharper> those could be valid in the schema since we're only specifying the encoded type as string or list or dict;