=== sambetts|afk is now known as sambetts [16:46] smoser: rharper : does cloud init always go through all cc_* modules when trying to process a config or does it attempt to filter to only those cc_* modules for which a section key is present. [16:46] err that was a ? [16:46] blackboxsw: no [16:46] there are several controlling factors [16:46] I think we've documented this; lemme find it [16:47] not that I can find easliy [16:47] =( [16:47] blackboxsw: so, in the system_info configuration (defined in /etc/cloud/cloud.cfg) there are lists of modules to run at various stages [16:47] cool yeah I just missed the docs/comments explaining why as i was wondering why cc/handle needs to check for key and log a "skip" [16:47] those are considered the default list of modules that run [16:51] so, for example, chef is defined in the final module stage, but if there is no chef config defined why would we attempt to run the module's handle() [16:52] do some modules actually perform work if no section key is present? [16:53] I know ntp performs work if both pools and server lists are empty.(creating default pools). But that at least requires an ntp key in your cloud-config [16:54] smoser: hi [16:54] blackboxsw: so, typically if you provide top-level config for a module, it will run; some modules (if in the default list) will run whether config is provided or not; [16:54] ntp, for example, is in the default list in /etc/cloud/cloud.cfg [16:54] chef, however, is not [16:58] I misunderstood, I thought cloud_final_modules list containing chef & puppet in /etc/cloud/cloud.cfg means chef will be run by default. Is that cloud_final_modules treated differently than cloud_config_modules in the cloud.cfg? [16:58] sorry, I didn't realize what was in the list [16:58] oh ok. good good [16:58] yes, it will run if in the list; typically the modules should have a check to see if they have config or not and quickly exit [16:59] yeah I was wondering about shortcutting that last part "e if they have config or not and quickly exit" so that they don't even get called if no config [17:00] so that the handlers get a little simpler (knowing that they aren't called if their schema-defined keys are not present). [17:00] but not something for this branch I'm working currently. Just thoughts bouncing around in my head [17:01] ... as there's a lot of empty space in there (my head === sambetts is now known as sambetts|afk [17:05] farcaller, hey. [17:05] whats up, farcaller ? [17:47] smoser: sorry, got afk for a moment [17:48] smoser: I got you a simple implementation of NoCloud via dmi: https://bugs.launchpad.net/cloud-init/+bug/1691772 [17:48] Ubuntu bug 1691772 in cloud-init "provide a way to seed NoCloud from network without image modification." [Medium,Confirmed] [17:48] I think it's safe enough in that you need to have nocloud magic string for it to work, and it's very simple to pass through via qemu / libvirt / whatever. [17:56] farcaller, that looks nice. [17:56] a few thoughts [17:56] a.) can you give an eexample of qemu cmdline that you use. (i'im guessing its like the kernel command line... you're puttting ds=nocloud;seedfrom=http://) [17:57] yep, I use exactly that [17:57] b.) not sure if serial number seems right or not here (admittedly we'll be abusing something) [17:58] and yes, it's definitely quite an abuse, I guess other fields could work better, but I'm not sure. It's up to "cloud" owners to see what they are willing to sacrifice, I guess? [17:58] well, if you're passing 'seed' in that way, your'e not really a "cloud" owner i dont think. right? [17:58] yep [17:58] ie, if you've *acutally* got a cloud, lets take a similar path to what brightbox did [17:59] that's what I think too [17:59] it's more useful for small scale / hobby stuff [17:59] its really good idea. and somethign iv'e wanted to do before. thanks for raising it. [18:00] (unfortunate that smbios is only x86... arm64 has it, but qemu bugs prevent using) [18:00] yep [18:01] it's just that ec2-like md server is still the simplest way to bootstrap [18:03] farcaller, you can also fairly easily modify your smbios to look like ec2 [18:04] not really, I tried that :) [18:04] cloud-init seems to look into sysfs for ec2-like uuid [18:04] well... thats because it was broken [18:04] I couldn't find a way to fake that [18:04] its fixed now i'm pretty sure (in trunk) [18:05] I see. I was experimenting with 16.10 image, I believe [18:05] you ahve to set product serial and uuid to the same value and they ahve to start with ec2 (case insensitive) [18:05] didn't work for me there [18:05] yep, I definitely tried doing that [18:06] commit was at 370a04e8d7b530c1ef8280e15eb628ff6880c736 [18:06] it went in last week, not in an image yet. [18:06] ack [18:07] farcaller, do you know how big that can be ? [18:07] how long the fied [18:07] field [18:08] nope. I didn't check tbh. I verified that it fits seedfrom with ip address, and that's it [18:37] smoser: I'm seeing unit test failures on fedora because DataSourceCloudStack is trying to access /var/lib/NetworkManager, which requires root access. [18:38] selinux=off [18:38] ;) [18:38] smoser: nope. [18:39] drwx------. 2 root root 4096 May 17 10:53 /var/lib/NetworkManager [18:39] Unit tests shouldn't be trying to access anything outside of the current directory in any case. [18:39] Or current directory + any tmpdir for the test. [18:39] agreed. [18:39] (i was just joking about selinux) [18:39] Is there any chance we can get automated ci running on a centos system? [18:39] larsks, there sure is! [18:40] Is there anything I can do to help? [18:40] https://jenkins.ubuntu.com/server/view/Cloud-init/ [18:40] the centos-6 and centos-7 are running now (and passing) [18:40] powersj just got that up a few days ago. [18:41] smoser: I am confused as to why they are passing, given the errors I am seeing locally. [18:41] they dont run on MP but on trunk nightly [18:41] My centos7 box has the same permissions on /var/lib/NetworkManager as the fedora system on which I'm working. [18:41] probably becausae they run in an lxd image that doesn't have networtk manager [18:41] yeah I'm cheating by using lxd, don't have hardware for dedicated centos system yet [18:42] powersj: just installing the NM package into the LXD container should trigger this problem, right? [18:42] But I guess the real solution is to fix the unit tests. [18:42] ii suspect it should trigger it, and yeah, we can fix the unit tests. i'll give it a try here. [18:43] i suspect i can just mkdir -p /var/lib/NetworkManager && chmod 700 [18:43] you need to touch a file in there [18:43] smoser: awesome, thanks. [18:43] if os.path.exists(d) and len(os.listdir(d)) > 0: [18:43] then it'll try to parse it [18:44] smoser: I would vote for just installing the package, rather than trying faking it. Because who knows what other permissions we might be missing? [18:45] hm.. well it wasnt that easy [18:45] sudo chmod go-rX /var/lib/NetworkManager/ [18:45] and [18:45] tox-venv py3 python3 -m nose tests/unittests/test_datasource/test_cloudstack.py [18:45] still works for me. [18:47] Iiiinteresting. I bet that https://github.com/cloud-init/cloud-init/blob/master/cloudinit/sources/DataSourceCloudStack.py#L179 is finding one of the other directories in that list. [18:47] oh. because i have a /var/lib/dhclient probably [18:47] Yeah. [18:47] yeah [18:48] :) [18:50] What if we just mocked out get_dhclient_d? [18:52] larsks, i think just mock get_vr_address [18:53] hi all, I tried to disable autmount of ephemeral storage to /dev/vdb via cloud.cfg but our instance is still creating ephemeral -> /dev/vdb, here is my example https://pastebin.com/8HkNbbEd , any other method I can take to preventing epehemeral storage from auto mounting to vdb? [18:53] smoser: or that. [18:53] but get_client_d would work to, and then you could even populate it [18:53] with a file that shoudl get correctly parsed. [18:54] Right, that was my thought. [19:11] smoser: https://code.launchpad.net/~larsks/cloud-init/+git/cloud-init/+merge/324277 [19:12] that mocks out get_latest_lease (which was simpler than mocking out get_vr_address, since we can just return None) [19:20] that'll do.t hanks. [19:24] hi all, anyway to disable mounting of ephemeral0 -> /dev/vdb? [19:32] raspado: that's not a "mount"; that's just the name of the block device. I don't think it gets mounted anywhere automatically. [19:37] cloud.cfg doesnt manage that? [19:38] No, the linux kernel automatically creates block devices entries for any devices it discovers. [19:38] That's just normal kernel behavior. [19:39] so if ephemeral storage is available, the kernel will auto discover and allocate it to a device id? hmm is that just cloud-aware os's? [19:39] No, that's just normal Linux. IF there is a block device attached, THEN you get an entry in /dev. [19:40] hmm so in my case when i build an instance that has ephemeral storage, it automounts to /dev/vdb, I was hoping I can manipulate that via cloud.cfg somehow [19:40] ah well.. [19:43] http://cloudinit.readthedocs.io/en/latest/topics/examples.html#adjust-mount-points-mounted [19:43] fs_setup: may be able to disable this? === rangerpb is now known as rangerpbzzzz === powersj is now known as powersj_ === powersj_ is now known as powersj [22:25] powersj: rharper smoser ok here's the start of the schema validation tool for ntp with sample failures (poor formatting from by for loop) http://pastebin.ubuntu.com/24601457/ [22:25] from *my* for loop [22:41] * rharper looks [23:20] blackboxsw: super interesting ... so, looking at the valid ones; and I'm not sure we can do much about it; the strings for hosts or ips... [23:21] those could be valid in the schema since we're only specifying the encoded type as string or list or dict;