=== smatzek_ is now known as smatzek [15:39] harlowja, i can get you one, send an email to me asking for it and i'll forward correctly [16:32] hi, I am trying the centos 7 cloud images on xenserver and was wondering how/which datasource they try to use [16:33] since xenserver obviously does not support cloud-init at all I tried sniffing with tcpdump to see if it tries the EC2 method etc… but nothing [17:09] apollo13, it depends on how the image is configured [17:09] i woudl have thoguht they'd try the ec2, but they might only try openstack or config drive. [17:09] smoser: I mounted the raw file, can I easily check? there is no datasource config in /etc/cloudinit, so it should fall back to the default list? [17:10] probably something like: [17:10] $ lxc exec x1 -- grep -r datasource /etc/cloud/cloud.cfg.d [17:10] /etc/cloud/cloud.cfg.d/90_dpkg.cfg:datasource_list: [ NoCloud, ConfigDrive, OpenNebula, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Ec2, CloudStack, None ] [17:10] if there is no config, it should actually try all of them. [17:11] mhm, what are the requirements for EC2? I put the VM onto a network with no DHCP in the hopes that it would assign itself a 169.254 ip addr and then try it [17:12] apollo13, no. it wont do that. that is to be done... and that is the ultimate goal. [17:12] but right now, it will fallback to dhcp [17:12] ah so I need to serve dhcp with those IPs? [17:13] or rather: is there any easy config mechanism I could use in a network which does not really support cloud-init [17:15] apollo13, http://ubuntu-smoser.blogspot.com/2013/02/using-ubuntu-cloud-images-without-cloud.html [17:15] or rather could I use a No Cloud datasource to bootstrap somethng else? [17:15] that works for ubuntu images. [17:15] right. [17:15] you can also actually just add the config inside the image if you're willing to do that [17:16] smoser: I would be willing to do that basically I could reconfigure the ec2 source to use an ip on my network, right? [17:17] you could do that probably yeah. [17:17] also this: [17:17] doc/examples/cloud-config-datasources.txt [17:17] assuming I could do this, are there any docs on the ec2 format? [17:18] so... easiest thing for you to do [17:18] is to mount the image and add /etc/cloud/cloud.cfg.d/my-stuff.cfg [17:18] with [17:18] https://git.launchpad.net/cloud-init/tree/doc/examples/cloud-config-datasources.txt [17:18] see lines 28 there... you can put user-data and meta-data right inside the image [17:18] its completely static at that point, but that migth be sufficient for you [17:18] for ec2 metadata [17:18] https://gist.github.com/smoser/1278651/ [17:19] oh I could also set seedfrom for the NoCloud source? [17:20] then it wouldn't even be static [17:20] https://git.launchpad.net/cloud-init/tree/cloudinit/sources/DataSourceEc2.py#n42 <-- there we go [17:20] I need instance-data somewhere :D [17:21] and there I can put your server, no? [17:21] yeah. you can do that. [17:21] nocloud can also 'seed_from' a url [17:21] yeah, but then I do not need to remaster the image and can work with everything [17:25] smoser: ok, I can play with that -- thank you so much [17:25] I'll report back later, lets see if I can get that to work [17:26] 8 [18:31] smoser: that thing is weird, it tries to query 20:30:04.279650 IP 172.22.1.225.50786 > gw01.bap.lan.domain: 54179+ A? does-not-exist.example.com. (44) and what not but not instance-data :D gotta setup a password and see what the log says [18:33] smoser cool, thx, will send u a offical email [18:33] oh it also tries metadata.google.internal -- that one looks like I could hook into [18:33] the google metadata server looks nicer anyways (looking at the cloudinit source) [18:34] and they actually document it :D [18:35] ah amazon too [18:35] apollo13, yeah, you can read the code for does-not-exist [18:36] thats trying to work around providers who do dns ... [18:36] whats that called. [18:36] jupp, though I wonder why it checks everything but not ec2, where is the default log? [18:36] where they give you their web server [18:36] dns redirects probably [18:36] so http://does-not-exist.example.com takes you to your service provider [18:36] yeah [18:45] smoser: I feel stupid now, the image seems to be buggy :( unexpected error nonetype object has no attribute status_code for the ec2 source :D [18:45] lets see where that could happen [19:05] does cloud-init also run after starting the machine? it seems that after the machine came up with fallback config it retries to request the config [19:20] it's part of init; so it runs every boot (but not everything is re-run each boot, there are different frequencies); I'm not sure; but if it didn't find a datasource then on next boot, it will attempt to find it again [19:21] if it does find a data source, it will cache that along with the instance id, and as long as that data is present, it won't attempt to acquire datasource again. [19:24] rharper: okay, and on the absolute first boot, when I get a login mask -- does cloud-init already have finished at that point? [19:24] yes [19:25] so nothing run in parallel or so… weird, cause I am getting requests for the ec2 data quite late [19:25] cloud-init runs through roughly 4 stages; init --local (looks for a local data source, like config drive, etc); init-network (bring up networking and look for datasources on the network); at this point if it doesn't find a data source, it goes to fallback, then modules mode=config (this runs the configuration modules); and then a modules mode=final which runs any configuration and final boot scripts before exiting a [19:25] nd letting it finish booting [19:26] no, cloud-init is quite serial by design [19:26] if you put ssh keys in your user-data, they need to be imported and available before networking comes up and sshd runs, etc... [19:26] okay, I just started a new vm -- lets see, I'll give it five minutes but I think after booting there is another service running requesting metadata [19:26] or still cloud-init periodically requesting data till it gets something [19:27] there's no service running; just cloud-init in 4 distinct phases [19:27] no [19:27] no background service [19:27] okay, maybe I indeed rebooted the machine, I'll give it a few minutes [19:27] 21:27:36.503771 IP 172.22.1.221.60060 > app01.bap.lan.8773: Flags [S], seq 541007663, win 29200, options [mss 1460,sackOK,TS val 4294777761 ecr 0,nop,wscale 5], length 0 [19:27] ha [19:27] that is a minute after the login prompt is there [19:28] that is a request to instance-data.:8773 [19:28] and yet another minute later another try, so something is clearly still running [19:29] any ideas where from that would be coming :D [19:29] I don't know your image [19:29] the centos generic cloud image [19:29] but cloud-init doenst' background any service [19:30] mhm, kinda chicken-egg like :D [19:31] I cannot get into the machine till cloud-init is through, but that fails currently :D [19:31] maybe I really modify the image to set a password for testing [19:31] can you get at the image offline? [19:31] mount it up and extract the /var/log/cloud-init.log; that'd be very useful [19:31] cloud-init also dumps processing to console-log, so having serial console output is informative too [19:32] yeah, let me see if I can kill it without shutting down [19:33] I'll be back in a bit, so please continue with questions and I'll reply when I can === rtheis_ is now known as rtheis [19:35] rharper: ok, can I shutdown the vm or would really killing be better? [19:35] oh wait, I just snapshot it, shut down and reset [19:41] rharper: http://apolloner.eu/~apollo13/.tmp/cloud-init.log [19:42] that 19:27 where it calls instance-data. is when I said " 21:27:36.503771 IP 172.22.1.221.60060 > app01.bap.lan.8773:" -- at this point I already had a login there [19:45] though the messages log seems to indicate that multi-user target is reached later, which makes sense, lets try a new vm :D [19:46] I tried logging in now, will post auth.log and messages soon [19:48] oh wait, I maybe should have mentioned that I am/was trying to login via the console from xenserver which probably shows the single user mode :D [19:58] jupp that was it, sorry for beeing so stupid [19:58] or not, I learned a lot about cloud-init and audit logs :D [19:59] I'll call it a win [20:00] apollo13: looking at the logs, cloud-init finished here: Cloud-init v. 0.7.5 finished at Thu, 22 Sep 2016 19:30:58 +0000. Datasource DataSourceNone. Up 312.13 seconds; after this anything that cloud-init blocked during start-up will continue; including reaching multi-user target later; [20:00] rharper: yes and I tried logging in __before__ that on tty1 :( [20:00] I just thought: hey vm is up already, wth didn't cloud-init do something [20:00] little bit embarrasing ^^ [20:01] no worries; it's rather complicated [20:01] at least the interplay between all of the systemd units and services [20:01] so, the next question is on whether to impelment a google metadata service or ec2, any suggestions? [20:02] can you attach disks ? [20:02] are there any nice docs somewhere on which keys my api should return etc? [20:02] jupp I can [20:02] then I'd use a config drive source [20:02] ah no, to lazy [20:02] and I need a sideproject :D [20:02] attaching a blob of yaml formatted as an iso seems a lot easier than implementing a metadata service [20:03] but boring [20:03] surely [20:04] google seems to be better documented and from the looks of it it doesn't need more than a few fields [20:04] /w/win 8 [20:05] ? [20:05] https://dpaste.de/5YWj/raw <-- pretty much al that is needed, then throw that behind a nice django iface and I am done [20:06] that + xenapi and colleagues in the office don't have to get on my nerves for a new vm, sounds like a win win [20:10] mhm, last but not least, can I somehow configure static networking via cloud-init? [20:11] yes [20:11] but for centos you'll need 0.7.7 [20:11] mhm, any docs :D [20:12] yes, one sec [20:12] (I apparently searched with the wrong terms) [20:12] ok smoser sent a formal request to yourubuntu email [20:12] harlowja, k [20:13] apollo13: doesn't look like it's quite made it into the 0.7.7 docs; but the format is yaml and looks like this: http://curtin.readthedocs.io/en/latest/topics/networking.html [20:14] its tricky though... that has to exist inside the image (which kind of defeats its purpose) [20:14] or be read from a static datasource. [20:14] or, the kernel command line. [20:14] no, network_data.json [20:14] from metadata service could work [20:14] but that looks different [20:14] :D [20:14] no. doesnt work. [20:14] rharper, because networking is only applied by local datasources. [20:15] bleh [20:15] which kinda makes sense, but reconfig would be nice [20:15] apollo13: cloud-init has to work with all sorts of cloudes [20:15] clouds [20:15] reconfig is on the roadmap , way down at the end [20:15] but, that's the idea [20:16] user modifies the instance (hotplug), cloud-init could ask the cloud for new metadata and update config [20:16] https://specs.openstack.org/openstack/nova-specs/specs/liberty/implemented/metadata-service-network-info.html [20:16] that's the network_data.json format, cloud-init will read that from a ConfigDrive; but as smoser said, it has to be attached (i.e. a local datasource) [20:16] crazy crazy :D [20:17] I guess I could just pull in networkd files and restart via that… [20:17] ah damn, not in centos7 yet, well dhcp it is then [20:17] oh, it is there, they rolled in 219 [20:20] the way to do this... getting network configuration data from a network datasource is as it is being proposed at [20:20] https://code.launchpad.net/~utlemming/cloud-init/+git/cloud-init/+merge/303471 [20:21] apollo13 perhaps nrezinorn can share a cent7 package with u at some point [20:21] apollo13 could add a datasource that brings up ipv4 link local networking, finds information about its config, and then reads data and metadata. [20:21] he's been rolling through the cent7 changes and adjustments recently [20:21] rolling through/into/over [20:21] lol [20:23] that sounds nice, but for now I can write a small service, that is enough for now [20:25] k [20:26] smoser: that merge from utlemming looks neat [20:27] rharper, yeah. that is what we want to get to. [20:27] and can do that on any cloud that clearly identifies itself. [20:27] yep, see the DMI bits [20:27] right. [20:28] dmi? that will be harder to fake for me I guess :D [20:28] apollo13: exposing the cloud provider via SMBIOS data (DMI table on x86) [20:29] linux does a dmidecode (flags to extract a specific field, like BIOS vendor) [20:29] yeah, but /me is no cloud, just hijacking requests to metadata servers^^ [20:29] with that, cloud-init can assume it's running on a particular cloud and can use that cloud's datasource code [20:29] if you launch VMs, you're a cloud =) [20:29] I am a rocket first and foremost [20:29] or spaceship if you are kind^^ [20:30] xen also supports supplying/injecting SMBios data (at least HVM) [20:30] hehe [20:30] * rharper notes nick of apollo13 checks out [20:30] * rharper relocates, bbiab [23:54] hi, is static network config supposed to work on ubuntu 16.04 ? this has worked fine for me with 14.04 and centos, but 16.04 it seems to just use dhcp and my static config from meta-data is nowhere to be found [23:54] the interface name is ens2, which i changed in meta-data, but that still does not fix it [23:54] (originally i was using eth0)