[15:39] <smoser> harlowja, i can get you one, send an email to me asking for it and i'll forward correctly
[16:32] <apollo13> hi, I am trying the centos 7 cloud images on xenserver and was wondering how/which datasource they try to use
[16:33] <apollo13> since xenserver obviously does not support cloud-init at all I tried sniffing with tcpdump to see if it tries the EC2 method etc… but nothing
[17:09] <smoser> apollo13, it depends on how the image is configured
[17:09] <smoser> i woudl have thoguht they'd try the ec2, but they might only try openstack or config drive.
[17:09] <apollo13> smoser: I mounted the raw file, can I easily check? there is no datasource config in /etc/cloudinit, so it should fall back to the default list?
[17:10] <smoser> probably something like:
[17:10] <smoser> $ lxc exec x1 -- grep -r datasource /etc/cloud/cloud.cfg.d
[17:10] <smoser> /etc/cloud/cloud.cfg.d/90_dpkg.cfg:datasource_list: [ NoCloud, ConfigDrive, OpenNebula, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Ec2, CloudStack, None ]
[17:10] <smoser> if there is no config, it should actually try all of them.
[17:11] <apollo13> mhm, what are the requirements for EC2? I put the VM onto a network with no DHCP in the hopes that it would assign itself a 169.254 ip addr and then try it
[17:12] <smoser> apollo13, no. it wont do that. that is to be done... and that is the ultimate goal.
[17:12] <smoser> but right now, it will fallback to dhcp
[17:12] <apollo13> ah so I need to serve dhcp with those IPs?
[17:13] <apollo13> or rather: is there any easy config mechanism I could use in a network which does not really support cloud-init
[17:15] <smoser> apollo13, http://ubuntu-smoser.blogspot.com/2013/02/using-ubuntu-cloud-images-without-cloud.html
[17:15] <apollo13> or rather could I use a No Cloud datasource to bootstrap somethng else?
[17:15] <smoser> that works for ubuntu images.
[17:15] <smoser> right.
[17:15] <smoser> you can also actually just add the config inside the image if you're willing to do that
[17:16] <apollo13> smoser: I would be willing to do that basically I could reconfigure the ec2 source to use an ip on my network, right?
[17:17] <smoser> you could do that probably yeah.
[17:17] <smoser> also this:
[17:17] <smoser>  doc/examples/cloud-config-datasources.txt
[17:17] <apollo13> assuming I could do this, are there any docs on the ec2 format?
[17:18] <smoser> so... easiest thing for you to do
[17:18] <smoser> is to mount the image and add /etc/cloud/cloud.cfg.d/my-stuff.cfg
[17:18] <smoser> with
[17:18] <smoser> https://git.launchpad.net/cloud-init/tree/doc/examples/cloud-config-datasources.txt
[17:18] <smoser> see lines 28 there... you can put user-data and meta-data right inside the image
[17:18] <smoser> its completely static at that point, but that migth be sufficient for you
[17:18] <smoser> for ec2 metadata
[17:18] <smoser> https://gist.github.com/smoser/1278651/
[17:19] <apollo13> oh I could also set seedfrom for the NoCloud source?
[17:20] <apollo13> then it wouldn't even be static
[17:20] <apollo13> https://git.launchpad.net/cloud-init/tree/cloudinit/sources/DataSourceEc2.py#n42 <-- there we go
[17:20] <apollo13> I need instance-data somewhere :D
[17:21] <apollo13> and there I can put your server, no?
[17:21] <smoser> yeah. you can do that.
[17:21] <smoser> nocloud can also 'seed_from' a url
[17:21] <apollo13> yeah, but then I do not need to remaster the image and can work with everything
[17:25] <apollo13> smoser: ok, I can play with that -- thank you so much
[17:25] <apollo13> I'll report back later, lets see if I can get that to work
[17:26] <roaksoax> 8
[18:31] <apollo13> smoser: that thing is weird, it tries to query 20:30:04.279650 IP 172.22.1.225.50786 > gw01.bap.lan.domain: 54179+ A? does-not-exist.example.com. (44) and what not but not instance-data :D gotta setup a password and see what the log says
[18:33] <harlowja> smoser cool, thx, will send u a offical email
[18:33] <apollo13> oh it also tries metadata.google.internal -- that one looks like I could hook into
[18:33] <apollo13> the google metadata server looks nicer anyways (looking at the cloudinit source)
[18:34] <apollo13> and they actually document it :D
[18:35] <apollo13> ah amazon too
[18:35] <smoser> apollo13, yeah, you can read the code for does-not-exist
[18:36] <smoser> thats trying to work around providers who do dns ...
[18:36] <smoser> whats that called.
[18:36] <apollo13> jupp, though I wonder why it checks everything but not ec2, where is the default log?
[18:36] <smoser> where they give you their web server
[18:36] <apollo13> dns redirects probably
[18:36] <smoser> so http://does-not-exist.example.com takes you to your service provider
[18:36] <apollo13> yeah
[18:45] <apollo13> smoser: I feel stupid now, the image seems to be buggy :( unexpected error nonetype object has no attribute status_code for the ec2 source :D
[18:45] <apollo13> lets see where that could happen
[19:05] <apollo13> does cloud-init also run after starting the machine? it seems that after the machine came up with fallback config it retries to request the config
[19:20] <rharper> it's part of init; so it runs every boot (but not everything is re-run each boot, there are different frequencies);  I'm not sure; but if it didn't find a datasource then on next boot, it will attempt to find it again
[19:21] <rharper> if it does find a data source, it will cache that along with the instance id, and as long as that data is present, it won't attempt to acquire datasource again.
[19:24] <apollo13> rharper: okay, and on the absolute first boot, when I get a login mask -- does cloud-init already have finished at that point?
[19:24] <rharper> yes
[19:25] <apollo13> so nothing run in parallel or so… weird, cause I am getting requests for the ec2 data quite late
[19:25] <rharper> cloud-init runs through roughly 4 stages;  init --local (looks for a local data source, like config drive, etc);  init-network (bring up networking and look for datasources on the network);  at this point if it doesn't find a data source, it goes to fallback, then modules mode=config (this runs the configuration modules); and then a modules mode=final  which runs any configuration and final boot scripts before exiting a
[19:25] <rharper> nd letting it finish booting
[19:26] <rharper> no, cloud-init is quite serial by design
[19:26] <rharper> if you put ssh keys in your user-data, they need to be imported and available before networking comes up and sshd runs, etc...
[19:26] <apollo13> okay, I just started a new vm -- lets see, I'll give it five minutes but I think after booting there is another service running requesting metadata
[19:26] <apollo13> or still cloud-init periodically requesting data till it gets something
[19:27] <rharper> there's no service running; just cloud-init in 4 distinct phases
[19:27] <rharper> no
[19:27] <rharper> no background service
[19:27] <apollo13> okay, maybe I indeed rebooted the machine, I'll give it a few minutes
[19:27] <apollo13> 21:27:36.503771 IP 172.22.1.221.60060 > app01.bap.lan.8773: Flags [S], seq 541007663, win 29200, options [mss 1460,sackOK,TS val 4294777761 ecr 0,nop,wscale 5], length 0
[19:27] <apollo13> ha
[19:27] <apollo13> that is a minute after the login prompt is there
[19:28] <apollo13> that is a request to instance-data.:8773
[19:28] <apollo13> and yet another minute later another try, so something is clearly still running
[19:29] <apollo13> any ideas where from that would be coming :D
[19:29] <rharper> I don't know your image
[19:29] <apollo13> the centos generic cloud image
[19:29] <rharper> but cloud-init  doenst' background any service
[19:30] <apollo13> mhm, kinda chicken-egg like :D
[19:31] <apollo13> I cannot get into the machine till cloud-init is through, but that fails currently :D
[19:31] <apollo13> maybe I really modify the image to set a password for testing
[19:31] <rharper> can you get at the image offline?
[19:31] <rharper> mount it up and extract the /var/log/cloud-init.log; that'd be very useful
[19:31] <rharper> cloud-init also dumps processing to console-log, so having serial console output is informative too
[19:32] <apollo13> yeah, let me see if I can kill it without shutting down
[19:33] <rharper> I'll be back in a bit, so please continue with questions and I'll reply when I can
[19:35] <apollo13> rharper: ok, can I shutdown the vm or would really killing be better?
[19:35] <apollo13> oh wait, I just snapshot it, shut down and reset
[19:41] <apollo13> rharper: http://apolloner.eu/~apollo13/.tmp/cloud-init.log
[19:42] <apollo13> that 19:27 where it calls instance-data. is when I said "<apollo13> 21:27:36.503771 IP 172.22.1.221.60060 > app01.bap.lan.8773:" -- at this point I already had a login there
[19:45] <apollo13> though the messages log seems to indicate that multi-user target is reached later, which makes sense, lets try a new vm :D
[19:46] <apollo13> I tried logging in now, will post auth.log and messages soon
[19:48] <apollo13> oh wait, I maybe should have mentioned that I am/was trying to login via the console from xenserver which probably shows the single user mode :D
[19:58] <apollo13> jupp that was it, sorry for beeing so stupid
[19:58] <apollo13> or not, I learned a lot about cloud-init and audit logs :D
[19:59] <apollo13> I'll call it a win
[20:00] <rharper> apollo13: looking at the logs, cloud-init finished here: Cloud-init v. 0.7.5 finished at Thu, 22 Sep 2016 19:30:58 +0000. Datasource DataSourceNone.  Up 312.13 seconds;  after this anything that cloud-init blocked during start-up will continue; including reaching multi-user target later;
[20:00] <apollo13> rharper: yes and I tried logging in __before__ that on tty1 :(
[20:00] <apollo13> I just thought: hey vm is up already, wth didn't cloud-init do something
[20:00] <apollo13> little bit embarrasing ^^
[20:01] <rharper> no worries; it's rather complicated
[20:01] <rharper> at least the interplay between all of the systemd units and services
[20:01] <apollo13> so, the next question is on whether to impelment a google metadata service or ec2, any suggestions?
[20:02] <rharper> can you attach disks ?
[20:02] <apollo13> are there any nice docs somewhere on which keys my api should return etc?
[20:02] <apollo13> jupp I can
[20:02] <rharper> then I'd use a config drive source
[20:02] <apollo13> ah no, to lazy
[20:02] <apollo13> and I need a sideproject :D
[20:02] <rharper> attaching a blob of yaml formatted as an iso seems a lot easier than implementing a metadata service
[20:03] <apollo13> but boring
[20:03] <rharper> surely
[20:04] <apollo13> google seems to be better documented and from the looks of it it doesn't need more than a few fields
[20:04] <roaksoax> /w/win 8
[20:05] <apollo13> ?
[20:05] <apollo13> https://dpaste.de/5YWj/raw <-- pretty much al that is needed, then throw that behind a nice django iface and I am done
[20:06] <apollo13> that + xenapi and colleagues in the office don't have to get on my nerves for a new vm, sounds like a win win
[20:10] <apollo13> mhm, last but not least, can I somehow configure static networking via cloud-init?
[20:11] <rharper> yes
[20:11] <rharper> but for centos you'll need 0.7.7
[20:11] <apollo13> mhm, any docs :D
[20:12] <rharper> yes, one sec
[20:12] <apollo13> (I apparently searched with the wrong terms)
[20:12] <harlowja> ok smoser sent a formal request to yourubuntu email
[20:12] <smoser> harlowja, k
[20:13] <rharper> apollo13: doesn't look like it's quite made it into the 0.7.7 docs; but the format is yaml and looks like this: http://curtin.readthedocs.io/en/latest/topics/networking.html
[20:14] <smoser> its tricky though... that has to exist inside the image (which kind of defeats its purpose)
[20:14] <smoser> or be read from a static datasource.
[20:14] <smoser> or, the kernel command line.
[20:14] <rharper> no, network_data.json
[20:14] <rharper> from metadata service could work
[20:14] <rharper> but that looks different
[20:14] <apollo13> :D
[20:14] <smoser> no. doesnt work.
[20:14] <smoser> rharper, because networking is only applied by local datasources.
[20:15] <rharper> bleh
[20:15] <apollo13> which kinda makes sense, but reconfig would be nice
[20:15] <rharper> apollo13: cloud-init has to work with all sorts of cloudes
[20:15] <rharper> clouds
[20:15] <rharper> reconfig is on the roadmap , way down at the end
[20:15] <rharper> but, that's the idea
[20:16] <rharper> user modifies the instance (hotplug), cloud-init could ask the cloud for new metadata and update config
[20:16] <rharper> https://specs.openstack.org/openstack/nova-specs/specs/liberty/implemented/metadata-service-network-info.html
[20:16] <rharper> that's the network_data.json format, cloud-init will read that from a ConfigDrive; but as smoser said, it has to be attached (i.e. a local datasource)
[20:16] <apollo13> crazy crazy :D
[20:17] <apollo13> I guess I could just pull in networkd files and restart via that…
[20:17] <apollo13> ah damn, not in centos7 yet, well dhcp it is then
[20:17] <apollo13> oh, it is there, they rolled in 219
[20:20] <smoser> the way to do this... getting network configuration data from a network datasource is as it is being proposed at
[20:20] <smoser>  https://code.launchpad.net/~utlemming/cloud-init/+git/cloud-init/+merge/303471
[20:21] <harlowja> apollo13 perhaps nrezinorn can share a cent7 package with u at some point
[20:21] <smoser> apollo13 could add a datasource that brings up ipv4 link local networking, finds information about its config, and then reads data and metadata.
[20:21] <harlowja> he's been rolling through the cent7 changes and adjustments recently
[20:21] <harlowja> rolling through/into/over
[20:21] <harlowja> lol
[20:23] <apollo13> that sounds nice, but for now I can write a small service, that is enough for now
[20:25] <harlowja> k
[20:26] <rharper> smoser: that merge from utlemming looks neat
[20:27] <smoser> rharper, yeah. that is what we want to get to.
[20:27] <smoser> and can do that on any cloud that clearly identifies itself.
[20:27] <rharper> yep, see the DMI bits
[20:27] <smoser> right.
[20:28] <apollo13> dmi? that will be harder to fake for me I guess :D
[20:28] <rharper> apollo13: exposing the cloud provider via SMBIOS data (DMI table on x86)
[20:29] <rharper> linux does a dmidecode (flags to extract a specific field, like BIOS vendor)
[20:29] <apollo13> yeah, but /me is no cloud, just hijacking requests to metadata servers^^
[20:29] <rharper> with that, cloud-init can assume it's running on a particular cloud and can use that cloud's datasource code
[20:29] <rharper> if you launch VMs, you're a cloud =)
[20:29] <apollo13> I am a rocket first and foremost
[20:29] <apollo13> or spaceship if you are kind^^
[20:30] <rharper> xen also supports supplying/injecting SMBios data  (at least HVM)
[20:30] <rharper> hehe
[20:30]  * rharper notes nick of apollo13 checks out 
[20:30]  * rharper relocates, bbiab 
[23:54] <smemsh> hi, is static network config supposed to work on ubuntu 16.04 ? this has worked fine for me with 14.04 and centos, but 16.04 it seems to just use dhcp and my static config from meta-data is nowhere to be found
[23:54] <smemsh> the interface name is ens2, which i changed in meta-data, but that still does not fix it
[23:54] <smemsh> (originally i was using eth0)