/srv/irclogs.ubuntu.com/2020/04/30/#cloud-init.txt

=== tds0 is now known as tds
amansi26Hi, I have a doubt, the network-details when we pass through openstack, how is that get handled by cloud init?11:14
amansi26Like where is that information get processed. I am using v19.1. Not able to figure out the exact place where it get processed11:15
sackrebutzhey again - i have yum_repos enabled but it seems it’s silently ignored by cloud-init when run on the server. Config: https://dpaste.org/xCEg // Log: https://dpaste.org/MafQ12:05
sackrebutzAlso no file is created within /etc/yum.repos.d/12:06
felfert_sackrebutz: Have a look in /var/lib/cloud/instance/12:15
felfert_In there are several files which represent your user-data and how it is recognized by cloud-init12:16
felfert_Your config looks goot to me but there might be yaml errors somewhere else and if there are errors, then /var/lib/cloud/instance/user-data.txt.i and /var/lib/cloud/instance/user-data.txt show the difference of the raw data and how cloud-init has interpreted it12:18
sackrebutzfelfert_: Thanks - i can see that in both files, there is the yum_repos config and it’s just as i have put it into the config12:20
sackrebutzis there any other log that would tell me what it’s doing ?12:23
sackrebutzat some point, cloud-init has do make a decision whether it takes the yum_repos config into account or not i guess?12:24
felfert_It does not specifically mention that in the logs12:26
sackrebutzI now copy-pasted the example from https://cloudinit.readthedocs.io/en/latest/topics/examples.html?highlight=yum#adding-a-yum-repository and it’s not workiing either - might be a bug ?12:36
felfert_Well, I use a much older cloud-init (that comes preinstalled with CentOS-7 stock cloud images and *that* one works)12:37
felfert_Is yours a stock CentOS-8 cloud image? Or did you install cloud-init yourself?12:39
sackrebutzit’s the centos8 stock12:40
sackrebutzv18.512:41
felfert_The only other difference: My yaml files use 2-space indentation, yours use 4-spaces. Oh perhaps check, if you have indented with tabs accidentially.12:44
felfert_Do you use multiple yaml files as userdata or just a single yaml?12:48
sackrebutzI already converted to 2-spaces for the sake of testing, but dodn’t make a difference12:48
sackrebutzonly 1 yml file12:48
felfert_Then to me this looks like a bug.12:50
rharpersackrebutz: can you check if  'yum-add-repo' is present in your /etc/cloud/cloud.cfg  ; do you have the complete cloud-init.log from your test ?12:56
sackrebutzrharper: I stumbled over the same on https://bugzilla.redhat.com/show_bug.cgi?id=1027406 - testing it right now12:57
ubot5bugzilla.redhat.com bug 1027406 in cloud-init "cloud-init should probably have yum_add_repo module enabled" [High,Closed: currentrelease]12:57
rharpersackrebutz: ah, ok12:59
sackrebutzrharper: it is presend below ‘cloud_config_modules’ is that corret ?12:59
sackrebutzpresent*12:59
rharperyes12:59
sackrebutzhm12:59
felfert_But 1027406 is ancient (and fixed in 0.7.5-1.el6)13:00
rharperok, so next, let's look if it ran in cloud-init.log13:00
sackrebutzyep, one sec13:00
sackrebutzhttps://dpaste.org/WctE13:03
rharperSkipping modules 'yum-add-repo' because they are not verified on distro 'centos'.  To run anyway, add them to 'unverified_modules' in config.13:03
rharpernasty13:03
rharperdistros = ['fedora', 'rhel']13:03
sackrebutzahhh13:03
rharpersackrebutz: so, you can add to your config,  unverified_modules: [yum-add-repos]   ;  let me confirm that13:04
sackrebutzyum-add-repo it would be, no?13:05
rharperyeah, a list with the module name13:05
rharperyes13:05
rharperI suspect that's worth an upstream patch13:06
sackrebutzyes! now it’s adding it correctly13:07
rharper\o/13:07
sackrebutzi was already doubting on my whitespace indentation skills even after 20years of python13:08
rharper=(13:08
sackrebutzonce again, thank you very much rharper  felfert_13:08
rharperI generally use something like python3 -c 'import sys, yaml; print(yaml.dump(yaml.load(open(sys.argv[1]))))' my.cfg13:09
rharpersackrebutz: sure13:10
rharperhttps://github.com/canonical/cloud-init/pull/34013:14
potofteaHi I'm trying to run single module "runcmd" with command: "cloud-init --file /root/cloud-init.yaml single --name runcmd --frequency=always". From logs I see that it commands are being written (Shellified 7 commands) but not execute.  Could this be a bug?13:16
rharperpotoftea: it's a two step process, runcmd writes it's output to a file and later in final, runparts is called on it, let me get the final handler name13:17
rharpercloud-init single --name scripts_user --frequency=always13:19
rharperpotoftea: ^13:20
potofteaohh ok, thank you for quick response and yeah it helped.  rharper Thank you13:20
rharpersure13:21
amansi26rharper: Hi, I have a doubt, the network-details when we pass through openstack, how is that get handled by cloud init?  Like where is that information get processed. I am using v19.1. Not able to figure out the exact place where it get processed13:48
rharperamansi26: in cloudinit/sources/DataSourceOpenStack.py the  'network_config' property  uses cloudinit/sources/helpers/openstack.py:convert_net_json() which converts network_data.json from openstack into network-config v1 format13:50
amansi26rharper: but I am using datasource as configDrive13:53
rharperConfigDrive has it's own network-config property, it has two network config formats it reads13:54
amansi26you mean version1 and version2. Right?13:54
rharperone is network-config which is a debian etc/network/interfaces file format, we convert that into v1 config, via  cloudinit.net.eni.convert_eni_data()  if network_data.json is present in the config drive, then we use the same openstack.convert_net_json13:54
rharperamansi26: no13:54
amansi26rharper: The issue I am facing is when I try deploying dhcp network. it shows type as static in netcfg (stages.py) http://paste.openstack.org/show/792946/13:59
rharperwhat config files are inside your config drive ?13:59
amansi26didn't get you rharper14:01
rharperamansi26: inside the configdrive, which is an iso attacted to your instance, it contains files that cloud-init reads, in particular, there may be a file 'network_config' or 'networkdata'  ;  they wil define what your network config will be14:02
rharperamansi26:  like so  https://paste.ubuntu.com/p/BGJ7fbM37p/14:06
amansi26So if I need to change something in the netcfg details it has to be done from openstack commands (like we need to pass a new parameter)14:07
rharperamansi26: typically yes, your tenent networks define this,  is it a dhcp subnet, or a static ip allocation, etc14:08
amansi26rharper: Thanks, cleared a lot of underline concepts14:09
rharperamansi26: great14:09
Odd_Blokeblackboxsw: Do we have any automated testing that validates our schemas?14:19
Odd_Bloke(e.g. ensures that the examples are valid under the schema.)14:19
Odd_Blokeblackboxsw: https://github.com/canonical/cloud-init/pull/335 <-- ready for your review, I believe14:29
mutantturkeyguten morgan16:31
blackboxswgood mornin :)16:35
blackboxswOdd_Bloke: approved yet needs rebase https://github.com/canonical/cloud-init/pull/34116:36
blackboxswOdd_Bloke: unittests CI runs perform schema validation against all our existing rtd examples to make sure they adhere to the schema we have just defined.16:37
blackboxswOdd_Bloke: I think it would be nice to extend that unittest run to also validate each of the schema['examples'] defined in the config modules too16:37
blackboxswto validate our docs16:37
blackboxswand the schema we claim will work16:38
=== felfert_ is now known as felfert
Odd_BlokeIt sounds like you're drawing a distinction between "our existing rtd examples" and the schema examples: don't the schema examples end up in our generated docs?16:57
Odd_Bloke(Just making sure I understand. :)16:57
bpoI wonder if someone here might point me in the right direction to analyze my cloud-init setup. I tried to get started with `cloud-init analyze blame` but that was empty - and `cloud-init analyze dump` returns `[]`. Any hints on how I should proceed? `systemd-analyze blame` shows `cloud-config.service` taking 8.5s, and `cloud-init.service` taking17:04
bpo1.092s. This system is running Amazon Linux 2.17:04
Odd_Blokebpo: Is /var/log/cloud-init.log readable by the user you're running these commands as?17:15
Odd_Bloke(And what version of cloud-init?)17:15
Odd_Bloke(I'm about to be AFK for a while, so either someone else will answer or you'll have to be patient. ;)17:16
bpo@Odd_Bloke Yes, /var/log/cloud-init.log is readable (running with sudo). `/usr/bin/cloud-init 19.3-2.amzn2`17:16
bpoNo rush17:16
bpo(and the log has entries)17:17
blackboxswOdd_Bloke: right, I was trying to draw the distinction. But I was misremembering our unittests actually only validate schema against our integration tests examples in tests/cloud_tests/testcases https://github.com/canonical/cloud-init/blob/master/tests/unittests/test_handler/test_schema.py#L400-L401217:19
blackboxswBut I think it would be helpful if we actually extended unit tests to validate both the rtd examples in doc/examples as well as each cc_*.py module's schema["examples"] as well17:21
blackboxswas I believe we have found a couple doc errors in the past17:21
blackboxsw... or been informed of17:21
blackboxswit'd help vet our new schema changes as well because example cloud-config  with top-level keys that are not covered by existing schema, don't error (as we aren't strict at that level).17:22
blackboxswlet's try proper grammar17:23
blackboxswHaving unittests attempt shema validation on doc/examples cloud-config as well as any config module schema["examples"] will help validate schema in CI as we add it to each config module.17:24
blackboxswPerforming schema validation against cloud-config from doc/examples that contains config keys not yet in schema will not warn about unknown top-level keys.17:25
LongLiveCHIEFis there a way to control the order of module execution during the final 2 boot stages?17:38
blackboxswparide: for tomorrow I think we can re-enable unittests on centos in your branch https://github.com/canonical/cloud-init/pull/231  ; a git rebase with those changes and it can land17:44
blackboxswLongLiveCHIEF: if you created your own image you can edit the order of modules named in /etc/cloud/cloud.cfg17:44
LongLiveCHIEFthanks!17:45
blackboxswLongLiveCHIEF: or more generally add an additional config in/etc/cloud/cloud.cfg.d with the full cloud_config_modules:  in whatever order you wanted and it would override the defaults in /etc/cloud/cloud.cfg17:46
LongLiveCHIEFwhat if I put them in a specific order in user-data of the datasource?17:47
LongLiveCHIEFi'm actually doing this with NoCloud dsmode local17:49
blackboxswLongLiveCHIEF: something like this https://paste.ubuntu.com/p/77pqCKjrBB/17:49
blackboxswso providing user-data that re-orders modules is trickier17:50
LongLiveCHIEFmakes sense17:50
blackboxswhrm, you could use #cloud-config's write_files  to emit that additional config file to /etc/cloud/cloud.cfg.d17:51
blackboxswhttps://cloudinit.readthedocs.io/en/latest/topics/modules.html#write-files17:51
LongLiveCHIEFi'm working on raspberry pi booting with ubuntu 20. Got it working pretty well, but only if i disable and mask systemd-networkd-wait-online, and then run netplan apply17:52
blackboxswwrite-files is run the init phase which would write that config out before you got to the later stages17:52
LongLiveCHIEFwhich means I can't use user-data modules for easy things and instead have to put them in a script called by runcmd module17:52
blackboxswnice on rasppi work. you mentioning running netplan apply cloud-init should be doing that for you pre-network bringup if you provide network configuration to it17:54
* blackboxsw looks up nocloud again to see the mechanism17:54
LongLiveCHIEFyeah, that's the trick i use for writing out and then running netplan17:55
LongLiveCHIEFi do, but it never resolves. I have a power_state module usage at the end that reboots, and the second boot is when the network finally resolves17:55
LongLiveCHIEF(wifis obviously... ethernet works fine)17:55
LongLiveCHIEFbut if I do it the workaround way, i can do it purely with wifi and it works prior to cloud-init finishing17:56
LongLiveCHIEFso I'm hoping to plug back in for other modules in the final stage17:56
blackboxswLongLiveCHIEF: hrm, so your seed directory or seed url contains a network-config file in it?17:57
LongLiveCHIEFs=/boot/firmware/cloud-init17:57
LongLiveCHIEFi'm creating the cloud-init directory on the boot partition, which gets mounted to /boot/firmware during initial boot17:58
LongLiveCHIEFand that /cloud-init directory has just user-data and meta-data17:58
blackboxsw+1 on that and /boot/firmwware/cloud-init/ contains user-data and meta-data and maybe a network-config?17:58
blackboxswok I *think* you cold provide your netplan configuration in the network-config file as network config v2 to cloud-init to represent your entire network config on the rasp https://cloudinit.readthedocs.io/en/latest/topics/network-config-format-v2.html#network-config-v217:59
LongLiveCHIEF/boot/firmware contains network-config... which winds up merging into 50-cloud-init.conf17:59
blackboxswrharper: or Odd_Bloke could correct me if I'm wrong there17:59
blackboxswLongLiveCHIEF: right I believe so17:59
LongLiveCHIEFyes, confirmed, i CAN18:00
LongLiveCHIEFBUT18:00
blackboxswwhen network config v2 is passed to cloud-init, it should be passed directly though to /etc/netplan/50-cloud-init.yaml18:00
LongLiveCHIEFif I do, then network won't fully resolve until after reboot, which means all the package modules in the final cloud-init stage will fail18:00
blackboxswhrm, that part lost me. why wouldn't it resolve if you define dhcp on eth0 and whatever wireless config you have?18:01
LongLiveCHIEFbut if I run netplan apply using the runcmd module during the final stage, the  network connection gets established18:02
LongLiveCHIEFno wait, i don't have eth018:02
LongLiveCHIEFit's the whole wifis with cloud-init thing18:03
blackboxswI was wondering about something like this https://paste.ubuntu.com/p/YrmftwVD9G/18:03
blackboxswbut I think I'm missing your problem18:03
LongLiveCHIEF<blackboxsw "hrm, that part lost me. why woul"> @blackbox that's the part I can't figure out either.  I admit, i haven't spent enough time digging through the logs yet though.18:03
LongLiveCHIEFi'm doing something like that, only i've removed eth0 entirely18:03
LongLiveCHIEFif i leave it in, then cloud init hangs forever18:04
LongLiveCHIEFif i take it out, then it finishes, but doesn't establish network connection until reboot18:04
LongLiveCHIEFwhich means I can't use modules that run in the final stage that require connection18:04
LongLiveCHIEFand the funny thing is, canonical has these instructions verbatim in their new 20.04 docs for IoT, despite literally everyone you see on youtube stating wifi networking will fail if you do this: https://ubuntu.com/tutorials/how-to-install-ubuntu-on-your-raspberry-pi#3-wifi-or-ethernet18:05
blackboxswLongLiveCHIEF: this reminds me of this bug https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/187034618:07
ubot5Ubuntu bug 1870346 in cloud-init (Ubuntu) "Wifi configuration" [Medium,Triaged]18:07
blackboxswis this also what you are experiencing?18:08
blackboxswI think it could be.... or it'll be what you hit next if you provide your network-config file.18:09
* blackboxsw has to jump away for a bit. will check later 18:09
LongLiveCHIEFhttps://github.com/HackerHappyHour/bootcc/tree/setup-config-options/system-boot18:10
LongLiveCHIEFyes, that's what I'm experiencing.  As the bug report states, it's not an inconvenience to reboot... however, that means that you can't use many of the cloud-init modules in the final stage that require a network connection18:11
LongLiveCHIEFi've got it working with the project i linked above so network will work even on first boot, but I want to make sure I can control the order of the final stage modules to ensure I can still use them, instead of having to run another script in the runcmd stage that will install packages and do other configs that cloud-init already has modules for18:13
LongLiveCHIEF*working so that even wifi networks will work on first boot18:15
LongLiveCHIEFIt would be cool if the docs for the modules included what stage each module was run in.  Some of the modules specify, but most don't18:21
LongLiveCHIEFhttps://bugs.launchpad.net/ubuntu/+source/netplan.io/+bug/187437718:26
ubot5Ubuntu bug 1874377 in netplan.io (Ubuntu Focal) "Netplan does not connect to Wireless after `sudo netplan apply` until reboot" [High,Fix released]18:26
LongLiveCHIEFI think i know how to fix this. I'll work on it. thanks for the help blackboxsw_18:27
LongLiveCHIEFscratch that, there's already a fix merged! https://github.com/CanonicalLtd/netplan/pull/13318:31
rharperblackboxsw: we don't netplan apply specifically because we run before networkd starts, so once we netplan generate (which creates the required systemd-networkd files) and naturally systemd-networkd finds what it needs.18:37
rharperLongLiveCHIEF: w.r.t netplan and wifi; and the linked PR; it's not yet clear to me that the fixed version is actually fixed, as in the bug blackboxsw mentioned,   (and in your case) we have a valid wifi config, and the "fixed" version of netplan. yet not seeing wifi come up;  so something is still missing in netplan;  that is, the apply fix is too late;  whatever needs to happen during apply should happen on the initial boot;  so maybe this clean up18:42
rharperof other networkd services mentioned needs to be figured out without a netplan apply command18:42
LongLiveCHIEFyeah, i'm seeing the same thing now that i've gone through the PR18:44
LongLiveCHIEFmy end goal is the same as it would be on a digital-ocean droplet really. To setup a specified user, and packages, on first boot/provision18:45
LongLiveCHIEFusing wifi only18:46
rharperyeah, AFAICT, the only issue is netplan itself;18:46
LongLiveCHIEFand I can actually do that now using cloud-init, it's just a hackier way of doing it than i prefer18:46
LongLiveCHIEFjust annoying because I need that netplan apply prior to the start of the config stage of cloud-init, (and it seems it would definitely work by that time), but I can't really hook into it until the final stage18:49
rharperwell, you shouldn't18:49
LongLiveCHIEFhaha, why, there's a reason network stage comes before config stage :p18:49
rharperright18:50
rharperwhen we generate, everything should be fine ... I'm going to see what gets written out on generate;  are you in a focal image ?18:50
LongLiveCHIEFyessir18:50
rharperk18:51
LongLiveCHIEFi can confirm that generate creates everything correctly18:51
rharperthat's confusing then ...18:51
LongLiveCHIEFno kidding!! haha18:52
rharperwe're still missing something if apply fixes things18:52
LongLiveCHIEFyep18:52
rharperok, so 0.99-ubuntu1 writes a /run/netplan/wpa-wlan0.conf (and stuff)18:53
rharperupgrading to ubuntu2;18:54
rharperand it still writes it18:54
LongLiveCHIEFnot sure which build i was on, downloaded it early in the day on the 24th, which is the day 0.99 was released18:55
rharperah, that;'s the config18:55
LongLiveCHIEFso i may have just missed it18:55
rharpercould you try the daily image from today ?18:55
LongLiveCHIEFpoint me to the dl?18:55
rharperhttps://cloud-images.ubuntu.com/daily/server/server/focal/current/18:56
LongLiveCHIEFthey don't make dailies for the preinstalled arm6419:01
LongLiveCHIEFnot sure how to work with the arm64 cloud root image19:01
rharperah, that's a bummer19:06
rharperwell, you can mount up the image, upgrade the package, and try with that19:06
rharperyou can use cloud-init --clean --logs19:07
rharperto clear out the previous run (mostly, some things like package instakks and user-adds will remain) but you can test the networking bit19:07
LongLiveCHIEFyeah, that's what I was going to do19:08
LongLiveCHIEFi have a hunch that I want to test first19:08
LongLiveCHIEFi'm telling netplan what IP to assign.  I wonder what will happen if I just let an IP get assigned by DHCP instead19:08
LongLiveCHIEFmight be a bug with managing resolve when using manually assigned IP's19:10
Odd_BlokeLongLiveCHIEF: (I'm catching up on backlog so apologies if this isn't useful info, but) there are known issues with wifi on Pi that I believe are being worked on.  I'll see if I can find a bug reference.19:11
LongLiveCHIEFno worries.19:13
LongLiveCHIEFno rush. I'm just waiting to put everything together for docs for github.com/octoprint/docker for a recommended setup guide for our less tech savvy users19:14
Odd_Blokebpo: Hmm, that's strange.  I'm not sure what's going; what does `cloud-init analyze blame -i /var/log/cloud-init.log` give you?  (The same, I expect.)19:16
bpoYep, same:19:16
bpo-- Boot Record 01 --1 boot records analyzed19:16
LongLiveCHIEFi get 419:17
LongLiveCHIEFah n/m, i've booted that particular one a few times19:17
Odd_Blokerharper: The netplan fix only landed 2 days ago, it's a better version of the pre-release wifi fix.  Have we definitely seen runs with that new version (that's only in groovy-proposed currently)?19:18
Odd_Blokebpo: Would you be able to pastebin your cloud-init.log?  (Or, alternatively, could you try copying that file onto an updated Ubuntu server, xenial or later, and running `cloud-init analyze blame -i that_file`?)19:21
LongLiveCHIEFI'm guessing if I change the cloud.cfg.g final_module order using write_files, it wouldn't take the new config in until service restart19:22
LongLiveCHIEF * I'm guessing if I change the cloud.cfg.d final_module order using write_files, it wouldn't take the new config in until service restart19:22
Odd_BlokeLongLiveCHIEF: cloud-init actually executes from scratch for each phase, so I believe later phases would pick up the new configuration.19:22
Odd_Blokerharper: (^ this is perhaps something we need to consider for the daemon plans, in terms of reloading configuration?)19:23
LongLiveCHIEFwrite_files happens in the final stage though, correct?19:23
LongLiveCHIEFso it would be too late19:23
Odd_BlokeLongLiveCHIEF: You can see the order by looking at /etc/cloud/cloud.cfg; write_files runs early in the init phase (i.e. the first one).19:23
Odd_Bloke(At least in the configuration we ship upstream and in Ubuntu by default. :)19:24
LongLiveCHIEFcool19:24
LongLiveCHIEFi have a workaround method that works for me atm, so I'm going to run with that for the next week, but lmk if you guys want me to test anything/share logs with you in the meantime19:26
LongLiveCHIEFi did test out my dhcp hunch, and it was a no-go. Still failed to resolve when enabling dhcp (both 4 and 6)19:27
LongLiveCHIEFi've learned a ton about cloud-init as a result of all this though, so at the very least, I'll likely be submitting a few docs contributions over the next 2 weeks19:28
Odd_Bloke\o/19:29
Odd_Bloke(That's why we leave the bugs in. ;)19:29
LongLiveCHIEFi'm also considering writing a cloud-init module that would allow you to do things like set the default pin states and such for pi19:29
Odd_BlokeOh, that would be cool!19:30
LongLiveCHIEFyou fix wifi, i'll add gpio. deal?19:30
LongLiveCHIEF😏19:31
LongLiveCHIEFmy pipedream goal however, is to enable raspberry-pi imager to be a datasource19:32
Odd_BlokeLongLiveCHIEF: I'm not familiar with the RPi ecosystem, I'd be interested to hear a little more about how you would see that working.19:33
LongLiveCHIEFraspberry pi imager is an electron based application that burns bootable usb's.  What that means though is that it could also use local network details to lookup and generate network-config and user-data at an endpoint accessible to your own internal network, and it could easily add that network endpoint to the ds seed in cmdline.txt19:35
LongLiveCHIEFso imagine the rpi imager having similar configuration screen as digital-ocean, and in one step it downloads, burns, and configures a bootable microsd for your pi that will install whatever packages you desire19:36
LongLiveCHIEFmany of the options I'm talking about i've organized into a project here: https://github.com/HackerHappyHour/bootcc/projects/119:37
LongLiveCHIEFi'm going to be hacking on a prototype for the next 4 days, and then see if it's something that makes sense to contribute to the https://github.com/raspberrypi/rpi-imager project19:38
LongLiveCHIEFannonced here: https://www.raspberrypi.org/blog/raspberry-pi-imager-imaging-utility/19:38
blackboxswgood point Odd_Bloke on daemon config reload. that'd be a gap vs current implementation19:39
LongLiveCHIEFi'll ping you guys when i have a simple demo ready. Probably about this time tomorrow19:40
blackboxswgood deal LongLiveCHIEF.19:40
Odd_BlokeLongLiveCHIEF: Cool!19:41
bpoOdd_Bloke: https://pastebin.com/uhJNysgm19:42
bpoMust be an issue with the distro, that's from a fresh AWS Linux 2 AMI with no customisations - same behavior.19:43
Odd_Blokebpo: That doesn't give me any output with cloud-init master so Something Strange is happening; I'll see if I can figure it out.19:44
bpoOdd_Bloke: thank you! I'll be curious to hear what you find. I will check back in periodically but will probably be slow to respond.19:47
LongLiveCHIEFbpo: you guys talking about the wifi/network issue or something else?19:51
Odd_BlokeSomething else. :)20:03
Odd_Blokebpo: OK, the issue has something to do with the log format differing between upstream and Amazon Linux (meaning that the upstream code doesn't identify any of the cloud-init.log lines as being cloud-init log lines).20:11
Odd_Blokebpo: Could you run `grep _log -R /etc/cloud` on an instance to find any cloud-init logging configuration (and if you find some, pastebin it)?20:11
blackboxswOdd_Bloke: https://github.com/canonical/cloud-init/pull/335 is good (needs rebase)20:42
Odd_Blokeblackboxsw: https://github.com/canonical/cloud-init/pull/329 is ready for your re-review.21:29
blackboxswhrm Odd_Bloke. so lxc images are all a different hash, how do we know if a hash is from a xenial image or a bionic. and if we've changed from xenial -> bionic across travis runs aren't we just removing all images and re-downloading the new series21:36
Odd_Blokeblackboxsw: We only use one lxd image in our builds, a xenial one.21:36
Odd_BlokeLet me add that to the explanatory comment.21:36
blackboxswahh right --os-name xenial21:38
blackboxswok I'm good with that then21:38
blackboxswbut comment welcome for future me21:39
Odd_Blokeblackboxsw: Pushed as a separate commit for ease of review: https://github.com/canonical/cloud-init/pull/329/commits/32b2ea23d28accf4c4c34f32865c8041712e248021:40
bpoOdd_Bloke: Here is /etc/cloud/cloud.cfg.d/05_logging.cfg: https://pastebin.com/x6NZ4NFJ22:01
Odd_Blokebpo: Thanks!  I'm just finishing up my day, so I'll take a look in the morning.22:03
bpoOdd_Bloke: sounds good, thanks22:04

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!