| powersj | blackboxsw, fyi it may not land as I gotta figure out what to do with the new vlan + proxy | 00:09 |
|---|---|---|
| powersj | I updated a bunch of jobs, but still need to debug the ci jobs | 00:10 |
| blackboxsw | No worries powersj. Thanks for the heada up | 00:35 |
| smoser | blackboxsw: jenkins didn't like your aure branch | 02:08 |
| braincod1 | Hello cloud-init devs! | 03:28 |
| braincod1 | I have a userdata.sh that I would like to drop into /var/lib/cloud... it is not entirely clear to me (from the docs), if I can do that as-is with a bash script without migrating to cloud-init YAML DSL? | 03:30 |
| braincod1 | I tried putting it under per-boot, but doesn't seem to work as expected | 03:34 |
| === braincod1 is now known as braincode | ||
| smoser | braincode: you're wanting shell script to be executed every boot ? | 03:42 |
| braincode | Yeah, it just provisions an EC2 instance | 03:43 |
| braincode | This one: https://github.com/umccr/umccrise/blob/master/deploy/roles/brainstorm.umccrise-docker/files/bootstrap-instance.sh | 03:43 |
| braincode | I started to migrate it to cloud-init YAML, but there are some parts I would need to refactor (namely, env vars) for it to work with cloud-init: https://github.com/umccr/umccrise/blob/master/deploy/roles/brainstorm.umccrise-docker/files/cloud-init.yml | 03:44 |
| smoser | that should generally work | 03:44 |
| smoser | http://paste.ubuntu.com/p/kBJCbWXdKJ/ | 03:44 |
| smoser | https://github.com/umccr/umccrise/blob/master/deploy/roles/brainstorm.umccrise-docker/files/cloud-init.yml is confused. | 03:45 |
| smoser | #cloud-config | 03:45 |
| smoser | must be yaml formatted, and can do things lke you have there 'fs_setup:' | 03:45 |
| smoser | but your 'sleep 10' is not useful there. | 03:46 |
| smoser | the first half of that script... can be in a per-boot script | 03:46 |
| smoser | you can put cloud-config things in /etc/cloud/cloud.cfg.d/99-your-stuff.cfg | 03:46 |
| smoser | but those things must be valid yaml. | 03:47 |
| braincode | Yeah, the cloud-init one is work in progress | 03:47 |
| braincode | That's why is half-baked, sorry | 03:47 |
| smoser | hhttp://paste.ubuntu.com/p/QdDYygjNzd/ | 03:48 |
| smoser | http://paste.ubuntu.com/p/QdDYygjNzd/ | 03:48 |
| braincode | @smoser, I'm getting this out of dropping the shell script in per-boot: https://paste.ubuntu.com/p/5Tc33RtnQQ/ | 03:48 |
| smoser | but you should absolutely be able to put that shell script into /var/lib/cloud/scripts/per-boot | 03:49 |
| braincode | Yes, I know, work in progress, I'm focusing now on just the shell script for now | 03:49 |
| smoser | (probalby you want per-instance... so it only runs once per instance) | 03:49 |
| smoser | then chmod 755 it | 03:49 |
| smoser | and it should be good | 03:49 |
| smoser | if its not executable it will be ignored. | 03:49 |
| smoser | if you have issues.... | 03:49 |
| smoser | almost certainly there is a 'WARN' in /var/log/cloud-init.log | 03:49 |
| smoser | and the output of such scripts is in /var/log/cloud-init-output.log | 03:49 |
| smoser | (if not redirected) | 03:49 |
| smoser | i might suggest in the script at the top you can redirect it yourself... as I did with the 'tee' | 03:50 |
| smoser | ( | 03:50 |
| smoser | long | 03:50 |
| smoser | script | 03:50 |
| smoser | here | 03:50 |
| smoser | ) 2>&1 | tee /run/this-is-my-output.log | 03:50 |
| smoser | make sense ? | 03:50 |
| braincode | Yeah, thanks a ton! :D | 03:50 |
| smoser | but if you weren't making the script executable, then cloud-init will ignore it. | 03:51 |
| braincode | Also not entirely clear to me from https://cloudinit.readthedocs.io/en/latest/topics/datasources.html how to get those AWS env vars I'm defining on the script? | 03:58 |
| braincode | gotta go, brb, thanks a lot @smoser! | 03:58 |
| braincode | What I mean is that it is well documented where that json with all the instance-data sits, but it's not clear how to i.e fetch instance-id or region and then using it on the cloud-init.yaml? Am I allowed to call the python methods shown somehow from the YAML? | 04:16 |
| blackboxsw | Braincode you are only a few days too early. https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/335290 | 04:18 |
| blackboxsw | Should be able to use it in jinja templates | 04:18 |
| blackboxsw | As.soon as we land that branch | 04:21 |
| braincode | Hoho, awesome! :) | 04:32 |
| braincode | Thanks for the suggestions on the 755... it went a little bit further but not all the way it seems: https://paste.ubuntu.com/p/54Rmy8jpJD/ | 06:19 |
| braincode | That is, with https://github.com/umccr/umccrise/blob/master/deploy/roles/brainstorm.umccrise-docker/files/bootstrap-instance.sh | 06:19 |
| braincode | ... and I did not cut any further output at the end, that's it, all cloud-init executes from the scrip... wait a second, yeah, for sure it's about the Instance Role permissions | 07:01 |
| nicolas17 | hi | 09:25 |
| nicolas17 | on one of kde.org's servers, cloud-init decided that it has never run before | 09:26 |
| nicolas17 | which means it eg. re-generated the ssh host keys | 09:26 |
| nicolas17 | any idea why that can happen? I have logs | 09:27 |
| nicolas17 | https://paste.kde.org/pl76fn3ks | 09:39 |
| nicolas17 | I guess the error on line 498 may be it? | 10:02 |
| powersj | smoser, blackboxsw ci is broken until I figure out the proxy | 14:31 |
| smoser | i noticed :) | 14:35 |
| blackboxsw | smoser: on your oracle instance I presume the commented #datasource_list: [ OpenStack ] in 99-oracle-bare-metal-datasource.cfg was you right for your dev/testing | 16:29 |
| smoser | blackboxsw: this is apita | 16:32 |
| smoser | right. i commented that out. | 16:32 |
| smoser | this negative skip is annoying | 16:33 |
| smoser | and i'm not sure the one in DataSourceConfigDrive is done right... | 16:34 |
| blackboxsw | yeah and $ python3 -c 'from cloudinit.sources.DataSourceOpenStack import detect_openstack; print(detect_openstack())' | 16:34 |
| blackboxsw | True | 16:34 |
| blackboxsw | yes, the negative skip as a policy due to ds detection order stinks. | 16:35 |
| blackboxsw | so I only recently added that OracleCloud.com asset_tag check to DSOpenStack. | 16:41 |
| blackboxsw | do we have different asset tags on OCI-classic vs. OCI | 16:42 |
| blackboxsw | doesn't appear to. /me grasps for some other env variable that'd be specific to oci-c only | 16:42 |
| blackboxsw | as a 'pass' for DataSourceOpenStack. | 16:43 |
| smoser | no asset tags on oci-classic | 16:43 |
| smoser | uses ec2 | 16:43 |
| smoser | not covered here, not negaively affected here. | 16:43 |
| blackboxsw | sooo, since the only add of 3cee0bf8 was enabling detection of DMI_ASSET_TAG_ORACLE_CLOUD, we can back that out with your Oracle Datasource introduction | 16:45 |
| blackboxsw | right? | 16:45 |
| smoser | no. | 16:45 |
| smoser | because then we'd have to deal with "transfer"ing | 16:46 |
| smoser | blackboxsw: i'm currently confused at this | 16:47 |
| smoser | $ sudo python3 -c 'from cloudinit.stages import _pkl_load; print(_pkl_load("/var/lib/cloud/instance/obj.pkl").sys_cfg.get("datasource_list", "why is there no datasource_list"))' | 16:50 |
| smoser | why is there no datasource_list | 16:50 |
| * blackboxsw hrm same on lxc... checking | 16:53 | |
| smoser | $ sudo python3 -c 'from cloudinit.stages import Init; x = Init(); print(x.cfg.get("datasource_list"))' | 16:56 |
| smoser | ['Oracle', 'None'] | 16:56 |
| smoser | i thought that that was basically where the datasource got its sys_cfg | 16:56 |
| smoser | blackboxsw: sorry. i rebooted. cleaned and rebooted. seeing if it comes up different. | 16:58 |
| blackboxsw | no worries smoser I'm on a container | 16:58 |
| smoser | well at least it recreates. | 16:58 |
| blackboxsw | sources.find_sources gets passed a valid/populated datasource_list | 17:02 |
| blackboxsw | checking to see what it does w/ self.cfg | 17:02 |
| blackboxsw | checking to see what it does w/ Init.cfg | 17:02 |
| blackboxsw | saves as ds.sys_cfg | 17:03 |
| blackboxsw | hrm | 17:03 |
| smoser | so | 17:04 |
| smoser | sudo cloud-init clean --logs | 17:04 |
| smoser | sudo cloud-init init --local | 17:04 |
| smoser | sudo python3 -c 'from cloudinit.stages import _pkl_load; print(_pkl_load("/var/lib/cloud/instance/obj.pkl").sys_cfg.get("datasource_list", "why is there no datasource_list"))' | 17:04 |
| smoser | you will see the datasource now | 17:05 |
| smoser | sudo cloud-init init | 17:05 |
| smoser | try again and it is not there. | 17:05 |
| smoser | s/datasource/datasource_list/ | 17:05 |
| blackboxsw | hrm something w/ restoring from cache? not seeing that on lxc's nocloud datasource between --local and network stage runs. | 17:11 |
| [42] | i'm doing something wrong somewhere i believe | 17:21 |
| [42] | i'm setting a hostname in my user data and i can see it's set in /etc/hosts | 17:21 |
| [42] | but it doesn't seem to set the system hostname as i can see in /etc/hostname / hostnamectl | 17:21 |
| [42] | preserve_hostname is false | 17:22 |
| [42] | any ideas what could be blocking this? | 17:22 |
| blackboxsw | dumb, lxc does reproduce | 17:25 |
| blackboxsw | i was not printing the pkl'd object datasource_list, but Init's.cfg['datasource_list'] | 17:25 |
| blackboxsw | so upon reload of cache in network stage, datasource_list is dropped | 17:26 |
| [42] | i found https://git.launchpad.net/cloud-init/tree/doc/examples/cloud-config.txt#n291 | 17:27 |
| [42] | how does it check for "not modified by user"? | 17:27 |
| [42] | does that even apply if it's set to false in /etc/cloud/cloud.cfg? | 17:28 |
| === brunobronosky is now known as Guest20404 | ||
| [42] | found it in /var/lib/cloud/data/previous-hostname | 17:32 |
| blackboxsw | so logic for hostname set is here : /usr/lib/python3/dist-packages/cloudinit/config/cc_set_hostname.py, it checks a previous-hostname cache file to compare hostname/fqdn against any previous cloud-init configured hostname. if cloud-init thinks hostname didn't change, it'll emit a log "No hostname changes. Skipping set-hostname" | 17:32 |
| blackboxsw | sorry took me a while to type. yes that's the cache cloud-init compares against | 17:33 |
| blackboxsw | I'd expect you are seeing that Skipping... log in /var/log/cloud-init.log | 17:33 |
| [42] | i didn't see that line, might be due to the old debian cloud-init | 17:34 |
| blackboxsw | [42] is debian on 0.7.9 still? we've definitely overhauled a bit of the cc_sethostname logic a bit after that. | 17:35 |
| [42] | yes | 17:35 |
| blackboxsw | :/ yeah I don't think we have the same problem/behavior anymore. I believe we made the cc_sethostname a bit more flexible. | 17:36 |
| blackboxsw | flexible/re-entrant | 17:38 |
| blackboxsw | cc_update_hostname is the other module that is at play here | 17:39 |
| blackboxsw | ... and there really shouldn't be two modules for this ultimately.... but we haven't yet aligned that behavior but we've talked about that a possible future | 17:41 |
| smoser | [42]:the intent is that it stores what it was previously | 17:43 |
| smoser | and if it is not what it was previously, then it assumes written by user. | 17:44 |
| [42] | smoser: yup figured that out | 17:44 |
| smoser | yeah.. /me catches up with backscroll | 17:44 |
| [42] | unfortunately others administrating these systems don't always invest enough time to figure out why things don't work to properly fix it rather than just quick fixing it on one system | 17:45 |
| [42] | which isn't useful when cloning a machine | 17:46 |
| smoser | blackboxsw: http://paste.ubuntu.com/p/cwBT2fNMHp/ | 18:00 |
| smoser | blackboxsw: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1787459 <-- thats a bug for the sys_cfg problem | 18:17 |
| ubot5 | Ubuntu bug 1787459 in cloud-init (Ubuntu) "datasource.sys_cfg gets different values in local stage and after." [Undecided,New] | 18:17 |
| blackboxsw | almost done w/ your review on original branch smoser. will check paste andbug in a few mins | 18:18 |
| smoser | blackboxsw: "a couple of nits about persisting variables" | 18:48 |
| smoser | but i dont see comments i line | 18:48 |
| smoser | inlin | 18:48 |
| smoser | inline | 18:48 |
| blackboxsw | bah, checking the submit button | 18:50 |
| blackboxsw | ok smoser so your paste you are allowing oracle as openstack if Oracle not in ds list. so old behavior persists for existing pets which are upgraded. and there is not a world where ds-identify would report OpenStack on OracleCloud.com per your new changes, so looks safe from that front. | 18:56 |
| blackboxsw | +1 on your paste | 18:56 |
| blackboxsw | smoser: 6 unsaved comments posted on your oci branch | 18:58 |
| smoser | blackboxsw: https://meet.google.com/bxm-azib-mtj | 19:39 |
| smoser | blackboxsw: http://paste.ubuntu.com/p/V2qQ4BWtVg/ | 20:52 |
| blackboxsw | smoser: looks good although the ci-b64: prefix not being processed to base64-encoded-keys. I can't decide if I care enough about the interim state of the data. | 21:04 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!