=== djhankb6 is now known as djhankb
[05:25] <thatsk> anyone is here
[05:25] <thatsk> to answer
[05:25] <thatsk> some question
[05:36] <thatsk> ?
[15:07] <Srijan32> Hello Everyone, I am trying to do an auto install using cloud-init version 22.4.2-0ubuntu0~22.04.1
[15:07] <Srijan32> Here is my user-data.
[15:07] <Srijan32>   late-commands:
[15:07] <Srijan32>     - curtin in-target -- wget https://www.python.org/ftp/python/2.7.18/Python-2.7.18.tgz
[15:07] <Srijan32>     - curtin in-target -- tar -zxvf Python-2.7.18.tgz
[15:07] <Srijan32>     - curtin in-target -- pwd
[15:07] <Srijan32>     - curtin in-target -- cd /target/Python-2.7.18
[15:07] <Srijan32>     - curtin in-target -- ./configure && make && checkinstall
[15:07] <Srijan32> When I try to boot the system, it gives me the following error:
[15:07] <Srijan32> chroot; failed to run command 'cd': No such file or directory
[15:09] <dbungert> the directory you changed to under target doesn't mean you are in the python directory for the configure step.  Those are seperate shells.
[15:09] <dbungert> for intricate scripting you may find it more convenient to make a secondary script, then download and run that
[15:12] <Srijan32> Thank you Dbungert, will try creating this on a separate script
[15:16] <Srijan32> When I get into the shell, I see the filesystem and also I see the /target directory. I also see the Python-2.7.18 drectory there, just cannot cd into it
=== Felix is now known as Guest8539
[17:15] <Guest8539> Hi folks
[17:17] <meena> who are all these impatient people?
[17:21] <Guest8539> We configure our machines using cloud-init in two phases/steps. Using user-data for the machine, let's say it's on AWS, we pass in the initial configuration. This cloud-init configuration, after running all the tasks, downloads another configuration and now we need to run the second configuration on the machine.
[17:21] <Guest8539> We were doing something like this at the end of the first configuration thinking that this is all that we need. But it seems like this still runs the first configuration?
[17:21] <Guest8539> ```
[17:21] <Guest8539> cloud-init clean
[17:21] <Guest8539> cloud-init --file /etc/cloud/cloud.cfg.d/my-custom.cfg init
[17:21] <Guest8539> ```
[17:22] <Guest8539> > But it seems like this still runs the first configuration?
[17:22] <Guest8539> I mean it does execute the configuration at `/etc/cloud/cloud.cfg.d/my-custom.cfg` but it also re-runs the first configuration.
[17:28] <blackboxsw> Guest8539: so, cloud-init clean means that cloud-init will treat the 2nd launch as a clean new system and re-run everything it sees from the discovered datasource (Ec2 IMDS @  http://169.254.169.254) plus overrides in your new config in /etc/cloud/cloud.cfg.d/my-custom.cfg
[17:30] <blackboxsw> cloud-init is generally  intended to run once in most cases (with the exception of opting in to network hotplug). What kind of use-case (cloud-config changes) are you trying to apply on the 2nd boot?
[17:30] <Guest8539> Thanks for the reply, what if instead of running `cloud-init clean`, I just delete `/etc/cloud/cloud.cfg` and then run `cloud-init --file /etc/cloud/cloud.cfg.d/my-custom.cfg init`?
[17:34] <Guest8539> > What kind of use-case (cloud-config changes) are you trying to apply on the 2nd boot?
[17:34] <Guest8539> We have some sensitive configurations that we don't inject using the user data. We download them using a single-use token at the end of the first run and then would like to just run that configuration. It serves us some other use cases as well like overcoming userdata limitations from some cloud providers end.
[17:35] <blackboxsw> Guest8539: I think we need to back up to understand what supplemental config you are looking for.. there are ways you could inject an executable /var/lib/cloud/scripts/per-boot/yourscript.sh that will get run on next boot even when cloud-init doesn't redetect that it needs to perform a full setup.
[17:36] <blackboxsw> hrm. trying to think this through.
[17:36] <jchittum> Guest8539: I'm guessing that this second cloud-init run is a manual step? that a human is logging in, inputting a token, etc? and you don't even want the token being transmitted in the original cloud-init?
[17:37] <Srijan15> Need help running the following using curtin
[17:37] <Srijan15>     - curtin in-target -- wget https://www.python.org/ftp/python/2.7.18/Python-2.7.18.tgz
[17:37] <Srijan15>     - curtin in-target -- tar -zxvf Python-2.7.18.tgz
[17:37] <Srijan15>     - curtin in-target -- sh -c cd /target/Python-2.7.18
[17:37] <Srijan15>     - curtin in-target -- pwd
[17:37] <Srijan15>     - curtin in-target -- sh -c ./configure && sh -c make && sh -c checkinstall
[17:37] <Srijan15> These doesnt seem to work, getting the error message no such file or directory. However, when I boot I am able to see Python-2.7.18 under the target directory
[17:39] <Guest8539> Sorry for causing a bit of confusion. I only want to run all the configurations on the first boot. To simplify what I want to achieve; cloud-init picks userdata and executes it. In the end, using a script, we want to switch to another cloud-config and execute it.
[17:42] <falcojr> Guest8539: Cloud-init really isn't intended to do that. You can clean and then change (or override) the various *_modules lists in /etc/cloud/cloud.cfg to decide what to run, but that's going to be fragile in the long term
[17:42] <blackboxsw> Srijan15: I think that's probably best placed question for the #curtin channel because it is not cloud-init ... but dbungert responded to you earlier with the solution there "the directory you changed to under target doesn't mean you are in the python directory for the configure step.  Those are separate shells.
[17:42] <blackboxsw> 08:09 for intricate scripting you may find it more convenient to make a secondary script, then download and run that"
[17:47] <blackboxsw> what dbungert pointed out was your separate `in-target` lines mean a completely different shell, thereforce any cd you are performing is lost on your next curtin in-target invocation you'd either need to do it all on one line in the single curtin in-target call, or wget some script you are hosting from elsewhere. 
[17:48] <Srijan15> Let me try putting them all in a single line using && and check
[17:49] <Guest8539> > Cloud-init really isn't intended to do that
[17:49] <Guest8539> Basically, it's a legacy software that we want to fix. We have long moved away from this dual configuration thing.
[17:50] <Guest8539> As an example, I'm not sure if it's okay to post code snippets here:
[17:50] <Guest8539> ```
[17:50] <Guest8539> #cloud-config
[17:50] <Guest8539> write_files:
[17:50] <Guest8539> - path: '/opt/bin/firstrun.sh'
[17:50] <Guest8539>   content: |-
[17:50] <Guest8539>     #!/bin/bash
[17:50] <Guest8539>     yum install -y curl jq
[17:50] <Guest8539>     echo "#cloud-config\nssh_pwauth: false" > /etc/cloud/cloud.cfg.d/my-custom.cfg
[17:51] <Guest8539>     cloud-init clean
[17:51] <Guest8539>     cloud-init --file /etc/cloud/cloud.cfg.d/my-custom.cfg init
[17:51] <Guest8539> - path: '/etc/systemd/system/firstrun.service'
[17:51] <Guest8539>   content: |-
[17:51] <Guest8539>     [Service]
[17:51] <Guest8539>     Type=oneshot
[17:51] <Guest8539>     RemainAfterExit=true
[17:51] <Guest8539>     ExecStart=/opt/bin/firstrun
[17:51] <Guest8539> runcmd:
[17:51] <falcojr> Guest8539: In IRC, please use a pastebin service
[17:52] <blackboxsw> Guest8539: per falcojr's message, the fragility comes from a permanently customized my-custom.cfg  will only allow certain subset of modules. If someone migrates or clones that VM in EC2 (or other datasources) the IMDS will change the instance-id that the IMDS gives in metadata. It tells cloud-init will try to re-run all modules to create new users/setup new host SSH keys etc.. But, my-custom.cfg will only permit running a subset.
[17:52] <blackboxsw> So your images will not fully realize any config changes requested by the datasource IMDS.
[17:52] <Guest8539> Here is the pastebin https://pastebin.pl/view/3411eb9e
[17:54] <Guest8539> blackboxsw I'm fine with that but is it possbile to do this? Like just run the custom configuration and not the actual configuration from userdata? I think with `cloud-init clean` it's not possible since in that case cloud-config will be re-fetched from the userdata as well.
[18:00] <blackboxsw> Guest8539: generally running `cloud-init init` outside of the normal boot process is a very fragile because it's only one of the bootstages applicable to initial system setup which need to be run in order https://cloudinit.readthedocs.io/en/latest/explanation/boot.html#boot-stages
[18:01] <blackboxsw> trying to invoke that while within cloud-init's inital boot in the runcmd is designed to burn you with many potential cornercases
[18:05] <blackboxsw> Guest8539: so that I understand: it seems like you can't take a completely clean base cloudimage and provide your custom user-data via write_files with whatever additional user-data. Are instead taking a legacy VM that was launched at one point with #cloud-config userdata and trying to additionally modify it to run some suppplemental configuration?
[18:07] <Guest8539> Yes that is one use case that we have unfortunately
[18:23] <blackboxsw> Guest8539: so somehow you are getting into the system, writing a file that emits a script, and trying to run that script across system reboot right?
[18:26] <blackboxsw> Guest8539: since you are in the system, you could just add your script to /var/lib/cloud/scripts/per-boot  and either run `sudo cloud-init single --name scripts-per-boot` or reboot the node and it'll perform supplemental config.  You'd have to ensure that /var/lib/cloud/scripts/per-boot/yourscript.sh is chmod 755  and also idempotent. But, that avoids the cloud-init clean mess from within cloud-init boot stages
[18:27] <falcojr> looks like it's a separate cloud-config though, not just a script
[18:31] <blackboxsw> get's dicey, could run `sudo cloud-init single --name runcmd --frequency always` but yeah it's not ideal.I was just reading the abbreviated cloud-config written and it seems to be just something could be contained in a one-off script. not really needing cloud-config in general. But, I realize the paste was probably redacted