| kikimora | Hi, I'm trying to figure out how can I configure cloud init to a) being able to install and configure software and b) don't let admins run arbitrary code at launch time. A bit of context - I have devops who run instances (ec2:RunInstance) and pass a role (ec2:PassRole) to enable the instance read some protected data. I want them to be able to do a) | 00:16 |
|---|---|---|
| kikimora | but if they can do b) then devops can escalate privileges and read secret data which I want to hide from them. | 00:16 |
| kikimora | Can anyone suggest a way to implement these requirements? | 00:16 |
| minimal | kikimora: so you want to let them have control over user-data contents but not be able to do things you don't want them to do despite them having control over user-data? | 00:21 |
| kikimora | sort of... I'm not entirely sure what cloud-init is capable of. One way I can imagine this could work is to limit cloud-init (or user data) to a single script that I can authorise by signing or providing a checksum. Devops then specify the script with parameters to setup a box. At least this way I can be sure what they are running. | 00:23 |
| minimal | cloud-init doesn't check for signatures etc, ut just uses whatever its passed | 00:24 |
| minimal | how are they creating VMs and passing user-data to them? | 00:25 |
| kikimora | They use normal AWS stuff - ec2:RunInstance from terraform or AWS CLI and pass user data as a parameter. Unfortunately I cannot limit use of user data with AWS IAM. | 00:26 |
| minimal | so unless you prevent them from having terraform access and come up with some frontend to terraform thath they use and then you somehow validate the user-data they pass before terraform is invoked then I don't see how you can manage that | 00:28 |
| kikimora | >cloud-init doesn't check for signatures etc, ut just uses whatever its passed | 00:28 |
| kikimora | Can it be limited to just a config data, not scripts? In this case I can limit them to AMIs with this particular cloud-init configuration. | 00:28 |
| minimal | even if it could be limited in such a way they could simply use "write_file" in user-data to create scripts and "runcmd" in user-data to execute those... | 00:29 |
| kikimora | I see... this is surprising. Cloud-init is mature product and being able to execute arbitrary code as root is a way to privilege escalation which supposedly should be addressed at this stage. Maybe I'm asking for something strange or everybody else are not concerned about security implications for some reason I don't understand? | 00:32 |
| minimal | kikimora: its a way for *admins* to perform initial bootstrapping/configuration of machines | 00:33 |
| minimal | so by its very nature its designed to let *admins* do such stuff | 00:33 |
| minimal | your are worried about security implications of the person "owning" a VM being able to control the VM that they "own"? | 00:34 |
| kikimora | The VM in question runs app that connects to a database with encrypted data. To make app work it has to decrypt the data and so it gets access to decryption keys. I don't want admins launching instances of this app to be able to read secret data and post credit card numbers on the internet. | 00:35 |
| kikimora | or you can imagine crypto wallt private keys, health records or whenever | 00:35 |
| minimal | ok but that's irrelevant to cloud-init specifically - it provides a method for the person(s) creating a machine to act on user-data provided by the person creating the machine to do various things upon 1st boot | 00:36 |
| minimal | so person creating the machine is effectively "root" (in theory) for that machine - they decide what user accounts are creating, what SSH keys are set up, what services run, what packages are installed, etc | 00:37 |
| kikimora | yeah, you right, I'm trying to use it the wrong way. Do you know if removing/disabling cloud-init interfere somehow with normal AWS operations? | 00:38 |
| minimal | basically yes - how else will a host obtain details of its IP addresses etc, it comes from EC2 Metadata which cloud-init (or equivalent) uses | 00:39 |
| minimal | for keys in AWS you can use their security services in general to control access but, bottom line the admin are *admins*, not end-users. If you want to try and control them you could set up remote logging, contralised user management, etc but again admins are admins so generally they need priviliges in order to do their job | 00:40 |
| minimal | its like wanting to have armed guards for your bank vault but wanting to prevent those guards from themselves robbing your vault... | 00:43 |
| kikimora | Not really. This means that any person who has RunInstance and PassRole permission in AWS (not necessarily admin) can escalate permission up to any role she can pass. | 00:45 |
| kikimora | I was afraid to get the answers I've got when I started the conversation but they are still very good answers. Thank you for your support, appreciate it. | 00:52 |
| minimal | my point was that the person creating a VM, assuming they can pass user-data, can do things like create a user account for themselves with sudo access to "root" as, after all as the person creating the VM they "own" it and should be able to do such thing with a VM that they "own". | 00:54 |
| minimal | rememnber that this user-data stuff is actioned at *FIRST boot*, its only relevant then - so if admin A creates VM A then it doesn't matter if admin B has RunInstance role once VM A is created and past 1st boot (unless admin A gave admin B's ssh public key access to VM A) | 00:57 |
| Vaelatern | except for ssh keys | 00:59 |
| Vaelatern | which I found out through some documentation :) | 01:00 |
| minimal | Vaelatern: not sure what you mean | 01:01 |
| Vaelatern | https://www.linkedin.com/pulse/lost-ssh-key-cloud-init-answer-himanshoo-wadhwa/ | 01:01 |
| Vaelatern | "documentation" | 01:01 |
| minimal | right, changing the user-data, yes it will check at boot if it has changed | 01:05 |
| minimal | there are ways to prevent such changes being acted upon | 01:05 |
| Vaelatern | Hey, where can I find out what's configurable at the kernel command line? | 01:10 |
| Vaelatern | such as settings | 01:10 |
| kikimora | I get it. My use case is different. I have users installing software into customer's account and they should not "own" VMs they create because they create them on customer's behalf. | 01:12 |
| kikimora | I get it. My use case is different. I have users installing software into customer's account and they should not "own" VMs they create because they create them on customer's behalf. | 01:12 |
| Vaelatern | kikimora: I'd recommend a rearchitecture | 01:13 |
| Vaelatern | if ownership of the VM can reveal your secrets, you should move secrets out of the VM | 01:13 |
| Vaelatern | or change the ownership | 01:13 |
| Vaelatern | no way to perform magic | 01:13 |
| minimal | kikimora: or provide an abstract way for these people to create the VMs in a way you can control, e.g. Rundeck driving terraform. I've worked in place where some support people needed to send logfile to vendors for problem diagnosis so rather than give support access to VMs Rundeck was implemented as a controlled frontend that they used to trigger log transfers without direct machine access | 01:25 |
| minimal | Vaelatern: not sure of 1 single doc but there's references here: https://cloudinit.readthedocs.io/en/latest/topics/boot.html?highlight=cmdline#generator and here: https://cloudinit.readthedocs.io/en/latest/topics/datasources/nocloud.html?highlight=seed#nocloud | 01:27 |
| minimal | kikimora: re app keys etc, that's why I made reference to Amazon Secrets Manager or Hashicorp Vault... | 01:28 |
| falcojr | you can also pass network configuration via kernel command line, https://cloudinit.readthedocs.io/en/latest/topics/network-config.html#default-behavior | 14:32 |
| Vaelatern | so my takeaway is "limited options for command-line edits, template user-data instead" | 16:02 |
| === nicolasbock_ is now known as nicolasbock | ||
| === cjp256_ is now known as cjp256 | ||
| === meena4 is now known as meena | ||
| === patviafore_ is now known as patviafore | ||
| === madhens_ is now known as madhens | ||
| === frickler_ is now known as frickler | ||
| === nahamu_ is now known as nahamu | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!