| viks____ | Hi, i'm trying to create a openstack baremetal instance. But it does not run cloud-init during the boot. I see the error message `cloud-init is enabled but no datasource found, disabling` in `/run/cloud-init/cloud-init-generator.log` within the instance. But when i curl metadata server within the instance, it is successful. Any suggestions why it is not able to find the datasource during the bootup? | 09:04 |
|---|---|---|
| meena | viks____: you could override the detection and set up a concrete data source in the config | 09:07 |
| viks____ | https://www.irccloud.com/pastebin/qElupQdg/ | 09:07 |
| viks____ | meena: how to do that? | 09:08 |
| meena | like that, but only seeing OpenStack | 09:22 |
| viks____ | meena: you meant like `datasource_list: [ OpenStack, None ]` ? | 09:27 |
| viks____ | i built the images using Disk image builder with `DIB_CLOUD_INIT_DATASOURCES="ConfigDrive, OpenStack"` because of which `datasource_list: [ ConfigDrive, OpenStack, None ]` created | 09:30 |
| Guest20 | hi there, what is the use of cloud init.. is it to launch commands whenever an instance boots up.. is that the only use? | 12:45 |
| meena | Guest20: it can do that, but in general, people use it to provision (cloud) servers: to go from a relatively generic image to one with resized disks, correctly configured networking and hostname, and whatever else you need extra. that extra can even be left to the end user of the cloud, in the form of user-data | 13:00 |
| Odd_Bloke | viks____: cloud-init will only run on a system if it identifies that it will have something to do; it does this via a script called ds-identify (ds == datasource, cloud-init's nomenclature for the code that interacts with a particular cloud platform). To avoid requiring network access, ds-identify relies only on local data. In OpenStack's case, it will look for a mountable config drive and, failing | 14:39 |
| Odd_Bloke | that, look for a variety of strings in DMI data. | 14:39 |
| viks____ | Odd_Bloke: doesn't it check for metadata server? i'm not using config drive as of nw as i faced some issue | 14:41 |
| Odd_Bloke | viks____: Aha, should have read all your messages: ds-identify has a special case for [OnlyOneDataSource, None] in that configuration: it will circumvent its usual logic, and detect OnlyOneDataSource. So I suspect that making `datasource_list: [OpenStack, None]` should do the trick. | 14:42 |
| Odd_Bloke | viks____: It can't check for the metadata server: (a) ds-identify runs at systemd generator time, which is very early in userspace boot (and well before network is configured, because cloud-init configures networking), and (b) ds-identify is designed to avoid making external HTTP requests if we don't know that we're on a platform where we should be making such requests (to avoid MITM attacks). | 14:45 |
| viks____ | Odd_Bloke: In case of non-baremetal openstack, cloud-init just works fine with metadata server.. it's giving problem in baremetal bootup.. | 14:53 |
| Odd_Bloke | viks____: Right, non-baremetal OpenStack will have the DMI attributes configured correctly by Nova by default. | 14:56 |
| viks____ | Odd_Bloke: can you plz elaborate a bit abt `DMI attributes`.. i using baremetal along with other openstack services along with Nova.. so Nova should have configure it right? | 14:59 |
| === hjensas is now known as hjensas|afk | ||
| falcojr | viks____: this is the OpenStack section of the ds-identify script mentioned earlier, https://github.com/canonical/cloud-init/blob/master/tools/ds-identify#L1164 | 15:13 |
| falcojr | DMI attributes are system attributes usually found in /sys/class/dmi/id | 15:14 |
| viks____ | falcojr: will check it.. thnx | 15:25 |
| viks____ | falcojr: i think it's being configured in mine in case of baremetal.. i see the values configured appropriately in baremetal instance in: `/sys/class/dmi/id/*` | 16:37 |
| Odd_Bloke | viks____: Would you be able to run `for file in /sys/class/dmi/id/*; do echo $file: $(sudo cat $file); done 2>/dev/null` on such an instance and pastebin the output at https://paste.ubuntu.com/ ? | 17:05 |
| viks____ | https://www.irccloud.com/pastebin/FXPqIobQ/ | 17:26 |
| Odd_Bloke | viks____: Right, there are values there, but they match the hardware in the system: for cloud-init to detect this as OpenStack, those values would need to be set by OpenStack to particular values. Given that this _is_ a bare metal system, it's not especially surprising that they aren't (because there's actual hardware for DMI to report on). | 17:58 |
| Odd_Bloke | This isn't a configuration I have experience with, so I don't know if there's a better way to get Nova to DTRT for you, but from a cloud-init POV, I expect that setting DIB_CLOUD_INIT_DATASOURCES="OpenStack" would address the issue; are you able to test that? | 17:59 |
| chillysurfer | hello all! i can't seem to remember (or find) where cloud-init writes out network config that it dynamically generates. anybody know the location? | 18:07 |
| Odd_Bloke | chillysurfer: Which distro? | 18:09 |
| chillysurfer | Odd_Bloke: this is centos | 18:09 |
| viks____ | Odd_Bloke: above image is created with that only | 18:11 |
| Odd_Bloke | chillysurfer: Then I think it'll be /etc/sysconfig. | 18:14 |
| Odd_Bloke | viks____: Huh, OK, I would expect that to cause the correct DS to be detected. Can you paste /run/cloud-init/ds-identify.log please? | 18:15 |
| chillysurfer | Odd_Bloke: hmmm /etc/sysconfig/network has no indication why interfaces to handle | 18:15 |
| chillysurfer | uncommented lines are: | 18:16 |
| chillysurfer | HOSTNAME=something | 18:16 |
| chillysurfer | NETWORKING=yes | 18:16 |
| Odd_Bloke | chillysurfer: And that's the only file in there? | 18:17 |
| Odd_Bloke | (I don't have a sysconfig system to poke at to hand, unfortunately; also I'm disappearing for lunch for a while. :) | 18:17 |
| chillysurfer | Odd_Bloke: there is also /etc/sysconfig/network-scripts/* which includes some config but just for successful interfaces | 18:18 |
| chillysurfer | not failed | 18:19 |
| chillysurfer | just not sure how config looks when cloud-init determines to "ignore this interface" | 18:19 |
| Odd_Bloke | chillysurfer: It feels like I'm missing some context on the problem you're seeing, could you fill some background in? | 18:20 |
| Odd_Bloke | (Actually gone this time. :p) | 18:20 |
| chillysurfer | Odd_Bloke: no problem :) feel free to wait until you get back to respond | 18:21 |
| chillysurfer | basically there is an interface that cloud-init is trying to configure that shouldn't be configured | 18:32 |
| chillysurfer | and i know with netplan it is easy to see the config and what interfaces that it will try to configure | 18:33 |
| chillysurfer | doesn't seem so straightforward to find that info here | 18:33 |
| === ijohnson is now known as ijohnson|lunch | ||
| blackboxsw | falcojr: Odd_Bloke what's our integration test answer through pycloudlib for instance.pull_file on files that require root perms? | 19:24 |
| blackboxsw | I can instance.execute("sudo cat /rooot/blah") and harvest things that way, but WDYT about just supporting a use_sudo flag for that ? | 19:25 |
| blackboxsw | I can instance.execute("sudo cat /rooot/blah") and harvest things that way, but WDYT about just supporting a use_sudo flag for pull_file ? | 19:25 |
| falcojr | I thought we solved that already...lemme check | 19:25 |
| blackboxsw | ahh there is a use_sudoe on execute() | 19:25 |
| blackboxsw | but not on pull_file | 19:26 |
| falcojr | blackboxsw: is there something not working? It should work as is for root owned files | 19:27 |
| blackboxsw | I'm getting EACCESS errors from paramiko File "/root/ubuntu-advantage-client/.tox/behave-gcppro-18.04/lib/python3.8/site-packages/paramiko/sftp_client.py", line 896, in _convert_status | 19:28 |
| blackboxsw | raise IOError(errno.EACCES, text) | 19:28 |
| blackboxsw | since we just use the sfpt.get() method | 19:28 |
| blackboxsw | since we just use the sfp.get() method | 19:28 |
| blackboxsw | bah you know what I mean | 19:29 |
| falcojr | I do XD | 19:29 |
| falcojr | from an integration test? | 19:29 |
| blackboxsw | yeah falcojr new integration tests on ua-client-side of the house. Since we are trying to capture root-read-only files, the ssh ubuntu user doesn't have access to them | 19:30 |
| falcojr | oh, so you're talking about changing things in pycloudlib? | 19:32 |
| blackboxsw | yet I don't see a perm upgrade option within paramiko to "sftp_client.get()" maybe pycloudlib instance.pull_file could either accept a use_sudo param and prefer inst.exectute(["cat blah", use_sudo=True) or try sftp.get() and fallback on EACCESS to use sudo | 19:32 |
| blackboxsw | yeah I'm talking about change pycloudlib to support pull_file for files that are root-read-only | 19:33 |
| falcojr | yeah, we did the tmp copy thing in cloud-init https://github.com/canonical/cloud-init/blob/master/tests/integration_tests/instances.py#L72 | 19:35 |
| blackboxsw | falcojr: ahh right and cloud-init's integration tests are use_sudo=True for all execute calls | 19:36 |
| blackboxsw | it seems bold that we are doing that for every pull_file command in cloud-init's integration tests given that we may only want sudo for a few file pulls | 19:37 |
| blackboxsw | but, again, it means that under the hood, we want pull_file to just succeed. | 19:38 |
| falcojr | that and most files we actually care about are owned by root | 19:38 |
| falcojr | you could add something similar in pycloudlib, but you'd probably need to handle permissions off the bat | 19:39 |
| falcojr | in cloud-init we're ignoring permissions until we can't anymore :P | 19:39 |
| blackboxsw | "handle permissions off the bat" do you mean exception handling on permissions issues assume non-root access but handle root-readonly files ? | 19:40 |
| falcojr | I mean setting/preserving proper permissinos | 19:41 |
| Odd_Bloke | If you want to test if the default user has access to a file, `assert client.execute("test -r {}".format(filename))` is more accurately expressive of the check than `with pytest.raises(IOError): client.pull_file(filename)` (and will have fewer false positives, as IOError is likely used by paramiko to indicate other error conditions too). If you don't want to test that, then I don't know that it matters | 20:17 |
| Odd_Bloke | what UID is used to read the file (and so root makes sense, as it can read ~everything). | 20:17 |
| Odd_Bloke | chillysurfer: OK, so if you look for "pplying net" in cloud-init.log, you should see the network configuration that cloud-init has found logged. | 20:19 |
| Odd_Bloke | (One log spelling uses Applying, one uses applying. >.<) | 20:19 |
| Odd_Bloke | chillysurfer: If looking at that yourself doesn't help, feel free to paste it and point us at it so we can discuss the issue more concretely. :) | 20:21 |
| chillysurfer | Odd_Bloke: no that makes total sense. so if bringup is set to False then that means cloud-init will not try to bring the interfaces up then right? | 20:24 |
| === ijohnson|lunch is now known as ijohnson | ||
| Odd_Bloke | chillysurfer: That only affects whether the cloud-init code will bring them up; if they're configured in a way that means the _system_ will bring them up, they'll still come up at that juncture. | 20:56 |
| chillysurfer | Odd_Bloke: thanks!! | 21:20 |
| Odd_Bloke | :) | 21:29 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!