| goten | script placed in /etc/cloud/cloud.cfg.d/my_script.cfg is not executed during boot up. Did not find any useful info in /var/log/cloud-init.log, is other places I can look for more info? Or what is the best way to troubleshoot this? | 05:34 |
|---|---|---|
| === smoser changed the topic of #cloud-init to: reviews: http://bit.ly/ci-reviews | mail: http://bit.ly/ci-mail-archive | docs: http://cloudinit.readthedocs.io/en/latest/ | next meeting Tue, 12 Sep 2017 16:30:00 +0000 | ||
| powersj | just to have them in here: these are my two MPs for KVM | 15:17 |
| powersj | commands as strings: https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/330535 | 15:17 |
| powersj | add xkvm: https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/330536 | 15:17 |
| dpb1 | xkvm? | 15:32 |
| powersj | wrapper around qemu command | 15:44 |
| powersj | rharper/smoser created for curtin's use | 15:44 |
| nacc | since qemu's parameters make little/no sense to most people, dpb1 :) | 15:45 |
| dpb1 | people who aren't cpaelzer | 15:45 |
| dpb1 | powersj: TIL, thx | 15:46 |
| smoser | rharper, https://blueprints.launchpad.net/nova/+spec/virt-device-tagged-attach-detach | 16:05 |
| smoser | that is what i think i remember. | 16:06 |
| rharper | \o/ | 16:06 |
| === smoser changed the topic of #cloud-init to: reviews: http://bit.ly/ci-reviews | mail: http://bit.ly/ci-mail-archive | docs: http://cloudinit.readthedocs.io/en/latest/ | next meeting Mon, 18 Sep 2017 16:00:00 +0000 | ||
| rharper | smoser: in that spec, it mentions the qemu firmware exposing metadata path; we have support for that now, right? the nocloud from smbios? do you think that's the same? | 17:33 |
| rharper | "Outside the scope of the Nova work, a simple tool will be created that can parse this metadata file and set tags against devices in the udev database. It is anticipated that cloud-init would trigger this tool. " | 17:34 |
| smoser | rharper, ? | 17:49 |
| smoser | i dont follow | 17:49 |
| rharper | I know we added some dmi path to getting datasource, or user-data, right ? | 17:50 |
| rharper | the DMI NoCloud mode ? | 17:50 |
| rharper | I was wondering if that was what the spec was referring to w.r.t qemu exposing metadata via firmware to instances; the spec has a general concern about providing network-based metadata to instances which may not have network configuration (the writers were unaware of how cloud-init handles this in other network-based datasources) | 17:51 |
| rharper | and the second quote was just an interesting tidbit; the use-case of 'tagging' devices for service discovery (sdc is for 'oracledb') and wanted to provide a more "General" way for applications to discover the tagging without forcing the app to learn "openstack device tagging json schema"; and suggested udev tags/attrs as such a method | 17:52 |
| smoser | well, the dmi nocloud stuff is not related to the openstack stuff for sure. | 17:53 |
| rharper | but it | 17:53 |
| rharper | but it was provided by qemu (aka firmware) | 17:54 |
| smoser | we just now support reading nocloud info from dmi | 17:54 |
| smoser | yeah | 17:54 |
| rharper | I wonder if it could be a ConfigDrive style support if they could stuff enough info in there | 17:54 |
| rharper | I guess it's only useful during boot; | 17:55 |
| rharper | hence NoCloud; I don't suppose qemu has an interface to update values in the dmi table dynamically | 17:55 |
| smoser | powersj, you just replaced my mp ? | 17:56 |
| smoser | https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/330459 -> https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/330535 | 17:57 |
| powersj | Based on your comment it seemed like you wanted a seperate MP. So we can merge yours and then put mine on top to change the other strings. | 17:57 |
| smoser | oh i see you added some of the string changes too | 17:57 |
| powersj | yeah | 17:57 |
| smoser | powersj, i'm sorry. | 18:34 |
| powersj | ? | 18:34 |
| powersj | smoser: I assume the sorry is for the review? | 18:38 |
| smoser | yeah | 18:42 |
| powersj | smoser: for KVM when we execute SSH expects a string, and mount-image-callback expect a string | 18:50 |
| powersj | should I then do the opposite as LXD, in that if it detects it is a of type array, convert it to a string? | 18:50 |
| smoser | powersj, mount-image-callback doesn't expect a string. it has a sane interface. | 19:08 |
| smoser | ssh interface sucks | 19:08 |
| smoser | it shoves you through sh (or rather the user's shell) more for security reasons than anything else. | 19:09 |
| smoser | that way the platform can manage what your user can do by setting the shell differently. | 19:09 |
| powersj | smoser: the way I was calling MIC https://paste.ubuntu.com/25522742/ | 19:10 |
| smoser | to fix ssh, you can do this: | 19:10 |
| smoser | https://gist.github.com/smoser/88a5a77ab0debf268b945d46314ea447 | 19:10 |
| smoser | you have to be creative. | 19:10 |
| smoser | (that does insert a dependency on 'base64', which i dont have a good way to work around) | 19:10 |
| powersj | sigh | 19:12 |
| powersj | smoser: I guess I'm not sure I understand why the usage of the python library doesn't work | 19:13 |
| smoser | that is basically what we use in tools/run-centos : inside_as | 19:14 |
| smoser | what python library ? i'ms orry. | 19:14 |
| powersj | the integration tests are in python | 19:14 |
| powersj | I am using paramiko to call ssh | 19:14 |
| powersj | from what you are saying you want me to get rid of that to call your ussh shell script | 19:14 |
| smoser | powersj, no. using paramiko is fine | 19:24 |
| smoser | we just wrap it sanely. | 19:24 |
| powersj | smoser: define wrapping sanely :) outside of your base64 encoded strings this is a new to me | 19:26 |
| smoser | :) | 19:26 |
| smoser | i want to provide an interface '.execute()' that takes a list. not a string. | 19:26 |
| smoser | (just like subprocess.call() does... its not a crazy idea) | 19:27 |
| dpb1 | powersj: you are familiar with pep8, right? | 19:27 |
| dpb1 | the pep, not the tool | 19:27 |
| powersj | dpb1: of course | 19:27 |
| dpb1 | ok | 19:27 |
| powersj | smoser: you are choosing to pass a list to replicate the behavior or because you feel passing the list is a more design decision? | 19:31 |
| powersj | is a better design decision* | 19:31 |
| smoser | i think it is a better design decision. because in the end for doing anything complex you fight the shell more than it helps you. | 19:45 |
| smoser | and... as shown if you want to invoke a shell, that is easy. | 19:45 |
| powersj | smoser: ok that is fair and makes sense when using subprocess calls, what I am trying to grawk is how to take that decision and apply it to my use cases. | 19:47 |
| powersj | 1) the use of the python ssh library which expects a string | 19:48 |
| powersj | 2) how I call MIC: https://paste.ubuntu.com/25522742/ | 19:48 |
| smoser | well you can just call mic with a list | 19:49 |
| smoser | mount-image-callback image -- command | 19:49 |
| smoser | mount-image-callback image -- command arg1 arg2 | 19:49 |
| smoser | so 2 is pretty easy | 19:49 |
| powersj | well, that's not so clear to me | 19:50 |
| powersj | I can't append a list to it, so I have to process the list more than simply ' '.join(list) right? | 19:50 |
| powersj | or append the list to the whole command, meaning something like: ['mic', 'image', '--'] + command | 19:51 |
| smoser | powersj, right. | 20:05 |
| powersj | smoser: https://paste.ubuntu.com/25523087/ that is where I get stuck | 20:08 |
| smoser | your /bin/sh is the problem there | 20:10 |
| smoser | chroot _MOUNTPOINT_ /bin/sh dpkg-query | 20:10 |
| smoser | you're telling /bin/sh to execute something named 'dpkg-query' | 20:10 |
| smoser | which is not a file | 20:10 |
| powersj | and sure enough that works | 20:11 |
| smoser | so you just have to say: | 20:11 |
| smoser | if command_is_a_string: | 20:11 |
| smoser | command = ["sh", "-c", command] | 20:12 |
| smoser | where 'command_is_a_string' is syntax to do that idea | 20:12 |
| powersj | ok, do you even want to allow strings at this point then? | 20:12 |
| smoser | we can allow them. | 20:13 |
| powersj | you mentioned in only certain places in the last review | 20:13 |
| smoser | its just that they're scary to me | 20:13 |
| powersj | "lets only fix strings where we were already using sh (via sh -c)." | 20:13 |
| smoser | right | 20:13 |
| powersj | so if using sh -c --> convert to string and remove sh -c from it | 20:13 |
| smoser | so the places where we *were* doing 'sh', '-c' | 20:13 |
| smoser | it makes sense to just drop the sh -c | 20:14 |
| smoser | and pass the thing in as a string | 20:14 |
| smoser | and let the "do the right thing for strings" logic take over | 20:14 |
| powersj | if we were not doing that, then use as a list | 20:14 |
| smoser | yeah. | 20:14 |
| smoser | the one you have theere in that paste has a bug. | 20:14 |
| smoser | --showformat='${Version}' | 20:14 |
| smoser | we really wanted | 20:14 |
| smoser | --showformat=${Version} | 20:14 |
| powersj | I'm not so sure those quotes are a bug ;) | 20:15 |
| powersj | that is getting the version to print it in the logs | 20:15 |
| smoser | its getting the version with '' around it | 20:15 |
| powersj | ok so yes not part of the version | 20:15 |
| powersj | but made the output look nice ;) | 20:15 |
| powersj | ok one more question to let me get going on this | 20:16 |
| powersj | you said this command is easily fixed into sh | 20:16 |
| powersj | cmd = ('for ((i=0;i<{time};i++)); do {test} && exit 0; sleep 1; done; exit 1;' | 20:16 |
| smoser | the for() is bash syntax. | 20:20 |
| smoser | cmd = ('i=0; while [ $i -lt {time} ] && i=$(($i+1)); do {test} && exit 0; sleep 1; done') | 20:21 |
| powersj | ah | 20:21 |
| smoser | exit 1 at the end. | 20:21 |
| powersj | ok I'll get these into a new merge for you to look at in the morning | 20:21 |
| powersj | then we need to talk about SSH | 20:21 |
| smoser | yeah. | 20:22 |
| smoser | it is doable | 20:22 |
| powersj | smoser: thx | 20:23 |
| smoser | powersj, http://paste.ubuntu.com/25523267/ | 20:45 |
| powersj | smoser: interesting | 20:50 |
| smoser | http://paste.ubuntu.com/25523299/ | 20:51 |
| smoser | powersj, so whatever you were going to pass to paramiko to execute | 20:51 |
| smoser | just pass paramiko.execute(shell_pack(cmd)) | 20:52 |
| powersj | smoser: but I can't pass a list | 20:52 |
| powersj | ;) so convert list to string? | 20:52 |
| powersj | or add another isinstance(cmd, list) and run that | 20:53 |
| smoser | you can pass shell_pack() a list or a string. or you can change it to only take a list | 20:53 |
| smoser | and then do the isinstance business elsewhere. | 20:53 |
| smoser | to be nice, we can wrap paramiko.execute() with a try/except and then lie about the commadn that we executed | 20:56 |
| smoser | so that the command that is printed looks like what was executed, rather than | 20:56 |
| smoser | eval set -- "$(echo J2VjaG8nICcoIGhpIHBvd2Vyc2ogKSc= | base64 --decode)" && exec "$@" | 20:57 |
| powersj | heh yeah... | 20:59 |
| dpb1 | bad memories there | 21:01 |
| powersj | one of my biggest complaints is debugging failed commands, especially when they look like lists | 21:01 |
| dpb1 | in my last last job, we had a windows service that did something like this on every incoming ssh attempt. | 21:01 |
| powersj | so to get a base64 encoding string is even worse | 21:01 |
| powersj | haha | 21:02 |
| dpb1 | in our case, it was an improvment | 21:02 |
| dpb1 | at least then you could cut and paste a string out | 21:02 |
| dpb1 | instead of a random mix of ASCII and non-ascii characters that windows programs would decide to show you | 21:02 |
| dpb1 | especially quotes | 21:02 |
| smoser | http://docs.paramiko.org/en/2.2/api/client.html | 21:04 |
| smoser | that is odd | 21:04 |
| smoser | it returns a file-like object representing stdin, stdout, stderr | 21:04 |
| smoser | how does it return stdin ? | 21:04 |
| rharper | smoser: https://bugs.launchpad.net/cloud-init/+bug/1716773 | 21:08 |
| ubot5 | Ubuntu bug 1716773 in cloud-init "cloud-init doesn't cache network_config property in cache" [Undecided,New] | 21:08 |
| * powersj loves the faster unit tests | 21:24 | |
| powersj | thx blackboxsw | 21:24 |
| blackboxsw | hopefully as we get more that continues to be the case | 21:25 |
| powersj | smoser: should the shell pack be a separate merge? | 21:44 |
| powersj | working quite nice so far | 21:44 |
| powersj | smoser: string merge is ready https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/330535 | 21:51 |
| === dhill__ is now known as dhill_ | ||
| blackboxsw | grr smoser https://bugs.launchpad.net/cloud-init/+bug/1691489 I just saw this on zesty on my nth redeployment iteration with our SRU candidate | 22:25 |
| ubot5 | Ubuntu bug 1691489 in cloud-init (Ubuntu Yakkety) "fstab entries written by cloud-config may not be mounted" [Medium,Confirmed] | 22:25 |
| blackboxsw | ubuntu@zesty1:~$ grep reformat /var/log/cloud-init.log 2017-09-12 22:23:52,316 - DataSourceAzure.py[DEBUG]: reformattable=True: partition 1 (/dev/sdb1) on device /dev/disk/cloud/azure_resource was ntfs formatted and had no important files. Safe for reformatting. | 22:25 |
| blackboxsw | this message came up with a reformated disk on an instance that I had already created | 22:25 |
| blackboxsw | cloud-init0.7.9-233-ge586fe35-0ubuntu1~17.04.1 | 22:25 |
| blackboxsw | hrm reproduced multiple times on zesty. not sure what we do w/ that bug then. the "fix" didn't fully resolve it, but it didn't break cloud-init either. | 22:34 |
| blackboxsw | ahh nevermind. even though reformatting happens /mnt is still formatted an no errors in cloud-init.log (per the original bug) | 22:41 |
| blackboxsw | I was taking the ephemeral reformat log as a symptom of failure, but it's an ephemeral disk, reformatting it is okay. Not mounting it is not okay | 22:42 |
| blackboxsw | ok gotta run | 22:42 |
| === caphrim007 is now known as caphrim007_ | ||
| === caphrim007_ is now known as caphrim00_ | ||
| === caphrim00_ is now known as caphrim007 | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!