| basilAB | I was trying to use cloud-init (version: 0.7.5) on ubuntu-14.04. I am finding issue in appending ssh key to authorized from 'OpenStack' metadata. But it is working with 'ConfigDrive' | 00:55 |
|---|---|---|
| basilAB | https://pastebin.com/0JPwU8fK | 00:56 |
| robjo | smoser: Can take another look at https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/329616 and merge or let me know what else needs fixing? | 11:58 |
| niluje | :< | 13:36 |
| smoser | whats up niluje | 13:46 |
| smoser | robjo, ... looking now | 13:47 |
| smoser | basilAB, is that your holw /var/log/cloud-init.log ? | 13:48 |
| basilAB | smoser: yes. 'grep' with 'failed' | 13:49 |
| basilAB | my host has python2.7 and python 3 on the host and can see all the modules called under python2.7 | 13:50 |
| afranc | When does cloud-init detect payload as `text/x-not-multipart` ? | 14:33 |
| afranc | I passing a shell script as a user data but when i start some extra commands, the ShellScriptPartHandler detects the content as `text/x-not-multipart` ending it up as an `Empty payload`, logs - https://pastebin.com/raw/sd32FqJw | 14:39 |
| smoser | fun | 14:41 |
| smoser | https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/329926 | 14:41 |
| smoser | robjo, i responded in mp | 14:41 |
| smoser | basilAB, can you paste the hole log ? | 14:42 |
| robjo | smoser: thanks will get back to it today | 14:42 |
| smoser | blackboxsw, see merge above. powersj interested in your feedback there too | 14:42 |
| smoser | afranc, what was the problem ?. can you post an entire log and tell me what you expected ot happen and what didn't? | 14:43 |
| blackboxsw | grabbing it now | 14:43 |
| powersj | smoser: ah yes I saw this fail recently on my artful box | 14:45 |
| smoser | i think that the MP above is really all we can do, but that means somehow we have to have jenkins test runners have python3.5 | 14:45 |
| smoser | as i really want c-i to report failures that would be seen in xenial | 14:46 |
| smoser | i guess alternatively we could un-pin jsonpatch | 14:46 |
| smoser | or pin at a newer (artful) version and just live with the risk of differences | 14:47 |
| basilAB | smoser: Here is the full log https://paste.ubuntu.com/25432350/ (The problem which I was checking was to see why ssh key available in the metadata is not getting appended to ubuntu user) | 14:50 |
| basilAB | I can see the user getting created | 14:50 |
| smoser | 2017-08-30 14:45:00,079 - cc_final_message.py[WARNING]: Used fallback datasource | 14:52 |
| smoser | that means it did not find a metadata service to use. | 14:52 |
| afranc | smoser i have posted both user-data and cloud-init.log which is failing - https://pastebin.com/raw/zEtB6XDf | 14:52 |
| blackboxsw | smoser: per your branch on xenial is it worth us adding a python3.6 tox env to better exercise 3.6 issues? | 14:53 |
| smoser | basilAB, your networking seems to not have come up. in 14.04 you have to have an /etc/network/interfaces that defines basically "dhcp on eth0". | 14:54 |
| smoser | cloud-init assumes the systme is set up to configure networking. | 14:54 |
| afranc | smoser if i have a user-data like this https://pastebin.com/raw/wiDx027d, the ShellScriptPartHandler detects the content as `text/x-shellscript` writes it to `/var/lib/cloud/instance/scripts/part-001` and executes it fine | 14:54 |
| smoser | blackboxsw, well that becomes hard to run then from xenial | 14:54 |
| afranc | smoser but if the user-data has additional commands like in https://pastebin.com/raw/zEtB6XDf, the `ShellScriptPartHandler` handler detects it as `text/x-not-multipart` and complains it as `Empty payload of type text/x-not-multipart` and writes it as `Writing to /var/lib/cloud/instances/af23b054-f125-4821-8639-084933ad7244/cloud-config.txt - wb: [384] 0 bytes` and do not execute the entire shell script from user-data | 14:56 |
| afranc | smoser i wanted to know why adding extra commands, make the `ShellScriptPartHandler` detect the content as `text/x-not-multipar` | 14:58 |
| -basilAB:#cloud-init- I have'source /etc/network/interfaces.d/*.cfg' in '/etc/network/interfaces . Interface configuration: https://paste.ubuntu.com/25432392/ | 14:58 | |
| smoser | afranc, i suspect you might not have an 'eth0' device? | 14:59 |
| smoser | something made networking not come up | 14:59 |
| smoser | er.. sorry. not afranc above, but basilAB . | 14:59 |
| afranc | ok | 14:59 |
| basilAB | hmm.. I do see eth0 up with an IP assigned from DHCP after cloud-init run. Are you saying it is a timing issue ? | 15:01 |
| smoser | afranc, nothign created the user 'ubuntu' i think | 15:01 |
| smoser | afranc, what image are you using ? | 15:01 |
| afranc | smoser Ubuntu 14.04.2 LTS | 15:02 |
| smoser | from cloud-images.ubuntu.com ? | 15:02 |
| smoser | or did you build one | 15:02 |
| afranc | smoser, i build one . | 15:03 |
| afranc | smoser i am confused because at some cases shell script as user-data works | 15:03 |
| smoser | afranc, well, cloud-init failed to find the datasource. | 15:18 |
| smoser | so it didn't get your user-data | 15:18 |
| afranc | smoser doesnt this confirm `2017-08-30 14:48:06,025 - util.py[DEBUG]: Crawl of openstack metadata service took 36.475 seconds` | 15:22 |
| afranc | and this log - https://pastebin.com/raw/v4215hr8 is a working case using the same cloud.cfg | 15:22 |
| smoser | afranc, i'im confused a bit on what happened there. | 15:26 |
| smoser | bouncing the networking is wierd. 14.04 doesn't do networking that well. i do not think that the format of the user-data is your problem. | 15:27 |
| afranc | smoser this is a log from file from a working use case - https://pastebin.com/raw/wh2yyfQZ | 15:29 |
| smoser | afranc, well, a log of *only* that boot would be good too | 15:29 |
| smoser | the one you posted has multiple boots | 15:29 |
| smoser | and i'd really suggest sanity checking *anything* against an official ubuntu image. | 15:30 |
| afranc | this is another log file from non-working use case - https://pastebin.com/raw/UESQGp7S | 15:30 |
| afranc | ok | 15:30 |
| afranc | i was only seeing the difference in the way the `ShellScriptPartHandler` detecting the user-data | 15:30 |
| afranc | for a working use case it detects the user-data as `text/x-shellscript` and the case where it doesn't work, it detects it as `text/x-not-multipart'` | 15:31 |
| afranc | ok. i will do the sanity check on the image | 15:32 |
| smoser | afranc, possibly you have a space at the beginning of it ? | 15:35 |
| smoser | it only will read the first line. and expects it shoudl start with '#!' to be a shellscript | 15:36 |
| afranc | i see. i will double check | 15:37 |
| afranc | how does it detect the content as `text/x-not-multipart` ? | 15:37 |
| smoser | it doesnt know what it is. thats just fallback. | 15:48 |
| afranc | i see | 15:54 |
| smoser | rharper, fwiw, i think i agree with xnox's mp for upstart in artful. | 15:55 |
| smoser | he doesnt delete the upstart/ jobs just removes the packaging of them in artful | 15:55 |
| smoser | and if all other ubuntu packages have done that, then cloud-init in ubuntu shoudl too. | 15:55 |
| rharper | smoser: ok | 15:56 |
| afranc | thanks smoser i will dig this further | 15:57 |
| rharper | smoser: IIUC, when there is an artful+1; it will get branched from master, so we'll need to re-apply the same MP to that from master; no? So, it seemed to me that upstream should need some changes so that a downstream would opt-in to those files being packaged; I dunno | 15:58 |
| smoser | no. master has no debian/ | 15:58 |
| smoser | it has packages/debian | 15:59 |
| smoser | and packages/debian/rules *does* have 'upstart' in it. but we can drop that. it doesnt really hurt anything. | 15:59 |
| smoser | basically the plan is that ubuntu/devel branch will always go on being ubuntu/devel (right now that is artful) | 16:00 |
| rharper | also, downstream branches also have a cloud.cfg that'll have the emit upstart and other upstart related runtime files | 16:00 |
| smoser | the first time we need to SRU to artful, we create ubuntu/artful branch. (or we can do taht whenever artful releases... doesnt really m atter). | 16:00 |
| rharper | generally I was hoping that we'd comb through master to find out what a 'non-upstart' code base would need to look like | 16:00 |
| blackboxsw | rharper: were you referring to this locale branch in standup today? https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/329152 | 16:06 |
| rharper | yeah | 16:06 |
| blackboxsw | that you mentioned you had updated yesterday? | 16:06 |
| rharper | y | 16:07 |
| blackboxsw | last commit pushed was 08-16 | 16:07 |
| rharper | can't be | 16:07 |
| blackboxsw | I don't see the changes that were discussed between you and smoser | 16:07 |
| blackboxsw | hmm refreshing | 16:07 |
| blackboxsw | maybe launchpad is falling apart. I'll try a clone and see if I see changes | 16:08 |
| rharper | ok | 16:08 |
| rharper | it looks fine to me in lp | 16:08 |
| rharper | maybe because of rebase? I dunno | 16:09 |
| nacc | commit authorship date != push date | 16:09 |
| nacc | and if you do amending, etc. the date won't necessarily change | 16:09 |
| rharper | nacc: thanks | 16:09 |
| nacc | i think LP uses author date, rather than commit date (or v.v). I can't recall | 16:10 |
| nacc | but i've seen similar artifacts | 16:10 |
| blackboxsw | hmm I'm only seeing cf1c4106545b6acceded30ae9f3f65910277335e | 16:10 |
| rharper | cf1c4106545b6acceded30ae9f3f65910277335e is where it's at | 16:11 |
| rharper | blackboxsw: see nacc's comment | 16:11 |
| nacc | blackboxsw: you could try and look at `git log --pretty=full` to see the commit and authorship dates both | 16:12 |
| nacc | i think that's the argument | 16:12 |
| nacc | oh 'fuller' | 16:12 |
| blackboxsw | nacc +1 on that. | 16:13 |
| blackboxsw | hmm. rharper I thought you made references to authorship changes you made yesterday. I was expecting to see some changes that we determine needs_regen value based on a call to 'locale -a' | 16:18 |
| blackboxsw | maybe I misunderstood from standup. Was your intent to avoid the extra exec in querying whether the local was already generated? | 16:19 |
| rharper | blackboxsw: we're not querying what's present via locale -a; rather the change was to read the distro default locale config file, and if that has a set value, to assume the image is built correctly and not regenerate then | 16:21 |
| rharper | it handles if the file is present but has no value, or if missing, and will regen as needed there | 16:21 |
| blackboxsw | ok gotcha, thanks for the correction. ok, makes sense to rely on images to be properly built | 16:22 |
| rharper | I think the path we took was that the image builder should be in control of what the default lang for the image is, and declare that in the correct distro location;, cloud-init will read that and respect it; if an image is built with locales but left unset, then cloud-init will regenerate the cloud-init default locale | 16:22 |
| rharper | blackboxsw: right | 16:22 |
| rharper | my most recent change was the keep around the value present in the default locale config file and have that be separate from the cloud-init default/fallback value so we can reason on whether we need to update the locale config file, as well as regenerate | 16:23 |
| blackboxsw | gotcha, and you must've squashed your commits then | 16:26 |
| blackboxsw | which is why I didn't see updates | 16:26 |
| blackboxsw | ok | 16:26 |
| rharper | blackboxsw: I did | 16:29 |
| rharper | one of the reasons smoser liked bzr (-n 0 would show you the evolution of the code) | 16:29 |
| dpb1 | cloud-init summit notes posted: https://lists.launchpad.net/cloud-init/msg00094.html | 16:35 |
| smoser | man. look at that active mailing list | 16:35 |
| smoser | 3 messages in 2 days! | 16:36 |
| * smoser just sent https://lists.launchpad.net/cloud-init/msg00095.html | 16:36 | |
| dpb1 | smoser: I almost couldn't find the archive link! | 16:36 |
| dpb1 | smoser: can I modify the review to be a bit.ly link? | 16:36 |
| dpb1 | (in topic) | 16:36 |
| Odd_Bloke | unsubscribe | 16:38 |
| dpb1 | Odd_Bloke: you always have to be the center of attention | 16:38 |
| smoser | dpb1, i will. | 16:40 |
| Odd_Bloke | dpb1: I find it's better for everyone. | 16:41 |
| smoser | Odd_Bloke, is that if you're the center of attention or if you leave the playground (unsubscribe) | 16:42 |
| dpb1 | smoser: thx, can we also put up a bitly for the email list? like, | mail list: http://pad.lv/~cloud-init | | 16:43 |
| === smoser changed the topic of #cloud-init to: reviews: http://bit.ly/ci-reviews | mail: http://bit.ly/ci-mail-archive | docs: http://cloudinit.readthedocs.io/en/latest/ | next meeting Tue, 05 Sep 2017 16:30:00 +0000 | ||
| smoser | how's that? | 16:45 |
| dpb1 | nice | 16:46 |
| Odd_Bloke | smoser: Little bit of column A, little bit of column B. | 16:46 |
| dpb1 | smoser: wish there was a way to inject text into this page: https://code.launchpad.net/~cloud-init-dev/cloud-init/+git/cloud-init/+ref/master/+activereviews | 16:47 |
| dpb1 | smoser: like describing what it meant for something to be wip | 16:47 |
| dpb1 | smoser: but, oh well | 16:47 |
| nacc | dpb1: per project? yeah, i agree | 16:47 |
| dpb1 | (going back to what we were discussing earlier about WIP branches) | 16:47 |
| nacc | dpb1: i'd file a bug on launchpad | 16:48 |
| nacc | tbh | 16:48 |
| dpb1 | nacc: you have the same ask for #ubuntu-server stuff too, right? | 16:48 |
| nacc | dpb1: yeah, and i imagine any team would like to put 'policy' somewhere that's well-defined and have LP display it | 16:48 |
| dpb1 | nacc: ok, I will no sense complaining about it in #cloud-init channel | 16:48 |
| dpb1 | lol | 16:49 |
| nacc | if they are using git on LP, i mean | 16:49 |
| dpb1 | thx, good suggestion | 16:49 |
| blackboxsw | rharper: approved with nits https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/329152 | 16:51 |
| robjo | smoser: https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/329616 should be all set now | 16:56 |
| blackboxsw | https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/329946 looks good. a doc nit and and a question for you smoser | 17:22 |
| blackboxsw | nice robjo, I'll glance over that too today to catch up on opensuse context. | 17:22 |
| smoser | blackboxsw, please do. i am basically good with it at this point. the tests bare minimum, but better than there *was* so i didnt want to tax robjo wit their creation. | 17:23 |
| blackboxsw | +1 landing it smoser don't wait on me for that :). but yeah I had missed his branch earlier so wanted to read it a bit more | 17:24 |
| smoser | blackboxsw, responded | 17:28 |
| smoser | https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/329946 | 17:28 |
| smoser | and updated with doc removal. good catch on that . thank you. | 17:28 |
| smoser | rharper, i had a question on your locale mp also | 17:28 |
| rharper | smoser: here | 17:33 |
| smoser | rharper, see mp. | 17:35 |
| rharper | k | 17:35 |
| smoser | you can answer here if you want more real time | 17:35 |
| rharper | smoser: hrm, you're right; I don't really like the idea of reading the same file twice in the default/common path; so either if we write an updated locale conf, we invalid the cached variable, or we just let it always read the file | 17:40 |
| smoser | rharper, oh. i was just suggesting when you successfully 'apply', then you can just set the new value. | 17:42 |
| smoser | but yeah, setting it to None would do the same thing and just force a re-read | 17:42 |
| rharper | ok; that works | 17:42 |
| smoser | yeah. just update it. since you're caching it you are guaranteed to possibly be out of date | 17:42 |
| smoser | so setting it (rather than setting it to None) doesn't make it any worse | 17:43 |
| rharper | right | 17:43 |
| rharper | we're not really protecting against file system writes underneath us | 17:43 |
| rharper | possibly if we were to be daemonized and called a second time to switch locales; | 17:44 |
| rharper | smoser: blackboxsw: thanks for review, working on an update | 17:51 |
| smoser | blackboxsw, so your change at https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/329872 | 17:55 |
| smoser | that does change to doing a 'import oathlib...' every time oauth_headers is called | 17:55 |
| smoser | i guess if you like the clarity it seems the performance is not terribly worse off. | 17:58 |
| smoser | >>> timeit.timeit("bool(oauth1 is None)", setup="import oauthlib.oauth1 as oauth1") | 17:59 |
| smoser | 0.2964163040742278 | 17:59 |
| smoser | versus | 17:59 |
| smoser | >>> timeit.timeit("import oauthlib.oauth1 as oauth1")0.531060776906088 | 17:59 |
| smoser | (default is 1M runs) | 17:59 |
| * rharper wants is 2 tenths of a second back | 18:04 | |
| smoser | powersj, updated https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/329926 | 18:09 |
| smoser | 2 tenths of a second per 1M runs, rharper | 18:09 |
| rharper | the average over 1M runs | 18:09 |
| rharper | no? | 18:09 |
| rharper | or total time? | 18:09 |
| rharper | yeah, you're right | 18:10 |
| rharper | or is that per loop? I'm confused now | 18:10 |
| smoser | that shows total time | 18:10 |
| rharper | python3 -m timeit '"-".join(str(n) for n in range(100))' | 18:10 |
| rharper | 10000 loops, best of 3: 30.2 usec per loop | 18:10 |
| smoser | timeit does | 18:10 |
| rharper | https://docs.python.org/3/library/timeit.html | 18:10 |
| smoser | the python -m might be different. | 18:11 |
| rharper | timeit.timeit('"-".join(str(n) for n in range(100))', number=10000) | 18:11 |
| rharper | 0.3018611848820001 | 18:11 |
| smoser | the above definitely ran 1M imports in .3 seconds | 18:11 |
| rharper | nope | 18:11 |
| rharper | pretty sure that's time per loop | 18:11 |
| smoser | nah. i'll show | 18:11 |
| rharper | I mean, there are words on that page | 18:11 |
| smoser | timeit.timeit("time.sleep(1)", setup="import time", number=3) | 18:12 |
| smoser | 3.0026550928596407 | 18:12 |
| rharper | that is really bizarre that the cli and the library differ ; =( | 18:14 |
| smoser | blackboxsw, rharper please re-review https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/329811 | 19:36 |
| rharper | k | 19:37 |
| smoser | and blackboxsw https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/329811 | 19:38 |
| blackboxsw | re-reading both | 19:39 |
| blackboxsw | hmm that's the same link | 19:39 |
| blackboxsw | and testing mkdtemp on permissions set. | 19:39 |
| smoser | blackboxsw, hm.. second ment to be | 19:40 |
| smoser | https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/329946 | 19:40 |
| smoser | blackboxsw, i will come back in later tonight and will pan on doing an artful upload. | 19:44 |
| blackboxsw | sounds good smoser thx | 19:44 |
| smoser | and https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/329872 is in now. | 19:45 |
| rharper | smoser: blackboxsw: I'll try to have an update to locale branch soon so it can get reviewed prior to smoser's return tonight | 19:51 |
| blackboxsw | smoser: thanks on the land. So, per import attempts inside oauth_headers (rharper too) this is a performance impact to every publish_event webhook in maas right? | 20:35 |
| rharper | ack | 20:35 |
| blackboxsw | roger. ok. | 20:36 |
| rharper | at least I think so | 20:36 |
| rharper | blackboxsw: well, I think we reasoned that 1) it only imports it once 2) systems without ouath pay the most (try to import again) 3) maas systems have oath | 20:36 |
| rharper | so, it should have no per-message impact on ubuntu (which is where the maas reporting with oauth is most used) | 20:37 |
| rharper | non-ubuntu are unlikely to use the MAAS Datasource, so won't pay for the re-import attempt | 20:37 |
| blackboxsw | right, ok that's reasonable. (and given that systems without oauth would break if they tried calling oauth_headers in the first place, we're safe there). | 20:38 |
| rharper | right, pay once and then stacktrace | 20:38 |
| powersj | blackboxsw: FYI https://bugs.launchpad.net/cloud-init/+bug/1714117 | 23:48 |
| ubot5 | Ubuntu bug 1714117 in cloud-init "tox: long CI times" [Undecided,Confirmed] | 23:48 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!