=== vrubiolo1 is now known as vrubiolo
Odd_Blokesmoser: I wonder if you have any thoughts on https://github.com/canonical/cloud-init/pull/369?  I don't love the idea of pushing more info into the log which will be unused in almost every situation.  I am considering suggesting writing out to a separate file (perhaps in /run/?) to avoid this.  What do you think?13:48
Odd_Bloke(Asking because I know you have Opinions about our logging. ^_^)13:49
Odd_Blokemruffell: Of course, thank you for your work!13:49
Odd_Blokemeena: Being able to run CI is a big step up from commit emails IMO, but I agree that a lot of the actual review functionality isn't much different (there's only so many different ways to comment on patches that are ultimately just text file, lol).13:51
lucasmouraHey everyone, does anyone have an idea to manually test this feature ? https://git.launchpad.net/cloud-init/commit/?id=87cd040e13:54
lucasmouraMy understanding on it is that we must fake the endpoint to return 403. But I could not find a way to properly test this13:55
Odd_Blokelucasmoura: I don't see why you would need 403 for that commit (it looks like it should affect all interactions with the AWS IMDS, to me), is that definitely the one you meant to link to?14:01
lucasmouraOdd_Bloke, Nope, my bad here. This is the right commit https://git.launchpad.net/cloud-init/commit/?id=1f860e5a14:02
Odd_Blokelucasmoura: Right, that one makes more sense! :p  Yeah, I think all we can test is the happy path (i.e. we still come up on AWS); we do sometimes ask the clouds to test changes that require messing with their IMDS, so we might be able to do that in this case.14:04
Odd_Blokeblackboxsw: Can you add anything about asking clouds to test changes that require IMDS manipulation?14:05
smoserOdd_Bloke: i had a partial comment from a week ago or so saying something to that effect.14:07
smoserits very easy to have a knee jerk reaction that the log should have shown you exactly the information that you found would have helped out in this very rare scenario14:08
smoseri've recently come more and more to agree with https://dave.cheney.net/2015/11/05/lets-talk-about-logging14:08
smoseri think the real solution is14:09
smosera.) cloud-init log a lot less by default14:09
smoserb.) easily turn on debug logging, which is then a firehose14:09
Odd_BlokeYeah, agreed, I like that as a long-term goal.14:21
Odd_Blokesmoser: So is your position that we shouldn't include this change at all because it's log spam, or that we should put it in a separate file, (or that we should allow it as-is, because it's a drop in the logging pond as things stand)?14:23
smoseri'd say its log spam14:24
smoser(which ... 90% of cloud-init log is, so its a hard position to hold)14:24
smoserhindsight is 20/2014:25
smoserbut i like the separate-file even less than i like the log spam14:25
smoserreally we need to be able to enable debug logs easily14:26
Odd_BlokeOK, fair enough; I was at least thinking that that would go away across boots, but it's not great either.14:26
Odd_BlokeSo I wonder if the thing to do is to accept this one as-is, and then come up with a Plan For Logging which we can add to HACKING.rst and then expect future submissions to follow?14:26
smoserits hard to tell someone "no that can't go in the log". when there is so much crap in the log.14:28
smoserso, mostly i agree with you. let this in, and then future policy.14:28
smoserbut people have wanted to (and have) put stuff in the log as a result of kernel bugs in one release or uninitalized memory usage in a subprogram.14:29
smoserstuff that is so very unlikely to ever have another 'hit' on usefullness14:30
Odd_BlokeSo I guess, at a minimum, we need to reach an agreement on what log levels should be used (and for what), and determine the mechanism(s?) by which you can opt into the firehose of debug messages.14:33
Odd_Bloke(I think we want users to be able to opt-in without image modification, for example, else there'll be a lot of bugs that we can't get enough info on.)14:34
smoseryeah. :-(14:35
smoserwhich leads you to "log a ton of non-important information by default in the event that sometime it might be useful"14:35
smoseri've said this before, but as i do things now, i'm much more strict than i was when cloud-init started.14:36
smoserfail loudly and exit failure.14:36
smoserbecause cloud-init's behavior of "well... stuff might have gone wrong, check the log for WARN messages"14:37
smoserresults in missing *more* bugs14:37
Odd_BlokeYeah, there's definitely a fine line to balance between "break very obviously" and "do enough that people can at least access the instance to debug".14:40
Odd_BlokeBecause if you don't do the latter, then you _need_ to log a bunch of stuff to the serial console to be sure that people will be able to debug the problem well enough.14:41
blackboxswfalcojr: thanks for the note at standup. I'll scrub your sru verification results now too15:30
falcojrcool, thanks15:31
blackboxswOdd_Bloke: lucasmoura agreed we have sometimes asked for assistance from the specific cloud author on certain functionality that we land. In the case of https://git.launchpad.net/cloud-init/commit/?id=1f860e5a the author was fred-lefebvre. i have in the past emailed microsoft folks notifying them that their patch landed and is in testing and that it could be validated by following15:35
blackboxswI mentioned microsoft, but the same applies to aws. If we can notify them of the change in -proposed. they can get a chance to test.  We could choose to highlight @fred-lefebrvre in a comment on the merged PR https://github.com/canonical/cloud-init/pull/216 that it is available for testing (then we are pretty sure the author gets a notification of this)15:38
falcojrpython 3.5 is cool15:57
falcojrit's mad about an unprintable unicode character16:24
falcojrif I try to print the variable in middle log line I get16:24
falcojrUnicodeEncodeError: 'ascii' codec can't encode character '\u03b5' in position 74: ordinal not in range(128)16:24
falcojrnot sure why it's trying to encode ascii when system locale stuff all says utf-8, and if I open a python shell manually I can print it fine16:25
falcojronly happens on xenial though16:25
Odd_BlokeIs it because it's writing to a file that's opened with ascii encoding?16:26
punkgeekIs there any way to detect os (linux or windows) in cloud-init?18:07
powersjpunkgeek, hey - cloud-init does not support Windows at this time. Inside cloud-init we do have various methods for determining which Linux or bsd distro we might be running on18:10
blackboxswpunkgeek: for windows you'd probably be using cloudbase-init instead of cloud-init as they are separate projects.  But in cloud-init upstream let's you detect what linux/bsd distribution you are running on via `cloud-init query distro`   or in jinja templates in your #cloud-config userdata file18:10
powersj^ nice18:10
blackboxswpunkgeek: https://cloudinit.readthedocs.io/en/latest/topics/instancedata.html#using-instance-data shows examples of providing #template: jinja\n#cloud-config18:11
blackboxswin that you can use {{ v1.distro }}      or even the short alias {{ distro }}18:11
blackboxswyeah, `cloud-init query --all` will give you a list of any keys which could be provided in #template: jinja\n#cloud-config user-data18:13
punkgeekAha thank you, what differences between cloudinit and cloudbase?18:13
blackboxswpunkgeek: my understanding is https://github.com/cloudbase/cloudbase-init   is windows-only and only supports a subset of the config modules that cloud-init upstream supports https://cloudinit.readthedocs.io/en/latest/topics/modules.html18:16
blackboxswthis channel is related to upstream cloud-init, which as powersj mentioned. does not support windows.18:16
blackboxswfrom their splashscreen https://cloudbase.it/cloudbase-init/  it looks like they have plans to support bsd at some point, but generally very windows focused.18:18
blackboxsw+ looks like they support 4 datasources(cloud platforms) instead of upstream cloud-init's 21 https://cloudinit.readthedocs.io/en/latest/topics/datasources.html18:19
punkgeekThank you so much18:20
blackboxswno worries18:20
blackboxswfalcojr: thanks for https://github.com/cloud-init/ubuntu-sru/pull/107#pullrequestreview-428284569   2 minor comments and we can land it18:31
punkgeekI want to deploy a shell script into the machine that does not have internet connectio, what is the best way? Here is my shell shell script: https://github.com/autovmnet/tools/blob/master/vm_config.sh18:39
lucasmouraHi everyone, I am trying to manually test this PR https://github.com/canonical/cloud-init/pull/234, but I am still experiencing issues with it, so I think I am missing something18:43
lucasmouraI am trying to replicate the exact same case that rharper used in the discussion18:43
lucasmouraAlthough I can see the error he describe when running cloud-init, I see another error when updating for the fixed version18:44
lucasmouraIt states the following error: Exec format error. Missing #! in script?'18:46
lucasmouraWhich makes sense, sice the example in the PR is not a shell script per se. So does the example in this PR supposed to work ? The one rharper uses to reproduce the error with a lxc container18:47
meenaOdd_Bloke: you should rebase https://github.com/canonical/cloud-init/pull/391 and… we should merge it, or i should start sending you patches19:03
Odd_Blokemeena: There are some comments on there that I need to read through and address, and unfortunately it's dropped down my list (for now). :(19:28
rharperlucasmoura: here, what's not working ?19:35
rharperlucasmoura: if you have a newer cloud-init with the fix, when cloud-init processes the mime-type of the payload (which is a cloud-config, not a shell-script) then it will get merged correctly into user-data, *instead* of being thought as a shell-script; which as you see, it isn't  and it cannot be executed19:36
rharperlucasmoura: for verifying that;  can you reproduce the failure with the steps in https://github.com/canonical/cloud-init/pull/234#issuecomment-604033345  ?19:36
lucasmourarharper, yes, I can reproduce the error perfectly. But the problem is that an error still happens when I try to run the same user-data on the newer cloud-init version19:38
lucasmouraJust give me a couple of minutes and I will add the script I am using and the error I am receiving19:38
lucasmouraI think it will be easier to explain the issue19:38
rharperlucasmoura: are you using the lxc-proposed-snapshot to create a new image with the updated cloud-init  ?19:39
lucasmouraNo, but I am manually installing the new version in the lxc container. First I reproduce the error, than I manually add the ppa where the newer cloudinit version is and try to run it again to see if no error is raised19:40
rharperand you run cloud-init clean --logs --reboot ?19:41
lucasmouraOh no, I just run cloud-init init19:41
rharperyou need to clean19:42
rharperotherwise it's not a "first boot" any more19:42
rharperso the already parsed user-data is written out as a shell-script and runparts will see the old file19:42
rharperyou can skip the reboot19:42
rharperbut you do need clean19:42
lucasmouraThat makes total sense19:42
rharperand I suggest; cloud-init clean --logs && cloud-init init --local && cloud-init init19:42
lucasmouraLet me update the script19:43
rharperthe safer way is to always use the lxc-proposed-snapshot;  and run a completely new container19:43
lucasmourathanks powersj19:43
lucasmourarharper, do you mean reproduce the error first and then launch a container with lxc-proposed-snapshot to verify the fix ?19:44
rharperlucasmoura: yeah; https://github.com/cloud-init/ubuntu-sru/pull/100/files19:52
rharperin there, I have a recreate()  and then a verify()19:53
lucasmourarharper, cool. Thanks for the suggestion. I will start using it :)19:56
haybillI am having a cloud-init/netplan networking problem with Ubuntu 20.04.  Cloud-init runs with local cloud-init datasource and sets up the networking configuration (static IP, vlan, etc) without any errors.  Networking is configured but not active after cloud-init completes.  I did notice that cloud-init finds all the Ethernet links down when it runs (Up column in Net device info).  I have discovered that doing a 'netplan apply'20:03
haybillafter cloud-init has run will made the network configuration active.  The Ethernet HW in question is a Intel 10Gb ixgbe interface.  Is this expected behavior? Is my cloud-init config missing something?20:03
rharperhaybill: not expected; couple of things 1) if you can, share you net config, ensuring you've a match on the ixgbe nic in your config to ensure that it's brought up  2) there was a netplan bug around bringing things online without an applyl; though it was related to wifi; might be related;  are you using -daily images or the released image (which will have -updates available) ?20:17
haybillrharper, #2 I am using released 20.04 with a fresh apt update && apt upgrade.  Should I be trying something newer/etc?20:25
rharperhaybill: no, I just new there was a release to -updates for netplan for the wifi issue;20:29
haybillrharper, I am having trouble getting you a sample networking config right now, but when I deploy 18.04 with the same tools networking is fine20:30
rharperhaybill: you have a server install? or using a cloud-image ?20:30
blackboxswfollowup on https://github.com/cloud-init/ubuntu-sru/pull/105#pullrequestreview-428359896 for you falcojr20:33
blackboxswsorry lucasmoura missed the conversation20:33
blackboxswthx rharper20:33
rharperblackboxsw: yw20:35
haybillrharper, it is a Dockerfile generated image (like https://github.com/packethost/packet-images).  Based on this conversation I am beginning to be worried that I am missing some new 20.04 packages for networking20:35
rharperhaybill: ok, are you booting a container ? and passing in the nic ?  or booting a vm ?20:39
rharpereither case, I would suggest testing with the Ubuntu cloud images, https://cloud-images.ubuntu.com/daily/server/focal/current/    there are VM images and root.tar.gz for containers ;  testing with this can help you track down if it's related to your image build or some other issue (or cloud-init/netplan bug)20:40
haybillrharper, thx I will look into the cloud images20:44
lucasmourablackboxsw, no worries20:52
lucasmourarharper, powersj thanks for the help, just confirming that the proposed fix solved the issue20:53
rharperlucasmoura: nice!20:54
blackboxswlucasmoura: ec2 sru pr reviewed https://github.com/cloud-init/ubuntu-sru/pull/102/files#21:00
lucasmourablackboxsw, ack21:00
blackboxswlucasmoura: another for you https://github.com/cloud-init/ubuntu-sru/pull/106/files#21:20
blackboxswhttps://github.com/cloud-init/ubuntu-sru/pull/107 merged21:22
blackboxswlucasmoura: same minor change request on https://github.com/cloud-init/ubuntu-sru/pull/108/files#21:28
blackboxswand we'll merge both21:28
falcojrblackboxsw: thanks for the reviews! I followed up to your followup on https://github.com/cloud-init/ubuntu-sru/pull/10521:29
blackboxswnp and strange falcojr ... I thought my eyes were seeing exactly the opposite (no system_info via cloud-config).  I'll double check again. I probably botched something21:32
falcojrit's possible I'm doing something crazy too! :D21:32
* blackboxsw doesn't mean falcojr is strange... I wouldn't say that out loud.... erm, I mean.21:33
* blackboxsw walks away21:33
falcojrI won't deny being strange ;)21:33
blackboxswwow falcojr ok, I swear I didn't see the Resolving logs before on my side,21:44
blackboxswok. I'm rerunning. and I think I'll take the debt of documenting the system_info via cloud-config example in cloudinit.readthedocs.io . so I actually remember that21:45
blackboxswI can see the pre-upgrade 'failure' case now21:45
blackboxswand verifying the fix per your script21:45
blackboxswfalcojr: ok +1 on your branch if we can add some valid URL can verify that we see the Resolving http://valid.com in logs and not foo.com21:51
blackboxswhttps://github.com/cloud-init/ubuntu-sru/pull/110 merged21:53

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!