[09:54] <seven-eleven> hi
[09:57] <seven-eleven> why is the hashed passwd not picked up by cloud init? login to user fails http://dpaste.com/3FQ5MDD
[12:23] <chillysurfer> rharper: good morning! you and i had a chat a couple of weeks ago about vendordata and userdata, and merging them. from my tests (also sent a mail on the mailing list yesterday) it looks like even with merge_how for both userdata and vendordata that the latter is still overwritten. i got a response exlpaining this behavior. but wanted to close the loop on it and see if you had any other ideas of ways to po
[12:23] <chillysurfer> ssibly not have userdata overwrite vendordata. thanks!
[12:57] <otubo> Odd_Bloke, any chance to have an official ics calendar entry file for the cloud-init meeting? I consistently missed *all* meetings so far :-) I think it could be useful for mor people as well.
[13:11] <Odd_Bloke> powersj: ^ You might be able to help out with getting some sort of public calendar entry for the cloud-init status meeting?
[13:16] <Odd_Bloke> seven-eleven: How did you create the hashed password?
=== logan_ is now known as logan-
[14:26] <rharper> chillysurfer: hey, thanks for following up;   I don't have another idea with existing code;  it's something that we want to support and I have some ideas on how we can do it
[14:30] <chillysurfer> rharper: awesome makes total sense! thanks again for all the help!
[17:37] <blackboxsw> hrm rharper Odd_Bloke, I'm struggling a bit with trying to pre-enabled debconf selections in cloud-init during early boot by running the shell
[17:38] <blackboxsw> db_x_loadtemplatefile   versus just using debconf-loadtemplate
[17:38] <blackboxsw> using shell, I run into Debconf.pm /tmp text file is busy issues, even if I set env['TMPDIR'] /var/tmp/cloud-init/* (as cloud-init does)
[17:39] <blackboxsw> errors like this: https://paste.ubuntu.com/p/hvfggHKFFF/
[17:40] <blackboxsw> yet if cloud-init just installs debconf-utils and runs [debconf-loadtemplate cloud-init mydebconf.template] it succeeds
[17:41] <blackboxsw> I'll try explicitly writing the cloud-init debconf script to /home/ubuntu to confirm that it's not cloud-init's script that is the problem, but the Debconf perl modules which are choking on file creation
[17:41] <blackboxsw> if I don't find an easy workaround for this today, I'd like to conditionally install debconf-utils package if #cloud-config has a 'drivers' configuration
[17:42] <Odd_Bloke> "Can't exec" doesn't suggest the issue is file creation to me.
[17:42] <blackboxsw> Odd_Bloke: yeah, I'm trying to sift through the perl code to find out how to rememdy the file creation logic in perl's open
[17:42] <blackboxsw> as it doesn't seem to be paying attention to my TMPDIR in environ
[17:43] <blackboxsw> I'd really like to avoid installing another package for htis
[17:43] <Odd_Bloke> "The Linux kernel will generate a bad interpreter: Text file busy error if your Perl script (or any other kind of script) is open for writing when you try to execute it." <-- https://stackoverflow.com/a/1384594
[17:44] <blackboxsw> Odd_Bloke: maybe I'm not closing/flushing the tmpf I've created within that ExtendedTemporaryFile context manager
[17:44] <Odd_Bloke> I don't really understand the moving parts here, but is it possible you're trying to execute inside a `with open(..., "w"):` block?
[17:44]  * blackboxsw tries to unwind that 
[17:46] <rharper> it certainly looks like an open file handle to me
[17:46] <rharper> blackboxsw: I would suggest using the temp_files mkdtmp to create a dir, then plain util.write_files() which should generate files with no handles left open to them
[17:47] <blackboxsw> rharper: yeah I'm unwinding/not using the context manager anymore along those lines to see if that resolves my issue
[17:47] <blackboxsw> thanks
[17:47] <blackboxsw> gents
[17:57] <blackboxsw> Odd_Bloke: rharper yeah thanks guys, I was bungling the ExtentedTempFile context manager
[17:57] <blackboxsw> using temp_utils.mkdtemp  and cleaning up afterward works fine
[17:58]  * blackboxsw fixes unit tests for this and tests on a clean gpu instance
[17:59] <rharper> \o/
[18:42] <chillysurfer> is there already a helper in cloudinit that retrieves the distro?
[18:44] <rharper> chillysurfer: cloudinit/distros/__init_.py is the base distro class, then there are distro specific implements of the distro class in the same directory
[18:45] <rharper> chillysurfer: the distro variant is set at build-time, see setup.py for details;  this is extracted from the util.get_linux_distro/system_info;  which reads python info + host related files (like os-release/redhat-release/lsb_release);
[18:56] <chillysurfer> nice thanks!
[19:55] <chillysurfer> i'm trying to use packages/bddeb to build a source pkg but also signed. with debuild i'd use -k to specify which gpg key id i want to sign it with, but i can't seem to pass through with bddeb
[19:55] <chillysurfer> am i missing something obvious?
[19:56] <chillysurfer> nvm i can debsign the file then
[19:56] <Odd_Bloke> Yeah, that's easiest.
[20:16] <seven-eleven> Odd_Bloke, i created the password hash for cloud-init like so http://dpaste.com/1RAP7VH
[20:17] <seven-eleven> i tried also setting the password hash in chpasswd key, not successful either :-)
[20:18] <Odd_Bloke> seven-eleven: The output I get from running that is more like `$6$zTJtXk035OHCYOBf$/GX3r335pk5z9Pa2TpDgG1nj/ES6kqQOU6QbnIVAMYBDFCjIGfj9yMkrmR8MQdeCEWCzFZz60xb8ySOQAkLJK1`, are you sure your password hash is correct?
[20:19] <seven-eleven> Odd_Bloke, hm, maybe I should try another one
[20:19] <seven-eleven> Odd_Bloke, i saw that multiple people recommended python to generate the hash
[20:19] <seven-eleven> my hash looked similar to yours
[20:19] <seven-eleven> quiete long
[20:25] <seven-eleven> im trying right now with `openssl passwd -1 -salt somesalt`
[20:26] <blackboxsw> rharper: I validated as far as I can on eoan-proposed ubuntu-drivers debcon-selections setup. The wiring still needs to be put into ubuntu-drivers-common to install linux-modules-nvidia-* on our behalf. https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/371546 is ready
[20:26] <rharper> blackboxsw: cool, I'll review
[20:26] <blackboxsw> our CI is unhappy at the moment, but tests all pass locally
[20:27] <seven-eleven> Odd_Bloke, didn't work with the openssl hash. the problem looks like that the hash is never added to /etc/shadow. the question is why
[20:27] <seven-eleven> Odd_Bloke, I use `kvm-install-vm` script to create my cloud-init image
[20:28] <seven-eleven> see: https://github.com/giovtorres/kvm-install-vm/blob/master/kvm-install-vm#L437-L537
[20:28] <Odd_Bloke> seven-eleven: OK, in that case I would suggest filing a bug with the output of `cloud-init collect-logs` attached and pointing to it here.
[20:28] <blackboxsw> Odd_Bloke: seven-eleven hashed pw issues reminds me of a bug that we had fixed
[20:29]  * blackboxsw digs it up... I'm thinking if we are on older cloud-init that'd be a known problem
[20:29] <seven-eleven> blackboxsw, i'm on version: /usr/bin/cloud-init 19.1-1-gbaa47854-0ubuntu1~18.04.1
[20:29] <blackboxsw> bummer, me was thinking https://bugs.launchpad.net/bugs/1811446
[20:29] <ubot5`> Ubuntu bug 1811446 in cloud-init (Ubuntu) "chpasswd: is mangling certain password hashes" [Undecided,Fix released]
[20:30] <blackboxsw> specifically that fix corrected hashes that started with $6
[20:30] <seven-eleven> "This bug is believed to be fixed in cloud-init in version 19.1"
[20:30] <blackboxsw> yeah no dice :/
[20:31] <seven-eleven> ok going to file a bug as Odd_Bloke suggested
[20:32] <Odd_Bloke> If it's not showing up at all, then it probably isn't a subtle password handling bug.
[20:35] <seven-eleven> i will verify by setting NOPASSWD in sudoers then I can read the shadow file
[20:38] <blackboxsw> seven-eleven: in your cloud-init log, I would also expect a    grep 'setting hashed password' /var/log/cloud-init.log to return a line for your user.   In our testing of the fix for 1811446 we provide chpasswd: with a 'list' value: https://pastebin.ubuntu.com/p/Mr77D6t9KG/
[20:38] <seven-eleven> oh shadow has a hash set for both root and my user, but the bash is totally different
[20:39] <seven-eleven> blackboxsw, i tried chpasswd too: http://dpaste.com/3986T25
[20:39] <blackboxsw> example of our chpasswd use for setting root user's hashed password: https://github.com/cloud-init/ubuntu-sru/blob/master/bugs/lp-1811446.txt
[20:40] <blackboxsw> hrm seven-eleven looks sane
[20:40] <seven-eleven> let me try list with quotes, didn't try that before
[20:43] <blackboxsw> seven-eleven: that forward slash at the end may end up funky or shell escaped which could affect behavior.
[20:43] <blackboxsw> you could also try leading hyphen before each chpasswd list item and then quotes;   list:\n  - 'root:<hash>'\n  - 'user2:<hash2>'\n
[20:44] <seven-eleven> ehm, it's actually setting a hash to shadow
[20:44] <seven-eleven> because this time I omitted the user, and now only the root hash is visible in shadow
[20:44] <seven-eleven> but the hash is different than the supplied one
[20:45] <seven-eleven> look likes $6$ELfTiASdsab$Zt6RUzzoio2lkwfhsldhfasdfl9rGKKH5tN9k/qYlca8/OoIBWbLmR0HYLVWLPo3s8YkZZJvv2j99426k/.:18419:0:99999:7:::   (slightly modified)
[20:46] <seven-eleven> not sure if cloud init converts the supplied hash
[20:46] <seven-eleven> or simply uses a newly random generated passwd
[20:46] <blackboxsw> one thing to also check from the system if you can get in:    sudo cloud-init query userdata to see if your userdata is being mangled
[20:47] <seven-eleven> let me try
[20:47] <seven-eleven> oh yeah
[20:47] <seven-eleven> see: http://dpaste.com/29MGBYY
[20:47] <seven-eleven> the hash is eaten up
[20:48] <blackboxsw> seven-eleven: yep, looks like that leading $ is actually being variable substituted
[20:48] <seven-eleven> oh
[20:49] <blackboxsw> single quote should prevent that I think.
[20:49] <seven-eleven> I have to probably escape it with \
[20:49] <seven-eleven> i try
[20:49] <blackboxsw> or escape it
[20:49] <seven-eleven> i used single quotes list: 'root:$1$someafk$taAISFjdAzgAb/'    (modified hash)
[20:50] <seven-eleven> i think this is a #bash question, I will google how to properly escape EOF text
[20:50] <rharper> for hashes we use the | inline content, right ?
[20:51] <rharper> that's quite strange
[20:55] <seven-eleven> blackboxsw, hehe with backslash escaping it worked :D
[20:56] <seven-eleven> interesting that single quotes were not sufficient
[21:00] <rharper> so, it's making it into subproces and we pass the the user:hash + newline into chpasswd -e
[21:02] <rharper> seems quite odd to have that get mangled;  looking at sub, we encode the data (into bytes) ;  what OS are we talking about?   python3 ? python2 ?
[21:06] <smoser> i think its probably a problem before it gets to cloud-init
[21:08] <seven-eleven> rharper, are you refering to https://bugs.launchpad.net/cloud-init/+bug/1811446 or my hash being eaten up? in my case my problem was that single quotes are not sufficient for EOF things, I need to use backlash. in case of blackboxsw's link to the bug it looks like ubuntu bionic
[21:08] <ubot5`> Ubuntu bug 1811446 in cloud-init (Ubuntu) "chpasswd: is mangling certain password hashes" [Undecided,Fix released]
[21:30] <rharper> seven-eleven: no,  just wondering what cloud-init actually got, did you file the bug with the cloud-init collect-logs tarball ?   your paste earlier showing that your list http://dpaste.com/29MGBYY  was truncated; that suggests to me (what smoser said) that the user-data isn't making cleaning into cloud-init;
[21:31] <rharper> if this is reproducible (sounds like it is) then filing a bug with your steps and the collect logs from within the instance would be most helpful to sort things out
[21:41] <seven-eleven> rharper, hm, cloud-init received an empty variable due to the missing escape in my bash script, that's why http://dpaste.com/29MGBYY is truncated, shouldn't it be truncated if cloud-init receives an empty hash?
[21:51] <rharper> seven-eleven: I'm at a loss with how you;re feeding cloud-init data;  do you have a /var/log/cloud-init.log that I can see?  depending on the datasource, you may want to base64 encode the user-data
[21:56] <seven-eleven> rharper, i'm using this script which feeds the cloud init with EOF to genisoimage, see line 529: https://github.com/giovtorres/kvm-install-vm/blob/master/kvm-install-vm#L435-L537
[21:57] <rharper> shell esacping
[21:57] <seven-eleven> interestingly `/var/log/cloud-init.log` is empty
[21:58] <seven-eleven> rharper, yep, single quote was not enough with EOF, I have to use \$, not '$'
[21:58] <seven-eleven> ah, the log is on the guest
[22:00] <seven-eleven> rharper, I found the log on the guest but we don't need it anymore. shell escaping :)
[22:00] <rharper> right =)