smosertribaal: on upgrade, that config file wont get re-written. on fresh install it will.13:37
smoseri'm pretty sure thats thecase at least.13:37
smoserso if you purged the package and then installed it, you should get the "fresh install" feel.13:37
smoserwhich is what will happen for new image builds13:37
smoserwell... not a purge, but they actually *are* fresh installs13:38
rharpersmoser: tribaal: thanks for the info; I was thinking that;  generally we wouldn't want an upgrade to migrate existing users to a new datasource;  but users could opt-in via the dpkg reconfigure;14:05
smoseri think what we'd really want to get to is no datasource_list int he packaging by default14:07
smoserand cloud-init just do the right thing14:07
tribaalyes, I guess trying / discovering datasources would make sense14:07
rharpersmoser: right;  can you think of a blocker here ?14:07
smoserand then if the user *did* configure the list, then the would have effectively opted out of new datasources in ugprades which makes sense.14:08
rharpersmoser: ideally it's empty; unless a user (or program) wrote a "local" override14:08
smoserrharper: i can't think of a blocker.14:09
tribaalone think that could be an isuue is for cases were ordering matters - for example if we maintain cloudstack compat and the cloudstack datasource is first to report "found". but that's not a blocker.14:09
smoseryou just have to consider how upgrades work and such. and have the packaging do the right thing there.14:09
tribaalor say clouds migrating away from EC2 compatibility (which I assume exists)14:09
smosertribaal: those points aren't reasons to drop datasource_list from the package i dont think.14:10
smoserthey're valid things that we have to consider (and we have had bugs where the same instance swithced datasources on reboot)14:11
rharpermmm, ibmcloud14:11
smoserbut having "one more thing" (the packaging) holding that list is just a pita14:11
smoserdry - don't repeat yourself14:12
smosercloud-init (thanks smoser) likes to have you repeat yourself ;)14:12
rharpersmoser: I think we could drop it; ds-identify already provides the "default" list if nothing is present14:12
rharperwould be worth experimenting with14:12
smoseryeah. i agree.14:13
smoserand event hen... i'd like to get rid of *that* list somehow.14:13
smoserbut at least that isnt end-user configurable14:13
rharperhrm, yes;  and we still find them  anyhow rather than just accepting the names14:18
rharperblackboxsw: did we have general sru testing steps/language somewhere?  or did we say we wanted to write that down to point folks at it ?16:43
blackboxswrharper: we don't but we said we wanted to create those docs.  I thought I added a trello card to todo soon, but I don't see it16:45
blackboxswadding one now16:45
rharperblackboxsw: was the bug verification script card updated with the 8 bugs for this SRU ?16:54
blackboxswrharper: sorry, done https://trello.com/c/WzfgsM2l/9-write-verfication-scripts-in-ubuntu-sru-repository17:06
blackboxswmight still need pruning17:06
blackboxswbut just grabbed via git diff ubuntu/xenial..HEAD | log2dch --trello | grep pad.lv (for bug-related lines)17:06
* blackboxsw has an errand. back in ~2017:07
chillysurferi'm adding some code for a merge request to cloud-init, and i'm making the (abbreviated) call to cloudinit.log.getLogger().debug('my message here'), but in my testing that doesn't actually log to /var/log/cloud-init.log. am i doing something wrong? i see debug log messages in cloud-init.log so i think the level is fine18:08
blackboxswhrm rharper I'm seeing strange behavior  trying to launch softlayer vms. The default user in cloud.cfg is ubuntu, yet that user is not being created. no tracebacks, and I see that the platform is also providing vendordata to reset root's password via a #!/bin/bash\necho 'root:$6$rounds=5000$5d6<redacted>' | chpasswd -e"18:10
rharperblackboxsw: eww18:10
rharperblackboxsw: let's collect-logs and look there18:10
blackboxswthis is OS-Code/Live18:10
rharperalso check previous softlayer sru results ?18:10
blackboxswrharper: yeah, will do on that18:11
rharperchillysurfer: deploying a newly installed patch, unittesting ?  can you provide more details on how you're testing ?18:11
blackboxswchillysurfer: if you are actually talking about grabbing logs from unit tests, you'll need to set with_logs = True in the unit test18:11
blackboxswthen you can check self.logs.getvalue() in the unit tests18:11
chillysurferrharper blackboxsw sorry i should've clarified. the code is written and i've generated a deb and testing it on a vm right now. not unit testing, actually running cloud-init through systemd18:12
chillysurferi don't even really see where we wire up logging to /var/log/cloud-init.log. i see the def_log_file but i don't see us addHandler on it18:13
rharperchillysurfer: typically the module that logs will have something like:  from cloudinit import log as logging;  LOG = logging.getLogger(__name__)  ;18:13
rharperchillysurfer: the connection happens via /etc/cloud/cloud.cfg.d/05_logging.cfg18:14
blackboxswchillysurfer: wiring logging comes config from /etc/cloud/cloud.cfg.d/05_logging.cfg18:14
chillysurferrharper: yep that's exactly what i did (except i didn't pass in __name__)18:14
blackboxswI lose :)18:14
chillysurferrharper: ah nice :)18:14
chillysurferin that case i should be logging18:14
chillysurferreally strange i'm not18:14
chillysurferno need to flush or anything right?18:14
rharperso either the path isn't being taken18:14
rharperor something else isn't quite right;  do you have a diff of your changes you can share ?18:15
chillysurferrharper: i don't have a diff at the moment. here's all i'm doing:18:16
chillysurferfrom cloudinit import log as logging; _LOG = logging.getLogger(); _LOG.debug('hello world')18:16
rharperhow is this module called ?18:16
chillysurferrharper: i wired it up from cloudinit.cmd.main18:17
rharpercloud-init runs through cloudinit/cmd/main.py which will run through cloudinit/stages.py:Init() object which runs a sequence of "stages";18:17
blackboxswrharper: looks like softlayer leaves around the /var/lib/cloud/seed/nocloud-net/user-data  which contains user: root definition, which overrides typical cloud-init behavior . That seed file is left over from earlier networklayer boot stages I think https://pastebin.ubuntu.com/p/SMWQvcZMhY/18:17
chillysurferrharper: yep that's what i did. i added an arg, and then imported and run the action18:18
rharperchillysurfer: so depending on how early you are, logging may not be initialized;18:18
rharperif you're calling LOG. before setUpBasicLogging() in main is called18:18
rharperthen you won't see that;  early main.py has it's own logging18:18
rharperwhich then gets replayed after setUpBasicLogging is called;  but previous calls to LOG before setup won't show up18:19
chillysurferahhhh wait yeah i see that we have `if args.debug: logging.setupBasiclLogging()`18:19
rharperwhich will write to stderr18:19
rharperwithout config18:19
rharperbut with the 05... it should show up in /var/log/cloud-init.log (but not in journalctl )18:19
chillysurferrharper: oh so you're saying it should show up in cloud-init.log?18:20
chillysurfereven before setupBasicLogging?18:20
chillysurferoh sorry got confused18:20
chillysurferso you say this is expected that i'm seeing now logged messages18:20
chillysurfer*no logged messages18:20
rharperdepending on where you are inserting your LOGs, it's possible.18:20
PetarisIs there anything special I need to do when using write_files with content that includes "#" in it?18:21
chillysurferah ok makes sense. i'll play around with this. thanks, rharper!18:21
rharpermain.py has it's own logging functions before it gets to parsing /etc/cloud/* for log setup;  which eventually get's configured to write to /var/log/cloud-init.log;18:21
blackboxswPetaris: you might want to base64 encode the content to avoid getting interpreted as yaml comments18:21
blackboxswPetaris: https://cloudinit.readthedocs.io/en/latest/topics/examples.html#writing-out-arbitrary-files18:22
blackboxswrharper: per softlayer, this root behavior is there regardless of the current SRU. I'll spend some time trying to debug and file a separate bug  if needed18:23
blackboxswit's happening on 19.1.1 on softlayer before the actual upgrade to 19.2.2418:23
rharperblackboxsw: hrm  I forget if softlayer has multiple datasources ...18:23
rharperthat really smells wrong to me18:23
rharperI thought they were configdrive, not a nocloud18:23
blackboxswrharper: agreed. I'll try the other 'flavors'18:23
rharperthat seems like a metadata bug18:24
blackboxswrharper: it does smell like metadata bug (or cloud-init not cleaning up earlier boot configs)18:24
blackboxswsince SL has multiple boot 'configurations18:24
rharperblackboxsw: right, there are like two stages18:25
rharperblackboxsw: but the seed file seems wrong, Ds is IBMCloud right ?18:25
rharperfor softlayer ?18:25
ubot5Launchpad bug 1673637 in cloud-init "cloud-init - Hosts in softlayer receiving warning" [High,Fix released]18:26
rharperblackboxsw: looks like it has some backgound18:26
Petarisblackboxsw, Thanks for the link.  I figured there would be need to either encode or escape them18:28
rharperPetaris: you don't need to encode for passing along # if you use the | marker in yaml , it will preserve the it as a string18:32
blackboxswthx for the extra option rharper18:38
Petarisrharper, ok, cool18:38
blackboxswrharper: per SL, yes datasource is DataSourceIBMCloud [OS-Code/Live /dev/xvdh]18:39
blackboxswwill RTD on softlayer behavior again18:40
* blackboxsw needs to swap that back into memory18:40
faahello i'm FreeBSD tester and simple patch creator ;)18:58
smoserblackboxsw: what did you think was wrong ?19:00
smoserwith ibmcloud19:00
blackboxswsmoser: the instances are coming up with root as the default user, instead of ubuntu (which is still configured default in /etc/cloud/cloud.cfg).19:01
blackboxswsmoser: and there is a v.l.c/seed/nocloud-net left around on the booted instance that is overriding user: root etc which is why my default ubuntu user isn't being set *I think*19:01
smoserblackboxsw: xenial or bionic19:03
blackboxswxenial: my expectation  was that I'd have an ubuntu user set up.19:03
blackboxswbut no ubuntu user, it's only trying to create root. root@SRU-worked-ibmcloud:~# grep User /var/log/cloud-init.log19:03
blackboxsw2019-09-03 16:57:52,405 - __init__.py[INFO]: User root already exists, skipping.19:03
blackboxsw2019-09-03 18:56:11,903 - __init__.py[INFO]: User root already exists, skipping19:03
* blackboxsw tries bionic too19:04
smoserunless something has changed, the cloud team builds those images in some odd way.19:04
blackboxswsmoser: yeah I was double checking cpc19:04
smoseribmcloud datasource ends up never being used.19:04
smoseri think if you poke around in /etc/cloud/ you'll see.19:07
blackboxswsmoser: looks like bionic works19:07
smoserand /etc/fstab and such19:08
smoserit is really bad19:08
blackboxswjust xenial falls over19:08
smoserand we decided not to fix it there.19:08
smoseror at least that it could happen later.19:08
blackboxswso bionic is good (and has no /v/l/c/seed/nocloud-net config) root@SRU-worked-ibmcloud:~# egrep -r 'Adding|User' /var/log/cloud-init.log19:10
blackboxsw2019-09-03 19:06:53,113 - __init__.py[DEBUG]: Adding user ubuntu19:10
smoserblackboxsw: iirc there is some udev hook that mounts the config disk to /var/lib/cloud/seed/nocloud19:10
blackboxswahh checking hooks19:10
smoseri wrote all this down somewhere i thikn19:10
smoserprobalb y  a google doc19:10
blackboxswyeah /me was looking through transition google docs atm19:10
blackboxswthank you scott19:11
blackboxswand hackmd to cover our bases :)19:11
smoseris all i could find on hackmd19:11
smoserwhich mostly just points to / mirrors the datasource doc19:11
smoser(in DataSourceIBMCloud)19:11
blackboxswhttps://hackmd.io/HQ4Brp_RTNyFPfopxtDFFw :)19:12
blackboxswyep just found it19:12
smoserwhcih only discusses how cloud-init works, not how you could modify an image with a bunch of changes so that cloud-init would be tricked19:12
ubot5Launchpad bug 1766401 in cloud-images "full config file wiped after apt-upgrade issued" [Undecided,New]19:13
blackboxswahh smoser ok, yet it seems on xenial we don't have all the magic we need to continue using the old datasource, as IBMCloud is being detected still19:15
blackboxswok I can file a bug on that and get it resolved19:15
* blackboxsw tries to remember if we had a followup to try to activate IBMCloud ds on xenial.19:16
smoser https://git.launchpad.net/cloud-init/commit/cloud-init/sources?id=11172924a48a47a7231d19d9cefe628dfddda8bf19:16
smoserlooks broken19:16
smoserDiffstat shows 0 files changed19:16
blackboxswok xenial I do see the following though datasource_list: [ IBMCloud ] in etc/cloud/cloud.cfg.d/99_networklayer_common.cfg19:17
smoserwell, that'd be a change19:17
blackboxswyeah because I don't think that was set originally19:18
blackboxswit may have been migrated back from bionic19:18
smoserblackboxsw: so do you see two sets of logs ?19:32
smosercloud-init logs / runs19:33
smosercan i ssh in?19:33
blackboxswsmoser: ok, there was a separate salesforce case requesting this change on xenial it seems. the shift from configdrive -> IBMCloud was requested by IBM on xenial19:34
blackboxswand they desire root user there it seems.19:34
smoserbecause inconsistency makes sure people stay employed19:34
smoserso it is working as desired?19:35
blackboxswsmoser: you can login if you like root@
blackboxsw"desired" for the moment, but I think we need to circle back w/ IBM/CPC on this19:35
Odd_BlokeMy understanding is that root should be the default for all instances, so the ubuntu user may be an accident.19:36
blackboxswOdd_Bloke: though I do wonder what the compelling need is to drop the ubuntu user altogether. I'm trying to get the background there on why that's a good thing to have the special case.19:44
Odd_BlokeWell, if the ubuntu user isn't cloud-init's default then it's going to have surprising behaviour compared to other places.19:45
blackboxswthough even in the Datasource itself IBM bakes in       {"cloud-init":"#!/bin/bash\necho 'root:$6$<snip>' | chpasswd -e"}19:45
Odd_BlokeWhereas if it doesn't exist at all, then people won't expect it to behave in any particular way.19:45
blackboxswso we were already doing some funkiness for root user in IBM19:45
smoserwlel, generally speaking this looks like it is functioning as designed.19:47
smosercpc could drop the hard coded datasource_list there. and i'd guess also the 'resize_rootfs' business19:48
blackboxswsmoser: yeah, I'm trying to shift back to the original datasource_list: [ConfigDrive, NoCloud]  to check on Xenial creating ubuntu user.19:49
* blackboxsw needs to rebooit19:49
smoserso we would have to backport the ibmcloud packaging bits.19:51
smoserneed abe6bcdfacdf2f6745e3acf5c76b332380b9c493 and 4f7bdee8cc37e0f36df888fbd79f0e68d6429431 back to xenial19:53
blackboxswsmoser: right, agreed, to enable IBM appropriately on new instances20:07
blackboxswyeah, so looking like this particular case is IBM-desired root-only login on xenial. bionic++ looks like it will have ubuntu & root useres.20:08
blackboxswusers* even20:08
smoseri think that probably that is the way it has always been on xenial20:09
smoserand bionic changed to be more like ubuntu "standard"20:09
smoserso from that perspectrive... yeah, you want to keep the old behavior20:10
rharperblackboxsw: yay for no regression, and let's document that expection in the sru manual template , ?20:12
smoseroh. i forgot. xenial is reporting only still for ds-identify20:18
blackboxswok added https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/184251020:18
ubot5Launchpad bug 1842510 in cloud-init (Ubuntu) "Xenial: Enable IBMCloud in default datasource_list template" [Undecided,Triaged]20:18
smoserblackboxsw: i'd say that is 'fix released' in cloud-init (Ubuntu)20:19
smosernot triaged20:19
blackboxswstill probably need to add IBMCloud to datasource_list config template even though image magic specifies an overried20:19
blackboxswsmoser: +120:19
blackboxswadded confirmed to the xenial task20:19
blackboxswok /me gets back to softlayer SRU verification now :)20:21
blackboxswhrm, probably about time for us to fix our rtd content generation.22:06
blackboxswit seems we are still out of date.22:06
blackboxswhrm github integrations seem successful22:13
blackboxswlast success 5 days ago, during our last commit22:13
blackboxswyet cc_set_passwords documentation still seems out of date w/ tip22:13
blackboxswthere, fixed RTD for the moment. now why did I have to manually loging to readthedocs.io and click the build button. that should have been triggered automatically on doc updates22:38
rharperblackboxsw: yeah, it's not quite working, I have a script that we could cron22:43
rharperI think the challenge is the github mirror, I'm not sure it picks up new branches/commits;  I know when we create new tags, those are missing as well22:44

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!