#cloud-init 2013-11-04
<harlowja> smoser u arrived?
<smoser> yeah.
<smoser> working on marks keynote.
<smoser> fun.
<smoser> que pasa senoir?
<harlowja> haha
<harlowja> well just wanted to say hi and all that crap
<harlowja> but not if u working on mr.boss stuff
<hltbra> how do I create notifications (email or whatever) if my cloudinit setup fails in Amazon EC2?
#cloud-init 2013-11-05
<gondoi> anyone know where boothooks.sh comes from? I don't see it as part of the package, but it's there and trying to run setenforce when selinux is not installed
<ypaq> hi
<ypaq> i was just trying to deploy a ubuntu node on rackspace with knife. in my bootstrap script i do a noninteractive apt-get update/upgrade. during the upgrade a command line dialog pops up and asks me how i want to proceed with the change of '/etc/cloud/cloud.cfg'.
<ypaq> this is a problem since i cannot interact with the shell. is there a way to make cloud init shut-up?
<ypaq> i have never used cloud-init before
<utlemming> ypaq: https://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/examples/cloud-config.txt
<utlemming> ypaq: let cloud-init handle the update via the #cloud-config directive
<utlemming> ypaq: or "export DEBIAN_FRONTEND; apt-get...."
<utlemming> ypaq: or "export DEBIAN_FRONTEND=noninteractive; apt-get...."
<ypaq> i already do the latter
<ypaq> DEBIAN_FRONTEND=noninteractive apt-get -q -y 
<utlemming> ypaq: did you change /etc/cloud/cloud.cfg? 
<ypaq> nope.. i didn't even know it existed
<utlemming> ypaq: run "apt-get -y install pastebinit; cat /etc/cloud/cloud.cfg | pastebinit"?
<utlemming> ypaq: that will give a pastebin of your cloud.cfg...I'm interested in what changed
<ypaq> ok... see what i can do
<utlemming> ypaq: any luck? 
<ypaq> it's a dragging process
<ypaq> http://paste.ubuntu.com/6367342/
<ypaq> guy from RS support says they are looking into the issue
<ypaq> can you tell anything from the pastebin?
<utlemming> yeah, RS didn't read the documentation
<ypaq> lol
<utlemming> its a RS issue. Basically they configured it wrong
<ypaq> thanks very much! anything i can tell them?
<utlemming> in order to prevent the error, they should be putting local changes in /etc/cloud/cloud.cfg.d/<NUM>_rackspace.cfg instead of change /etc/cloud/cloud.cfg
<utlemming> changing that file, which _is_ packaged, will result in errors
<utlemming> er, s/errors/config diffs/ on upgrade
<ypaq> thanks again! :)
<ypaq> i pasted this to RS support
<utlemming> ypaq: np...its an easy mistake to make...I did it that way for a while myself
<utlemming> ypaq: although, you want to get RS to fix that...because if you accept the package changes then the instance will behave differently than you would expect if you rebundle it. 
<ypaq> right now i just need to be able to deploy nodes
<ypaq> not that i'm going to set up cloud init now
<ypaq> RS says i'd have to run my upgrade interactive as workaround.. wtf
<utlemming> :|
<utlemming> ypaq: which release? 
<utlemming> ypaq: put "apt-mark hold cloud-init" before you do anything else. That will prevent the cloud-init stuff...but if they have mangled anything else it may not hold. Honestly, they should configure cloud-init with a cfg.d file.
<utlemming> s/it may not hold/then other packages may require interactive upgrades/
<ypaq> 12.04
<utlemming> ypaq: yeah, "apt-mark hold cloud-init" should get you through this
<ypaq> cool.. trying that
<ypaq> one sec
<ypaq> excellent.. seems like it made it 
<utlemming> ypaq: I would recommend removing that once RS gets this straightened out...I'm not a big fan of apt-mark'ing something since you tend to forget about it. 
<ypaq> thank you very much!! 
<ypaq> yeah.. makes sense
#cloud-init 2013-11-06
<gondoi> utlemming or smoser: you around?
 * utlemming is here
<gondoi> ohai
<gondoi> so i'm trying to use config-drive with user-data, but it cloud-init doesn't seem to execute my scripts
<utlemming> can you paste your script? 
<gondoi> i can see that it's getting the file because if i use invalid syntax at the beginning (ala no #!) it errors
<gondoi> super simple one just for testing: https://gist.github.com/gondoi/7342051
<utlemming> and /var/log/cloud-init.log
<gondoi> umm it's in debug so it has tons of output.. just a min
<gondoi> okay, refresh that gist
<gondoi> utlemming: any thoughts?
<utlemming> gondoi: what cloud is this on?
<gondoi> rackspace
<utlemming> gondoi: is this from a fresh instance? or have you rebooted it? 
<gondoi> from a fresh instance
<gondoi> i tried putting that script in the user-data
<utlemming> is your file in /var/lib/cloud/instance/user-data.txt?
<gondoi> yes
<utlemming> gondoi: with out taking a look, I'm not exactly sure why its not firing
<gondoi> is it a fair simulation to blow away the instances directory and run cloud-init again?
<utlemming> gondoi: 12.04? 
<gondoi> yes
<utlemming> rm -rf /var/lib/cloud; cloud-init-cfg start; cloud-init all
<utlemming> gondoi: did you do _anything_ else to this instance? 
<gondoi> i tried copying the original cloud.cfg from the package but i just put back the one from the base image
<utlemming> gondoi: do you have launchpad account?
<gondoi> eys
<gondoi> yes
<gondoi> https://launchpad.net/~bkbox
<utlemming> can you run "ubuntu-bug cloud-init"? 
<gondoi> sure
<utlemming> and then follow the propmpts. When given a choice with "C", pick that. 
<utlemming> I forget the prompt
<gondoi> cancel?
<gondoi> *** It seems you have modified the contents of "/etc/cloud/cloud.cfg".  Would you like to add the contents of it to your bug report?
<utlemming> Yes
<gondoi> oky
<gondoi> umm it trashed it.. want me to view or save it for you?
<utlemming> gondoi: meh, don't worry
<utlemming> gondoi: the problem is here: http://paste.ubuntu.com/6373051/
<utlemming> gondoi: RS has imposed a 13.04 or 13.10 /etc/cloud/cloud.cfg onto 12.04, which is confusing cloud-init.
<gondoi> hmm thought about that and tried the 0.6.3 release of the config
<gondoi> want to give that one a shot?
<utlemming> apt-get -y purge cloud-init; rm -rf /var/lib/cloud /etc/cloud; apt-get update; apt-get -y install cloud-init
<utlemming> then "cloud-init-cfg start" and "cloud-init all"
<gondoi> bad command all
<gondoi> start perhaps?
<utlemming> whoops
<utlemming> try "cloud-init start" and "cloud-init-cfg all"
<gondoi> stil nothing :/
<gondoi> user-data.txt file is there too
<utlemming> no, check again
<gondoi> oh.. 
<utlemming> so your problem is that RS has used an improper configuration and that is causing it to crap out on you
<gondoi> :(
<gondoi> k
<utlemming> you are the 2nd person who has been burned by this
<gondoi> :D
<gondoi> yeah i saw ypaq's messages yesterday I think? wasn't sure if it was the same issue for me
<utlemming> I would recommend that you file a support ticket and ask that they use a proper configuration file for cloud-init
<utlemming> gondoi: I would be interested to know their response....
<gondoi> me too :D
<gondoi> i'll see what I can find out
<gondoi> sorry, i'm trying to execute it again so I can reproduceâ¦ i removed the file /var/lib/cloud and ran cloud-init start cloud-init-cfg all and it's not showing now
<utlemming> I would run: "cloud-init start", "cloud-init-cfg all", "cloud-init-cfg all final"
<gondoi> there we go
<gondoi> awesome
<gondoi> one more questionâ¦ i noticed you mentioned before that they should use the cfg.d to place the config.. if they put the values there that are in cloud.cfg, will they override the main cloud.cfg?
<utlemming> correct
<gondoi> cool
<utlemming> presidence is /etc/cloud.cfg, /etc/cloud.cfg.d and then your user-data
#cloud-init 2013-11-07
<harlowja> smoser https://bugs.launchpad.net/cloud-init/+bug/1248625 
<harlowja> "/proc/pid/mountinfo was introduced in linux kernel version 2.6.26" :-/
<harlowja> damn RHEL
<harlowja> maybe have to add alternate path for old kernels
<harlowja> let me see what i can do
#cloud-init 2013-11-08
<gondoi> utlemming: ping
#cloud-init 2014-11-03
<harlowja> smoser where u
<harlowja> this is zzzzz (keynote, lol)
#cloud-init 2014-11-04
<smoser> JayF, you in paris ?
<mhroncok> hi, any chance https://code.launchpad.net/~harlowja/cloud-init/py2-3/+merge/225240 will be merged soon?
<smoser> harlowja, https://github.com/dprince/os-net-config/tree/master/etc/os-net-config/samples
<harlowja> smoser https://github.com/dprince/os-net-config/blob/master/os_net_config/objects.py#L136 
<harlowja> seems like the implemnetations convert down those objects into different formats
<harlowja> and thats it; read from yaml/json, make objects, write objects using different formatters
<harlowja> and run some restart commands
<harlowja> ifup/down...
<harlowja> although i like templates and not big string functions :-P
<harlowja> * https://github.com/dprince/os-net-config/blob/master/os_net_config/impl_ifcfg.py#L73
#cloud-init 2014-11-05
<harmw> hiren_: did you checkout the new cloudinit pkg on fbsd?
<harmw> it installed and ran just fine a few days ago, absolutely wicked stuff :)
<hiren_> harmw: heh, not my self but it's being integrated in freebsd release process right now :-)
<hiren_> I mean, right now now.
<smoser> hiren_, you have a link to something i could read on "its being integrated in .."
<hiren_> smoser: it's all in testing phase so I am a bit hesitant but... https://svnweb.freebsd.org/base/projects/release-vmimage/release/amd64/mk-openstack.sh?view=markup&pathrev=274133
<hiren_> I'd want it to get into the final/good shape before announcing anything.
<gholms> Fancy
<smoser> hiren_, thanks.
<hiren_> sure.
<hiren_> you'll are at the conf right now?
<smoser> gholms, i ran into nurmi here.
<smoser> i'm here in paris, so is harlowja.
<gholms> Excellent.
<gholms> Did you run across grze as well?
<smoser> i told him you should have gotten to come
<smoser> not seen grze. 
<smoser> chris (dont remember last name) and one other guy..
<smoser> they gave me a cab ride from airport
<gholms> Was his last name Grzegorczyk?  :)
<harmw> hiren_: thats some nice stuff you have there
<harmw> hiren_: https://github.com/clalancette/oz/blob/master/oz/auto/FreeBSD10.0.auto
<harmw> perhaps that's got some usefull stuff as well
<smoser> gholms, yeah, i think so.
<gholms> That'd be grze.
<smoser> oh. hm.. then i confused him (or the namek 'grze') with grazziano (sp?)
<gholms> Ah, could be.  He's obino on IRC.
<smoser> yeah. thats right.
#cloud-init 2014-11-06
<fmoreau> hi, sorry for the dumb question, but how can I disable datasource fetching, or in other words how can I set cloud-init to use empty datasource only ? thanks.
#cloud-init 2014-11-07
<fmoreau> hi, how can I disable datasource fetching entirely ?
<harlowja> smoser u are leaving today right?
<smoser> yeah
<smoser> on a bus at 10:50
<harlowja> kk
<harlowja> well it was fun smoser 
<harlowja> ha
<smoser> harlowja, yeah. nice to see you man. 
<harlowja> def :-P
<fmoreau> hi, is package_command() really needed, or simply implementing update_packages_sources() and install_pakages is enough ?
<wumpus42> I have been successfully using cloud-init 0.7.4 in a centos 6.5 image I created under an icehouse environment. When I created a new centos 6.5 image and yum installed cloud-init, I got 0.7.5. 0.7.5 is failing in setting up the ssh keys for the user.
<wumpus42> I turned up/on DEBUG for cloud-init console logging
<wumpus42> Here's the contents of /var/log/cloud-init-output.log http://pastebin.ubuntu.com/8869675/
<wumpus42> ok. wait. something's funky about that pastebin. let me try again
<wumpus42> http://pastebin.ubuntu.com/8869713/
<wumpus42> line 683 seems to show the error. Does cloud-init 0.7.5 work with icehouse? does it need a particular patch to icehouse?
<wumpus42> it seems that the error is occurring in /usr/lib/python2.6/site-packages/cloudinit/ssh_util.py while trying to parse through /etc/ssh/sshd_config
<wumpus42> I have been successfully using cloud-init 0.7.4 in a centos 6.5 image I created under an icehouse environment we're running in a lab. When I created a new centos 6.5 image and yum installed cloud-init, I got 0.7.5. 0.7.5 is failing in places 0.7.4 wasn't like setting up the ssh keys for the user. I turned up/on DEBUG for cloud-init console logging. Here's /var/log/cloud-init-output.log...
<wumpus42> ...http://pastebin.ubuntu.com/8869713/ . If I'm reading the source correctly, line 683 of /usr/lib/python2.6/site-packages/cloudinit/ssh_util.py seems to be having a problem while trying to parse through /etc/ssh/sshd_config.
#cloud-init 2015-11-02
* You're now known as ubuntulog2
<nosleep77> hi guys, i'm trying to find a way to create a 2nd eth file (for ex ifcfg-eth1) if the instance has 2 nics. is there a way to automate this?
<natorious> hi nosleep77.   Is it "ln -s /etc/init.d/net.lo /etc/init.d/net.eth2" ?
<natorious> thats what it is on gentoo anyways
<nosleep77> natorious: no its centos 6.6
<natorious> thats gonna be in /etc/sysconfig/network-scripts then.  Is it for static or dhcp networking?
<natorious> if this is specific to a cloud env too, you could remove the hwaddr and attr lines from your /etc/sysconfig/network-scripts/ifcfg-ehtx and /etc/udev/rules.d/70-persistent-net.rules file after getting both interfaces working then take snaps in that cloud env to use
<natorious> oh, looks like you can pass the network config via cloud-config to set that up for you too
<natorious> though the datasource would need to support it to call apply_network.  Looks like OpenNebula, NoCloud and ConfigDrive support it
#cloud-init 2015-11-04
<smoser> natorious, hey. i can promise you some time later today, but the meeting in 2 minutes i can't really attend.
<natorious> smoser: k, sounds good.
<natorious> did find a bug w/ ephemeral disks that are not formatted w/ a filesystem in 0.7.x.  The mounts module seems to add an fstab entry thats invalid causing reboots to break in certain cases.
<natorious> https://bugs.launchpad.net/cloud-init/+bug/1513109
<smoser> natorious, clearly making reboot hang sucks
<smoser> but if you declare a ephemeral disk, cloud-init assumes a filesystem on it.
<smoser> but if mount fails, shoudl rpboably warn and not leave the fstab entry there.
<natorious> right.  From a provider standpoint, secure erase would surely be done.  Our pub cloud ephemeral disks I believe come formatted ntfs too
<natorious> it shouldn't be adding a raw block dev that isn't a partition to fstab thou
<natorious> oh, I'll add some log data to that issue, one sec
<flyinbut1s> Hello! Quick question...  when I'm creating a new image...  how do I tell cloud-init to run the init stuff on next boot?
<natorious> flyinbut1s: if you build a new instance from that image, it should detect that its different and run everything as new
<natorious> if your just testing things out, you can rm /var/lib/cloud and either reboot or run your init stages again
<flyinbut1s> ahah, excellent. Wasn't clear if I had to toggle a flag somewhere or not.
<flyinbut1s> thanks!
<natorious> it tracks whats been ran via /var/lib/cloud/instances/{uuid}/data etc
<natorious> so your datasource should be changing the instance info
#cloud-init 2016-11-08
<fktFOnQHgWiiEHwZ> https://www.youtube.com/watch?v=3EsJLNGVJ7E & https://wikileaks.org/podesta-emails/emailid/15893 & http://i.imgur.com/TmzXvvz.jpg
#cloud-init 2016-11-10
<Odd_Bloke> rharper: http://paste.ubuntu.com/23456363/ is the modifications I've just made locally to cloudinit-analyze to get it to run over `journalctl -o short-precise -u "cloud-*"` output.
<Odd_Bloke> Basically the line format is different (because it has the process name/PID in there), and it's a syslog-style timestamp but with microseconds.
<Odd_Bloke> And also I was getting a repeated "finish" event, which the second hunk ignores.
<Odd_Bloke> But that was just a hack, I didn't actually think about it.
<rharper> Odd_Bloke: I have short-precise working
<Odd_Bloke> rharper: Nice!  Can you point me at the code?
<rharper> yes, lemme get it pushed into the repo
<rharper> Odd_Bloke: ok, pushed to cloudinit_analyze trunk
<rharper> and I think I already sent you the bits about extracting the systemd journal directory ?
<rharper> if we have that as an artifact, then we can reconstruct the data via -o short-precise offline
<Odd_Bloke> You did, but I didn't actually finish doing it.
<Odd_Bloke> Let me add a work item for that.
<rharper> ok
<rharper> smoser: re: systemd unit files, the following changes would address slangasek and pitti's request to use sysinit.target;  I've tested in Xenial and Yakkety Openstack instances, in Yakkety, tested both ifupdown and netplan versions, and it's all working fine.  I still need to test those changes in UC16 images and ideally in an Azure instance with the early disk mount (but I don't have access to an instance)
<rharper> http://paste.ubuntu.com/23456512/
<smoser> rharper, there are some others too though, local-filesystems isnt right.
<Odd_Bloke> rharper: Can I launch an instance for you?  Or do you have some specific requirements?
<rharper> smoser: I only took the suggestion of replacing Before=basic.target dbus.sock, to Before=sysinit.target; and then the networking changes to support both networking and networkd
<rharper> Odd_Bloke: that'd be perfect; I just install the latest cloud-init. modify the services files reboot (and hope for the best)
<rharper> smoser: this is what it looks like after the change: http://paste.ubuntu.com/23456557/
<smoser> rharper, ok. reading... i was responding to pitti and updating https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/310386
<smoser> now i'll read back
<smoser> Odd_Bloke, thats nice (the journalctl stuff)
<rharper> smoser: yeah, pitti showed me the -o short-precise
<rharper> and I realized that I won't need changes to the logging conf file
<rharper> which makes merging cloudinit-analyze less bothersome
<rharper> Odd_Bloke: I'm going pull in that last bit to help drop that extra finish event that showed up (looks cleaner than what I recall I did)
<Odd_Bloke> rharper: What's your LP ID?
<smoser> raharper
<smoser> rharper is is long-lost account that he does not have creds to
<smoser> :)
<Odd_Bloke> rharper: ubuntu@yakkety-161110-1518.cloudapp.net
<Odd_Bloke> smoser: Thanks. :)
<Odd_Bloke> rharper: (Love your hostname on that key.)
<rharper> heh
<Odd_Bloke> rharper: And ubuntu@xenial-161110-1527.cloudapp.net
<smoser> rharper, going to ask pitti some in -devel
<rharper> ok
<Odd_Bloke> rharper: Well, except now I'm thinking about beer.
<rharper> =)
<Odd_Bloke> (Specifically https://untappd.com/b/cloudwater-brew-co-cosweisse/1757056)
<rharper> did you see the upcoming Jester King Collab with cloudwater ?
<rharper> http://jesterkingbrewery.com/cloudwater-brew-co-collaboration
 * rharper likes how #cloud-init is now #cloud-init-beer 
<rharper> Odd_Bloke: wow, that one sounds amazing
<rharper> Brettier than a bretty thing on St. Brett's Day.
<rharper> lol
<rharper> Odd_Bloke: smoser: on the changes to cloud-init service;  the After=systemd-networkd-wait-online.service is *dependent* on systemd-networkd.service dropping dbus.service from it's After= value;  This only affects instances that do networkd *and* need early mount (so netplan image in Azure, for example)
<smoser> rharper, https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/310547
<rharper> Odd_Bloke: the changes I proposed work fine in azure yakkety; testing xenial now
<rharper> smoser: cool
<smoser> rharper, so confused by the above.
<smoser> you're saying systemd-networkd.service has to change to have these work, right ?
<rharper> to support all scenarios, yes
<rharper> but pitti knows this
<rharper> it's mentioned in the one bug
<rharper> I've commented it
<rharper> and it only affects the "mount stuff early"; so networkd/netplan enabled instances in Azure (which needs the early mount)
<rharper> so UC16 is affected
<smoser> why does azure need the early mount ?
<rharper> I think it was just the place where early mount/reformat of disk occurred
<rharper> other clouds could also want to do that; but none do by default?
<rharper> IIRC< the bug was that fstab was updated before cloud-init reformatted the drive ?
<rharper> due to not running cloud-init early enough ?
<rharper> something to that effect
<rharper> that was why we had Before=basic.target
<smoser> smartos does it, rharper
<smoser> but user-data can affect it too
<rharper> ah
<rharper> Odd_Bloke: where do we file bugs against waagent ?
<Odd_Bloke> rharper: In the Ubuntu package?
<rharper> in the code that runs in the instance
<rharper> python stacktrace
<rharper> just wanted to pass it along
<Odd_Bloke> rharper: Sorry, I meant: in the Ubuntu package (unless that doesn't make sense).
<rharper> ah, sure
<Odd_Bloke> That question mark was doing a lot of work. ;)
<rharper> hehe
<rharper> Odd_Bloke: smoser: thanks for the azure instances;  the changes in smosers branch appear to work fine for xenial and yakkety;  I'm going to test out UC16 on openstack next
<rharper> Odd_Bloke: I'm done with those, if you want to kill em off
<smoser> rharper, did you reboot ?
<rharper> yeah
<smoser> after having that entry in /etc/fstab
<smoser> ok. yeah
<smoser> rharper, just pushed to trunk the merge for Martin,
<smoser> to be clear, I can't commit this (and upload to Ubuntu) until bug 163692 is fixed in the the given release. Thats what you're saying, right?
<smoser> bah
<smoser> the fix for https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/310547
<rharper> smoser: it only affects UC16
<rharper> but in general, yes that's right
<rharper> we should wait
<smoser> well, we can definitely go to zesty
<smoser> and i feel safe saying it can go to azure also
<smoser> er... to xenial
<smoser> as we dont really support networkd in xenial outside of core.
<smoser> and that fix is comming.
<smoser> rharper, the docs are up at http://cloudinit.readthedocs.io/en/latest/
<smoser> now also
<smoser>  http://cloudinit.readthedocs.io/en/latest/topics/boot.html
<rharper> smoser: yeah
<smoser> and separated out the datasource to their own files so we can document each of the search criteria in them.
<smoser> and i got to go.
<smoser> later
#cloud-init 2016-11-13
<vans163> Hello I see you need to use genisoimage like so:  genisoimage -output atomic01-cidata.iso -volid cidata -joliet -rock user-data meta-data  to create the iso.  But is there a way to pipe user-data and meta-data?
#cloud-init 2017-11-06
<thomasfedb> I have an Ubuntu VM on OpenStack with cloud-init. Although the instance is named "xyz.domain.com" in OpenStack, I find that /etc/hostname is just "xyz" rather than the FQDN. Is this expected behaviour? Configurable behaviour?
<mbwe> i get a message in my cloud-init log, failed to shellify, that is with my rundcmd line
<smoser> powersj: do you know about
<smoser>  https://jenkins.ubuntu.com/server/job/cloud-init-ci/473/console
<smoser> mbwe: can you give an  example of the runcmd ?
<smoser> i've never seen that error and really wouldnt have expected that it could be buggy.
<smoser> also paste your full /var/log/cloud-init.log would be hepful
<smoser> helipful
<smoser> helpful
<smoser> third times a charm
<powersj> smoser: Failed to destroy ZFS filesystem: cannot destroy 'default/containers/cloud-test-ubuntu-xenial-modules-set-password-list-t1tvm1dther3': dataset is busy
<powersj> I have not seen that. I've seen something similar long ago
<smoser> :-(
<rharper> unlucky
<smoser> rharper powersj :-(
<smoser> twice in a row
<smoser> https://jenkins.ubuntu.com/server/job/cloud-init-ci/474/console
 * rharper asks smoser for lotto numbers
<smoser> Failed to destroy ZFS filesystem: cannot destroy 'default/containers/cloud-test-ubuntu-xenial-modules-set-password-list-5rceznb9hsr9': dataset is busy\n"
<rharper> smoser: https://github.com/zfsonlinux/zfs/issues/1810
<rharper> interesting related to mounted namespaces in the containers
<smoser> rharper: in #lxc-dev was talking to stgraber
<smoser> there are 3 unmount processes eathing 100% of cpu on that system now
<smoser> (representing 3 contaienrs that failed to die)
<rharper> eww
<rharper> I wonder if there is a new systemd in xenial recently
<msaikia> Hi, can anyone please take a look at this changeset
<msaikia> https://code.launchpad.net/~msaikia/cloud-init/+git/cloud-init/+merge/330105
<smoser> rharper: how do i run lp:~raharper/curtin/trunk.vmtests-v3-streams on diglett
<smoser> youo have imates there ?
<rharper> smoser: one sec
<rharper> IMAGE_DIR=/srv/tmp/rharper/images
<blackboxsw> smoser: I wonder why lxc doesn't set hostname fqdn in /etc/hosts as our integration tests seem to expect.
<blackboxsw> tried hostname --fqdn on the lxc instance when providing cloud-config   hostname: def\n fqdn: abc.domain.com and hostname --fqdn is alsways lxd.
<blackboxsw> your cii-fixup branch is approved. msaikia I'm on your branch now for real
<smoser> blackboxsw: i dont think i follwo.
<blackboxsw> smoser: np I know you're working something else. Getting a pastebin together.
<blackboxsw> http://pastebin.ubuntu.com/25906048/
<blackboxsw> smoser: ^ I expected to see the fqdn set (or at least hostname --fqdn as we test in integration tests)
<blackboxsw> I was trying to validate what our integration test is also 'testing'   in tests/cloud_tests/testcases/modules/set_hostname_fqdn.py
<blackboxsw> I also note that redhat behaves differently than ubuntu in this regard. if hostname and fqdn are provided. redhat/cent uses fqdn, ubuntu _select_hostname() returns just the hostname when both are provided
<blackboxsw> on first blush.
<blackboxsw> maybe I'm wrong there.
<smoser> blackboxsw: i'm going to pull ci
<smoser> and yes, hostname is messed up
<smoser> due largely to silly things like sysatemd-hostnamectl and sudo
<powersj> smoser: can you let me test this evening/tomorrow morning?
#cloud-init 2017-11-07
<smoser> powersj: ?
<smoser> soryr i merged ifyou were not ready for me too
<smoser> :-(
<blackboxsw> ok seeing progress on the cloud-init sru
<blackboxsw> looks like they just got accepted over in #ubuntu-release
<dpb1> blackboxsw: what is that markdown shared editing tool
<dpb1> ?
<blackboxsw> hack.md
<blackboxsw> hackmd.io
<blackboxsw> sorry
<dpb1> thx
<blackboxsw> 'cause io is hip
<dpb1> blackboxsw: does it have vi keybindings?
<blackboxsw> yeah dpb1 bottom right corner in the 'ui'
<blackboxsw> a toggle
<blackboxsw> sublime, vim and if you are crazy emacs
<smoser> ok. rharper blackboxsw so i verified that just dropping systemd-resolved from the ipcutre makes it fast
<smoser> (fast fail on lookup)
<rharper> time to file a bug
<rharper> *regression*
<smoser> i'll probably file a bug.
<smoser> its somewhere in ipv6 land
<blackboxsw> ok I see 17.1.27 in proposed now w/ rmadison and on local containers for x,z and a. I'm going through re-verification now
<blackboxsw> resetting the checklist
<blackboxsw> https://trello.com/c/wROS4mKT/458-sru-171
#cloud-init 2017-11-08
<sinanp> I am new to cloud-init, I launched a VM with the default cloud.cfg. It look likes it added a swap entry to fstab, but can't find in the config which line that does. Any suggestions?
<smoser> sinanp, ec2?
<smoser> if ther eis a swap device provided to it from the metadata service, it will make use of the swap space . some instances provide that.
<smoser> cloud-init *can* create and use a swap file, but wont do that by default.
<cetex> So, i've tried figuring out how people solve cloud-init on-premise
<cetex> with pxe-booted (ramdisk only) instances
<cetex> Can't find any good solution for this scenario, but i'm thinking i could easily implement the magic 169.254.169.254 magic ip.
<cetex> so, will cloud-init just try to ask for that ip? or does it figure out it's on ec2 before even asking?
<smoser> cetex: earlier versions of cloud-init will just look for that eventually.
<smoser> newer versions require some identification of the platform beore they'll go polling there.
<smoser> but you can tell it to look there (or another IP address) if you're configuring it
<cetex> smoser: how would i do that? tell it to go to some url, or assune ec2?
<smoser> you can modify the image ?
<smoser> cetex: well, if you're building your own images and they're mostly static, then the easiest thing to do is put data in /var/lib/cloud/nocloud/
<smoser> if you want it to use a web service that would provide custom information to each node, then you could mimic ec2. but i  might suggest instead doing "nocloud" and a "seed"
<smoser> http://cloudinit.readthedocs.io/en/latest/topics/datasources/nocloud.html
<smoser> and then some  more example of nocloud http://bit.ly/ci-ubuntu-netconfig
<smoser> blackboxsw, powersj so. i'm thinking about the nocloud integration, and thinking how nice the c-i is witih that pretty pipeline thingy
<smoser> but i can't bring myself to write and duplicate a bunch of 'stages' since i found jenkins job builder that can make jobs out of re-usable parts much more nicer.
<powersj> smoser: yeah the syntax of jjb for the pipeline sucks
<smoser> where do i read about variable substituion and such for pipelines
<smoser> does jjb have some support for pipelines ?
<powersj> yes see the cloud-init-ci.yaml
<powersj> that's why I require a specific version of jenkins-jobs
<powersj> i.e. the one that supports pipelines
<smoser> hm.. thinking. i thoguht that was basically just blobs to jjb
<powersj> here is the doc on pipelines themselves: https://jenkins.io/doc/book/pipeline/
<powersj> and here is the doc for jjb pipelines: https://docs.openstack.org/infra/jenkins-job-builder/project_pipeline.html
<smoser> ah.
<smoser> ok. the {{ that make sense.
<smoser> fun
<blackboxsw> yeah that link is the only documentation I'd looked at for pipelines https://jenkins.io/doc/book/pipeline/ as basics
<blackboxsw> I can dig up another example of pipelines from livepatch I think.. lemme check
<blackboxsw> hmm nope https://github.com/CanonicalLtd/livepatch-client/blob/master/Jenkinsfile
<smoser> blackboxsw: i just didn't realize how i'd gert variable substitution on a per-job (or per-THING) basis
<smoser> but the jjb gives me that at least somehow.
<smoser> ie, i'm going to have 3 jobs each that differ in only the name of the release.
<smoser> and coudlnt bring myself to duplicate big pipeline blobs to get that difference
<cetex> smoser: if i'm building my own images through ansible (which we're doing) i can easily just write ansible-stuff for it.
<cetex> smoser: but if i'm building my own images through ansible, and then want the hosts which boot from them to download a user-data script and run it on boot to adapt the image for the host?
<cetex> for example, to setup consul to join the local consul-cluster, nomad as well, maybe grab credentials to use to authenticate to s3 and stuff.
<cetex> but still being on premise.
<smoser> yeah. so that should be doable via /var/lib/cloud/seed . the noclodu source is what i'd recommend.
<cetex> I've done it with dhcp-options earlier, but there's too many assumptions in there.
<cetex> i'd like cloud-init to be given a url where it can find everything of this
<cetex> and then have cloud-init download and run it.
<smoser> so you'll assume "dhcp on the first interface" for network configuration
<smoser> right?
<blackboxsw> could we just define a function  that script that the pipeline could call w/ a list? like this? http://pastebin.ubuntu.com/25919298/
<cetex> we provision the network config through ansible, it's hard-coded into the image
<cetex> and i'd like to not touch that usually.
<smoser> cetex: sure. i guess . you can provision the network howeve ryou want as long as it comes up i guess. i dont understand how you'd re-use an image with specific network info other than dhcp built in
<cetex> well, everything is dhcp
<cetex> and everything has interface 1 & 2 bonded into lacp
<cetex> so, that's it.
<smoser> ok.
<smoser> so 2 things
<smoser>  https://git.launchpad.net/cloud-init/tree/doc/examples/cloud-config-datasources.txt
<smoser> see the 'NoCloud' section there.
<smoser> i'm suggesting that you put a file like this (with just the NoCloud) section and a 'seedfrom' url with whatever you want there.
<smoser> and then for networking, you'll have to tell cloud-init not to configure networking
<smoser> or you could declare to cloud-init the networkign that you wanted. but since you're already doing that i figure let that be as it is.
<smoser> to disable networking configuration in clodu-init put in
<smoser>  /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg
<smoser> network: {config: disabled}
<blackboxsw> or pipelines related even something like http://pastebin.ubuntu.com/25919350/
<smoser> cetex: http://paste.ubuntu.com/25919368/
<smoser> blackboxsw: can you hangout a minute ? and explain to me ?
<blackboxsw> yes let's powersj feel free to join if you like meetings too
<blackboxsw> :)
<cetex> smoser: alright. cool :D
<cetex> smoser: so, could i make cloud-init pick-up additional network-configuration as well even though i don't want cloud-init to do initial network-config?
<cetex> Ideally i'm thinking it would be best if cloud-init just ran the user-data script though. then we could solve * there.
<cetex> I mean, just add a "bond0.xxxx" interface where xxxx is a vlan tag and do dhcp/static ip on it
<smoser> cetex: cloud-init doesn't currently really "hotplug" network stuff.
<smoser> that is on the roadmap but right now it is just first boot only.
<smoser> cetex: oh. and you're pxe booting too arent you
<smoser> this is kind of easier then.
<smoser> you can put seed on the kernel command line
<robjo> could someone take a quick look at config/cc_update_etc_hosts please?
<robjo> the doc string says when manage_etc_hosts is set to local host then 127.0.1.1 fqdn hostname will be added to the hosts file
<robjo> however, if the distro implementation does not overwrite _get_localhost_ip() then the IP is really 127.0.0.1
<robjo> is that a typo or is the distro implementation expected to overwrite _get_localhost_ip() ?
<robjo> smoser: are you around ^^^^
<nacc> robjo: i think smoser is afk right now
<robjo> blackboxsw: see above, thoughts?
<blackboxsw> robjo: checking ... (muxing w/ a meeting right now)
<blackboxsw> robjo: correct assessment by you
<robjo> I'll fix the typo
<blackboxsw> looks like debian (and thereforce ubuntu distro subclass) do update get_localhost_ip to  127.0.1.1
<blackboxsw> but centos,rhel,suse don't
<blackboxsw> robjo: sorry, so ubuntu/debian is 127.0.1.1 rhel,centos,suse,freebsd use 127.0.0.1
 * blackboxsw wonders how we really *want* this to behave on all distros (I'd like to have same behavior on all distros if it makes sense)
<blackboxsw> if you are updating the text robjo, I'll make sure we have a talk prior to landing the merge proposal to make sure behavior is what we expect/document on all distros  (if there are differences)
<robjo> For now I updated the doc, merge request comming
<blackboxsw> +1 th
<blackboxsw> +1 thx
<robjo> blackboxsw: https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/333418
<robjo> also include fix for the suse hosts template expansion bug which is why I stumbled across this to begin with ;)
<blackboxsw> robjo: will have a review for that by tomorrow. thx
<blackboxsw> btw robjo we also have an issue on ubuntu in this space (I don't like the behavior if both fqdn & hostname are both provided) so we might have a follow-up that properly handles fqdn and hostname behavior shortly thereafter. will make sure to ping you on it for review/thoughts
<robjo> thanks, there's a test now, well for sles anyway :)
#cloud-init 2017-11-09
<smoser> blackboxsw: rharper some interesting numbers collected from my runs for artful zesty and xenial
<smoser> http://paste.ubuntu.com/25922247/
<smoser> and with that i go to bed.
<smoser> rharper: how do i "dump" the journal
<smoser> keeping the original format or whatever
<smoser> so that i can then later feed it to journalctl commands
<rharper> oh, you copy it from /run/log/journal
<rharper> then you use journalctl --directory=/path/to/copy/of/journal
<rharper> smoser: ^
<smoser> rharper: http://paste.ubuntu.com/25926898/
<rharper> smoser: oi; that's unfortunate;  would be nicer to have journalctl apply and offset
<smoser> well, yeah, but. if you have the offset and have the journall og, yhou can do whatever you want.
<smoser> we could also read the journal format and update it.
<rharper> well, that's what the offset would be fore
<rharper> journalctl --offset foo and it just applies that value to the timestamp;  but what you have is useful
<smoser> right. but such a feature is kind of not expecte.
<rharper> no;  it's a namespace bug
<rharper> and yes , not expected
<blkadder> Hi, are there any restrictions on the characters in the content section of a file that is written as plain text?
<blkadder> I am getting an error about unable to load yaml blob...
<rharper> blkadder: not that I know of; generally for write files, I try to use the content: |  to inline text with minimal formatting
<blkadder> Hrmm...
<blkadder> What about blank lines in the file?
<blkadder> It's busting on something...
<rharper> as long as it's indented I think it's fine
<blkadder> Mind taking a peek?
<rharper> pastebin away
<blkadder> https://pastebin.com/YYrGGd3t
<blkadder> I've just been using base64 which works fine, but one of the other folks here did this...
<rharper> http://paste.ubuntu.com/25927045/
<rharper> that's my inline with newlines and how I check it
<blkadder> Hmm, wondering if it doesn't like some character or other.
<rharper> the ?
<rharper> it seems it doesn't like
<rharper> ahh
<rharper> your path is not indented I think
<rharper> there you go
<blkadder> Ohh...
<blkadder> Thanks!
<rharper> http://paste.ubuntu.com/25927066/
 * blkadder mumbles something about white space and scope
<blkadder> Madness. :-)
<smoser> blkadder: easiest thing to do is to check that things are at least loadable as yaml
<rharper> I included my yprint which loads and dumps yaml
<smoser> we are working towards having a cloud-init tool to validate config
<smoser> ah. i didnt see that rharper
<smoser> yeah
<smoser> basically:
<blkadder> Cool, thanks.
<blkadder> Would love a linter. :-)
<blackboxsw> blkadder: cloud-init devel schame --config-file <your-config> will at least validate yaml for you (and some cloud-init keys currently)
<blackboxsw> blkadder: cloud-init devel schema --config-file <your-config> will at least validate yaml for you (and some cloud-init keys currently)
<blackboxsw> it in progress currently, (as a 'devel' subcommand) we're building it up as smoser suggested.
<blkadder> Awesome, thanks.
<blkadder> That is going in the toolbox.
<smoser> blackboxsw: could you https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/333350
<blackboxsw> smoser: rharper how best to yeap
<blackboxsw> oops
<blackboxsw> yeap
<rharper> yeap ?
<smoser> best way to yeap is just to do it. let your personality ring out.
<rharper> lol
<smoser> each person has their own 'yeap'
<rharper> *yawn*
<rharper> I need more yeap
 * smoser thinks "theres a million ways to yeap"
<smoser> https://www.youtube.com/watch?v=dDZyJTDZi0w
<blackboxsw> ha!
<blackboxsw> ok approved smoser want me to merge?
<rharper> smoser: quick question, the CI_DOMAIN value is not resolvable; is that expected ?  later in the code there is a cloudinit2.<same string as CI_DOMAIN but does not reference variable)>
<rharper> the subdomain is resolvable
<rharper> so I suppose I'm missing something
<smoser> hm.
<smoser> we use CI_DOMAIN in python code
<smoser> oh i see. yeahk, i didnt change the set_hostname_fqdn to use CI_DOMAIN
<smoser> i could have...
<rharper> -    ex_fqdn = "cloudinit2.i9n.brickies.net"
<rharper> 26	+    ex_fqdn = "cloudinit2.i9n.cloud-init.io"
<rharper> 27	
<smoser> but since we'd need some further magic to make the set_hostname_fqdn.yaml
<smoser> then its kind of 50/50 i think
<rharper> well, if we changed the domain in the future we might miss
<rharper> no ?
<smoser> ie, there is usefulness in those strings being simple identical strings there.
<rharper> so ex_fqdn = "subdomain." + CI_DOMAIN  ?
<rharper> or setting CI_DOMAIN to just the full string that's resolvable ?
<smoser> but you have to change tests/cloud_tests/testcases/modules/set_hostname_fqdn.yaml anyway
<rharper> again, I'm prolly missing something
<rharper> just the diff by itself looks odd to me
<smoser> well, line 39 there...
<smoser> thats yaml
<rharper> well, the yaml could use a template if we waned
<smoser> so we can't as easily change that.
<smoser> right. yes we could do something like that. but we dont do anything like that.
<rharper> not in cloud-init
<rharper> we do in vmtest for curtin
<rharper> for similar reasons
<rharper> we could include a # should match CI_DOMAIN from nocloudkvm.py  in the yaml for a marker/hint
<smoser> well. i dont knwo. i dont think there is a lot of value in it. its a string. tests cases are full of string duplication. if you think it shoudl say 'cloudinit2.' + CI_DOMAIN
<smoser> then i can make it say that
<rharper> so yaml aside, why doesn't CI_DOMAIN just equal the fqdn we want ?
<smoser> but i'm not really interested in tmemplating yaml just for that.
<rharper> I don't have a strong opinion other than it looked strange to me
<rharper> that's fair on the template
<smoser> blackboxsw: you think its worht the change there?
<smoser> rharper is suggesting line 26 in the web diff would show:
<smoser>  ex_fqdn = "cloudinit2." + CI_DOMAIN
<smoser> (probably an import to get it too)
<blackboxsw> it's a good point, either that + CI_DOMAIN or a comment cross-referencing the why
<smoser> k. i'll push that
<blackboxsw> yes, don't want just magic, as then we have to fgrep
<blackboxsw> the "later we" who forget this
<smoser> but you ahve to grep anyway
<smoser> because of the yaml
<blackboxsw> true story
<smoser> and i thought the inability to change the yaml and the proximity of the two meant it was more straight forward to have the same string
<smoser> but aywahy.
<blackboxsw> hold up a sec. :/ hmm. I'm onboard w/ the import in python for CI_DOMAIN.
<blackboxsw> in yaml just want that string comparison
<blackboxsw> those files are side-by-side and easy to Xreference
<blackboxsw> no need to do much work for the yaml as I want just plain expected strings etc.
<blackboxsw> I kindof liken it to our unit test  assertEquals('some-string', result_of_somefunc) it's nice to see the full expected string unaltered by having to decode what CI_DOMAIN means
<blackboxsw> smoser: rharper ^.   I don't really mind one way or another. Just wanted a stance
<blackboxsw> I liken 'it'  (the yaml file as the plain string assertions)
<rharper> I suggested a comment in the yaml to point back to the variable source
<rharper> IMO; I'd see CI_DOMAIN='cloudinit2.i9n.cloud-init.io';  then the test py code import CI_DOMAIN; and the yaml use the string value of the variable and add # CI_DOMAIN from nocloud_kvm.py
<rharper> so, one variable, one string; all .py code uses the same variable; and the yaml points to the variable via comment string
<blackboxsw> ok +1 /me should go back to reading comprehension school
<smoser> ok. udpated.
<smoser> rharper:
<rharper> k
<rharper> good enough
#cloud-init 2017-11-10
<harlowja> smoser yt
<harlowja> ah, nm, u already found my gihub boo boo
<harlowja> lol
<harlowja> and fixed it, lol
<smoser> harlowja: here now. i didnt fix it. maybe someone else did. whats up?
<harlowja> i clicked a fork button of https://github.com/shmup/miniboa/tree/master/src/miniboa and it went into the cloud-init org, lol
<harlowja> by accident :-P
<harlowja> i swear
<harlowja> ha
<harlowja> but someone seems to have wiped it (i couldn't)
<smoser> yeah. saw chatter about that.
<harlowja> lol
<harlowja> oops
<smoser> your name came up
<smoser> https://bugs.launchpad.net/cloud-init/+bug/1671927
<ubot5> Launchpad bug 1671927 in cloud-init (CentOS) "init local crash - unknown subnet type 'loopback'" [Undecided,In progress]
<harlowja> ya
<harlowja> i'll get around to that, idk why others on that bug from godaddy didn't just submit the patch
<harlowja> lol
<harlowja> sorta odd....
<harlowja> ah, 'That said, you wont see this bug with master or 17.1.'
<harlowja> well nm then
<harlowja> ha
<harlowja> but ya, 'any recent openstack should have provided that.'
<harlowja> sounds about right, lol
<smoser> yeah. i'm not sure how they saw the issue. something else must have been busted.
<smoser> they wont see the issue any more.
<harlowja> cool
<smoser> but i just dont understand why they were down that path.
<harlowja> yaa, not sure, i can look into more if that config drive upload doesn't have enough
<harlowja> cause `network_data.json` is in there
<harlowja> https://gist.github.com/harlowja/ccd24cd802bdf54a0061a5a02e2f8e2a
<blackboxsw> heya harlowja, yeah, we can thank mr powersj for seeing that and cleanup in github. I think he's subscribed to all new repos that show up (and has github-superhero perms to remove)
<blackboxsw> figured if it was intentional, someone would push it again
<blackboxsw> and then we'd have to talk about what telnet MUDs mean for cloud-init rev 17.2 :)
<harlowja> blackboxsw :-P
<harlowja> blackboxsw ya, thx, wrong button click
<harlowja> i'm gonna blame microsoft, the company who made my mouse at home
<harlowja> lol
<smoser> someone from redmond drove to your house and released a rodent.  they hate you more than me.
<harlowja> yup
<harlowja> lol
#cloud-init 2017-11-12
<kszarlej> hey guys. is it possible to run cloud init network configuration on every boot>
<kszarlej> so after I add another IP to NoCloud config drive for an existing KVM VM cloud-init after reboot will
<kszarlej> set this interface up?
<kszarlej> Hey guys. I am using cloud-init NoCloud on my KVM VMS to setup networking using network-config. On first boot cloud-init sets up networking interfaces correctly. After first boot modifying network settings (adding another IP address to interface) doesnt work - the address is not added after reboot. Is it by design (so cloud-init reads networking info only on first boot?)
<kszarlej> If yes then is there any possible way to "rerun" network configuration on every boot using cloud-init?
<kszarlej> besides rebooting I was also trying 'systemctl stop cloud-init-local.service  && systemctl start cloud-init-local.service'
<kszarlej> but it also doesnt add the new IP address. I am sure that new network-config is in the ISO /dev/sr0
<blkadder> cloud-init is designed to run once.
<kszarlej> blkadder: but it has 'per-always' 'per-instance' etc
<kszarlej> I would like to set that interfaces are being setup 'per-always'
<blkadder> There is a way to run something on subsequent boots, I was looking it up....
<kszarlej> but I cannot find which "module" is setting up networking
<kszarlej> none of the /etc/cloud/cloud.cfg surely
<kszarlej> of the listed in*
<kszarlej> there isnt any 'network' module
<kszarlej> it looks like it is done my cloud-init-local
<blkadder> I think (not positive someone smarter here may know) that this is what you'd use some sort of config management tool for. Cloud-init provides initial config then you hand off to config management for things you need to do repeatedly.
<blkadder> That's my understanding anyways...
<kszarlej> Ye but then I duplicate networking configuration between two tools
<kszarlej> cloud-init and ie. ansible
<kszarlej> it would be much better if cloud-init could do that
<blkadder> So don't have cloud-init handle the network config and just use ansible?
<kszarlej> How will I access an host using ansible+ssh when it has network not configured on start? :)
<kszarlej> I dont want to log to every KVM VM and set manually IP address
<kszarlej> its 21th century :)
<blkadder> kszarlej, Distribute your network config playbook as part of the initial cloud-init config and run it.
