#cloud-init 2014-02-10
<smoser> harlowja, for the tox changes...
<smoser> i want / need to be able to run 'make test' in package build
<smoser> which doesn't have general access to inter-tubes
<harlowja> smoser hmmm
<harlowja> durn intertupes
<harlowja> *tubes
<astol> hi all, how do I tell cloud-init to run first boot commands again on next reboot? (ie reset first boot attribute) 
<smoser> astol, you can configure that in /etc/cloud/cloud.cfg
<smoser> if you're just wanting '#!' things to run again, then you can change 'scripts_user' to be 'always'
<smoser>  - [scripts_user, always]
<astol> smoser: I am building an ami base image and since I have already booted once, I need to somehow reset that flag
<astol> so when I create instances again from that ami first boot sequence will be executed
<astol> I don't need to run them always, just one more time
<natorious> wouldn't deleting the /var/lib/cloud dir be what you want then?
<astol> natorious: well that's kind of what I did, but this felt so crude I decided to ask
<astol> natorious: I honestly scanned cloud-init sources for 15 minutes, but saw nothing (being a non-pythonista)
<astol> so I just delete /var/lib/cloud and that's all?
<smoser> astol, thats fine. you dont need to change anything.
<smoser> it will do that by design.
<smoser> the user-scripts run "per-instance"
<smoser> and "per-instance" is "whenever an instance-id changes".
<smoser> 'rm -Rf /var/lib/cloud' would suffice, but it is not necessary.
<astol> smoser: wow! that's actually even more nice, thanks :) I wonder if I'm the only guy asking about that
<smoser> pretty much its all supposed to "just work" :)
<smoser> this is pretty cool, we moight have 2 more datasources sometime ssoon
<smoser> GCE and cloud-sigma.
<harlowja> smoser u forogot mine! :-P
<harlowja> thats 3
<harlowja> ll
<harlowja> *lol
#cloud-init 2014-02-12
<dgarstang> What was that file I could remove that would make cloud-init run again?
<harlowja> @ /var/lib/cloud
<smoser> natorious, ping
<smoser> natorious, hm...
<smoser> harlowja, what do you think about http://paste.ubuntu.com/6922017/
<smoser> i think i'm going to do that.
<smoser> http://paste.ubuntu.com/6922056/
<natorious> smoser: whats up?
<smoser> oh. sent you a mail.
<smoser> please sign cla, and theen i'll pull your merge proposl
<natorious> awesome, thnx
<natorious> signed
<smoser> natorious, merged. and pushed.
<smoser> please give it a whirl and test and fix anythign that falls out.
<natorious> thnx.  I also like the idea of having the distro author go thru verifying the modules.  
<harlowja> smoser seems fine with me
#cloud-init 2014-02-13
<smoser> utlemming, https://code.launchpad.net/~utlemming/cloud-init/sdc-op-script/+merge/206065
<smoser> the only thing wrong with a boothook is that it doesnt guarantee network.
<smoser> harlowja_away, one comment on datasource openstack
<smoser> can't 'wait_for_url' 
<smoser> i know it sucks
<smoser> but you can't
<smoser> only ec2 can be annoying
<smoser> ie, we want to be able to have both OpenStack and EC2 enabled
<smoser> and not block boot for some stupid seconds
<smoser> harlowja_away, wake up. man.
<smoser> utlemming, did you see my half written response to the smart os change?
<utlemming> smoser: yeah, and it makes sense
<utlemming> smoser: I'll look at it later
<smoser> ok. i really, really, really want an upload of cloud-init today
<smoser> there is a bunch of stuff in
<harmw> smoser: will we see an new cirros release this month aswell?
<smoser> :)
<smoser> i've been meaning to do that.
<smoser> pathetically, one of my motivations for doing so is so i can get in a icehouse commit log :)
<harmw> nice, though how does cirros fit into icehouse commitlogs? ;p
<smoser> devstack
<smoser> i'll change the default cirros used to be the released version.
<harmw> nice nice
<smoser> what do you use cirros for harmw?
<harmw> testing openstack, or something else where I 'just want a vm shell'
<harmw> instead of booting fedora/centos 
<harmw> which take considerably longer to boot :)
<harlowja_> smoser waked up, lol
<harlowja_> smoser wait_for_url, can be removed :-P
<mfisch> is $INSTANCE_ID only available at bootcmd? It didn't seem to be there at runcmd time
<smoser> harlowja_, i just about have it i think. was testing that it actually worked
<smoser> then gonna merge it.
<smoser> hm..
<smoser> mfisch, the doc says it is.
<smoser> i'm ok to put it in at runcmd also.
<harlowja_> smoser did u get the overall change, basically unifying cfgdrive and osmetadata datasource, being backed by a common class, and then only really changing in the child classes how the data is fetched and from where (url or disk)
<smoser> mfisch, it does appear that it should be there for you in bootcmd and in boothooks, but it is only done there.
<smoser> harlowja_, yeah, it looks good.
<harlowja_> smoser k
<harlowja_> thx boss
<harlowja_> :-P
<smoser> harlowja_, http://paste.ubuntu.com/6927014/
<smoser> thats my changes so far.
<harlowja_> smoser seems ok, although should i just remove 'wait_for_url'
<harlowja_> cause i know we all love waiting for datasources to come online, lol
<harlowja_> and watchign the log 
<smoser> harlowja_, well, i had removed wait_for_url
<smoser> but then i put it back
<harlowja_> :-P
<smoser> i figured its more useful being configurable
<harlowja_> ah
<smoser> if you had a openstack where it just wasn't working
<smoser> ie, it didn't come up right way, you could turn max_wait up 
<smoser> and it would retry
<harlowja_> ya
<harlowja_> looks good to me then
<harlowja_> shall i patch or u?
<smoser> merged.
<harlowja_> k
<harlowja_> i know the heat folks and others wanted , https://code.launchpad.net/~harlowja/cloud-init/ec2-ssl ?
<harlowja_> although in general since i don't use ssl for this stuff anyway (cfgdrive stil..) up to u :-P
<smoser> harlowja_, didn't i pull that lareday?
<harlowja_> oh
<harlowja_> possibly
<harlowja_> will double check
<smoser> if not i will
<smoser> seems not
<smoser> harlowja_, there are merge conflicts there
<smoser> can you resolve those really quick ?
<harlowja_> k, one sec, writing up little speaker summary/abstract for atalanta summit stuff
<smoser> ooh. fancy.
<smoser> harlowja_, i'd also like to get the tox stuff too. but can't really pull it if it means it doesn't work "offline" 
<harlowja_> ya, understandable
<smoser> harlowja_, random question.
<harlowja_> yo
<smoser> i'd like to have cloud-init write some "state" to /run/cloud-init
<harlowja_> k
<smoser> i dont have any ideas for a mechanism on doing tha tthat doesn't result in potential race conditions on reading it
<smoser> ie, cloud-init writing the state file while a reader is reading it
<smoser> do you?
<harlowja_> write to temp state file, then swap it?
<harlowja_> mv is atomic i think in linux
<harlowja_> who's the readers in this case? users
<harlowja_> ?
<smoser> readers is users, yes (or tools)
<smoser> mv is atomic, yes. but if i write state to 'cloud-init.state', then go to update it.
<smoser> then that is not atomic.
<smoser> mv cloud-inti.state cloud-init.state.old && mv cloud-init.state.new cloud-init.state 
<smoser> would leave a race.
<smoser> probably over thinking
<smoser> :)
<med_> smoser, how do I ask cloud-init to re-run
<smoser> :)
<mfisch> med_ is asking because I want to test a feature w/o rebuilding an image
<med_> (ie, on an image after I locally hack the cloud-init code0
 * med_ is mfisch proxy
<smoser> what i do very commonly is
<smoser> rm -Rf /var/lib/cloud /var/log/cloud-init*
<smoser> reboot
<med_> He made me drive him to work and now he asks me to ask questions
<med_> nod. that's it.
<smoser> you can be less invasive too
 * med_ knew there was a way
<smoser> and you can run modules by themselves
<smoser> cloud-init single
<mfisch> smoser: I'm toying with the idea of having access to INSTANCE_ID inside the landscape module
<mfisch> although I wonder if landscape uses hostname by default, /me checks
<med_> mfisch, sounds like you're still working for (C)
<med_> j/k
<smoser> mfisch, so 2 things there..
<smoser> a.) i'm not opposed to having INSTANCE_ID just in the environment whenever we set it
<smoser> b.) you do have access to the instance id in the landscape module
<mfisch> b) not currently
<mfisch> and its not a shell script like bootcmd is
<mfisch> so I'd have to do a pass of the dictionary we're building
<smoser> you do have access.
<smoser> you get 'cloud' passed in
<smoser> that has it.
<mfisch> ah cool
<mfisch> would a simple check like if the dict has a key of $INSTANCE_ID set it there work?
<mfisch> may want to make that more generic
<smoser> mfisch, ?
<smoser> the 'cloud' object has 'get_instance_id()'
<smoser> and it proxies most things through to cloud.datasource.
<mfisch> smoser: I mean in the config file, we need a token that says "substitute this with the instance ID"
<smoser> i'm confused now.
<mfisch> smoser: the landscape stuff gets loaded as a dictionary in that file
<mfisch> account: foo
<mfisch> for example
<mfisch> computer-title could default to instance id if unset
<mfisch> or something like if dict[computer-title] == "$INSTANCE_ID" then do the substitution
<mfisch> smoser: let me propose a fix and maybe it makes more sense
<smoser> http://paste.ubuntu.com/6927388/ ?
<mfisch> smoser: yes, exactly
<smoser> that doesn't do any template/rendering, but functional
<mfisch> smoser: I will test that
<mfisch> smoser: it screwed up the ordering in the config file to change it there, I'll look into it 
<mfisch> also instance human readable name, whatever that is would work better there
<smoser> mfisch, yeah, yaml wont write in any predictable order
<mfisch> smoser: it's sticking it above [client] which confused landscape
<smoser> oh?
<mfisch> yeah it comes out as
<mfisch> computer-title = foo
<mfisch> [client]
<mfisch> ...
<smoser> oh. yeah, it would
<smoser> :)
<smoser> cause i put it there.
<smoser> i didn't realize that. 
<smoser> just put it in the right place.
<smoser> mfisch, http://paste.ubuntu.com/6927527/
<mfisch> smoser: I
<mfisch> smoser: I just moved it up and checked if it was in ls_cloudcfg
<mfisch> above the merge
<smoser> ewll that paste there should basically fill it into instance id if its not present.
<mfisch> smoser: yep and that explains why my idea didnt work
<mfisch> smoser: sorry for the delay, this is what I ended up doing based on the 12.04 version:
<mfisch> http://paste.ubuntu.com/6927655/
<smoser> mfisch, well, if you do it after the mergeTogether
<smoser> then you ensure that you're lowest priority
<smoser> ie, merge all the stuff the way it does
<smoser> then only fill it if its not set elsewhere
<zooz> hi smoser, It's Vaidas. We've been talking over email re GCE data source
<mfisch> smoser: true
<smoser> zooz, hey.
<smoser> trunk has gce now. woohoo.
<smoser> (possibly not working as i might have broke your code :)
<zooz> cool, I am about to look at your improvements and add them to my changes
<zooz> I have got proper tests now, handling of user-data and more
<zooz> smoser, no worries, I will test in GCE before I do a pull request
<zooz> in fact I have been using cloud-init in GCE without any issues for some time now
<zooz> smoser, a merge request is on the way for your review. I have tested it in GCE and ran unit tests. No issues.
<smoser> zooz, thank you.
<smoser> i plan on a cloud-init upload to ubuntu tonight.
<zooz> great
<smoser> so this will make its way into cloud images for trusty tonight
<zooz> awesome!
<zooz> done
<zooz> let me know if any issues
<med_> thanks smoser.
<med_> does cloud-init get SRUd (or have a MRE for prior releases)?
<smoser> well, SRU.
<smoser> but for "hardware" enablement, sru team has let things in.
<med_> nodz.
<smoser> it is honestly though, somewhat of apain to port back to 12.04 at this point. although datasources are fairly standalone.
<med_> 'k
 * med_ will just take a look at rmadison in the morning
<smoser> zooz, still there?
<zooz> smoser, yeah
<smoser> is there  a reason you'd not want the dns resolution?
<smoser> the way i did it it actually works (should work) in gce
<smoser> and will fail quickliy (dns resolution) just about anywhere else
<zooz> smoser, no particular reason, I can test it using dns name
<smoser> without attempting to get access to a 169.254.169.254 that might be firewalled off (and socket level timeout)
<zooz> good point
<zooz> let me do a quick test
<zooz> smoser, it does work in GCE using dns name
<smoser> ok. so lets keep that part, that will save people elsewhere from doing a request to that url.
<smoser> (unless there is a dns entry external there, which would be unlikely)
<zooz> smoser, do you want me to quickly add it back?
<smoser> sure.
<smoser> i have to go now for a couple hours.
<smoser> so i'll take a look again later.
<zooz> thanks
<harlowja_> smoser alrihgt, i got some free time
#cloud-init 2014-02-14
<smoser> zooz, still around ?
<smoser> zooz, i merged your changes, with some of mine though.
<smoser> some general things about datasources:
<smoser> a.) don't say you're "found" if you're not sure
<smoser> b.) WARN is annoying, and should only be used for something that the user really should see.
<smoser> utlemming, ping
<smoser> ok.
<smoser> please , everyone test trunk
<smoser> zooz for GCE
<smoser> harlowja_away, for openstack
<smoser> utlemming, for smartos
<zooz> smoser, great. Thanks
<harlowja_at_home> smoser: fyi, i'll be out till next thuresday (Skiinggg), don't think i let u know, but now am letting u know (just incase)
<smoser> harlowja_away, have fun
<smoser> harlowja_away, around ?
<smoser> zooz, did you try on google compute?
<zooz> smoser, yeah
<zooz> it fails
<smoser> boo
<smoser> what'd i break?
<smoser> i'm sorry for breaking it.
<zooz> is_resolvable() fails
<zooz> oh no, no worries
<smoser> really?
<smoser> you have an instance up now ?
<zooz> yeah socket.getaddrinfo does take a hostname
<smoser> you can 'ssh-import-id smoser' to let me in if you'd like.
<zooz> you're giving it a URL instead of a hostname
<zooz> sure, I can spin up a box on GCE for you
<zooz> give me a sec
<smoser> zooz, is_resolvable_url
<smoser> is what it should have used.
<smoser> can you test that really quick ?
<zooz> sure
<smoser> http://paste.ubuntu.com/6932936/
<zooz> http://paste.ubuntu.com/6932940/
<zooz> works now
<smoser> zooz, you got that error ?
<smoser> that shouldt be a warning.
<smoser> hm..
<zooz> user-data was not available
<zooz> I just added user-data item and it does work now
<zooz> so that warning is expected, unless it should use a different log level
<smoser> yeah. thats what i meant. 
<smoser> generally i dont want WARN
<smoser> as that goes to console
<zooz> oh, OK, I was not aware of that
<smoser> well, i did that one :)
<zooz> it's easy to change it
<smoser> but generally, WARN only things that really should be seen.
<smoser> yeah, i'll fix that
<smoser> http://paste.ubuntu.com/6932974/
<smoser> thanks.
<zooz> smoser, can you give me your ssh public key?
<smoser> oh
<smoser> ssh-import-id 
<smoser> (ubuntu ?
<zooz> no, fedora
<smoser> its also in debian
<smoser> oh for petes sake.
<zooz> :-)
<smoser> https://launchpad.net/%7Esmoser/+sshkeys
<zooz> I am not a big fan of ubuntu :-)
<smoser> ssh-import-id basically takes a launchpad (or github) username and does what you'd expect.
<zooz> ah that's pretty cool
<smoser> yeah, its extremely conveneint.
<zooz> ssh -l fedora 107.167.165.4
<zooz> you'd have to scp the DataSourceGCE.py file over
<zooz> smoser, I just added user-data item for you to test
<smoser> zooz, actually, i'm good.
<smoser> you can add or remove user-data ?
<smoser> i guess they're just keys.
<smoser> but you can add and remove them.
<smoser> hm..
<smoser> we shoudl think about that.
<smoser> the other thing to think about, is doing base64 encoding in user-data
<smoser> maybe useful, maybe not.
<zooz> removed
<smoser> zooz, ok. i'm done there. i just uploaded to ubuntu.
<smoser> thanks.
<zooz> cool thanks
#cloud-init 2014-02-16
<astol> hi all, anybody has a working solution for getting aws tags on cloud-init, maybe?
<astol> I want to assign hostname = name + instance id
#cloud-init 2015-02-09
<smoser> harlowja_at_home, ping
<smoser> you want to be on a cloud-init talk submission for ods-vancouver
<smoser>  http://paste.ubuntu.com/10144643/ <-- abstract
<harlowja_at_home> yo yo, will check when i get into work, bb
<larsks> smoser: are new commits still going to lp:cloud-init, or has all work (even maintenance) moved to github?
<smoser> larsks, my plan is to keep cloud-init 0.7.x on launchpad/bzr for near term.
<larsks> smoser: Thanks.
<robjo> Hi, how do I prevent host key regeneration?
<robjo> The situation is that I am trying to switch running instances from initialization framework $HOMEBREW to cloud-init
<robjo> after reboot the host key is regenerated, but there was a hostkey already, and I need to prevent this from heppening?
<harlowja> smoser whats up dawg
<larsks> robjo: it looks like you should be able set 'ssh_deletekeys' to 'false' to prevent cloud-init from deleteing and re-generating keys.
<robjo> larsks: thanks will test this
<smoser> you want to be on a cloud-init talk submission for ods-vancouver
<smoser> harlowja, ^
<harlowja> smoser ohhh, sureeee
<harlowja> smoser like a talk or like a presentation?
<harlowja> *not a session right?
<smoser> presentation, yeah.
<smoser> as in https://www.openstack.org/summit/vancouver-2015/call-for-speakers/TalkDetails/
<smoser> i need a catchy title.
<harlowja> Click and Clack talk Cloud-init
<harlowja> haha
<harlowja> *u can be click
<harlowja> i'll be clack
<smoser> deal
<smoser> i didn't realize californians listened to npr
 * harlowja not in a while, ha
<harlowja> sometimes :-P
<harlowja> i thought car-talk over though
<robjo> larsks: Thanks, worked :)
<larsks> robjo: \o/
<jseun> hi! I am facing a strange behaviour with cloud-init: it seems that I cannot change the default username 'ubuntu' in the /etc/cloud/cloud.cfg file to something else.  I get a bunch of error like Failed to set passwords, etc... as if the user creation failed earlier in the boot process.
<jseun> but naming it 'ubuntu' again works flawlessly
<larsks> jseun: what version of cloud-init is in use?
<jseun> larsks: 0.7.5-0ubuntu1.3
<larsks> jseun: Interesting.  If I boot a Trusty image on my openstack environment, using this user-data file: http://chunk.io/f/c170788bd870459185833da5fbcfd3b8
<larsks> ...it just works.  That is, the 'example' user is created instead of 'ubuntu',  and I can log into the 'example' account using my ssh key.
<jseun> larsks: interesting indeed.  I'll disable clearing the console so I can see what's going on
<jseun> thanks for having a look
<jseun> hmm.. I did not use that syntax, larsks (looking at your user-data)
<jseun> system_info:
<jseun>   default_user:
<jseun>     name: docker
<jseun> that's how I specified the default user
<jseun> I'll try your way
<larsks> Hope it works! :)
<smoser> jseun, that should work also
<smoser> and actually is the "new" er way to do that.
<jseun> smoser: neither worked for me. cc_ssh_authkey_fingerprints.py and friends fail if I change the default username, which is strange.  I am trying to get more debug informations from the console buffer since I am locked out.
<smoser> jseun, http://paste.ubuntu.com/10147330/
<smoser> that just "worked for me" on a trusty instance
<smoser> jseun, http://paste.ubuntu.com/10147350/ and that one for vivid
<jseun> smoser: allright, so it does work.  I'll have to see what happens on my side.  I am using cloud-init to configure a custom live CD image based on Ubuntu Core 14.04 (see http://github.com/jseun/ucldh)
<harmw> harlowja: any experience with glusterfs?
<harmw> (or anyone else, for that matter)
<harlowja> negative
<larsks> harmw: a *litttte* experience with glusterfs.
<larsks> s/tte/ttle/
<harmw> ok, fair enough
<harmw> and how was that experience :)
<larsks> harmw: well...it seemed super easy to setup, and I was happy with my little experiement.  I wasn't doing anything performance (I was using it to provide shared storage in a test kubernetes cluster).
<harmw> hm hm hm
<larsks> harmw: there is a #gluster channel, too :)
<harmw> sure, but I'm not after biased answers :P
<harmw> I had a tiny ceph deploymnt that worked, kinda
<harmw> interested in gluster, if it's any good to replace ceph
<smoser> jseun, ah. nice. maybe you're not finding a datasource?
<smoser> jseun, one thing i might do for debuggin is use 'backdoor-image' 
<smoser> https://code.launchpad.net/~smoser/+junk/backdoor-image
<smoser> basically will just inject another user (backdoor user) into a system
<smoser> into an image or into a target root directory
<jseun> oh yes! that's just what i need right now
<smoser> see the usage http://bazaar.launchpad.net/~smoser/+junk/backdoor-image/view/head:/backdoor-image
 * harmw cloud insert some random NSA-based joke here
 * harmw allows harlowja to do just that
 * harlowja runs away
<smoser> well, that tool by default inserts the user 'backdoor', so it tries to be up front about everything
<smoser> HEY! i'm putting a backdoor into your system !
<smoser> then later, you say:
<smoser>  hm... why is there a user 'backdoor' here... i wonder if thats a good thing.
<harmw> consipiray and clever mindtricks
<jseun> lol
<jseun> smoser: i could not add the docker user because the docker group already exists : http://paste.ubuntu.com/10147940/
<jseun> having docker group in primary-group directive should fix it i think
<jseun> http://paste.ubuntu.com/10148316/ <-- how would you guys tell cloud-init to create this default user with a primary-group already created?
#cloud-init 2015-02-10
<jseun> guys, where's the .py file that plays with networking configuration, especially, /etc/network/interfaces ?
<jseun> ok, this is handled in distro classes
<jseun> it looks like cloud-init writes to /etc/network/interfaces from a template where eth0 is not marked as auto.  a) can I tell cloud-init to not alter /etc/network/interfaces ? or b) where can I modify the template, if any?
<jseun> I have put eth0 configuration into /etc/network/interfaces.d/eth0 and would like cloud-init to not overwrite /etc/network/interfaces unless I put some configuration directive in my meta-data file
<larsks> jseun: are you booting with a config drive? Or just using the metadata service? With a cursory glance at the source, it looks as if cloud-init only ever writes that file when using config drive, nocloud, or OpenNebula data sources.
<larsks> ...but I could be wrong.
<jseun> larsks: it's meta-data and user-data file written to an ISO, I guess this is NoCloud
<larsks> No, that's config drive :)
<jseun> ah! :)
<larsks> Huh, it *looks* as if the network configuration is only written if your metadata includes network configuraiton information.  I'm looking in cloudinit/sources/DataSourceConfigDrive.py, around line 215.
<larsks> Similarly, in .../DataSourceNoCloud.py around line 186.
<larsks> I'm actually not sure which data source you're hitting :/
<jseun> larsks: my meta-data file does not specify any network configuration, yet /etc/network/interfaces got smashed
<larsks> Hmmm, in the nocloud datasource, there is also a check against (self.dsmode in ("local", seeded_interfaces)), and I don't know the code well enough to know what that does.
<larsks> I guess I would stick some debugging code in there and run it from the command line to see what happens.
<jseun> I fixed it, for now, having auto eth0\niface eth0 inet dhcp\n in the meta-data file
<smoser> that is config drive'
<smoser> meta-data and user-data.
<smoser> cloud-init in 0.7.X really expects that eth0 is 'auto' in the OS.
<larsks> Weird, then, because it looks like the network config is only written if there is a network_config key available.
<smoser> networking is one thing that really needs improving.
<smoser> it is complex though, to block system boot waiting for networking information from somewhere that might not be there... 
<smoser> at least for making generic images, and the goal for ubuntu is "one image that works everywhere".
<jseun> any ways I could provide the eth0 ip address in final_message or is it more a runcmd thing?
<tennis> hi.... How can you determine what cloud-init thinks is the default user? 
<tennis> That is, how can I be sure my "default_user" parm in /etc/cloud/cloud.cfg worked?
<smoser> http://paste.ubuntu.com/10164529/
<smoser> harlowja, arent we missing the return on the not case tehre?
<harlowja> surely would seem like it, lol
<harlowja> good catch
<tennis> smoser: Does my above query make any sense? 
<smoser> tennis, /var/log/cloud-init.log has a line like:
<smoser> 2015-02-10 21:37:24,737 - __init__.py[INFO]: User ubuntu already exists, skipping.
<tennis> smoser: ah, ok.  Thanks!
<smoser> harlowja, lp:~smoser/cloud-init/py2-3 we're closer.
<harlowja> woot
<harlowja> progress
<smoser> i'm down to a failure around random_seed.
<smoser> i really hate bytes and strings and encode and decode
<harlowja> :)
<harlowja> ya
<harlowja> me too
<harlowja> waste of my life, lol
<smoser> alright.
<smoser> i hope to upload to vivid tonight. 
<smoser> later.
<harlowja> cool
<harlowja> lata
#cloud-init 2015-02-11
<Odd_Bloke> smoser: https://code.launchpad.net/~daniel-thewatkins/cloud-init/fix-multi_log/+merge/249308
<Odd_Bloke> Anyone else who wants to pitch in with a review is also welcome. :)
<Odd_Bloke> smoser: I'm not sure how to reproduce that template warning; I've tried on vivid on GCE.
<smoser> woot.
<smoser> cloud-init uploaded to vivid running on python3.
<smoser> thanks to harlowja and barry and Odd_Bloke 
<larsks> Congrats!
<harlowja> np
<harlowja> smoser partay time right
<harlowja> lol
#cloud-init 2015-02-12
<Odd_Bloke> smoser: \o/
<harmw> smoser: new version out?
#cloud-init 2015-02-14
<j12t> I'm getting an exit code 1 from 'cloud-init --debug init --local'. But nothing resembling an error message.
#cloud-init 2016-02-15
<Odd_Bloke> smoser: Re-review of https://code.launchpad.net/~daniel-thewatkins/cloud-init/shim_fixes/+merge/273977 would be appreciated. :)
#cloud-init 2016-02-16
<smoser> Odd_Bloke, thank you for reviewing https://code.launchpad.net/~sankaraditya/cloud-init/topic-stanguturi-vmware-support/+merge/277933
#cloud-init 2016-02-17
<arkin> I'm trying to factor wget http://dev.mysql.com/get/mysql-apt-config_0.6.0-1_all.deb && dpkg -i mysql-apt-config_0.6.0-1_all.deb
<arkin> into my init file
<arkin> but dpkg pops up a configurable option (which mysql version)
<arkin> can I bypass this ?
<smoser> arkin, you have a couple options.
<smoser> cloud-init does its package installation with:
<smoser>  DEBIAN_FRONTEND=noninteractive
<smoser> in the environment
<smoser> and a install of
<smoser> APT_GET_COMMAND = ('apt-get', '--option=Dpkg::Options::=--force-confold',
<smoser>                    '--option=Dpkg::options::=--force-unsafe-io',
<smoser>                    '--assume-yes', '--quiet')
<smoser> so you should be able to
<smoser> DEBIAN_FRONTEND=noninteractive dpkg --force-confold --force-unsafe-io
<smoser> and get the same behavior
<smoser> to dpkg
#cloud-init 2016-02-18
<vhosakot> Hi, I have a question about cloud-init in OpenStack images.
<vhosakot> Anybody there ? :)
<Odd_Bloke> smoser: Where are we on moving 0.7.x to git?  I've got a few bzr branches hanging around that I'll hold off on merging/proposing if it's happening...
<Odd_Bloke> smoser: Unrelately, the clear_dhcp and if_down_up methods in https://code.launchpad.net/~sankaraditya/cloud-init/topic-stanguturi-vmware-support/+merge/277933 make me a little nervous.  Do you know if there is a "proper" way of doing this via the OS?
<jgrimm> Odd_Bloke, fyi, smoser is out on vacation today, will be back tomorrow tho
<Odd_Bloke> jgrimm: Ah, thanks. :)
<jgrimm> np
<vhosakot> Hi, would it possible for cloud-init to make a cloud instance join a multicast group by sending an IGMP join message while the instance boots ?  This will enable cloud instances receive multicast traffic in a cloud.
<vhosakot> Hello, anybody there :)
<vhosakot> ? :)
<Odd_Bloke> vhosakot: o/
<vhosakot> Odd_Bloke: \o/
<Odd_Bloke> vhosakot: cloud-init can run any scripts you give it, so you should be able to make it do that. :)
<Odd_Bloke> vhosakot: So if you construct appropriate user-data and pass that in to the instance, it should work.
<vhosakot> Odd_Bloke: ok, so I can use the --user-data or --user-data-file argument of cloud-init to make the instance send IGMP join message while it boots. Is this right ?
<Odd_Bloke> vhosakot: Yeah, assuming that the cloud you're on will allow it etc. :)
<vhosakot> Odd_Bloke: Yes, OpenStack's nova has --user-data argument that is passed to cloud-init when the instance boots
<Odd_Bloke> vhosakot: Oh, yeah, I mean assuming that IGMP multicast-y things work. :)
<vhosakot> Odd_Bloke: yes, there will be a multicast sender in our cloud that responds to the IGMP join message sent by the instance
<Odd_Bloke> vhosakot: Cool!  Good luck getting it set up!
<vhosakot> Odd_Bloke: thanks!  do you anyone that has tried/done multicast with cloud-init
<Odd_Bloke> vhosakot: I don't know anyone who's tried it, no.
<vhosakot> Odd_Bloke: wow, I'm the first one.. thanks! have a great day :)
<Odd_Bloke> :)
#cloud-init 2016-02-19
<tdurakov> hey
<tdurakov> got question for cloud-init cloud-config usage for no-cloud case
<tdurakov> I'm trying to set up static ip for vm, by setting it configuration over metadata. it works weel for ubuntu 14.04 but fails for 15.10, are there any known bugs for this?
<tdurakov> oh, found one: https://bugs.launchpad.net/cloud-init/+bug/1315501
<fxpester> hi all
<fxpester> can you help me to figure something out, is it possible to use cloud-init on already running system ?
<Odd_Bloke> fxpester: You can get it to re-run, but as it expects to run on first boot you might see some unexpected behaviour.
<fxpester> I see, I was about to pass some orchestration commands with openstack-heat...
<Odd_Bloke> fxpester: I don't really know enough about openstack-heat to know what that means. ^_^
<fxpester> let`s say I running 1000 virtual machines with IRC client in it, but my IRC server is closing so I need to tell all my VMs that they need to change server - I need to execute from shell `connect 'new.server.org'` so if it was at server start, cloud-init could help me
<Odd_Bloke> There has been discussion of having cloud-init run as an agent, so it could do things other than at boot, but that hasn't yet happened.
<Odd_Bloke> So I think you're best off finding another tool better suited to the job. :)
<smoser> Odd_Bloke, i'd love to move over to git right now.
<smoser> you want to do that ?
<Odd_Bloke> smoser: What would be involved?
<smoser> i realy just have to dig up notes from harlowja.
<smoser> i've looked at this a couple times, and almost been there but never pulled the trigger.
#cloud-init 2017-02-13
<NostawRm> info on it seems a little sparse, any way to make cloud-init use openstack's admin_pass from metadata? I could just be missing it in my searches
<NostawRm> the options I have found involve changing parts of cloud-init which I'd rather not do so we can update without issue later
<smoser> NostawRm, patches are welcome, but cloud-init will pull the users ssh keys.
<smoser> which.. seems  more secure and desireable.
<NostawRm> smoser, wanting to fight with networking configs via vnc
<NostawRm> you're definitely right, but it seems to have its place, and I did just look through the source, its not there so woo
<smoser> NostawRm, well, if cloud-init is finding the datasource (which would be required to use the adminpass anyway)
<smoser> then you can pretty easily just tell it to set a root password
<smoser>       #cloud-config
<smoser>       password: passw0rd
<smoser>       chpasswd: { expire: False }
<smoser>       ssh_pwauth: True
<smoser> that'll let you log in on console or ssh with ubuntu:passw0rd
<dgarstang> I have an Ubuntu instance that boots with 2x2Tb disks attached. Cloud-init only formats them to 1Tb. Is this a known issue with cloud-init?
#cloud-init 2017-02-14
<linforpros> setting manage_resolv_conf: true? where should it go? cloud.cfg.d ... ?
<linforpros> Also be sure that your cloud.cfg file includes this# configuration module in the appropirate section.
<linforpros> Is it in the cloud.cfg? or in the user-data?
<larsks> smoser: is it expected that a newer version of cloud-init should work with content in /var/lib/cloud/instance from an older version of cloud-init? I am just starting to look into a bug someone has reported and I am curious if this situation is expected to work.
<smoser> yes, it should work.
<larsks> Okay, thanks. Time for some investigatin', I guess.
<smoser> its kind of a pain, in that we have to maintain basically forward compatibility of old object.pkl stuff
<larsks> I wonder if something like json + a version number would be usable (because it wouldn't have actual python objects, and you could write translations from version n-1 to version n).
<smoser> backwards compatibility, not forwards compatibility... basically if we add a method to a class in the source, we have to add a fallback implementation too so that we dont call a fucntion that does not exist in the loaded object.
<smoser> if i had this to do again, i'd like to get have something other than a python object pickled there.
<smoser> but rather a json object that would get loaded.
<larsks> Yeah.  Plus it would be consumable by non-python tools, which might be useful.
<smoser> yes.
<smoser> i'm definitely open to transition to that :)
<smoser> which is more work than your bug fix
<smoser> larsks, example of what i was talking about is at c92f02037afc6b0434c9498904f7d888e00cd55b
<larsks> Right.  Not surprising, and a maintenance pain :)
<jmelvin> ping cloud-init, can someone please help I'm having issues following: http://cloudinit.readthedocs.io/en/latest/topics/hacking.html , seeing this error:  https://paste.fedoraproject.org/558081/70881101/
<Odd_Bloke> jmelvin: LP_USER should be replaced with your Launchpad username.
<Odd_Bloke> Oh, wait, didn't read the whole paste.
<Odd_Bloke> >.<
<Odd_Bloke> jmelvin: You'll need to register your local SSH key with Launchpad.
<smoser> :). jmelvin on your http://launchpad.net/~jmelvin see 'SSH keys'
<jmelvin> oh man the directions don't say that
<smoser> and the pencil there.
<smoser> sorry, yeah.
<smoser> MP for that is welcome :)
<jmelvin> thanks guys!
<smoser> (hope that didn't sound rude, i agree we can add that. if you want to walk through that simple change, then i can pick it up quick)
#cloud-init 2017-02-16
<freakynl> Hi, would cloud-init be a good choice to use for configuring an ova/ovf template after deployment? It won't be running on public clouds. The default config seems to want to pull information from the default gw which seems very amazon EC2 like
<smoser> freakynl, well,. it will look to the ec2 metadata service currently if it didnt find any ovf data.
<smoser> but if you have an ovf iso transport cdrom attached, it should use that.
<freakynl> smoser: Thanks for the info. Was more hoping of a yes this is the right tool or no you should look elsewhere kind of answer before diving in the docs :)
<smoser> freakynl, yes!
<smoser> that better ?
<freakynl> smoser: hell yea :) thanks
<larsks> harlowja: around?
<larsks> smoser: cloudinit/net/sysconfig.py seems to be broken, particularly right here: https://github.com/cloud-init/cloud-init/blob/master/cloudinit/net/sysconfig.py#L286
<larsks> At that point, "iface" is just a dict, and has no "children" attribute, so it all blows up.
<smoser> :(
<larsks> https://bugs.launchpad.net/cloud-init/+bug/1665441
<smoser> do you have network config that showed this ?
<larsks> I do, although I am pretty sure that any network config would show this...I don't see how you would get anything other than a dict at this point.
<larsks> But I will update the bug with my network_data.json.
<smoser> must of regressed. i am surprised harlowja woudlnt have cried.
<rharper> looks like a nice candidate for integration tests
<larsks> rharper: looks like a nice candidate for *unit* tests, even, I think.
<rharper> oh, that too
<rharper> there are sysconfig unittests
<larsks> rharper: I am pretty sure this stems from someone writing "iface" when they meant "iface_cfg".
<rharper> so that might be an easy way to expose it
<rharper> y
<larsks> With a couple small fixes I have it not failing and generating a semi-valid config file, although it appears to be missing an ip address.
<larsks> Oh hey, actually I think that fixed it.  It just generates more alias configurations than I expected.
<harlowja> who what
#cloud-init 2017-02-17
<larsks> harlowja: still around?
<smoser> larsks, i just responded to your MP
<smoser> i'm sorry.
<larsks> smoser: yeah, I expected the unit test request, since obviously having one would probably have avoided the problem in the first place :).
<larsks> I will try to add that tomorrow.
<smoser> i really hate it when people do that to me
<smoser> so i pass the love on to you
<smoser> :)
<larsks> I was partly considering a more substantial rewrite, but then the networkmanager folks got all responsive and went and fixed the second problem when I pointed it out today.
<rharper> smoser: have you uploaded master to zesty ?  specifically for those ds-identify label fixes ?
<smoser> rharper, the label fixes are in zesty, yes.
<rharper> hrm, I'm not seeing the comma fix
<smoser> zesty has 0.7.9-26-g1cd8cfa-0ubuntu1
<rharper> ok
 * rharper looks at his image build
<smoser> (if we hadn't gotten it in, then curtin woudl still be failing)
<rharper> yeah; that's right
<larsks> smoser: I added tests to https://code.launchpad.net/~larsks/cloud-init/+git/cloud-init/+merge/317553
<smoser> larsks, yeah, i saw. i'lll get that today.
<larsks> Spiffy.
<smoser> larsks, ok. i reviewed. i'll pull.
<larsks> \o/
<smoser> in general, please try to set commit message to git style commit message to be squashed
<smoser> subject
<smoser> <blank line>
<smoser> description
<larsks> It is?
<larsks> Already?
<smoser> ah. yes. your top commit *is* , and thats good.
<larsks> The second one, too.
<smoser> but in the merge proposal
<smoser> a combined one... to squash to
<smoser> as i'll just squash them.
<larsks> Sure.  I specifically *didn't* squash them so that the test could be tested before the fix.
<larsks> Otherwise there's no demonstration that the test is testing anything.
<smoser> yeah, thats fine. and good.
<smoser> but my general flow is that i'd
<smoser>  git checkout larsks/branch
<smoser>  git rebase -i master
<smoser>  <squash everything>
<smoser>  grab the 'Commit message' from the merge proposal web ui
<smoser>  git commit
<smoser>  git checkout master
<smoser>  git merge <that head>
<smoser> so there is only one commit addd to master
<larsks> Ah, I see.  Well, you could just take the commit message from the top commit, and discard the second.
<larsks> But I get it, and will note that in the future.
<smoser> thats what i did
<smoser> and just pushed
<smoser> the pain in that process above is that it is painful
<smoser> and also that i then have to manually tag the MP as 'merged'
<larsks> Fair enough.
<larsks> Like I said, noted for the future.
<smoser> as my rebase now has no lineage with the head, so laucnhpad doesn't just recognize it got pulled.
<smoser> i need tools for this :)
<larsks> I can squash and re-push the merge request if you'd ilke.
<smoser> ist fine. but i shoudl have run tox fully before pusing :-( .
<larsks> smoser: I think your fingers just had too much to drink.
<rharper> mmm, coffee
<smoser> i dont usually bother correcting typos. i just leave that to the readers brain.  faster that way.
<nacc> or smoser is half-german
<nacc> ist at least means something then :)
<harlowja> larsks am around
<larsks> harlowja: I fixed some bugs in cloudinit/net/sysconfig.py, which I think is your work.  It's been merged as f81d6c7.
<harlowja> sweet
<larsks> I hope it looks sane to you :)
<harlowja> will try to check it out
<smoser> the failure iiuc was with multiple ip addresses (networks) on a single link
<smoser> jgrimm, or rharper if you have some time, read this
<smoser> http://paste.ubuntu.com/24015364/
<jgrimm> ack
<jgrimm> smoser, line 24? not sure i grok the connection between reading 1-5 and not being able to change via user-data/vendor-data?
<jgrimm> possibly worded funk that i'm not understanding what you are saying
<jgrimm> funky
<jgrimm> ah, i get it (after reading cloud-init:) nvm
<smoser> bah. should be 1-3 there.
<jgrimm> ok, i wondered that
<smoser> 1-4
<smoser> man. typos abound
<jgrimm> ack. happens
<jgrimm> smoser, initial setting for xenial will be false? and configured via?
<jgrimm> not mentioned lines 34-36 area.
<jgrimm> but otherwise looks good
<smoser> when we initially put it back into xenial, we'll be in 'report' mode
<smoser> which means nothing will be affected at all.
<smoser> cloud-init's builtin default will be 'false', so it woudl then warn without a timeout
<smoser> which i think is kind of what we'd want.
<jgrimm> ack, just verifying as it wasn't mentioned
<smoser> jgrimm, i'm running out the door, probably poke inlater
<smoser> http://paste.ubuntu.com/24016244/
<smoser> i uploaded to zesty with some of that.
<jgrimm> smoser, ack thanks
#cloud-init 2017-02-18
<nimbius> sup cloud-init.
<nimbius> cloud-init it not fetching ip addresses for my node.  as far as i can tell, it not even mounting the config drive
<nimbius> do i need to have the config drive mounted and ready?
<nimbius> tried mounting it, this isnt read.
<nimbius> the ssh keys in my openstack config on the config drive arent imported into anything meaningful, and network config isnt being read from the configdrive.
<nimbius> does cloud-init support gentoo??
<magicalChicken> nimbius: cloud-init does have some support for gentoo
<magicalChicken> most people who might be able to help you are not in right now because its a long weekend
<magicalChicken> so it may be easier to get support on tuesday
<nimbius> k
 * nimbius sighs
<nimbius> i have to open an ubuntu account to report a bug.
<nimbius> wtf.
#cloud-init 2018-02-12
<rauno> hi
<rauno> struggling with cloudinit with debian8 virtual machine, somewhy it doesn't find the config provided via configdrive
<rauno> seems to be quite old version also 0.7.6 in this image
<rauno> it still tries to find it from  Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed
<mjh> Hi all, I hope this is the right place to ask a question about cloud-init usage particularly with regards to aws. The number of additional block devices for our instances is a variable depending upon environment. I would like to format and mount all unused block devices under /data/0 /data/1 etc. I cant see a way to do this in the given examples and my google foo has not been powerful enough.
<cesarjorge> Hi, in my cloud-init logs I see the following:
<cesarjorge> stages.py[WARNING]: Could not find module named cc_resolv_conf_authkey_fingerprints (searched ['cc_resolv_conf_authkey_fingerprints', u'cloudinit.config.cc_resolv_conf_authkey_fingerprints'])
<cesarjorge> What does this warning mean and how is it solved?
<mjh> hi cesarjorge. I'm new to cloud-init but I cant find a authkey_fingerprints key for cc_resolv_conf. Is this a typo maybe somehow pulling in cc_ssh_authkey_fingerprints?
<cesarjorge> I don't know, but I use in cloud.cnf the following init module - resolv-conf
<cesarjorge> And not use nothing for cc_ssh_auth...
<mjh> are you able to post your configuration - I'm happy to compare with what I have.
<mjh> ah...
<cesarjorge> I use the default config file, except, that I add this - resolv.conf in init section
<mjh> sorry, we both appear to be a bit lost :-/
<cesarjorge> cloud-init-0.7.9-9.el7.centos.2.x86_64
<cesarjorge> Other fail:
<cesarjorge> My image by default use swap partition as:
<cesarjorge> # /dev/mapper/centos_centos-swap swap                    swap    defaults        0 0
<cesarjorge> When cloud-init start:
<cesarjorge> systemd-fstab-generator[1334]: Failed to create swap unit file /run/systemd/generator/dev-mapper-centos_centos\x2dswap.swap, as it already exists. Duplicate entry in /etc/fstab?
<cesarjorge> Then in VM:
<cesarjorge> The cloud-init add this:
<cesarjorge> # /dev/mapper/centos_centos-swap	none	swap	sw,comment=cloudconfig	0	0
<cesarjorge> Howto workaround this problem?
<mdt[m]> hi, i just struggled over cloud-init and would like to understand its concepts. i wonder where i can find a highlevel document about its flows and concepts. did i miss something?
<CodexRaptr> go for cloud-init.io
<mdt[m]> i checked that page, sure. as i do not run cloud-init in aws or such but on my notebook i need some deeper insides in the flows. i am missing this from that page.
<mdt[m]> i just installed the cloud-init package into my debian based virtual machine image - i assume i need a server outside providing the meta data? where is that concept explained?
<smoser> mdt[m]: cloud-init needs to find a "datasource".  a datasource is
<smoser>  http://cloudinit.readthedocs.io/en/latest/topics/datasources.html
<mdt[m]> smoser: ok, now i can relate that... thank you. so first step (before cloud-init) is always dhcp to get a working network?and how will the different nodes be differentiated on the meta-data-server? by mac?
<smoser> mdt[m]: well that is entirely up to the platform.
<smoser> cloud-init does not always dhcp on the first nic now. on certain platforms it reads metadata for network configuration from the datasource (not all datasources require network).
<mdt[m]> smoser: so what are usual patterns?
<mdt[m]> ah, ok, which other datasources (i saw kernel params already) are possible?
<mdt[m]> (ah, and i saw iso media as well)
<smoser> well, i'm not privy to how Azure or GCE or Amazon platforms hook up  networking and all.
<smoser> to automate things locally without a "full blown cloud", you can use the NoCloud datasource
<smoser> i have some examples of using those at https://asciinema.org/~smoser
<mdt[m]> my current interest is to learn here on my notebook (where i could establish a network datasource as well) and be prepared to either a vmware environment or aws
<mdt[m]> oh, thank you, i will take a look...
<mdt[m]> (and cool stuff with asciinema, it allows cut&paste, i personally dont like these yt vids from screens)
<smoser> yeah, asciinema (outside of being really hard to type) is really nice.
<smoser> also, see the urls in the descriptions, they point to gists on github that have the fiels there for you.
<smoser> oh. i see that the debian does not. https://asciinema.org/a/145772 points to gist.
<mdt[m]> ok (debian was the first i looked at)
<mjh> hi, please can I have a quick hand with the cloud-init commandline. I wish to re-run configuration of disks from a local configuration file. I run: /usr/bin/cloud-init --debug single --name disk_setup --frequency always and I get the volumes formatted but not mounted.
<mjh> sorry - found it, I did not twig that is was a separate module.
<mjh> It would be really nice if I could mount _all_ unformatted disks under a given dir. please let me know if i am missing something.
<dpb1> mjh: mount all unformatted disks? what kind of magic is this? :)
<mjh> :-)
<mjh> depending on aws environment I may have 0 1 or 2 additional block devices for elasticsearch. I want all of them formatted and mounted under /data/ with the minimum of customisation between environments. So as it stands I am using a pre-service to elasticsearch to write out a could init configuration which I am then running. feels yuck but probably more rubust than writing a format / mount script myself.
<smoser> blackboxsw: you left bug numbers in on the changelog.  we cna just roll them into the existing sru. did you want to do those 3 individually?
<smoser> wrt https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/337515
<smoser> mjh: there isnt anything there. i agree that'd be a nice feature.
<mjh> :-) hey ho
<dpb1> mjh: ya, I think it's a neat idea.  sounds like zfs. :)
<dpb1> (one aspect of zfs anyway)
<mjh> not sure about zfs (or lvm) for this, it would still require robust scripting and I'm not sure that I like the idea of striping over remote aws block devices (I probably did not mention this was on aws...)
<dpb1> mjh: I mean that when you create zfs filesystems, they automatically get mounted under a particular path.
<mjh> I get you.
<dpb1> mjh: I like your idea about a nice easy way of saying "automount all unformatted storage using ext4" in the config.
<blackboxsw> smoser: artful and xenial pushed for 17.2.35 https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/337516 https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/337515
<blackboxsw> testing an azure deployment with preprovision flag now
<smoser> blackboxsw: sory...
<smoser> you need to have the new chagnelog entry and version
<smoser> (we can't re-upload the same version again)
<smoser> you can just strip the bug numbers out
<smoser> and we'll add the new chagnelog entry to the sru branch
<blackboxsw> smoser: I misremembered fixing
<blackboxsw> and making not
<blackboxsw> and making note
<blackboxsw> for some reason I thought since we were replacing and not officially releasing to non-proposed that was an option. but I suppose we already officially published to -proposed. so, makes sense
<blackboxsw> smoser: shall I move the SRU bug to the new changelog entry or leave it on the previous changelog New upstream snapshot. (LP: #####)
<smoser> blackboxsw: sorry for slow reply.
<smoser> leave the old one in place. just put the sru bug on the new commit also i guess.
* blackboxsw changed the topic of #cloud-init to: Reviews: http://bit.ly/ci-reviews | Meeting minutes: https://goo.gl/mrHdaj | Next status meeting: Monday 2/19 16:00 UTC | cloud-init 17.2 released (Dec 14, 2017)
<blackboxsw> smoser: just pused
<blackboxsw> pushed even
<blackboxsw> thx smoser
<blackboxsw> saw unapproved float by
<smoser> the other is on its way
<blackboxsw> rharper: confirmed my azure pre-provision instance is still timing out /looping :)
<blackboxsw> so it is indefinite wait until IMDS tells the instance things are ready /accessible
<blackboxsw> non-404
 * blackboxsw grabs a bit of lunch
<rharper> blackboxsw: cool, yeah, the updated branch is a replacement for what's in trunk now; that was one of the reasons I was confused.  I'm reviewing it now.
<rharper> https://code.launchpad.net/~tamilmani1989/cloud-init/+git/cloud-init/+merge/336392
<smoser> blackboxsw or rharper can i get a review of https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/337100
<smoser> i'd like to have that landed.
<smoser> https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/337471 should also be striaght forward
<blackboxsw> reviewing them
<blackboxsw> smoser: why are we not patching sys.exit for other with self.assertRaises calls throughout cloud-init unit tests? https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/337471
<blackboxsw> ha! n/m there aren't any
<blackboxsw> disregard
<blackboxsw> landing
<blackboxsw> smoser: do we need  comparable writable/system-data support in cloudinit/sources/DataSourceNoCloud.py for your https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/337100 or do we assume the race condition w/ bindmount is long since past by the time cloud-init datasources get there
<blackboxsw> n/m: "It is guaranteed at cloud-init-local.service time"
<blackboxsw> ok landing
#cloud-init 2018-02-13
<smoser> robjo: didnt we have a LP: # like prefix for suse bug ?
<smoser> i remember some cute name. but i dont see it in logs.
<smoser> nice. i find it.
<smoser> but it looks like i typod it.
<smoser> in 8f162b6603aefef400b784ab70dc57080978cffc
<smoser> goo: http://bugzilla.opensuse.org/show_bug.cgi?id=1069635
<ubot5> bugzilla.opensuse.org bug 1069635 in Other "cloud-init packages differ for each build" [Normal,Resolved: fixed]
<robjo> Its "boo" for opensuse bugs
<smoser> we had agreed on 'boo:' (bugzilla.opensuse.org)
<smoser> yeah.
<blackboxsw> goo :) hah
<robjo> thius bug was filed on the SUSE bugzilla instance, thus "bsc" => Bugzilla.suse.com
<robjo> "boo" => bugzilla.opensuse.org
<robjo> Also accessible via boo :) I'll fix the description
<smoser> oh. ok. we can do 'bsc:' i guess ?
<robjo> we have both, "boo" and "bsc" just depends on who files the bug and against which product
<robjo> SLES usually ends up in "bsc" and openSUSE usually ends up in "boo" but not always, user training is difficult ;)
<robjo> plus "bsc" is not accessible for everyone
<robjo> smoser: all set I hope
<smoser> robjo maybe you missed my inline comment.
<smoser> can you use 'elif'
<robjo> smoser: done
<blackboxsw> well junk, I need something from jsonschema draft 6. and latest support in python jsonschema is draft4. looks like that project is blocked a bit on getting full draft6 support. /me might have some upstream contributions to push into jsonschema in short order.
<powersj> blackboxsw: https://github.com/Julian/jsonschema/issues/337
<blackboxsw> powersj: that's what inspired my  'junk'  comment.
<powersj> heh
<blackboxsw> I saw that support request came in in June '17 and kinda  has some work around it, but looks stagnant
<blackboxsw> I can get around this by making cloud-init's schema too permissive. but supporting arbitrary dictionary keys is not going to happen without a lift in python jsonschema or using an external non-python tool for validation
<blackboxsw> after this pass on snap configuration I'll check how much work the schema version 6 support in python-jsonschema would require
<blackboxsw> though nice that someone else is also interested in that working.
#cloud-init 2018-02-14
<blackboxsw> powersj: smoser can you op me?
<blackboxsw> I must have to relogin to the nickserv
<blackboxsw> thx
* blackboxsw changed the topic of #cloud-init to: Reviews: http://bit.ly/ci-reviews | Meeting minutes: https://goo.gl/mrHdaj | Next status meeting: Friday 2/16 16:00 UTC | cloud-init 17.2 released (Dec 14, 2017)
<blackboxsw> hey folks, note we've changed cloud-init status meeting to this Friday instead of the upcoming Monday due to the US holiday
<blackboxsw> email sent to the cloud-init mailinglist
* blackboxsw changed the topic of #cloud-init to: Reviews: http://bit.ly/ci-reviews | Meeting minutes: https://goo.gl/mrHdaj | Next status meeting: Friday 2/16 16:00 UTC | cloud-init 17.2 released (Dec 14, 2017) | cloud-init 18.1 pending-release (Feb 22)
<kpcyrd> hello, can somebody help me figure out how the versioning scheme in cloud-init works? There are tags that look like releases (17.2, 17.1, 0.7.9), and there are also a whole bunch of tags with ubuntu prefix. What's up with those?
<nacc> kpcyrd: where are you looking?
#cloud-init 2018-02-15
<kpcyrd> nacc: https://github.com/cloud-init/cloud-init/releases
<powersj> kpcyrd: here is where we changed the version schema: https://lists.launchpad.net/cloud-init/msg00097.html
<powersj> basically we follow a YY.release method now
<powersj> That means 18.1 will be next up
<powersj> As far as those releases/tags that are showing up in GitHub, every time we make a release to the development release I beleive a tag gets made, similarly when we make an SRU to 16.04 and 17.10.
<nacc> i would assume hte ubuntu/ namespace is for uploads to ubuntu as well
<nacc> but not sure
<kpcyrd> powersj: I see. So https://github.com/cloud-init/cloud-init/releases/tag/17.2 is the latest "portable" release for now (ignoring everything with ubuntu)?
<kpcyrd> context: I was about to use cloud-init on archlinux and noticed the package was moved back into AUR and up for adoption, so I'm currently trying to get it back on track
<powersj> kpcyrd: I'd prefer smoser or blackboxsw answer that one, but my understanding is even the latest tag should be useable, it will just have additional fixes and changes since 17.2
<powersj> ah cool!
<powersj> kpcyrd: smoser usually comes back in in a few hours to see if anything needs his attention
<powersj> fwiw here is the 17.2 announcement https://lists.launchpad.net/cloud-init/msg00117.html
<kpcyrd> nice! I think I have to do some testing before I upload a new version anyway, feel free to ping me
<smoser> kpcyrd: for sure 17.2 is latest "stable" release.  the ubuntu tags are just things that have been uploaded to  ubuntu.
<smoser> we generally strive for "always releasable trunk".
<smoser> we expect to have 18.1 next thursday (Feb 22)
<smoser> if you find things quickly that are blockers for you we can get them in.
<kpcyrd> smoser: there have been a few bugreports (see comment section https://aur.archlinux.org/packages/cloud-init/), I think most of them are resolved by bumping the version
<kpcyrd> the package currently also explicitly sets python2, but the code reads like python3 is supported, so I would try to change that as well
<catmando> hey all
<catmando> i've got a couple of problems
<catmando> 1: cloud-init keeps resetting my hostname to localhost even though i've set preserve_hostname: False and hostname: some_hostname
<smoser> kpcyrd: yes, python3 is supported, that is what is used on ubuntu. you should move that.
#cloud-init 2018-02-16
<stanguturi> Hi, Is this correct channel for the bi-weekly meeting for 18.1 or do I need to login in any other channel. Thans.
<blackboxsw> stanguturi: absolutely. probably going to start it in a couple minutes
<stanguturi> ok Great. Thanks.
<blackboxsw> ok here goes
<blackboxsw>  #startmeeting Cloud-init bi-weekly status meeting
<blackboxsw> hey folks thanks for joining in to another cloud-init biweekly status meeting
<blackboxsw> the early meeting day this week is to avoid hitting the upcoming US holiday on Monday
<blackboxsw> This meeting is probably going to be short, but we wanted to generate any discussion around the release we have scheduled for next week. I'll go through the following topics
<blackboxsw> recent changes, In-progress development, Release 18.1 Discussion, Office hours (30 mins)
<blackboxsw> Without further ado...
<blackboxsw> #topic Recent changes
 * blackboxsw is sad I don't think meetingology is logging this meeting
<nacc> blackboxsw: i saw a leading space in #startmeeting
<nacc> not sure if it matters
<blackboxsw> ahh nacc I'll try again
<blackboxsw> #startmeeting Cloud-init bi-weekly status meeting
<meetingology> Meeting started Fri Feb 16 16:08:08 2018 UTC.  The chair is blackboxsw. Information about MeetBot at http://wiki.ubuntu.com/meetingology.
<meetingology> Available commands: action commands idea info link nick
<blackboxsw> much better thanks nacc
<nacc> blackboxsw: yw
<blackboxsw> #topic Recent changes
<blackboxsw> Cloud-init upstream team has been working on an SRU for Artful and Xenial.
<blackboxsw> We discovered a couple of SRU-blocking bugs on EC2 as well as cloud-init subcommands so we've landed a couple of fixes there which are queued for SRU now
<blackboxsw> * cloud-init status --wait blocks until all stages complete (LP: #1747965)
<blackboxsw> * SRU EC2 upgrade path fix for 'systemctl restart cloud-init.service' (LP:1748354)
<blackboxsw> * Fix ds-identify nocloud detection with bind mounted writable/system-data directory (LP: #1747070)
<blackboxsw> * Tests: include missing unitests in python2.6 environments. Fix py2.6 incompatilibilies
<ubot5> Launchpad bug 1747965 in cloud-init (Ubuntu) "cloud-init status reports done before boot is finished" [High,Fix released] https://launchpad.net/bugs/1747965
<ubot5> Launchpad bug 1747070 in cloud-init "ds-identify does not see nocloud seed in core snap" [Medium,Fix committed] https://launchpad.net/bugs/1747070
<blackboxsw> * Fixed centos cloud-init build and test tooliing
<blackboxsw> * SUSE: Fix groups used for ownership of cloud-init.log [RobertS]
<blackboxsw> thanks folks for continuing to push on quality of cloud-init releases.
<smoser> o/ thanks for starting blackboxsw
<blackboxsw> not sure if I'm missing any other content that has landed in the last week and a half
<blackboxsw> I also think powersj rharper may have sorted a couple of issues with storage on our common CI on Jenkins
<blackboxsw> #link https://jenkins.ubuntu.com/server/view/cloud-init/
<powersj> Yes CI is up and running again, I have more defensive statements in to prevent us from running out of storage
 * blackboxsw is not sure, are there rumors we might have more hardware dedicated to jenkins in the future powersj ?
<powersj> We do, however it is our jenkins master that runs out of storage :\
<blackboxsw> ahh gotcha, SPOF
<powersj> yeah
<blackboxsw> ok, if no other work is 'complete'; let's  jump topics
<blackboxsw> ahh forgot ryan landed
<blackboxsw>     net: accept network-config in netplan format for renaming interfaces
<blackboxsw> per LP: #1709715
<ubot5> Launchpad bug 1709715 in cloud-init "cloud-init apply_net_config_names doesn't grok v2 configs" [Medium,Confirmed] https://launchpad.net/bugs/1709715
<blackboxsw> #topic In-progress Development
<blackboxsw> So we are working toward quality on the 18.1 release for next week.
<blackboxsw> Ubuntu specifically is finalizing verification on cloud-init 17.2.35  update for Xenial and Artful series (expectation is that this SRU will be public in 1 week).   17.2.35 is a snapshot of tip from a couple days ago
<blackboxsw> we've also published tip of cloud-init master to bionic to keep the development release up to date with latest cloud-init
<blackboxsw> current ongoing work as always is on our trello board. we tried tidying up the cards a bit
<blackboxsw> #link https://trello.com/b/hFtWKUn3/daily-cloud-init-curtin
<blackboxsw> we have upcoming branches for a new snap cloud-config module for configuring and maintaining snap packages
<rharper> https://bugs.launchpad.net/cloud-init/+bug/1749722
<ubot5> Ubuntu bug 1749722 in cloud-init "NTP: take into account systemd-timesyncd where present" [Medium,In progress]
<rharper> I'm actively working on that
<blackboxsw> this snap work will obsolete snappy and snap_config modules, so expect that they'll be deprecated. in 18.1 and dropped completely in 18.2
<smoser> https://code.launchpad.net/~rski/cloud-init/+git/cloud-init/+merge/312284
<smoser> i just moved that back into review
<blackboxsw> #link https://bugs.launchpad.net/cloud-init/+bug/1749722
<smoser> hope to take a lookc at it today.
<blackboxsw> #link https://code.launchpad.net/~rski/cloud-init/+git/cloud-init/+merge/312284
<blackboxsw> so per rharper's; chrony will be first class citizen in cloud-init
<blackboxsw> per cards in our trello board TODO lane, any card above the 18.1 release card (and anything in Doing/Review  lane) is something we want to land in the 18.1 release
<blackboxsw> ... next topic so we can talk about release
<blackboxsw> #topic cloud-init version 18.1 release (2/23/2018)
<blackboxsw> next thursday we want to cut tip of cloud-init with any features we want to fold into the 18.1 release
<blackboxsw> this point in the meeting is a good opportunity for us to discuss features and bugs that any folks think are a priority for this release
<blackboxsw> smoser we saw some talk about archlinux support/updates, do we know whether we've gotten any updates about gaps/needs/bugs there?
<stanguturi> @blackboxsw: I have two requests. One for the merge request and one about the bug.
<smoser> blackboxsw: i've not seen any more than that developer asked about here in the channel.
<stanguturi> @blackboxsw: Let me know if I can post my questions here or discuss them offline.
<blackboxsw> stanguturi: please do discuss here. open forum :)
<blackboxsw> if it gets too long a discussion, we can take it to your branch or email
<blackboxsw> #link https://code.launchpad.net/~sankaraditya/cloud-init/+git/cloud-init/+merge/337736
<blackboxsw> for reference right ?
<stanguturi> @blackboxsw: Thanks. I have a merge request posted at https://code.launchpad.net/~sankaraditya/cloud-init/+git/cloud-init/+merge/337736
<stanguturi> Want this to get into 18.1 It's a low-risk fix. Should not break anything.
<stanguturi> Also, found a bug in ds-identify . https://bugs.launchpad.net/cloud-init/+bug/1749980
<ubot5> Ubuntu bug 1749980 in cloud-init "ds-identify doesn't properly detect ISO" [Undecided,New]
<blackboxsw> ok just glancing at your branch now stanguturi looks fairly straight forward, and as always I'd like to see some unit tests covering that changeset
<stanguturi> @blackboxsw: We already have unit tests for DataSourceOVF. This actually doesn't add any new functionality. The existing test cases should be sufficient enough.
<blackboxsw> we have existing unit tests in tests/unittests/test_ds_identify.py which should be easy to extend for the additional detection
<blackboxsw> in ds-identify
<blackboxsw> yeah I was thinking more about ds-identify specifically
<blackboxsw> all said though, that branch looks low-risk and we can probably get that landed before release.
<blackboxsw> I'll add a card to trello for us to shepherd that in.
<stanguturi> @blackboxsw: Great. Thanks.
<stanguturi> @blackboxsw: Also I have a question about https://bugs.launchpad.net/cloud-init/+bug/1749980 Any inputs will be great.
<ubot5> Ubuntu bug 1749980 in cloud-init "ds-identify doesn't properly detect ISO" [Undecided,New]
<blackboxsw> #link https://bugs.launchpad.net/cloud-init/+bug/1749980
<blackboxsw> looking
<blackboxsw> ohh good stanguturi we'll sort that bug and either provide more information on this
<blackboxsw> for that bug discussion, let's move it to the "office hours" topic which comes up next
<blackboxsw> I'd like smoser rharper to peek at that too
<stanguturi> @blackboxsw: Ok. Sure. Thanks
<blackboxsw> any other topics, branches or bugs that folks are itching to get in for 18.1 release?
<blackboxsw> kpcyrd: any opdates or concerns on archlinux that you are aware of currently?
<smoser> stanguturi: you can run a command there now ?
<blackboxsw> let's transition to office hours now
<smoser> 2 things
<blackboxsw> #topic Office hours (next ~30 mins)
<stanguturi> @smoser: Sorry. Didn't quite get the question.
<blackboxsw> And thanks all for joining. Any burning questions, bugs, branches that need discussion can be brought up now.
<stanguturi> @smoser: Oh. Are you asking if I can run any commands in my virtual machine right now.? Yeah. Sure.
<smoser> stanguturi: can you run stuff int hat system ?
<smoser> a.) cat /run/cloud-init/ds-identify.log
<smoser> b.) idstr="http://schemas.dmtf.org/ovf/environment/1"
<smoser> grep --quiet --ignore-case "$idstr" /dev/sr0
<smoser> grep --quiet --ignore-case "$idstr" /dev/sr0 && echo y || echo n
<smoser> stanguturi: basically the 'is_cdrom_ovf' should have gone down the path into that grep of the cdrom block device
<stanguturi> @smoser: grep --quiet --ignore-case "$idstr" /dev/sr0 returned "grep: /dev/sr0: Input/output error"
<stanguturi> @smoser: grep --quiet --ignore-case "$idstr" /dev/sr0 && echo y || echo n returned "grep: /dev/sr0: Input/output error and then new line and then n'
<meetingology> stanguturi: Error: No closing quotation
<blackboxsw> heh thanks meetingology
<stanguturi> @smoser: Actually, read_fs_info doesn't DI_ISO9660_DEVS in my system. and because of this, dscheck_OVF returns DS_NOT_FOUND.
<smoser> stanguturi: what release are you on ?
<stanguturi> Trying it on 17.04 zesty desktop
<stanguturi> and tried with top of the tree code in cloud-init.
<smoser> stanguturi: could you potentially let me in via ssh ?
<stanguturi> @smoser: Sorry. It's on my private network. Will not be able to provide ssh.
<stanguturi> @smoser: We can do a webex conference if you want.
<smoser> stanguturi: can you ssh out of the node ?
<stanguturi> @smoser: Yes.
<smoser> ok. /query window
<blackboxsw> ok this triage will continue. if there are no other pressing bugs/concerns, we'll close out this meeting and keep pushing toward 18.1 upstream release next thursday
<blackboxsw> thanks again for your time folks. I'll post these minutes to the cloud-init github page
<blackboxsw> #link https://cloud-init.github.io
<blackboxsw> next meeting march 5th same "bat time" same "bat channel"
<blackboxsw> #endmeeting
<meetingology> Meeting ended Fri Feb 16 17:18:00 2018 UTC.
<meetingology> Minutes:        http://ubottu.com/meetingology/logs/cloud-init/2018/cloud-init.2018-02-16-16.08.moin.txt
* blackboxsw changed the topic of #cloud-init to: Reviews: http://bit.ly/ci-reviews | Meeting minutes: https://goo.gl/mrHdaj | Next status meeting: Monday 3/5 16:00 UTC | cloud-init 17.2 released (Dec 14, 2017) | cloud-init 18.1 pending-release (Feb 22)
<rharper> blackboxsw: thanks!
<kpcyrd> blackboxsw: I didn't have the time to test it yet. Do you have any recommendations for a test-bed?
<kpcyrd> also, if somebody knows somebody at OVH I would be very interested in their custom cloud-init build for archlinux :)
<blackboxsw> kpcyrd: not sure exactly what you are looking for but lxc looks to have an archlinux image:
<blackboxsw> | archlinux (5 more)            | 03a5245fd014 | yes    | Archlinux current amd64 (20180216_01:27) | x86_64  | 130.49MB | Feb 16, 2018 at 12:00am (UTC) |
<blackboxsw> that's from 'lxc image list images:'
<blackboxsw> so if you are on ubuntu lxc launch images:archlinux myarch-container might get you setup
<blackboxsw> I'm not sure if cloud-init is installed in that archlinux container image though :/
<blackboxsw> rharper: smoser ok, so I have a puzzle to ponder for the snap module
<blackboxsw> installing snaps on containers requires that squashfuse get installed. so I thought I'd provide the following config
<blackboxsw> https://pastebin.ubuntu.com/p/FzXYSJyX2d/
<blackboxsw> I need apt upgrade to run first on an image so I can apt install squashfuse
<blackboxsw> but uprades are run in cloud-init final stage
<blackboxsw> and I have snap module schedule at module config timefreame
<blackboxsw> any suggestions? though I could just provide snap: {commands: [apt-get update, apt-get install squashfuse]
<blackboxsw> bah and reminds me I needed to bring up again whether we actually want to try limiting snap:commands to only running snappy commands to avoid potential abuse/bad practices
<blackboxsw> or since squashfuse is a known dependency, maybe the snap module should apt update && apt install squashfuse for us if snap configuration is provided
<blackboxsw> ... and we know that we are in a container
<smoser> blackboxsw: you might not have seen my thread on that.
<smoser>  https://bugs.launchpad.net/ubuntu/+source/squashfuse/+bug/1628289
<ubot5> Ubuntu bug 1628289 in Snappy "snapd should depend on squashfuse (for use in containers)" [Undecided,In progress]
<rharper> blackboxsw: yeah;  I think it would be OK to install snapfuse if it's not already present and we're in a container
<smoser> blackboxsw: its busted. dont work around it in cloud-init.
<blackboxsw> "it's busted"  you mean snappy requiring squashfuse to run in a container?
<smoser> the container image should have it.
<smoser> for unknown reason snapd does not want to fix that.
<smoser> https://github.com/snapcore/snapd/pull/3605
<smoser> https://github.com/snapcore/snapd/pull/2856
<smoser> they are not able to sru it because it would cause issue (due to an apt bug that would hold snapd rather than grab the new recommends)
<rharper> let's raise it again
<smoser> but that has no bearing against bionic , where they're just ignoring it
<smoser> see the second. i asked on jan 17
<rharper> to kirkland
<smoser> you saw my thread, rharper. i raised to Beret
<smoser> blackboxsw: so cloud-init should not try to fix this broken scenario
<smoser> i too went down that route which is how i got to know this stuff.
<blackboxsw> ok, so cloud-init should expect breakage on lxd platform
<rharper> smoser: yes, I'm fine with not working around it, should we at least raise a warning if we detect snap installs and on-container in cloud-init and reference the LP and PRs ?
<rharper> I'm sorta of the mind that we shove it in for testing for now; it's just much nicer to test cloud-config with lxd
<smoser> whats annoying is that if they do not fix it in bionic, then it will have another 2 years (at least) until fixed
<smoser> how about this
<blackboxsw> cloud-init may not need logic to forcibly install squashfuse on a container, but we could present an example doc that says this is how to install snaps on a container.
<smoser> a.) do not restrict snap commands to 'snap' ...
<blackboxsw> smoser my current branch doesn't restrict
<smoser> b.) warn if snap commands have a arg0 other than snap
<blackboxsw> sure
<smoser> (to discourage as you suggest)
<smoser> c.) then we can feed 00_apt: [apt-get, install, -qy, squashfs]
<blackboxsw> and cloudinit module example will list an lxd-supported snap:commands  cloud-config that notes the bug
<blackboxsw> yeah
<blackboxsw> docs are already in place for this. just wanted to check if we wanted to bake that 'fix' into cloud-init module logic instead of exposing it
<smoser> i guess i'm even open to taking it another step
<smoser>  snap/install_squashfs: true
<smoser> or
<smoser>  snap/squashfuse_in_container: true
<blackboxsw> hrm since we know it's a bug, I agree we probably shouldn't bake in a workaround into our new snap module configuration properties. but I can be swayed.
<smoser> as rharper suggested, for test... its just so useful
<smoser> you can document that thing as E_NO_RELY_ON_THIS_TESTING_ONLY
<blackboxsw> secondary (unrelated) question about snap module:     our old snap_config module ran a "snap managed" check  before trying to create a snap user, I'm planning on an line shell conditional example to check snap managed before "snap known system-user" call to cover this case
<smoser> even warn if its used.
<blackboxsw> .. planning an *inline shell conditional documentation example* which would showcase check managed before create user
<rharper> blackboxsw: I;ve not tried in a while, maybe the create-user does the check now
<rharper> that'd be ideal
<blackboxsw> agreed , I'll poke at it
<blackboxsw> ok, so back to snaps on lxd for testing. rharper/smoser you are of the mind that snap config module just does the lift for us on containers with a warning message?
<rharper> blackboxsw: I dunno how I feel about leaving a warning in about brokenness; which we'd need to update;
<rharper> maybe in exception handler we could check if incontainer and no squashfuse but that still feels wrong
<rharper> we don't really know what container environment we may be in
<rharper> some may be privledged and others not
<blackboxsw> vsm
<blackboxsw> oops
<blackboxsw> can't we know what container we're in checking /run/systemd/container
<blackboxsw> and installing only if that's "lxc"
<rharper> well, it's not the container name
<rharper> it's the capabilities
<smoser> wait now.
<smoser> i w was not clear
<smoser> snap/squasfuse_in_container: false
<blackboxsw> ok, hmm. so explicit better than implicit side-effects based on perceived environment I suppose.  Shall we just surface the snap/squashfuse_in_container: true option then?
<smoser> default to false
<blackboxsw> yeah that's easy enough and easy to deprecate that option when that bug is fixed
<smoser> and use util.is_container
<blackboxsw> +1 on is_container. BTW I'm adding deprecation warning messages to both snappy and snap_config modules in my snap branch. I wanted to warn existing users that those modules will be removed in 18.2
<blackboxsw> sound ok?
<smoser> sure.
 * smoser has to run.
<blackboxsw> what about snapuser config option under users_and_groups?
<blackboxsw> have a good one smoser
<blackboxsw> can talk about this against my branch when I post it
#cloud-init 2018-02-17
<StefanS_> Hi there! I'm just starting to learn cloud-init and I have two or three task to manage, but I have absolutely no clue how to start. I already read lot's of documentation about cloud-init, but I feel somehow lost :-(
<StefanS_> One task is to write the IP address of the VMs network "foo_net" into a specific file. The VW is member in the networks "foo_net" and "bar_net"...
<StefanS_> Another task is to execute the one or another script depending on userdata or (better) instance metadata
#cloud-init 2018-02-18
<kpcyrd> blackboxsw: one issue that I ran into is that a file is installed to /lib/udev... this doesn't work in archlinux, as /lib is a symlink to /usr/lib
<kpcyrd> blackboxsw: `LIB = "usr/lib"` in setup.py
#cloud-init 2020-02-10
<DanyC> hi all, anyone has any tips/ thoughts as to why cloud-init wouldn't run on a new EC2 instance created from an AMI which was created from a running EC2 instance? Pls not before creating the image i've run "cloud-init clean" to let cloud-init know/ think that is a first boot
<meena> DanyC: and it didn't believe youâ¦ hrmâ¦
<meena> DanyC: cloudinit started on boot? what's /var/run/cloud-init look like? what do the logs say?
<DanyC> meena: yes cloudinit started on boot. nothing odd in cloud-init.log while in -init-output.log it doesn't show me my user-data script
<meena> DanyC: so what happens when you do a cloud-init clean --logs --reboot now?
<DanyC> meena: same thing, i don't see any chance in init-output nor in the configuration of the instance.
<meena> DanyC: any chance of/for what?
<meena> oh, change.
<DanyC> in short i have the same issue described (maybe better than me here) at https://www.reddit.com/r/aws/comments/bwfvtz/when_does_user_data_run_in_ec2_instances_whose/
<DanyC> *change sorry for typo meena
<meena> DanyC: at the very least, your hostname should have been set.
<meena> n.b.: I have been lucky enough to have never worked with AWS. so take everything i say with a bag of salt.
<meena> (i have worked with Salt however)
<ananke> DanyC: interesting, I'll have to test this when I get to the office. I'm working now on a project involving that exact scenario: creating new AMIs from existing ones with packer. cloud-init does run in the results, but I haven't tested if user-data can be passed
<DanyC> ananke: let me know your findings. Is a bit hit and miss on my side: as 2 our of 10 works. I've even tried to set/ do https://aws.amazon.com/premiumsupport/knowledge-center/execute-user-data-ec2/ but no luck
<ananke> will do. I'll be in the office in a couple of hours, and I'll get to that first. this potentially means I'll have to bake in some cloud-init cleanup at the end of baking the image with packer
<ananke> DanyC: what distros are you testing this with? I can check on amazon linux 2, centos 7, & ubuntu 18.04 lts
<DanyC> ubuntu 16.04 ananke
<ananke> didn't that reach EOL last year?
<ananke> ahh, nevermind, studio did. 16.04 lts is still in maintenance update phase until 2021
<DanyC> ananke: happy to provide the cloud-init version if that helps.
<ananke> DanyC: how are you creating those AMIs? packer or some other process?
<DanyC> ananke: custom, i create a vmdk from an iso using qemu and then import the main AMI. Then at T1 create new instance from the main AMI (cloud-init user data kicks in), do some manual config, run "cloud-init clean", then create image from the running EC2 (w/o reboot btw).  Then at T2 i create a new EC2 from the snapshotted AMI, userdata doesn't kick in
<ananke> sounds like packer could help you automate that
<DanyC> ananke: it could but that won't solve my prob. (there is more tech debt i need to tackle to get it going).
<ananke> DanyC: btw, how do you pass your user-data to that new instance? using aws cli, their web console, or something else?
<DanyC> ananke: through cfn, creating a basic EC2 instance and passing a bash script to it. Note i'm not using cfn-hup/ cfn-init etc, keep it dead simple
<ananke> DanyC: and you're using the base64 encoding function right?
<ananke> something like:       UserData:
<ananke>         Fn::Base64: !Sub |
<DanyC> ananke: that is correct, that is what i'm doing
<ananke> that's actually on my to-do list this week, to have a new cloud formation template for testing the AMIs I create, including sending user-data. right now I just have the cloudformation template surrounding the packer creation
<Odd_Bloke> DanyC: Could you file a bug and attach the tarball that `cloud-init collect-logs` creates on an affected instance, please?
<Odd_Bloke> (File a bug at https://bugs.launchpad.net/cloud-init/+filebug :)
<DanyC> Odd_Bloke: sure thing i can do so
<ahosmanMSFT> @rharper I tried your suggestion and it should work in theory, but I get
<ahosmanMSFT> type object 'PlatformComponent' has no attribute 'preserve_instance'
<ahosmanMSFT> https://paste.ubuntu.com/p/KTSXVh5TRy/
<rharper> ahosmanMSFT: yes, I was going to reply
<rharper> you cannot reference a property of the definition
<rharper> the object gets an instance in collect where it creates a PlatformComponent
<rharper> in collect_snapshot IIRC
<rharper> somewhere from the args.preserve_instance where that value is known, we'll need to propigate that through to the platform
<rharper> possibly through the instance constructor; I was thinking it could be set in the config which is passed into the Image creation, but I'm not quite sure yet
<ahosmanMSFT> rharper: I'll target the source then and pull the property from there instead
#cloud-init 2020-02-11
<malasfar> GoodMorning Folks. i have two questions.. in VMware KBs it states to enable cloud-init customization use âdisable_vmware_customization: falseâ . shouldn't t his be set to true.. otherwise what does this property do if it was set to true when you define in /etc/cloud/cloud.cfg ?
<malasfar> 2nd Questions i have when  you set âdisable_vmware_customization: falseâ the datasource changes from being OVF which is the only datasource configured to seed=vmware-tools ?
<malasfar> any help is appreciated answering the above two questions. i m very close. like when provisioning a VM from vRA ( vRealize Automation ) our automation tool with static IP assignment it connects the NIC and customize the machine without rebooting it , I just have no user data  coming in with my cloud config code because datasource is changing to
<malasfar> vmware-tools for some reason
<meena> malasfar: link to those docs?
<malasfar> @meena https://kb.vmware.com/s/article/59557
<Odd_Bloke> malasfar: seed=vmware-tools is something that the OVF data source would display, could you pastebin the output of `cloud-init status --long` before and after making your change, please?
<malasfar> @odd_bloke  before status is none.. after its status: donetime: Tue, 11 Feb 2020 13:40:08 +0000detail:DataSourceOVF [seed=vmware-tools]
<malasfar> running sudo cloud-init query userdata  doesn't return anything . so i have no userdata present
<Odd_Bloke> malasfar: Please pastebin the output, I want to see it exactly.
<Odd_Bloke> malasfar: (And by pastebin I mean use https://paste.ubuntu.com/, for example.)
<malasfar> cloudadmin@vra-vmwlab-mysql-892:~$ sudo cloud-init status --longsudo: unable to resolve host vra-vmwlab-mysql-892: Connection timed out[sudo] password for cloudadmin:status: donetime: Tue, 11 Feb 2020 13:40:08 +0000detail:DataSourceOVF [seed=vmware-tools]
<Odd_Bloke> malasfar: OK, I'm going to ask a third time: please use a pastebin.
<malasfar> i dont know what that is.. do you want me to paste the content in pastebin
<malasfar> excuse my ignorance
<Odd_Bloke> malasfar: 10:08:46   @Odd_Bloke | malasfar: (And by pastebin I mean use https://paste.ubuntu.com/, for example.)
<Odd_Bloke> I don't mind ignorance, but I would like to feel like you're reading the messages I send. :p
<malasfar> you could have said .. paste in there and send me the url . if this what i think it is . all you said is use pastebin   .. https://paste.ubuntu.com/p/hd8mjH7CWz/
<malasfar> anyway i think i got it now.. appreciate your help in understanding the behaviour i m experiencing
<Odd_Bloke> malasfar: Cool, thanks for pasting. :)  It sounded like you were seeing the datasource _change_.  Is that pastebin from before or after the "change"?
<malasfar> thats after.. before is usually none since i use cloud-init clean --logs before shutting down the VM and converting to a template
<malasfar> the template is configured with dpgk-reconfigure cloud-init selecting only OVF as the datasource just not sure why it keeps changing to vmware-tools
<Odd_Bloke> It is using OVF, that's what "DataSourceOVF" is telling you. :)
<Odd_Bloke> The OVF data source, that is.
<malasfar> the type usually is ISO from the testing i have seen before .. this only happens if i m using cloud-init as the customization engine
<malasfar> thats when i can actually see the userdata using the query userdata command
<Odd_Bloke> malasfar: Can we take a step back to help me understand the problem?  What are you trying to achieve by changing the value of disable_vmware_customization?
<malasfar> i m trying to use cloud-init as the Guest OS customization engine based on this KB https://kb.vmware.com/s/article/59557
<malasfar> setting disable_vmware_customization: false to enable Guest OS customization with cloud-init
<malasfar> looks like from my research so far Setting this to false invokes the Cloud-Init GOS customization workflow.Setting this to true invokes the traditional GOSC script based customization workflow.   would you agree ?
<malasfar> there is an interesting blog that came up recently talks about how to hack cloud-init for it work with vSphere when provision VMs with an automation tool like vRealize Automation. here it is for reference https://vnuggets.com/2020/01/29/vra-with-cloud-init-and-static-networking/
<malasfar> would love to know your opinion on what's being proposed here in the article
<Odd_Bloke> I'm out of my depth with this level of VMWare stuff, I'm afraid.
<malasfar> i can help there . i just need someone not he cloud-init side.. i m happy to do a workshop
<malasfar> someone on the cloud-init side *** correction
<malasfar> we can have a zoom if you can spare an hour, you will be doing the community a favour :)
<malasfar> i can explain what i m facing here , which similar to what other people are experiencing working with VMware
<Odd_Bloke> The VMWare data source was mostly contributed by VMWare, so the core team don't have a depth of knowledge there, I'm afraid.  I suggest taking this up with your VMWare contacts, to ask them to improve the experience.
<Odd_Bloke> There's only so much we can do when working with proprietary platforms. :)
<malasfar> no one knows .. its already with engineering for the passed year
<malasfar> like i would love to use cloud-init with vSphere and our automation tool and i already spent a lot hours trying to get it to work .. i guess no light at the tunnel for this here
<malasfar> end of the tunnel that is *..  okay anyway i ll continue on my own .. thanks
<malasfar> cheers all .. thank you for your time
<meena> anyone fixed all my bugs yet?
<smoser> Odd_Bloke: around ?
<smoser> you'll know this.
<smoser> pip install <something>
<smoser> i can use '--no-deps', but that just feels dirty
<Odd_Bloke> smoser: Yep, that's how you install with pip.
<Odd_Bloke> *runs*
<smoser> :)
<smoser> is there a way to have it use dpkg provided modules ?
<smoser> like... only install deps if you have to ?
<smoser> the scenario here is that we know all the dependencies are satisfied (by distro-installed)
<Odd_Bloke> So I would _expect_ pip to detect the system-installed packages and know that the dependencies are already satisfied.
 * smoser tries that
<Odd_Bloke> (If you're in a virtualenv, you'll need to create it with --system-site-packages.)
<smoser> right. this is not a virtual env.
<smoser> so starting a container :)
 * smoser afraid of pip
<smoser> Odd_Bloke: thank you. you appear to be right. at least in the minimal case that i tested.
<Odd_Bloke> \o/
<Odd_Bloke> I love to appear correct. ;)
#cloud-init 2020-02-12
<otubo> Do we ever check the kernel version inside cloud-init? I'm trying to grep for an example but can't find it.
<otubo> Long story short, on my (already closed) PR[0] I recently found out that the fallocate on XFS bug is hit only on kernels < 4.18 (reference: man 8 swapon).
<otubo> So I'd like to introduce a condition ` and util.system_info()['release'] >= '4.18'' to the if statement to cloudinit/config/cc_mounts.py:252
<otubo> Couldn't find any trace of packaging.version.parse for that matter, though. Not sure if cloud-init already does this kind of version comparisons in the code base.
<otubo> [0] https://github.com/canonical/cloud-init/pull/70
<meena> otubo: what would packaging.version.parse contain? do we already produce a system_info()['release']?
<otubo> meena: packaging.version.parse contains parsers for standard versions systems, we could do things like version.parser(util.system_info()['release']) > version.parse("4.18") for example
<otubo> meena: yes we already have util.system_info()['release'] in place
<meena> otubo: cut it down to only have a max of one `.` and then compare them like floats :P
<meena> I think i did that somewhere in puppet codeâ¦ andâ¦ i'm not proud of it, but not ashamed of it either :P
<otubo> :-D if it works, it works
<meena> >>> float(str.join('.', os.uname()[2].split('.')[0:2]))
<meena> 4.15
<meena> this is ugly, and not safe.
<otubo> meena: my dear god
<meena> otubo: i'm sure there's a less worse way to do this
<meena> otubo: https://regex101.com/r/6XkWS3/2
<meena> rather, /3. no one should have negative versions >_>
<meena> and /4 more performant without those flags.
<meena> now with tests, https://regex101.com/r/6XkWS3/5/tests â that website is wonderful ð
<ananke> wonder if the sample test I added now shows up for others
<ananke> nah, looks like it would have to be forked
<otubo> meena: OMG! That's helpful. Thanks! :-)
<DanyC> hi, question - running "cloud-init clean --log --seed" will that be enough for cloud-init to run again the "modules-final/config-scripts" if i'm cloning from this VM ?
<DanyC> or do i need to delete "/run/cloud-init/" dir too to erase any state which might make cloud-init think is not a "first time boot" ?
<meena> DanyC: i'd think that's what `clean` does. though i don't know what --seed does
<DanyC> meena: thanks
<Odd_Bloke> otubo: So a danger with matching on kernel versions is that "4.18" on one distro can be very different to "4.18" on others.  However, in this case, I think the two cases we need to worry about are: (a) a version older than 4.18 which has support/fixes backported, and (b) a version that claims to be (at least) 4.18 but doesn't have the support/fixes.  I feel like we can safely dismiss (b), because that
<Odd_Bloke> seems likely to be a kernel bug/regression.  For (a), I think the worst case is that we use dd when we could have used fallocate, which isn't a very bad case at all.
<Odd_Bloke> meena: I don't think you can compare kernel versions as floats, because kernel version 4.18 > kernel version 4.4 but 4.18 < 4.4.
<meena> Odd_Bloke: wat
<meena> Odd_Bloke: aah, shite.
<Odd_Bloke> :)
<rharper> the cheaty way is to break major minor into ints
<Odd_Bloke> Yeah, (4, 18) > (4, 4).
 * rharper may have cheated a few times
<meena> yeah, that way you can compare all dots? (4, 18, 1)
<meena> whooaaaah.
<meena> rharper: that's so cool
<Odd_Bloke> It's barely even cheating, that's why `sys.version_info` is a tuple, so you can do things like `sys.version_info > (3, 3)` for Python version checks.
<meena> TIL
<meena> wonder if that works in ruby, too
<meena> cuz i'm having to teach a developer here what Semver isâ¦
<meena> or, all developers.?
<Odd_Bloke> It's probably sufficient for kernel major/minor/patch, but it falls over if your versioning has prereleases or postreleases (like apt packages do).
<rharper> it just gets strange with extra parts, 4.15.0-72-generic
<meena> rharper: LA LA LA LAAAAA
<Odd_Bloke> I don't know what you're talking about meena, we just happen to rev major versions on January 1st every year. ;)
<rharper> =)
<meena> Odd_Bloke: that's ð¯ % valid
<meena> http://sentimentalversioning.org/
<Odd_Bloke> I can only aspire to have an idea as good as using pi for version numbers.
#cloud-init 2020-02-13
<AnhVoMSFT> Hi folks, typically I see Looking for data source in: ['Azure'], via packages ['', u'cloudinit.sources'] that matches dependencies ['FILESYSTEM'] in a working image. We have some cloud-init package created by a distro that is showing in log "Looking for data source in: ['Azure', 'None'], via packages ['', u'cloudinit.sources'] that matches
<AnhVoMSFT> dependencies ['FILESYSTEM'] and it's not working as expected. Where should I begin to look at? /etc/cloud/cloud.cfg.d/ has the same file between two deployment and indicating Azure in the datasource list. How did 'None' get in there?
<AnhVoMSFT> I kept getting disconnected from this webirc and keep losing messages. Is there anyway to configure webirc to not disconnect?
<meena> ahosmanMSFT: uhm, you coud pay for cloud-irc?
<AnhVoMSFT> Switched to a client. Hopefully this works better.
 * meena uses thelounge
<Odd_Bloke> AnhVoMSFT: My guess is that the issue is not to do with that line, but something else.  Could you file a bug and attach the `cloud-init collect-logs` tarball from an affected instance, please?
<AnhVoMSFT> Odd_Bloke: he's generating another package for it. I felt like something else was wrong too; but from the log line that was the only thing difference up until that point. We will test the new package later today and see if issue persists.
<AnhVoMSFT> I am curious though, on how the datasource list is constructed, when does None get added?
<Odd_Bloke> AnhVoMSFT: OK, cool, let us know. :)
<Odd_Bloke> If it isn't in the datasource_list configuration in /etc/cloud/..., then I'm not sure off the top of my head.
<rharper> Odd_Bloke: blackboxsw:  https://github.com/canonical/cloud-init/pull/205
<Odd_Bloke> rharper: Approved, had to merge master so waiting for CI to land.
<Odd_Bloke> (Thanks!)
<rharper> Odd_Bloke: \o/
<pfak> any tips for troubleshooting why static ip address informatin isnt being set? my localDS specificies eth0 in 'network-config' but cloud-init is writing out ens18 to /etc/netplan/*
<pfak> and using dhcp instead of static
<pfak> of course i figured it out,now .. curtin wrote something out to /etc/cloud/cloud.cfg.d/ .
<rharper> pfak: curtin will write out the network-config that maas provided;
<pfak> rharper: this was the -live dvd
<pfak> but yeah, i blew that away and now cloud-init is working perfectly with new ifnames disabled
<rharper> subiquity writes network config directly to /etc/netplan/  IIRC, I don't believe subiquity provides network-config to curtin to write through
<pfak> /etc/cloud/cloud.cfg.d/50-curtin-networking.cfg was writen out by the ubuntu live installer
<rharper> which version of subiquity ?
<pfak> whatever is shipped with 18.04 installer
<pfak> 18.04.3
<rharper> hrm, ok
<pfak> idk how to tell, the package isnt installed
<pfak> and i blew away /var/log/installer
<rharper> ok
<pfak> sorry i couldnt be more helpful
<rharper> no worries
<Odd_Bloke> rharper: blackboxsw: powersj: I have a branch to convert cloud-init to use pytest (instead of the long-deprecated nose).  Given that it may cause unexpected issues, I propose aiming to land it soon after we cut 19.1.  What do you think?
<rharper> 20.1 ?
<Odd_Bloke> Yes.
<rharper> I think makes sense to me
<powersj> I'm good with it
<rharper> Odd_Bloke: what impact do you think this will have on existing branches which add/modify unittests ?
<Odd_Bloke> rharper: I haven't changed any test code in my branch, so I believe it will be minimal.
<Odd_Bloke> (I will send an email out to the list, but figured that shouldn't be the first that anyone else had heard of it. ;)
<powersj> Odd_Bloke, what would come of say... the xenial branch?
<powersj> the packaging branches
<Odd_Bloke> We would need to update Build-Depends there, at least.
<powersj> deps are in main?
<rharper> Odd_Bloke: ok
<Odd_Bloke> This is just changing Build-Depends, so they don't need to be.
<Odd_Bloke> (They do for trusty, but we don't need to worry about that for cloud-init.)
<powersj> aahh right
<powersj> thanks!
<rharper> Odd_Bloke: and they are available on xenial (out side of main) at levels that work ?
<Odd_Bloke> rharper: I have Travis passing, which includes the xenial package build for cloud tests.
<rharper> \o/
<rharper> perfect
<rharper> that's a big +1 for landing post 20.1; definitely mention that for downstreams
<Odd_Bloke> That's using bddeb, but I would be surprised if we saw anything different in the packaging branches.
<rharper> I wonder about non-ubuntu though
<rharper> likely worth seeing what robjo and otubo think of the branch
<Odd_Bloke> pytest was in main in trusty, so it's been Around for long enough that we're probably fine.
<rharper> yeah
<robjo> rharper: Odd_Bloke What branch?
<Odd_Bloke> robjo: Converting to pytest for testing.
<Odd_Bloke> The branch is not yet proposed, I'm about to do that then send an email out to the list notifying and asking for feedback.
<robjo> That should be fun
<robjo> Glad I have no active work :D
<Odd_Bloke> Can anyone remind me how to build a cloud-init package from the packaging branches?
<Odd_Bloke> I don't have an orig tarball, and make-tarball produces something that includes .tox.
<rharper> Odd_Bloke: one sec;
<rharper>  https://launchpad.net/ubuntu/+source/cloud-init  has links to orig tarballs;
<rharper> and then I use build-package after checking out the release branch,  https://github.com/CanonicalLtd/uss-tableflip/blob/master/doc/ubuntu_release_process.md  , search for "build-package"
<rharper> it has references to downloading an orig tarball and build-package script and example usage
<Odd_Bloke> OH, I had forgotten about build-package.
<rharper> Odd_Bloke: I'm surprised make-tarball included .tox
<rharper> it didn't for me
<rharper> Odd_Bloke: I've also done:  git checkout -b mybuid; git merge origin/ubuntu/$release; ./tools/bddeb;
<Odd_Bloke> Doesn't bddeb generate its own debian/ dir?  Or does it skip that if one is already present?
<rharper> Odd_Bloke: it looks like it copies from the topdir, so if you merge over it; then it should be the "release branch" version of debian/
<rharper> and you should bddeb --release $release
<Odd_Bloke> OK, I've drafted the email but I'm well EOD now, so I'll reread it with fresh eyes in the morning and send it then.
<blackboxsw> tiny tiny pr for doc update related to swapfile and cc_mounts configuration.
<blackboxsw> https://github.com/canonical/cloud-init/pull/212
 * blackboxsw is reviewing the netbsd branch, I think Goneri got back to the review comments
<blackboxsw> https://github.com/canonical/cloud-init/pull/62 for me
<blackboxsw> hrm I think there are more commits to come there, like consolidation of FreeBSD and NetBSD distro modules into a common parent class maybe. I'm not sure there, will ping the author
#cloud-init 2020-02-14
<otubo> rharper: Odd_Bloke we don't run unittests for RHELat the moment, but we have plans to run them in a near future for Fedora releases. But since we don't have anything in place right now, I guess we're fine with the changes :-) thanks for pinging
<otubo> mhayden: dustymabe the pull-request to rebase to 19.4 for fedora rawhide is open for review :-) https://src.fedoraproject.org/rpms/cloud-init/pull-request/6
<dustymabe> otubo: woot!
<dustymabe> that's awesome
<dustymabe> otubo: i'll try to have it reviewed by Monday
<otubo> dustymabe: no worries. Actually I just found out cloud-init upstream will cut 20.1 next 18th of Feb. Perhaps we should wait a couple more days and have the bleeding edge? :-)
<dustymabe> otubo: how about we get 19.4 into rawhide and (f32) first
<dustymabe> and then we leave 19.4 for f32 and go ahead and put 20.1 in rawhide after a month or two
<otubo> dustymabe: yeah, I think perhaps that's better. No need to rush :-) Fedora is in 17.1 for two years now
<dustymabe> otubo: ð
<smoser> why does tox environment creation take *so* long
<smoser> oh.
<smoser> ipv6 messup here.
<smoser> suck
<blackboxsw> nice email to the list btw Odd_Bloke on pytest
<smoser> Odd_Bloke: you or i have some difference in lxc
<smoser> http://paste.ubuntu.com/p/SgggZJChqV/
<smoser> and fwiw.. at least before moving to github (and i think even still) some jenkins c-i runs run-container
<smoser> to build centos
<smoser> of course i can't see jenkins so i cant show you that
<rharper> smoser:  at least the -proposed runs use run-container
<rharper> and ctool wrappers run-container I think for the centos build
<rharper> so we do need to sort those out
<rharper> Odd_Bloke: ^
<blackboxsw> also, in reviewing dan's code, I've found that our run-container centos (which I use during copr-build updates @ SRU time) have mismatched pkg deps for centos/7/8 that previously worked. I'm trying to sort that now as a separate PR from the pytest work
<blackboxsw> and our copr build test in jenkins is using run-centos instead of run-container so checking that too as I think we should drop this depreceated tools/run-centos now as it's been deprecated since 2018-05-23
<blackboxsw> that said, run-centos calls run-container anyway, so no big change there
<blackboxsw> I'll put a PR up for jenkins jobs to fix that aspect
<blackboxsw> and we can then drop tools/run-centos
<blackboxsw> btw the shift to python3 (and dropping py2 support) looks like it is a significant problem for finding other python3 dependencies in CentOS 7 and 8. our last successful copr-build jenkins job worked when we were still using python2 packages on centos7. now it seems I'm unable to find deps like: python3-httpretty and python3-tox etc on CentOS 7 or 8.
<blackboxsw> otubo: do you know offhand what repos I might be missing in our test vm setup to allow centos 8 or 7 vms to find python3-httpretty python3-unittest2 packages?  We generally rely on enabling epel repo. But, I see something about a PowerTools repo for CentOS 8 (not 7) that may contain more rpms cloud-init depends on for testing.
<blackboxsw> otubo: question 1, is CentOS 8's PowerTools repo something that is generally public and recommended for access various python development packages?
<Odd_Bloke> smoser: I see the same output from those lxc commands.
<Odd_Bloke> Oh actually, I'm on a different system now, let me see if I can get run-container to work here.
<blackboxsw> Question 2: Is there any way to access python3 rpms on CentOS 7 for httpretty, unittest2 and contextlib2
<Odd_Bloke> OK, it's working on this system, I'll have to dig into the issue on the other system.
<Odd_Bloke> I've definitely run into issues where the tooling assumes a local lxd remote before.
<Odd_Bloke> So maybe it's that.
<blackboxsw> otubo: those 2 questions may impact whether you'll have to patch cloud-init 20.1 etc if you are also somehow involved in CentOS updates when you are doing rawhide
<blackboxsw> though not sure if CentOS is something you are typically involved in.
<blackboxsw> here's the patchset I was starting for centos 8  https://paste.ubuntu.com/p/ft3QdgFqX7/
<blackboxsw> against Dan's branch
<blackboxsw> but I think it's still hitting Error: Unable to find a match: python3-httpretty python3-unittest2 python36-PyYAML python3-contextlib2 python-tox
<rharper> blackboxsw: you have epel enabled ?
<blackboxsw> strange is that Cent7 has python36-contextlib2-0.5.1-3.el7.noarch.rpm & python-contextlib2-0.5.1-3.el7.noarch.rpm  but Cent8 has no packages for this
<blackboxsw> rharper: I do
<rharper> ok, it may well be that real py3 on cent7 isn't easily available
<blackboxsw> I'm thinking Cent8 may need that powertools repo too.
<rharper> I'm sort of *ok* not producing cent7 py3; we've never done it in the past; and maybe the cent7 build should be done from the 19.4 release branch
<rharper> ie, py2 support only
<rharper> and then tip can be py3 cent8
<blackboxsw> rharper: and yes i'm thinking py3 is only partially supported on Cent7 which means our recent drop of any py2 in Dec 2019  will prevent us from adding more copr builds for cent7 from here on forward
<blackboxsw> yep
<rharper> blackboxsw: unless we build from stable release branch
<Odd_Bloke> Isn't paride looking at these failures?
<rharper> we can update our jenkins jobs / copr build repos
<rharper> Odd_Bloke: he was
<rharper> I pointed him at my py3/fedora-build branch
<Odd_Bloke> Let's make sure we aren't duplicating work here.
<rharper> blackboxsw: yes
<rharper> best to circle with paride next week
<blackboxsw> right https://github.com/canonical/cloud-init/tree/stable-19.4
<rharper> but I do think cent7 + py3 is not likely to happen
<blackboxsw> I think that too
<Odd_Bloke> This isn't urgent by any means, the builds have been broken for a minute.
<rharper> nope
<Odd_Bloke> Wasn't Cent7 the reason we haven't dropped 3.4 too?
<rharper> no
<rharper> SLes
<blackboxsw> sles
<blackboxsw> yeah
<Odd_Bloke> Ah, my bad.
<blackboxsw> -> lunch
<Odd_Bloke> OK, so run-container doesn't work at all for CentOS 7 on master.  So my branch definitely doesn't regress it. :p
<Odd_Bloke> That said, I've pushed up one change to pkg-deps.json which means that pytest runs and fails, rather than not being found at all.
<Odd_Bloke> (Which appears to be a step up from master, actually, where I see: /usr/bin/python3: No module named nose)
<blackboxsw> Odd_Bloke: +1 on centos not being related to your branch. I think generally the pkg-deps.json is broken since end of-jan when we  dropped py2 support in that tool
<blackboxsw> I'd be fine with us not dealing with the centos deps issues in your branch as long as we are certain we aren't regressing specifically on the pytest package dependency.
<blackboxsw> which is, I guess, what you just said
 * blackboxsw really goes to make some lunch now
<Odd_Bloke> :)
<Odd_Bloke> Yep, and I've pushed up a change so we aren't regressing on the pytest change.
<meena> anyone fixed all my bugs yet?
<meena> actually, just one "bug" networking.
<meena> would love to read some â¦. official thoughts on the mailinglist.
<blackboxsw> meena ,is this related to cloudinit.net refactor and potentially blowing up cloudinit.net modules in favor of network-aware distro classes versus keeping cloudinit.net modules/functions a place where the modules themselves know a bit about distro-specific needs? per https://lists.launchpad.net/cloud-init/msg00237.html
<meena> blackboxsw: in particular, re robjo 's, whos' suggestion is pretty, good, i think.
<blackboxsw> shhh, it'll go to his head ;)
<meena> Â¯\(Â°_o)/Â¯
<johnsonshi> Is there currently a way to specify the size of a partition in terms of actual bytes and not as a percentage of disk space using the disk_setup module? From the docs: "The size for partitions is specified in percentage of disk space, not in bytes (e.g. a size of 33 would take up 1/3 of the disk space)."
<Odd_Bloke> johnsonshi: I believe the docs are correct, I'm afraid.
<johnsonshi> Odd_Bloke: thanks!
<rharper> Odd_Bloke: hehe
<Odd_Bloke> rharper: We can't even win when they're accurate!
<Odd_Bloke> ;)
<rharper> =)
<rharper> lol
<blackboxsw> haha
<meena> you can't win with FLOSS
<meena> just, surrender.
<meena> (and fix all my bugs, please)
<blackboxsw> meena, man this cloudinit.net discussion is a tough one. I can see the need for making things distro-specific for sysconfig render cases because rhel and suse differ significantly in how they handle sysconfig options for various needs.  But, from the perspective of eni, freebsd(/etc/rc.conf)  and netplan (/etc/netplan/*) these network renders are specific to the single network backend that they are talking to. In
<blackboxsw> eni's case, not much difference between debian and ubuntu. netplan is it's own thing for netplan backend and freebsd renderer is it's own thing for /etc/rc.conf .  Given the netbsd support branch  https://github.com/canonical/cloud-init/pull/62 it doesen't seem like too much difference there in how rc.conf is rendered on ehandled
<blackboxsw> *on each "flavor"
#cloud-init 2020-02-15
<meena> blackboxsw: so the problem then is reduced to sysconfig, and how to extract Linux specific functionality from it, by, say, using the distro classes, or building a new hierarchy, even? for ephemeral, since those actions aren't supposed to be permanent
<and-win> Hello
<and-win> Could someone clarify how I can determine OS type with cloud-init to use it in Jinja-template of user-data? The goal is use different `runcmd` commands depending on OS
<blackboxsw> and-win: I don't think offhand s
<blackboxsw> That support is there. But resolving this bug would surface that info in jinja templates. https://bugs.launchpad.net/cloud-init/+bug/1829441
<ubot5> Ubuntu bug 1829441 in cloud-init "instance-data: surface merged cloud-config through cloud-init query command" [Wishlist,Triaged]
<blackboxsw> As merged cloud config defines system_info.distro
<blackboxsw> and-win: alternatively, you could have runcmd check the distro with something like [ `python3 -c 'from cloudinit.util import get_linux_distro; print(get_linux_distro()[0])'` == "redhat" ] && echo "YES"
<blackboxsw> I realize that'll be painful. it's be best if we can simply extend cloud-init builtin variables to source cloudinit.util.get_platform_info() (which also contains get_linux_distro values too)
#cloud-init 2020-02-16
<blackboxsw> and-win: I couldn't help myself :) https://github.com/canonical/cloud-init/pull/214   this is probably what you wanted
<blackboxsw> I'll fixup unit tests and make it pretty for next week
<malasfar> For the Cloud-init - vSphere customization issue using any automation tool . I found my own workaround and here is my blog . https://vmwarelab.org/2020/02/14/vsphere-customization-with-cloud-init-while-using-vrealize-automation-8-or-cloud/
<malasfar> was hoping for more support from this group but nobody wants to touch vSphere cause they are afraid according to some of you. .. but got it done on my own thinking out of the box which is something you may want to thing about some time. . this is the last time i join this channel.   Thank you!
