#cloud-init 2014-10-20
<smoser> harmw, 'make test' is failing for me. :-(
<smoser> and in test_simple_write_freebsd
<smoser> i'm not sure why. o rhow i didn't see it before.
<harmw> smoser: harlowja did something in that area recently
<harmw> *to fix some fault
<harlowja> smoser ya i fixed it
<harlowja> smoser try https://code.launchpad.net/~harlowja/cloud-init/test-fixups
<harlowja> connected to https://bugs.launchpad.net/cloud-init/+bug/1380424
<smoser> harlowja, you rock. 
<harlowja> ha
<harlowja> no u do
<harlowja> smoser https://code.launchpad.net/~harlowja/cloud-init/rpm-spec-fixups also if u want
<harlowja> discovered those when rebuilding the rpm for y!
<harlowja> and https://code.launchpad.net/~harlowja/cloud-init/fixed-rhel7 if u want
<smoser> harlowja, thanks. merged.
<harlowja> cool
<harlowja> https://code.launchpad.net/~harlowja/cloud-init/rpm-spec-fixups/+merge/238219 might need a freebsd specific distro variable though, not sure
<harlowja> harmw let me know and i can adjust that
<harlowja> basically distros get a new constant 'usr_lib_exec'
<harlowja> which i guess varies between rhel and others
<harlowja> mainly used to locate where 'write-ssh-key-fingerprints' is
<harmw> hm?
<smoser> maybe its time to replace that. :)
<harlowja> thats another option, ha
<harlowja> cant 'write-ssh-key-fingerprints' just use python to do it all (i'm not to sure why we need it actually)?
<harlowja> its not so complex script
<smoser> its just vestigial from smoser's love of /bin/sh
<harlowja> like that 3rd arm of yours
<harlowja> don't make me shake the 3rd arm again in paris
<harlowja> lol
<harlowja> so thats the other option, i can just replace that, lol
<harmw> you both updated to Juno already?
<harlowja> no
 * harlowja we aren't that quick, ha
<harmw> :)
<smoser> i'm deploying a juno *right now*!
<smoser> and after ditching horizon from my devstack config, devstack actually works a fair aount of the time.
<harmw> horizon.. puh
<smoser> (rather htan failing due to pip on some horizon dependency)
<harmw> I hardly ever use horizon
<harmw> Ill probably update to Juno somewhre thursday, to finally go dualstack ipv4/ipv6
<harlowja> users like horizon though, so i'll/someday at y! probably have to figure out why horizon broken there, lol
<harlowja> *internal users
<harmw> :P
<harlowja> *i'll/someday -> i'll/somebody
<smoser> harlowja, that lockstuff
<smoser> tha tyou poitned me at
<smoser> thats an entirely local operation right ?
<harlowja> define local
<harlowja> local to same host?
<smoser> right
<harlowja> yes
<harlowja> it is
<smoser> not sending messages to say "hey, let me lock a file please!"
<harlowja> do u want more than that?
<smoser> no. i do not.
<harlowja> kk
<smoser> it just seemed slow.
<harlowja> i can point u at another library i work on for remote locks, lol
<harlowja> https://github.com/stackforge/tooz 
<harlowja> :-P
<harlowja> smoser as for speed, ya, i'm not sure, it not doing anything to crazy
<harlowja> mainly https://github.com/openstack/oslo-incubator/blob/master/openstack/common/lockutils.py#L141
<harlowja> although u might want to see how much contention is happening @ https://github.com/openstack/oslo-incubator/blob/master/openstack/common/lockutils.py#L86
<harlowja> which depending on the contention rate could be significaent
<harlowja> also time.sleep(0.01) there...
<harlowja> ^ which is like an eternity in computer time
<harlowja> so u might try tweaking that down
<smoser> k. yeah, that loks sane.
<j12t> Is there some place that describes the difference between "cloud-init init" and "cloud-init modules"? Or in general how to invoke the thing?
#cloud-init 2014-10-21
<smoser> j12t, i'd suggest looking at the upstart jobs or sysvinit jobs.
<smoser> but largely 
<smoser>  cloud-init init --local
<smoser>  cloud-init init
<smoser>  cloud-init modules --mode=config
<smoser>  cloud-init modules --mode=final
<j12t> I since found those in the Arch/systemd service files. I think I understand the difference between init and modules, but what about --local and the --modes?
<j12t> Why does cloud-init but a (slightly scrambled) version of the abc into .ssh/authorized_keys instead of my public key? Having trouble trusting my eyes here ...
<harlowja> whats the 'abc'?
<j12t> basically ascii table
<j12t> each character separated by newline
<j12t> od says:    \n   +  \n   -  \n   /  \n   1  \n   0  \n   3  \n   2  \n
<j12t> 5  \n   4  \n   7  \n   6  \n   9  \n   8  \n   =  \n   A  \n
<j12t> and so it continues through uppercase and then lowercase letters
<harlowja> that seems odd, ha
<harlowja> whats the user-data u used to make this happen?
<j12t> I don't really understand how this is supposed to work, but /var/lib/cloud/instance/user-data.txt has the correct value
<j12t> http://pastebin.com/VTpTbNTS
<j12t> it comes via /dev/sdb which is a vfat drive labeled cidata
<smoser> well, its the key.
<smoser> i suspect.
<j12t> If you look at the pastebin, does this file look correct?
<smoser> well, i think i probalby intended to support that... but maybe not.
<j12t> "that" being ...?
<smoser> make the 'ssh-authorized-keys' be a list
<smoser> http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/examples/cloud-config.txt#L205
<smoser> the our ssh_authorized_keys is a string
<smoser> which is then being iterated over as a list of chars
<smoser> rather than a [<your-string>]
<j12t> are you saying that if I make it a list, it might work?
<smoser> i expect it will work, yeah.
<smoser> if your'e by hand writing that.
<smoser> then you can just wrap it in ["<your stuff>"]
<smoser> if you're yaml.dump() ing something then just make it a list.
<j12t> ha! you are correct
<smoser> http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/cloudinit/distros/__init__.py#L389
<smoser> thats the code that is going wrong there. 
<smoser> could definitely do:
<smoser>  if isinstance(kwargs['ssh_authorized_keys'], str)...
<j12t> Just an error message would be good enough already :-)
<j12t> As bugs go, this is a funny one. Was betting on some kind of prank first, but couldn
<j12t> t think of a prankster doing this :-)
<harlowja> i fix it
<harlowja> https://code.launchpad.net/~harlowja/cloud-init/ssh-key-types/+merge/239121
<harlowja> j12t ^
<j12t> harlowja: very nice, thank you.
<harlowja> np
<harmw> harlowja: ever seen an instance's disk switch to some other storagebackend?
<harlowja> i haven't :-/
<harlowja> sounds bad
<smoser> i don thitnk multi_log is righ thtere.
<harmw> can't remember I did something, but this instance now thinks it needs to use a qcow2 image when booting, instead of rgular ceph rbd
<smoser> "regular".
<harmw> hehe
<smoser> linux, harmw ?
<harmw> my cloud? yep, centos7
<harlowja> smoser want me to just use something other than multi_log (or not use it at all?)
<smoser> just log i guess. you can warn that.
<smoser> LOG.warn that seems potentially bad.  as long as 'None' is not going to generate a warning
<smoser> that must get filled in somewhere.
<smoser> ie, doest raise KeyError on kwargs['ssh_authorized_keys']
<harlowja> k
<harlowja> updated
<gkze> hi guys
<gkze> question
<gkze> why is there no option to set the UID of a user upon creation in cloud-config?
<JayF> Because you haven't added support for it yet? :P
<JayF> Mostly kidding. I don't know why, but I don't think it was explicitly excluded
<smoser> gkze, its possible it would work.
<smoser> hm.. no. thought it might .
<gkze> The change is simple, you just have to add a k/v item to the dict for user creation and then it will work
<smoser> have to just add to adduser_opts in cloudinit/distros/__init__.py
<smoser> yeah.
<gkze> maybe a few other changes
<smoser> i have to run.
<smoser> late
#cloud-init 2014-10-22
<championofcyrodi> Hi guys.  I'm running ubuntu 14.04 and everytime i reboot a nova instance using SSH i get: WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!, where the ECDSA key is changed and i have to clean it from my known hosts file.  Is there a way or document describing how to disable this ssh key injection so that it is always the same, or that it doesnt not change?
<harlowja> i'd ask your cloud provider, they are likely the ones that are changing the keys around and doing injection
<harlowja> or doing something else (probably isn't cloud-init doing this)
<championofcyrodi> i'm running my own fuel+mantis cluster w/ openstack
<championofcyrodi> (which i guess makes me the only cloud provider i can ask)
<harmw> this isn't keyinjection, it's c-i resetting your ssh host keys. Isn't that configurable in c-i.conf?
<championofcyrodi> I would think it is, but I'm not sure what the key/value pair is to configure.
<harmw> you're just using the default config?
<championofcyrodi> yea
<harmw> hmk, well I doubt thats the problem then
<harmw> did you check the logs?
<championofcyrodi> i pass in my own user-data to install some packages. but that's it.
<harmw> sounds fairly harmless :)
<championofcyrodi> let me check the logs...
<harmw> btw harlowja, my instance is only again after applying some hardcore raw sql :p
<harlowja> hardcore sql
<harlowja> sounds naughty
<harlowja> lol
<harmw> damn right
<harlowja> *hardcore raw sql
<harlowja> lol
<championofcyrodi> maybe this?  http://pastebin.com/zDYVkGnP
<harmw> well there is the reason why you keep having to edit your known_hosts file
<harmw> but what causes it (my guess, something in c-i.conf)
<championofcyrodi> I see an 'ssh' module set in the init stage...
<championofcyrodi> hmm this is frustrating.  I am seeing a module named 'ssh_config' is performing the action(s) in the DEBUG logs from cloud-init.log.   However, I'm not finding "c-i.conf" anywhere in this distro, nor am I finding anything matching the string "ssh_config"
<championofcyrodi> only 'ssh', 'ssh-authkey-fingerprints', and 'ssh-import-id'
<championofcyrodi> clear
<championofcyrodi> oops
<kwadronaut> well, the import-id is something you want to run only *once* per instance
<championofcyrodi> i think i found it...
<championofcyrodi> http://cloudinit.readthedocs.org/en/latest/topics/examples.html#configure-instances-ssh-keys
<championofcyrodi> it looks like i'll need to define it in the #cloud-config, otherwise it's randomly generated everytime.
<smoser> championofcyrodi, cloud-init will re-run the ssh key creation on 'per-instance' basis.
<smoser> its not "every time".
<championofcyrodi> so maybe the known key was just an issue for instances i terminated and re-created.
<smoser> its every time it sees a new instance-id.
<smoser> well, that would be very much by design :)
<championofcyrodi> thanks for telling me that.  is there a doc that describes the modules and when they are used?
<championofcyrodi> i found this, which has been helpful: http://cloudinit.readthedocs.org/en/latest/topics/modules.html  but the modules section is empty
<smoser> http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/examples/cloud-config.txt#L300
<smoser> then look at the ocnfig on your system (which cna be changed in user-data) in /etc/cloud/cloud.cfg and /etc/cloud/cloud.cfg.d/*.cfg
<championofcyrodi> thank you!!! you have saved my bacon.
<smoser> the dfeault frequency is 'per-instance'.
<smoser> you can change that ifyou'd like. but generally you do not want ot have multiple systems with the same ssh host keys.
<smoser> if you change it to 'once', it will write a file /var/lib/cloud/something-or-other/ssh.once
<smoser> and if that file is still there, it will never run it again
<harlowja> hmmm, need to work on that module.html doc 
<smoser> harlowja, 2.0
<harlowja> ya
<smoser> think about how to do it well.
<harlowja> lol
<harlowja> hmmm
<smoser> and then tell dumb people like smoser
<harlowja> :-P
<harlowja> smoser modules that have self-contained docs would be cool, then can use that in online docs :)
<smoser> yeah, that is what i want. 
<smoser> config modules with python comment that describe them.
<harlowja> >>> from cloudinit.config import cc_ssh
<harlowja> >>> cc_ssh.__doc__
<smoser> yeah.
<harlowja> put a module level comment/docstring and it can be found
<harlowja> by magic!
<harlowja> ha
<harlowja> then sphinx can read that afaik
<harlowja> doesn't seem so hard
<harlowja> get er' done
#cloud-init 2014-10-23
<smoser> harmw, any status on https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=194295 ?
<smoser> what does "reaady" mean ?
 * hiren_ is resurfacing slowly after paternity leave.
<harmw> smoser: it's waiting to be committed :)
<harmw> tbh, I've been way to busy with other stuff this week..
<smoser> oh wow. congratulations, hiren_.
<hiren_> smoser: thanks!
<smoser> what does that mean, harmw ?
<hiren_> harmw: want me to poke someone? 
<smoser> hiren_, you'll be in paris ?
<hiren_> smoser: paris? I'd like to but no, I wont be. 
<hiren_> some conference?
<hiren_> harmw: Are we just waiting for marino@ to commit the port?
<smoser> oh. yeah, some conference.
<smoser> openstack summit :)
<hiren_> smoser: heh, didn't know :-)
<hiren_> but paris sounds nice :-)
<hiren_> have fun.
<harmw> hiren_: yep
<harmw> he told me yestrday there are 10 pending ones, and c-i is one of thm
<harmw> OperationalError: (OperationalError) (1054, "Unknown column 'endpoint.region_id' in 'field list'")
<harmw> damn you, keystone!
<harlowja> hiren_ openstack summit in paris :-P
<harmw> hiren_: looking good :)
<smoser> i assumed with all the money that yahoo has everyone got to go.
<harmw> oh no wait, that was like ages ago already
<smoser> buying up companies for billions
<smoser> thanks harmw 
<harmw> what for?
<smoser> hiren_, you're first ? boy ? girl ?
<smoser> chicken ?
<smoser> steak ?
<harmw> lol
<harlowja> lol
<harlowja> smoser ya, i wish everyone could go, i fought for it, didn't work out :-P
<harlowja> stupid budgets and crap
<smoser> definitely hard to fund someone to go to paris for a conference if you have to scrape together 700 million for some videos
<harlowja> :)
<harlowja> i think i'll see what we can do about those module docs, let me see how hard it is to make sphinx generate those
<hiren_> haha
<hiren_> girl.
<hiren_> harmw: cool. If things get delayed, let me know. I'll try to push. I've cced myself on the bug.
#cloud-init 2014-10-24
<harlowja> smoser https://code.launchpad.net/~harlowja/cloud-init/start-module-docs/+merge/239523 begun
<harlowja> seems to work as expected
<harlowja> let me know what u think, will be nice to start to fill that out and have it on readthedocs
<smoser> thanks harlowja 
<harlowja> np
<harlowja> if i could show u it, i would, but it looks good :-P
<harlowja> its a start at least, document one module a day, keep the doctor away
<harlowja> but if u run `sphinx-build . <OUTPUTDIR>`  in cloud-init/doc/rtd u can see
<harlowja> and then open the outputdir html
<harlowja> anyway, feel free to merge it, i'm done editing i think :)
<harlowja> smoser if u can check out that module docs review, i'll start adding the rest of the docs
<harlowja> get that one merged, then start adding the rest
<harlowja> if u got some time
<harlowja> eventually then they will all be doced
<harlowja> and people will be happy, ha
<smoser> harlowja, i really like the idea.
<harlowja> u better!
<harlowja> ha
<smoser> i'm sorry i dont have time at the moment to look... it sure doesnt sound like it can be anything wrong
<smoser> all just magic goodness
<harlowja> np
<harlowja> its all ok
<harlowja> take your time
<harlowja> smoser https://review.openstack.org/#/c/130872/ may also be useful for u, if u are using that code
#cloud-init 2015-10-19
<Odd_Bloke> smoser: I'm looking at switching MBR disk setup to use sectors (rather than MB), because sfdisk in wily has dropped support for units other than sectors.
<smoser> yes please.
<Odd_Bloke> smoser: How would you suggest I test this?  I've done [33, [66, 82]], which is the canonical example we use.
<Odd_Bloke> But I don't really know what corner cases we might have that I should look at.
<smoser> mainly i think you dont have to care.
<smoser> CHS is garbage.
<smoser> thats waht you're asking about, right?
<smoser> what did '33, 66, 82' mean ?
<Odd_Bloke> smoser: No, at the moment we do (essentially) 'cat <list of partitions> | sfdisk -uM <device>'.
<Odd_Bloke> smoser: But sfdisk no longer supports megabytes as units, so I'm switching to using sectors as the units.
<Odd_Bloke> smoser: This is https://bugs.launchpad.net/cloud-init/+bug/1460715
<smoser> ah. ok.
<smoser> what did you mean 33, 66, 82 ?
<Odd_Bloke> smoser: So I've tested using 'layout: [33, [66, 82]' (i.e. one-third ext4, two-thirds swap), but I don't really know what else to try to check that it works in practice.
<smoser> Odd_Bloke, ok. now reading code and understand more.
<smoser> well you basically have to turn everything into sectors.
<smoser> now, to deal with sfdisk
<Odd_Bloke> smoser: Yeah, it's actually surprisingly painless.
<Odd_Bloke> smoser: I've got a branch prepped, I'm just trying to test it before pushing it up for review.
<smoser> i've never really used that code.
<smoser> in the future the plan is to support the disk layout settings taht curtin can do.
<Odd_Bloke> smoser: OK, I'll just push this up for review then.
<Odd_Bloke> I've done some basic testing.
<Odd_Bloke> smoser: That's https://code.launchpad.net/~daniel-thewatkins/cloud-init/lp1460715/+merge/274897
<smoser> Odd_Bloke, it'd be good to know if this code runs on old sfdisk
<Odd_Bloke> smoser: Ah, good point.
<Odd_Bloke> smoser: The existing trusty code doesn't work even without my changes. /o\
<Odd_Bloke> Ah, I'm hitting https://bugs.launchpad.net/cloud-init/+bug/1311463.
<smoser> Odd_Bloke, added a review
<Odd_Bloke> smoser: Thanks!  Replied to the inline comment.
<smoser> i've always been confused by sfdisk also
<smoser> lets find the real ansewr.
<smoser> but i suspect it is 512
<Odd_Bloke> smoser: I've been poking around in that code; let me have a dig before you switch to this.
<Odd_Bloke> (As in, I've been poking in the sfdisk code)
<smoser> reading http://karelzak.blogspot.com/2014/10/new-sfdisk.html
<Odd_Bloke> smoser: So my code doesn't backport to trusty nicely, because old 'sfdisk -l' output is different.
<smoser> yeah. i'm aware of such pain .
<smoser> from growpart
<smoser> it does support both. but its a pita
<smoser> http://bazaar.launchpad.net/~cloud-utils-dev/cloud-utils/trunk/view/head:/bin/growpart
<smoser> Odd_Bloke, the thing is that if we update to only supporting new, then anyone who picks this up is busted on old sfdisk
<smoser> in growpart, SFDISK_VERSION madness.
<Odd_Bloke> smoser: ;.;
<Odd_Bloke> smoser: So libfdisk definitely supports variable sector sizes; I don't know if we'd ever see them in practice.
<smoser> but then its not clear to me if it woudl be used anyway.
<smoser> logical sector size versus physical sector size
<Odd_Bloke> smoser: Right.
<Odd_Bloke> smoser: I have a meeting coming up, but I'll pick this up after that.
<smoser> Odd_Bloke, so.
<smoser> through some experimentation, i can verify that sfdisk uses 512 byte sectors
<Odd_Bloke> smoser: You're sure we won't hit funky hardware somewhere where it doesn't?
<smoser> i dont think so
<smoser> http://paste.ubuntu.com/12863364/
<smoser> Odd_Bloke, ^ that boots a guest with 4096 physical size and 512 logical (interestingly, it wont boot if you set logical to 4096).
<smoser> verified that the kernel says my physical size is 4096
<smoser> and that size reported (in your 'sfdisk --list') is in 512
<smoser> Odd_Bloke,  http://paste.ubuntu.com/12863544/
<harlowja> smoser https://github.com/openstack/cloud-init
<harlowja> guess now cloud-init == openstack
<harlowja> lol
<harlowja> Odd_Bloke ^  claudiupopa
<smoser> oh well. it is. as it is.
<harlowja> :-P
<natorious> oh wow
<natorious> did they merge the stackforge org into openstack?
<natorious> I suppose
<harlowja> yup
<harlowja> stackforge gone...
<harlowja> and/or retired
<natorious> fwiw, I report cloud-init hours worked as openstack dev hours
<harlowja> nice
<harlowja> well now u can do that without feeling bad, ha
<natorious> lol, no bad feelings here :P
<Odd_Bloke> smoser: So looking at the util-linux code; sector size defaults to 512, but it does do an 'ioctl(fd, BLKSSZGET, sector_size)' call to actually query the device (https://git.kernel.org/cgit/utils/util-linux/util-linux.git/tree/lib/blkdev.c#n193).
<Odd_Bloke> smoser: But that does mean that a `blockdev --getss $DEVICE` is definitely going to return the same answer, which would save on parsing sfdisk output.
<Odd_Bloke> Let me check that makes sense on trusty.
<smoser> ok. i'm wrong
<smoser> http://paste.ubuntu.com/12864078/
<smoser> your'e right about blockdev --getss
<smoser> but i think thats the same as woudl be returned by *logical*
<smoser> http://paste.ubuntu.com/12864114/
<smoser> ^--
<smoser> so reading logical in sys shoudl give you what sfdisk does.
<smoser> i managed to create a device with 4096 logical
<smoser> so ... sfdisk uses logical sectors. not physical sectors.
#cloud-init 2015-10-20
<openstackgerrit> XiaBing Yao proposed openstack/cloud-init: Update stackforge to openstack  https://review.openstack.org/237452
<Odd_Bloke> smoser: https://code.launchpad.net/~daniel-thewatkins/cloud-init/lp1460715/+merge/274897 is ready for re-review. :)
<smoser> Odd_Bloke, so, one pick.
<smoser> if we can avoid the fork, if there is data in /sys it seems fastest to read it from there.
<smoser> doesnt it ?
<Odd_Bloke> smoser: I'm happy to move to that; I was thinking that (a) blockdev will maintain an interface whereas /sys might change, and (b) blockdev might be available on platforms (BSD?) where /sys is not.
<smoser> b is almost certainly true
<smoser> but generally i dont think sys is a moving target
<smoser> i have a few other comments i'll put in review.
<mwak> hello
<smoser> hey
<mwak> I created a PR yesterday to merge the Scaleway datasource into cloud-init, https://code.launchpad.net/~edouardb/cloud-init/scaleway-datasource/+merge/274861. Any idea of the ETA to get it merged :) ?
<smoser> mwak, i reviewed. let me know if you n eed any helip.
<smoser> help
<mwak> smoser: thx!
<mwak> smoser: a) b) c) concerning d, is it possible to do that in another PR as we do not use that atm
<mwak> *Alright for a/b/c
<smoser> sure. if its not there, no point in looking for it.  i'd think it'd be great if you have it though.
<natorious> smoser: I'd been thinking more about network json and am not 100% that base.BaseOpenStackSource.network_config should return json data
<smoser> :)
<natorious> as is, it returns a network conf file when not network_json
<natorious> so should we process the network_json into a file to return specific to ea distro?
<natorious> also been throwing around the idea of vendor plugins and how they would fit into all this
<natorious> that is extensibility based on cloud providers specific feature set
<smoser> natorious, well, there is no 'file' that is specific to each distro
<smoser> unless you consider a tarball. but even then, not really
<smoser> ie, /etc/sysconfig/ you'd potentially have to write and delete multiple files
<natorious> right
<natorious> so passing content as network config as currently is is needing some eyes
<smoser> natorious, around ?
<smoser> http://paste.ubuntu.com/12879626/
<smoser> something to think about %
<smoser> ^
<natorious> smoser: your talking to the right guy then ;)
<natorious> so, persistent datasource w/ the openstack metadata service could call the same module for doing them after init
<natorious> thats the idea anyways
<natorious> I believe the intel osic cloud thats being spun up w/ have onmetal resources for all of us to play with
<natorious> whenever that comes around
#cloud-init 2015-10-21
<Odd_Bloke> smoser: So I'm looking at /sys, and I can't see how to find the logical sector size of a block device.
<ndonegan> Is it possible to make cloud-init run an abitrary script after it has gone through it's other sources?
<larsks> Odd_Bloke: I don't know about /sys, but this works, I think: blockdev --getbsz /dev/sda
<Odd_Bloke> larsks: Yep, blockdev would do it; we're trying to avoid forking if we can.
<larsks> It's just calling the BLKBSZGET ioctl.  I guess you could do that natively in python w/ the 'fcntl' module...
<smoser> Odd_Bloke, i put iot in my comments.
<smoser> oh carp. looks like i didn't hit 'submit' on some comments.
<smoser> but anyway, its /sys/class/block/<devname>/qeueu/logical_block_size
<smoser> and physical_block_size
<smoser> ndonegan, yes. just send script as user-data . if it starts with '#!' it gets executed.
<smoser> sucks that i lost that comment.
<smoser> Odd_Bloke, i had also rquessted that we round reasonably well (to 1MB or 4MB) increments so we land on fs alignment. and also start at 1MB in.
<ndonegan> smoser: Having issues with a certain meta data provider, hence why I want to be able to run a script after which verifies stuff happened.
<Odd_Bloke> smoser: If {logical,physical}_block_size are actually sector sizes, why is there also hw_sector_size?
<smoser> well, i dont know. in my tests, logical and physical match up with what i fed through qemu-system-x86
<smoser> and qemu calls them 'logical_block_size' and 'physical_block_size'
<smoser> so i'm justified in my assumption (as those names line up). but i admittedly did make an assumption :)
<smoser> ndonegan, i dont really follow.
<ndonegan> smoser: Having great fun with metadata proxy where it occasionally sends back crap.
<ndonegan> As an example, the request it should be sending to nova-metadata-api gets sent as the resonse to the client.
<ndonegan> So looking for some way to try and very that some valid data was sent back
<ndonegan> *verify
<Odd_Bloke> ndonegan: You're looking to change the image in order to do this, or configure cloud-init to do it from outside the instance?
<ndonegan> Odd_Bloke: Exploring any option, including edits to the image, to see if there's some way to verify that what comes back from the metadata service is correct.
<smoser> ndonegan, as in openstack ?
<smoser> yeah.
<ndonegan> It would actually be better if the proxy just failed entirely and timed out. That way I can have the "None" data source as a fallback, and let it write a raw userdata
<ndonegan> However, it sends back just enough before screwing up that the Ec2 source doesn't fail outright.
<ndonegan> smoser: Openstack with Contrail.
<smoser> right.
<ndonegan> And it's Contrail that's the issue.
<smoser> 2 things you could try
<smoser> a.) cirros
<smoser> b.) if you can enable config drive in launching... then you can probably avoid cloud-init hitting the md so you can get in and debug from there.
<smoser> ie, launch with 'config_drive=1'.
<smoser> i suggest cirros because it will time out faster and you can log in with password : cirros/cubswin:)
<Odd_Bloke> smoser: So, looking at the kernel source, hw_sector_size is the same value as... logical_block_size.
<ndonegan> We already have a base image we use everywhere (mini Centos setup for our environment).
<ndonegan> ConfigDrive might be a good option.
<smoser> Odd_Bloke, i would not have thoguth that :)
<mwak> smoser: I updated the PR to ensure we're on Scaleway when using the datasource, let me know if it's ok for you. I'll add unit test asap
<smoser> mwak, and sign cla ?
<smoser> mwak, the tewst there is fine, and does help some.
<smoser> but.. we'd like to avoid hitting a network service if we can
<smoser> as any connection to a network service might hang indefinitely
<Odd_Bloke> smoser: OK, the BLKSSZGET ioctl calls bdev_logical_block_size, which calls queue_logical_block_size which is also called for the logical_block_size sysfs entry.
<smoser> ie, in many environments a 'wget http://169.254.169.254/' will just block due to firewall rules and the like.
<Odd_Bloke> smoser: So it looks like we can use /sys/.../logical_block_size. :)
<smoser> Odd_Bloke, i'd like to fall back if that file isnt present. you plan on that , right ?
<Odd_Bloke> smoser: Yeah, that was the plan.
<smoser> ok. the other thing i didnt' say, that got lost in my browser.
<smoser> if we can start first partition at 1MB and only use 1MB units in sizes, that'd be good.
<smoser> i'm tempted to use 4MB units.
<smoser> https://bugs.launchpad.net/maas/+bug/1502259
<smoser> lvm uses 4MB extents by default.
<natorious> morning
<mwak> smoser: yep sign the cla too
<smoser> hey
<smoser> mwak, so does that make sense above ^
<smoser> is there any way that we can determine "am i on scaleway" without hitting a network resource ?
<smoser> hey. it is time for cloud-inti 2.0 meeting
<mwak> smoser: not really
<jgrimm> o/
 * Odd_Bloke is in a partner meeting, so is MIA.
<natorious> channel topic links need to be updated w/ new gerrit paths
<natorious> ie - https://review.openstack.org/#/q/project:openstack/cloud-init+status:open,n,z
<smoser> ah. yeah, i'll do that natorious
<smoser> ok. so looking at that fine url that natiorisou added.
<natorious> lol
<smoser> a couple that i shoudl grab quickly
<smoser> as they address the namespace change.
<smoser> natorious, had you made any more progress on yours?
<natorious> was just looking over harlowja's comments
<smoser> k
<natorious> ideally it would be nice if we had a utils.execute or similar shared subprocess call for exec
<smoser> natorious, yeah.  there is a fairly good one in cluod-init
<smoser> utils.subp
<smoser> in 0.7
<smoser> that i'd suggest we pull forward
<natorious> k
<natorious> assuming it is compatible with windows and all
<smoser> ah. yeah, it would need some windows work likely.
<smoser> it wrapps subprocess.
<smoser> subprocess.popen
<smoser> but i'd guess we could address anything non windows specific there
<niluje> @smoser | ie, in many environments a 'wget http://169.254.169.254/' will just block due to firewall rules and the like. >> do you think we should add a timeout for the test?
<niluje> that can easily be done with requests
<smoser> niluje, for tests ?
<niluje> to check the server's is on scaleway
<niluje> (I'm working with mwak there)
<smoser> yes, for sure.
<smoser> timeouts suck
<niluje> we thought about it for a while, and doing something else than a network test didn't seem straightforward
<smoser> because you're either safe (meaning unnecessary long wait) or aggressive (meaning your endpoint might not return in time)
<niluje> yes, I agree
<smoser> niluje, are you in a place wehere you could feed some dmi data to the guest ?
<smoser> it seems like generally good practice to identify your platform in such a way
<smoser> wrt timeouts... i dont want to enable a datasource by default that could block boot as it waits.  I'd *like* to change EC2 to not do that.
<smoser> but EC2 was "first" so i'm ok giving them a legacy grandfather-in.
<niluje> I totally understand your point of view :)
<niluje> unfortunately, at the moment, we can't really rely on something else than network resource and if we do, we won't be sure the test will still work in a few months/years
<niluje> we prefered to add this not-so-good-test rather than a better test that might not work in a while
<niluje> however if that's a blocker, we'll think about another solution
<natorious> smoser: any ideas on windows configdrive detection?
<smoser> natorious, i might look at how cloudbase has done it.
<smoser> i think they read the device with bsd tar
<smoser> !
<smoser> you cant make that stuff up :)
<smoser> but at least at one point that was what alexpilloti settled on.
<smoser> (honestly, i wuldnt mind having a vfat and iso9660 filesystem reader in pure python)
<natorious> oh wow, lol
<natorious> yeah, looked at ctypes methods a bit
<natorious> should be universal but alot less readable
<smoser> i suspect also faster than 'mount ... read ... unmont' for the amount of data we're pulling.
<natorious> uni to windows that is
<natorious> k, I'll see how their doing it and go from there
#cloud-init 2015-10-22
<akostadinov> Hello, I'm hitting an issue with systemd vs cloud-init. cloud-init runs as a systemd service. So when trying to start/stop services like docker from inside cloud-init scripts, the operation hangs in a deadlock. i.e. script waits for command to finish but command never finish as systemctl waiting for cloud-init to complete. The --no-block option is not helping as I don't only want to start a service but also interact with it and configure i
<stanguturi> HI I made some changes to cloudinit package, deployed the debian file. How to restart the cloudinit service to see if my changes are reflected. I can do a simple reboot and check out. But wanted to know of a simple solution instead of a full reboot
<smoser> akostadinov, hm.. can you give an example of exactly what you're doing ?
<smoser> stanguturi, depending on waht you changed...
<smoser> rm -Rf /var/lib/cloud/ /var/log/cloud-init*
<smoser> then
<smoser> cloud-init init --local
<smoser> cloud-init init
<smoser> cloud-init modules --mode=config
<smoser> cloud-init modules --mode=final
<smoser> or, if you just changed a config module
<smoser> easiest thing to do is:
<akostadinov> smoser: yes - starting docker service and pulling some images. It looks like docker-storage-setup service is required by the docker service and the docker storage-setup service should run after cloud-init finish. That's the deadlock issue. Read the man page and now trying to workaround this with systemctl --ignore-dependencies
<smoser>  cloud-init init single --name=<name> --frequency=always
<smoser> where 'name' is the name like 'cc_timezone'
<akostadinov> smoser: it makes sense for services to run after cloud-init so that machine setup is done. But there's better mechanism to run scripts at different stages of boot process. So that one can execute proper initialization at proper times
<stanguturi> @smoser: I changed some code in DataSourceOVF.py file. I want to restart the cloudinit service and see if my changes are loaded.
<smoser> stanguturi, yeah, follow that above ^
<smoser> init --lcoal and then init
<smoser> will run your ds
<smoser> akostadinov, how are you doing that ?
<smoser> via '#!' user-data or cloud-config install packages ? ..
<akostadinov> smoser:
<akostadinov> runcmd:
<akostadinov> ...
<akostadinov> - [ systemctl, enable, docker.service ]
<akostadinov> - [ systemctl, start, docker-storage-setup.service, --ignore-dependencies ]
<akostadinov> - [ systemctl, start, docker.service, --ignore-dependencies ]
<akostadinov> ...
<smoser> ah. k. so you installed it i suppose ?
<akostadinov> smoser: yep, installed with packages:...
<smoser> it should start on its own. shouldnt it ?
<akostadinov> smoser: don't know, but it won't start before cloud-init finish either case
<akostadinov> because docker-storage-setup should run after cloud-init final
<smoser> oh. i see.
<smoser> maybe drop a sytemd file that should start after docker.service
<smoser> drop it in before installation (as a boothook perhaps)
<smoser> and then it should start when it can on its own
<akostadinov> smoser: how do I register a boothook?
<smoser> bootcmd: []
<smoser> just like runcmd
<smoser> is the easiest way
<smoser> note, though, that boothooks run every boot
<smoser> rather than once per instance
<corb00> Hi I'm having a problem adding a yum repository to a CentOS #cloud-config , I am using yum_repos: â¦ seeing "the requested URL returned error: 404.."â¦ it works manually though
<corb00> baseurl: http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
<akostadinov> smoser: you mean to do something like:
<akostadinov> echo '...' > /etc/systemd/system/my.service
<akostadinov> systemctl ...
<akostadinov> correct?
<corb00> am I missing any entries? I have specified PHP_epel:
<corb00> baseurl:, enabled: true, name: some name, gpgcheck: false
<corb00> never tried this before so excuse my ignorance..
<smoser> akostadinov, well, yeah. except i was saying you dont have to call systemctl
<smoser> as if you've written the job correctly
<smoser> it will start when docker.service is running
<smoser> that make sense ?
<smoser> harlowja_, ^ can you help out corb00 ? not sure myself.
<harlowja_> hmmm
<akostadinov> smoser: yes, that's my next bet if --ignore-dependencies fails. I guess I'd need `systemctl enable my.service` though.. or maybe it will be picket up automatically, never tried. I guess I'll just try
<harlowja_> let's see here
<akostadinov> thank you for suggestions
<smoser> akostadinov, it should just pick it up automaticall
<harlowja_> corb00 can u pastebin the yaml u are using, or gist
<corb00> sure- give me a sec
<corb001> here you go harlowia (had to switch machine) â http://pastebin.com/fftj7ayn
<harlowja_> soooo
<harlowja_> think i see it
<harlowja_> http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm isn't the repo url, but is itself a repo file
<corb001> ahhh - bloody beginner here, okâ¦
<harlowja_> u want to use the url that is in that rpm
<corb001> ok so just leave out the last bit..
<harlowja_> what that url is? :-P
<harlowja_> http://paste.openstack.org/show/477195/
<corb001> wait I got you - okâ¦ have to check whats in it :-)
<harlowja_> pick 'http://download.fedoraproject.org/pub/epel/6/$basearch'
<harlowja_> i think its ^
<harlowja_> or 'https://mirrors.fedoraproject.org/metalink?repo=epel-6&arch=$basearch'
<harlowja_> not really sure, ha
<harlowja_> one of those :-P
<corb001> cool- let me try thatâ¦ thanks harlowja!!
#cloud-init 2015-10-23
<stanguturi> Thanks smoser for your valuable inputs.
<smoser> did it work ?
<smoser> stanguturi,
<stanguturi> smoser, no that didn't work. I had to reboot...
<akostadinov> smoser: thanks for help yesterday. FYI the --ignore-dependencies approach for starting docker from runcmd worked fine.
#cloud-init 2015-10-25
<alexpilotti_> smoser: morning!
#cloud-init 2016-10-25
<rharper> smoser: https://launchpad.net/~cloud-init-dev/+archive/ubuntu/daily/+packages  is this the daily ppa?  last publish was 8/18 -- was that the git switch over?
<smoser> :-(
<smoser> it has definitely built since git. i'll look.
<smoser> :-(
<rharper> I was going to get an updated core snap that points to cloud-init daily PPA and have it build core snap daily against that so we always have a fresh core.snap
<smoser> https://code.launchpad.net/~cloud-init-dev/+recipe/cloud-init-daily
<smoser> it sucks... that contents sometimes gets changed like randomly
<smoser> 'bzr-builder' i think is what is killing it
<smoser> there are no changes to bzr 'lp:cloud-init'
<smoser> will look at what its supposed to have
<rharper> huh
<smoser> yeah, i'm not making it up. just now changed it to http://paste.ubuntu.com/23379003/
<smoser> click the green thing and it goes to bzr-builder
<smoser> i'm going to delete it and hope recreating makes better
<rharper> whoa
<smoser> thought i'd seen a bug at some point
<rharper> we should get powersj to have a jenkins job to boot lxc container, add this ppa and install it
 * powersj likes that idea
<jgrimm> powersj, good morning!
<powersj> Can do something similar to my daily ISO test and compare when a new one comes out too
<powersj> morning :)
<smoser> well, *not* boot an lxc container and install the thing
<smoser> thats absically the first proof of integration test
<smoser> wgrant in launchpad pointed me at https://bugs.launchpad.net/launchpad/+bug/1623924 with work around . so looking.
<powersj> right, so we could have a daily integration test based on the daily ppa to run a subset of tests.
<smoser> alright. building now.
<smoser> rharper, thanks for noticing.
<rharper> np
<NerdyBiker> Hey all, does anyone know of any issues using Cloud-init chef function on a HVM type instance in AWS? As soon as we use a HVM instance, we get "Running chef (<module 'cloudinit.config.cc_chef' from '/usr/lib/python2.6/site-packages/cloudinit/config/cc_chef.pyc'>) failed" in the logs
<smoser> NerdyBiker, more info ?
<smoser> it works on non-hvm ?
<NerdyBiker> yup works on PV types
<smoser> distro ? cloud-init version ?
<smoser> that is really odd
<NerdyBiker> Oh sorry :D CentOS 6.5
<smoser> can you paste the cloud-init.log ?
<NerdyBiker> version 0.7.5 cloud-init
<smoser> or at least the stack trace
<NerdyBiker> Perferred method of pasting?
<smoser> pastebin other than pastebin.com :)
<NerdyBiker> http://pastebin.com/kGLPkXZf
<smoser> that was just rude :)
<NerdyBiker> damn
<NerdyBiker> haha!
<NerdyBiker> terrible timing
<smoser> thats fine.
<smoser> its just ad-everywhere
<smoser> /var/lib/cloud/instance/scripts/runcmd: line 3: chef-client: command not found
<smoser> well that'd be it.
<smoser> i guess your ami that yul're running for vhm does not have that package installed and the pvm does
<NerdyBiker> the user data is set to install chef
<smoser> user-data runs after package install
<smoser> but cc_chef.py might install it.
<smoser> looking
<smoser> hm..
<NerdyBiker> https://gist.github.com/MattDevUK/064ece1a309616176695b9b67a866aa6
<smoser> actually, its your runcmd that is failing.
<NerdyBiker> thats the user data we pass in
<smoser> you have /var/log/cloud-init-output.log ?
<NerdyBiker> that was the first pass
<NerdyBiker> paste*
<smoser> oh. yeah.
<smoser> do you ahve a /var/log/cloud-init.log ?
<NerdyBiker> in every other image we've used it on, this installs chef automatically using the params set in the chef block, then runs chef-client.
<smoser> that one will have the stack trace and generally more info
<NerdyBiker> sure :D
<NerdyBiker> it's empty :(
<NerdyBiker> any log levels we can change to try and get any content?
<smoser> NerdyBiker, its a bug in c loud-init that leaves it empty.
<smoser> can you get into that instance ?
<NerdyBiker> yup, I'm on it
<smoser> vi /etc/cloud/cloud.cfg.d/05*
<smoser> the logging file
<smoser> comment out
<smoser>  
<smoser>  - [ *log_base, *log_syslog ]
<smoser> then reboot
<smoser> maybe rm -Rf /var/lib/cloud /var/log/cloud*
<smoser> and reboot
<smoser> that should make it think its a first instance
<NerdyBiker> ok so just leave &log_file in there?
<NerdyBiker> ooo
<NerdyBiker> no i see it now :D
<NerdyBiker> booting now
<NerdyBiker> so... it apparently worked that time
<smoser> hm..
<smoser> maybe the omnibus installer is doing something int he background ?
<smoser> and we're just racing it
<NerdyBiker> the output after the reboot; https://gist.github.com/MattDevUK/9ad23f20e3570e82a1b2912e21d57ef0
<NerdyBiker> you can see where the "running chef failed" line was previously, now says installing Chef
<NerdyBiker> on a positive note, that's the first time we've managed to get that to happen on a HVM image, out of the 10 or so we've tried.
<smoser> follow bill's advice.
<smoser> reboot!
<smoser> and reinstall
<NerdyBiker> so uninstallec chef, remvoed cloud files, rebooted and success again. wonder if it's timing when starting a fresh instance
<NerdyBiker> @smoser just launched 2 fresh instances, both of them failed the cc_chef part on first run.
<smoser> first boot io is very slow
<smoser> but if the omnibus installer is backgrounding... thats a pita
<smoser> and it should not do that
<NerdyBiker> hm, just tried a basic restart, and it failed again
<NerdyBiker> seems to be somehting timing related. cleared the /var/lib/cloud, rebooted and it worked. :(
<NerdyBiker> damn that sucks
<NerdyBiker> I'll tets out some other scenarios and see what comes up
<NerdyBiker> test*
<NerdyBiker> @smoser so just got the logging change into a base AMI, started an instance and it failed, this is in the cloud-init.log; https://gist.github.com/MattDevUK/9f58b9716a6914406536839a77cadcf0
<smoser> hm..
<NerdyBiker> there is also a stack for the cc_locale run, but don't think that impacts the chef issue as that fails een when chef succeeds
<NerdyBiker> even*
<smoser> right.
<NerdyBiker> is there any more info I can give? Is this a valid problem?
<smoser> NerdyBiker, i think probably valid
<smoser> it seems likely to me to be related to omnibus
<smoser> woudl be nice to try with newer cloud-init.
<smoser> magicalChicken, https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/306731 fails rebase for me
<magicalChicken> smoser: I'll look into it, I hadn't noticed issues before
#cloud-init 2016-10-26
<rharper> smoser: please apply https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/309304
<NerdyBiker> @smoser I'll try updating cloud init and seeing how it behaves :)
<NerdyBiker> @smoser so, found a link talking about some issues running cloud init http://docs.rightscale.com/rl10/reference/10.5.2/rl10_cloud_init_installation.html followed it and upgraded our 0.7.5-10 to -20 but still not fix :( 0.7.5 seems to be the latest CentOS supported version at the moment.
<fxpester> hi all, is it possible to replace ip 169.254.169.254 with my own ? I can see only options to switch provider like 'Ec2, OpenStack' but in my case I can`t use nor router, nor dhcp, nor 169 ips
<smoser> fxpester, you can change the metadata_urls in Ec2:
<smoser> doc/examples/cloud-config-datasources.txt
<smoser> and also same way for openstack
<smoser> so in a file inside the image (/etc/cloud/cloud.cfg.d/your-mods.cfg) you need something like:
<smoser> http://paste.ubuntu.com/23383531/
<fxpester> smoser: looks awesome) thank you!
#cloud-init 2016-10-27
<rharper> smoser: when you get a change, https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1636912/ has some details re: networkd and cloud-init.service that pitti tracked down.  I'm going to see about testing out the change to cloud-init.service to include an After=systemd-networkd.service and see if that's resolves the ordering issue we're seeing
<rharper> smoser: playing with the networkd target and cloud-init.service;  I noticed that we say Before=network-online.target ; however, don't we want networking to be *up* when cloud-init.service runs (it may scan for network metadata services) ?
<smoser> its weird.
<smoser> network-online is an target
<smoser> that other things would wait for
<smoser> to know that the network is up
<smoser> and we want to block those things running until after cloud-init has run
<rharper> hrm
<rharper> ok
<smoser> we want to force that bottleneck in boot
<rharper> so, how do we handle the race between 'networking.service' being complete and the devices being UP
<rharper> and when cloud-init running
<rharper> and checking if networking is up (in the route output info)
<smoser> you're gonna make me think, eh
<smoser> we have walked through this, and the stuff is sane. but let me look
<rharper> so, in my case, networkd runs, kicks of DHCP, takes 2 to 3 seconds; cloud-init.service runs almost the same time as networkd (just right after)
<rharper> but races the DHCP response
<rharper> ok
<smoser> i'd not be completely sure, but pitti has helped me to be sure int he past
<rharper> ok, I updated the bug
<smoser> but obviously this is all really complex (and unforunatlye brittle)
<rharper> so, if you recall any discussion about the ordering, add it there so we can clarify
<rharper> I wonder if the serial execution of ifup  in networking.service naturally delays things
<rharper> vs. networkd going daemon mode and async bringing up networking
<smoser> so in ifupdown world, there is no race on 'networking.service'
<smoser> if its done ('After') then stuff that is supposed to be up is up
<smoser> which kind of makes sense.
<rharper> right, but we have no daemon
<rharper> rather there are two Exec=
<rharper> statements
<rharper> both have to run
<rharper> systemd-networkd can just spawn itself and report that it's done
<rharper> it gets out of the way faster
<smoser> well it shouldnt do that
<rharper> but it takes roughly 3 seconds for the DHCP response to complete (according to the journal)
<rharper> but
<rharper> there's a sepatarate systemd-networkd-wait-online.service
<rharper> which waits
<rharper> but again, that's a *start* of a unit
<rharper> not a completion
<rharper> so things are "up" after -wait-online exits
<rharper> which is roughly network-online.target
<smoser> well, it says (/lib/systemd/systemd-networkd-wait-online)
<smoser> which *says* "Block until network is configured":
<rharper> so maybe we can say Requires systemd-networkd-wait-online
<smoser> so maybe we just want to be After systemd-networkd-wait-online
<smoser> yeah
<rharper> lemme try that
<smoser> except... if htat is ogoing to cause issues when networkd is *not* used.
<rharper> maybe
<rharper> I'll test both
<smoser>  /lib/systemd/systemd-networkd-wait-online
<smoser> is blocking on my desktop right now
<smoser> :-(
<rharper> hrm
<rharper> may need to generate that
<rharper> based on if networkd is going to run
<rharper> or something like that
<rharper> ah
<rharper> [ INFO ] Network Service is not active.
<rharper> [DEPEND] Dependency failed for Wait for Network to be Configured.
<rharper> [DEPEND] Dependency failed for Initial cloud... job (metadata service crawler).
<rharper> it's disabled by default if ifupdown is installed IIRC
<rharper> bleh, I have it activated now, but it blocks continuously; and no idea why
<NerdyBiker> Anyone know how I can reference an EC2 instance ID in a cloud config on CentOS6?
<NerdyBiker> I've tried just using $INSTANCE_ID, want to use write_files to put it in a file
<NerdyBiker> been suggested to set a variable in bootcmd section, but not a fan of how it looks, wasn't sure if that's the best/only way
<rharper> NerdyBiker: you can readlink -f /var/lib/cloud/instance
<rharper> that's a symlink to the instances/$INSTANCE_ID of the current session
<rharper> NerdyBiker: it also looks like /var/lib/cloud/data/instance-id includes the current instance id value
<rharper> smoser: well, after most of the day here; I'd say were in a wedge;  networkd really wants dbus.service; otherwise it just adds boot time;  in Xenial, we don't have resolved service; I suspect there we could drop the requirement; but it means divergent config and possible behavior
<smoser> :-(
<smoser> rharper, but well what about the After when networkd is not running
<smoser> that would block, right ?
<rharper> I've not tested the non-networkd setup yet
<rharper> just trying to figure out how to get networkd and cloud-init to run in the right order (and without blocking excessively)
<smoser> ok.
<rharper> I wish there was an easy way to find out why a unit was skipped
<rharper> ok; I think have something;    if we drop the Requires , but use After=networking.service and After=systemd-networkd.service;  then there's no hang
<rharper> in the case of networkd/nplan, the netplan generater will add systemd-networkd as a wanted target if /etc/netplan/*.yaml is present
<rharper> the one catch here is that, networking service still runs as well (ie, if you have both ifupdown and networkd/netplan installed)
<rharper> I did have to update the cloud-config.target to Require cloud-init.service ;  it wasn't getting picked up after we dropped the networking requirement
<rharper> this stuff is just painful
<rharper> in the bug, still discussing with pitti and slangasek as the UC16 image is "unique"
<smoser> rharper, i dont know why you'd need 'Require' rather than just 'After' for cloud-config.target
<smoser> the difference (since they're clearly always installed together)  is that Require means only run if the other succeeds
<smoser> we want cloud-config.target to run anyway, as it may be needed.
<smoser> its kind of failsafe.
<smoser> but that is my understanding of Require and After
<rharper> smoser: I agree
<rharper> there unfortunately isn't a systemctl --who-wants=cloud-init.service
<rharper> so I can't tell *why* it isn't run
<rharper> it just _isnt_
<rharper> if I add just that line to the cloud-config.target
<rharper> it works
<rharper> there's not much reason to run cloud-config if cloud-init.service fails though, right?
<rharper> smoser: note that cloud-init.service is *already* in the After= list  for cloud-config.target
 * rharper fiddles some more for the smallest set of changes
<smoser> right.
<rharper> so, I'm left wondering why it didn't run
<smoser> look at the journal
<rharper> it says *nothing* about what didn't run
<smoser> for 'reaking'
<rharper> there wasn't a loop
<smoser> (not freaking)
<smoser> ok
<rharper> it's just not *wanted*
<rharper> which is bizare
<smoser> breaking. that is what i was looking for.
<rharper> cloud-init.target wants it
#cloud-init 2016-10-28
<NerdyBiker> rharper thanks! I totally forgot the instance-specific cloud places
<aixtools> o/ anyone listening
<aixtools> I am trying to clone cloud-init, but nothing seems to be 'cloning'
<aixtools> my local directory gets made, then quiet. While with github - works fine.
<aixtools> I am using: git clone myrealuid@git.launchpad.net:cloud-init to clone.
<aixtools> and in case noone has noticed http(s)://cloudinit.readthedocs.io no longer seems to exist.
<aixtools> afk but hope to read a hint later :)
<aixtools> awake anyone?
<aixtools> after forever, my git clone .... command ended with (excerpt)
<aixtools> ]Read from socket failed: Connection reset by peer
<aixtools> fatal: Could not read from remote repository.
<aixtools> it never asked for a password, and actually I am wondering how USERNAME is supposed to work here, if I have nothing to clone from
<aixtools> git clone YOUR_USERNAME@git.launchpad.net:cloud-init - shouldn't that just be
<aixtools> git clone git.launchpad.net:cloud-init ? trying it anyway...
<aixtools> ok, that fails with: Launchpad user 'michael' doesn't have a registered SSH key
<aixtools> so, since I do not get that message with I prefix my launchpad userid - how can I test that my ssh key is owrking properly?
<NerdyBiker> Anyone have an example as to how to input an ec2 instance ID into a write_file cloud-config? I can't seem to make it work at all. I know it can be found in the /var/lib/cloud/data/instance-id file, but need a way to put that value into a write_file content string.
<smoser> NerdyBiker, i ddont really think you can. it'd be nice, but there is no template on write file content.
<smoser> what you'd have to do is use boothook to execute data that read that and wrote your file.
<smoser> aixtools, well, you can only use git+ssh if you have a launchpad user and registered keys.
<smoser> but you can read-only clone with git://git.launchpad.net/cloud-init
<smoser> or http://git.launchpad.net/cloud-init (or https)
<aixtools> thank you @smoser - using the git:// for the copy - er clone - as I do not expect to have write perms :)
<smoser> aixtools, well, with git+ssh:// you can push back to launchpad for your branch.
<smoser> (independent of having commit access to master)
<smoser> s/branch/repo/ ; s/master/cloud-init official repo/
<aixtools> well, i thought that was what the next line was for... "git remote add USERID USERID@git.launchpad.net:~USERID/cloud-init
<aixtools> -- trying to follow your documentation.
<aixtools> thus - as I am only getting started - which procedure do you recommend. My goal is to a) remake the patch (or hack) someone made for AIX, and b) remade in such a way that it might be accepted for mainstream because c) do nopt want to wait for IBM to get around to it.
<aixtools> however, if d) happens (IBM finishes first AND it is in the official branch - I shall just sit back and relax :)
<aixtools> @smoser - I am still trying to get the "feeling" for how git works - so while I do read re not able to place your "s" command, aka I am almost comfortable with the principle of a pull request.
<smoser> aixtools, sorry. sed syntax an aixer should know that ;)
<aixtools> but not clear on what you mean. It is "whatever works" for collaboration with the intent to introduce a new platform.
<smoser> so you can always clone from git://git.launchpad.net/cloud-init
<smoser> that will work, and then you can commit locally and be happy
<smoser> but if you want to push to a remote git repo, then you can push to:
<aixtools> i know sed - used to use it years ago to automate some of my assembly code on the PDP-11.
<smoser>   git+ssh://aixtools@git.launchpad.net/cloud-init
<smoser> shoot... sorry. you can push to
<aixtools> but it is knowing what 'branch', 'repo', etc. mean. Still grasping with those :)
<smoser>   git+ssh://aixtools@git.launchpad.net/~aixtools/cloud-init
<smoser> a git "repo" (repository) contains multiple branches.
<aixtools> nods: does the "add remote" do anything remote, or is that to set one of the 'references' (I keep forgetting the git idiom)
<aixtools> i am trying to compare this to the process I have followed twice on github: a) fork on github; clone the fork made; commit to fork, and then pull-request.
<aixtools> (with making a branch directly after cloning the fork, and pushing to the branch)
<aixtools> i would guess that git+ssh://aixtools@git.launchpad.net/~aixtools/cloud-init is air atm - my actual id is currently something else though. I should probably try to 'get' aixtools.
<nacc> aixtools: a remote is just a definition of a repository (typically one you care to track)
<nacc> aixtools: you should always be able to read a defined remote (fetch) and you can sometimes write (push)
<nacc> aixtools: so often you have at least one remote, 'origin' e.g., if you use git-clone
<aixtools> reading and thinking... thanks...
<nacc> aixtools: you can have multiple, though, e.g., in launchpad, you might have one for origin (git://) (which I often call upstream) for the main cloud-init repository, then you'd have one for your lp user's repository itself (git+ssh://) (which you'd be able to right to)
<jgrimm> nacc, for context, i believe aixtools is reading this and trying to grok ->https://github.com/openstack/cloud-init/blob/master/HACKING.rst
<nacc> jgrimm: ack, that should probably be updated to the lp git tree?
<aixtools> e.g., the one line that should be two lines, or at least have ";" before cd cloud-init.
<jgrimm> bah, that' s the wrong one indeed!
<jgrimm> google got me, sorry
<aixtools> well, I found a pdf, labeled ...0.7.8 as the cloudinit.readthedocs.io URL seems to be gone
<rharper> http://cloudinit.readthedocs.io/en/latest/
<jgrimm> https://git.launchpad.net/cloud-init/tree/HACKING.rst
<aixtools> it is back! yes, http://cloudinit.readthedocs.io/en/latest/topics/hacking.html
<jgrimm> there we go, that's better
<rharper> git clone YOUR_USERNAME@git.launchpad.net:cloud-init cd cloud-init
<aixtools> same content as the PDF I found
<rharper> yeah that's just mal-formated markdown
<aixtools> FYI, not meaning to be fussy
<nacc> the hacking.rst has it has newline separated
<aixtools> or - I take being a noob seriously ;)
<aixtools> a bit serious - git is still a bit of 'magic' to me. I have been speaking with people who have been using it for 7+ years and there are, I expect many things that seem natural and logical. For me, atm, they are just key-strokes. There is probably one or more excellent books - I just have not found them yet.
<aixtools> Since we are, more or less on the same page - you know what I am reading - is now a good time to have a noob attempt something else?
<smoser> git for sure takes time to become comfortable with.
<smoser> yeah, that HACKING.rst is just wrong i think
<NerdyBiker> @smoser ah okay :( getting a bit tricky now. also, that previous issue with omnibus, si that an issue for someone to take a loko at? is it trackable?
<aixtools> well, I am willing to be a guinea-pig, if it helps to get it right.
<smoser> NerdyBiker, i'm not really sure. you can file a bug against cloud-init.
<smoser> i'd be interestedi in knowing if 16.04 works for you
<smoser> and if it works for you with packages rather than omnibus
<smoser> omnibus really amounts to a moving target
<smoser> basically wget <url> | sudo sh
<smoser> (i realize they're managing the thing that gets executed, and that its probably more secure than that , but the point is it can change at any point and i've not ever really looked at how it works)
<NerdyBiker> @smoser we're using centos so ubuntu isn't a useful test case unfortunately :( I'll see with packages and test that
<smoser> NerdyBiker, i knew you were centos.  you can try packages on centos too
<smoser> i just didn't know if that path works at all, or if omnibus is all that works there.
<NerdyBiker> @smoser was referring to the 16.04 comment, unless that wasn't to me :D
<smoser> yeahm, i know.
<smoser> i dont know if use-packaging path works on centos
<smoser> it *should* work on ubuntu
<smoser> i have more confidence in that :)
<aixtools> still having issues, will come back when I have more time.
<aixtools> thx for assistence.
<aixtools> afk
<NerdyBiker> @smoser haha okay :D finding a work around for now then will dedicate soem time to checking it out and filing an issue
<robjo> :q
<robjo> Could use some help sorting out an issue
<robjo> in order to break the systemd cycle that was present in the 0.7.8 release I patched my package with the cloud-init.service file from master
<robjo> problem is I cannot get that phase, i.e. cloud-init to run thus there is no ssh key added to the image
<robjo> I also tried to add it to the multi-user target instead of cloud-init target but to no avail
<robjo> I am about out of answers and somewhat tired of banging my head against the wall
<robjo> when I create a volume from the failed system and then attache it to a working system, chroot to the volume and then run cloud-init init the ssh key gets added, thus I think it is a problem with getting the code executed
<smoser> robjo, suse ?
<smoser> i suspect,as you do that its not getting run.
<robjo> smoser: yes, I am still trying to get 0.7.8 to work properly :(
<smoser> so i'd backdoor the image in some way so you can get in even if it does not run
<smoser> and then look around at journalctl for some help
<robjo> so what is the reason behind using DefaultDependencies=no in the unit files?
<smoser> https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1611074
<smoser> i really do hope that 'git log systemd/' will tell you such things or at very least point you to bugs.
<smoser> and we've had some annoying fallout also, wrt systemd-resolved
<robjo> smoser: my cloud-init.service file already looks like this patch, with the exception that I still had a Before-dbus.socket entry
<smoser> that probably wasnt causing issue, robjo
<smoser> unless you were using dns in the cloud-init.service and you're running systemd-resolvd. but anway.
<smoser> what i'd do is make sure you can get intot he image either via ssh or local log in and then collect journalctl and hope to find some help there.
<smoser> i suspect something is getting dropped or fails
<robjo> well the messages file has Cloud-init v. 0.7.8 running 'init-local'
<robjo> Starting Cloud-config
<robjo> cloud-init[7479]: Cloud-init v. 0.7.8 running 'modules:config'
<robjo> cloud-init[7492]: Cloud-init v. 0.7.8 running 'modules:final'
<robjo> and also:
<robjo> cloud-init[7492]: ci-info: no authorized ssh keys fingerprints found for user root.
<robjo> this last one is a bit curious
<smoser> hmm.
<smoser> you dont get cloud-init to run
<smoser> because networkign isnt coming up
<smoser> i think or is failing.
<robjo> Yes, that's the conclusion I came to, and looking at the service file again it has: Before=network-online.target
<smoser> right.
<smoser> that is by design.
<smoser> that is a target that other things would wait on
<smoser> and cloud-init wants to run after networking is up, but before that
<robjo> but according to https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/ that is the target the means the newtork is up and running, thus we can reach out and get the ssh key
<smoser> so that other things waiting on networking will have cloud-init.service run before them.
<smoser> what brings up your networking on suse ?
<robjo> well the network is up after ifup, but it's not useful until wicked does it's magic
<smoser> ok..
<smoser> so how do we know that wicked is finished doing its magic
<smoser> After=wicked-did-magic.service
<robjo> but that should be equivalent to After=network-onlie
<robjo> clearly I am not understanding one of the concepts here :(
<smoser> sure. thats true.
<smoser> what we *want* is:
<smoser>  a.) something brings networking fully up
<smoser>  b.) cloud-init.service runs
<smoser>  c.) network-online.target is reached
<smoser> that way, cloud-init gets a bottleneck point as early in boot as it can
<smoser> if cloud-init.service ran 'After=network-online.target' then anything else that ran at that point would be racing with cloud-init.
<smoser> and if the user had provided some user-data to modify behavior of those things, then they'd be racey
<robjo> OK, so I'll try Before: network-online.service and After: wicked.service
<robjo> wicked.service is in network-online.target.wants
<robjo> so that should give us what we are after, fingers crossed
<smoser> you'd think that wicked.service should be Required by network-onlknie.target
<smoser> other wise, network-online.target would possibly fire when there was no network on line
<robjo> wicked.service runs Before=network-online.target network.target
<robjo> so I think the order is going to turn out to be correct
<robjo> and here we alsways thought sysvinit ordering was hell
<smoser> robjo, i have a soft spot in my heart for sysvinit.
<robjo> I was never a big fan simply because I am not a fan of shell scripts, but it did have its positives
<robjo> we the exception of course that we on the distribution end made a mess of it and all had little tweaks that were slightly diferent and annoying to deal with
<robjo> systemd at least resolved that issue
<robjo> with the exception that in ubuntu it appears to be networking.service instead of network.service :(
<robjo> now lets see what this next round of builds produces.....
<smoser> they're different.
<smoser> networking.service is like your wiked
<smoser> wicked
<smoser> its what brings up the networking.
<robjo> OK, so I need to sed -i s/networking/wicked/ and not sed -i s/networking/network/
<robjo> alright I'll fiddle with the patch again..... my changelog is getting really ugly
<robjo> someone will love me for it one day ;)
#cloud-init 2017-10-23
<blackboxsw> ok will have a branch up tomorrow on the SRU https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+ref/fix-device-path-from-cmdline-regression. Just need to complete a test and the download rates I'm getting right now are painful (56K/b) so I'm leaving the download running til morning
<dpb1> blackboxsw: is there a bug associated with that fix?
<blackboxsw> yep dpb1 I'll dig it up and tie it in.
<blackboxsw> https://bugs.launchpad.net/cloud-init/+bug/1725067
<ubot5> Ubuntu bug 1725067 in cloud-init "cloud-init resizefs fails when booting with root=PARTUUID=" [High,Triaged]
<blackboxsw> dpb1: ^
<blackboxsw> the fix will land today
<dpb1> oh, cool
<dpb1> ah, ok, so caught during proposed testing
<dpb1> ?
<smoser> rharper: http://paste.ubuntu.com/25802105/
<smoser> and blackboxsw
<smoser> that *does* pass tests.
<rharper> k
<smoser> we could ideally make some metadata to the patching that would maek that work.
<smoser> would make the "block device" return for a stat or something.
<smoser> but thats getting fancy
 * rharper likes fancy
<blackboxsw> dpb1: yep caught before SRU, so it's where we want to catch it
<blackboxsw> well, we'd like to catch that with unit tests/integration tests
<blackboxsw> but, before release is better than the alternative
<dpb1> blackboxsw: can it be added to a unit test?
<dpb1> s/added to/covered by/
<blackboxsw> dpb1: I added a unit test that gets most of it. I have time to add a bit more concise unit test
<blackboxsw> yeah, my work that introduced this regression was to restructure the code so it was a bit easier to unit test (as it didn't have much in the way of testing). I just didn't add enough tests to validate this case. more to come
 * dpb1 nods
<smoser> blackboxsw: i think you forgot to push on fix-device-paht-from-cmdline-regression
<smoser> https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/332634
<blackboxsw> oops pushed 129beef2..445ee933
<blackboxsw> was debating about adding one more unit test for happy path handle(), but I think I'll add that in a separate merge proposal as it'll be some more rework
<blackboxsw> I want a test that shows we are properly attempting to resize a disk by uuid
<blackboxsw> not just the test that detects the disk/by-uuid from the commandline
<blackboxsw> but I think we don't have to block the SRU for this additional test, since we are manually covering that
<blackboxsw> what do you think smoser
<smoser> i'm fine with  your thinking there.
<nacc> rharper: smoser: if i have a cloud-config snippet in my default lxd profile for user.vendor-data a la http://paste.ubuntu.com/25802522/ does that mean no ubuntu user gets created (that is the behavior I am seeing)
<rharper> nacc: you're declaring the list of users to add
<rharper> so yes, you;re overriding the default user
<nacc> rharper: ok
<nacc> rharper: I think I must have misunderstood that :)
<nacc> rharper: is there an easy way for met to just modify the xisting users?
<nacc> *existing
<rharper> are you just wanting to import your keys?
<rharper> you can use top-level ssh_import_keys: [list of keys here]
<rharper> which will get imported into the default user (Ubuntu)
<nacc> rharper: ah ok
<nacc> rharper: but there is not a top-level ssh-import-ids?
<rharper> ssh_import_id: raharper
<blackboxsw> ok adding a backlog bug for unit test coverage of resizefs
<nacc> rharper: ok, thanks
<blackboxsw> ok filed https://bugs.launchpad.net/cloud-init/+bug/1726489
<ubot5> Ubuntu bug 1726489 in cloud-init "Need more happy path unit test coverage of cc_resizefs " [Medium,Triaged]
<blackboxsw> we need it by 17.2 to avoid any repeated regressions
<blackboxsw> (if we were touching resizefs module)
<nacc> rharper: does that go in user-data or vendor-data?
<rharper> nacc: either will work
<nacc> rharper: hrm ok
<rharper> nacc: generally you're a user
<rharper> things that lxd (or a cloud vendor would set by default) would go into vendor-data; but for your use/scripts/etc user-data is fine
<nacc> rharper: ack
<rharper> http://paste.ubuntu.com/25802644/
<dpb1> they get merged following a very well documented process.
<rharper> nacc: that' usually what I put in my user-data configs for vms and such
<smoser> powersj: https://jenkins.ubuntu.com/server/job/cloud-init-ci/
<smoser> why  oh why. we've gotten so much less reliable recently. :-(
<rharper> dpb1: well, it's well documented but not always what folks expect;
<dpb1> rharper: :)
<blackboxsw> grr FAIL: failed to install dependencies with read-dependencies ret=1
<dpb1> yes, the documentation is there, the testing thing is what we need. :)
<powersj> smoser: it is largely due to the MAAS tests
<powersj> e.g. centos
<blackboxsw> maybe we put in a retry on the yum package installs?
<powersj> err i.e. centos
<smoser> yes, but why did those get less reliable ?
<smoser> we tried to make them more reliable with the no-fastest-mirror
<blackboxsw> smoser: well we dropped the fastest mirror plugin (which we thought was the cause of our pains).
<rharper> until its not
<nacc> rharper: hrm, it's not added my user's key -- but i'll debug it a bit
<rharper> when the "fast" mirror was fast
<blackboxsw> heh, right exactly
<rharper> it helped
<powersj> yes, that was committed on the 19th, even before then things were getting worse
<rharper> nacc: are you launching new instances or modifying an existing one ?
<nacc> rharper: new insntances
<rharper> yes, let's see a cloud-init.log
<nacc> rharper: i'm assuming it would write the appropriate key into .ssh/authorized_keys?
<rharper> yes, under ubuntu
<nacc> rharper: one sec
<nacc> http://paste.ubuntu.com/25802666/
<nacc> for this profile: http://paste.ubuntu.com/25802674/
<smoser> nacc: you can also just reference 'default' in your list.
<smoser> http://paste.ubuntu.com/25802680/ i think does what you want
<smoser> you can use that as vendor-data. user-data would override it.
<nacc> smoser: where default is a special string that means keep whatever the default user is?
<rharper> nacc: I think you want to change your '-' to '_' in the ssh_import_id key; at least that's what's in mine;
<smoser> nacc: yes. default is special, means "the os configured default user"
<nacc> rharper: that would be odd that the top level field is different than the users field?
<nacc> or is that a yaml quirk
<nacc> i'll try it
<nacc> rharper: (you can see in smoser's paste ssh-import-id is used there)
<nacc> rharper: that worked
<nacc> fwiw, I had smoser's variant before and c&p it up
<nacc> that's why i had '-' instead of '_'
<nacc> not sure if that's a bug or not :)
<rharper> one could file a bug and ask for - instead of _ but I believe the manual/rtd shows _
<nacc> rharper: oh it's ok -- just may want to update smoser :-P
<rharper> lol
<rharper> ironic
<nacc> :)
<smoser> rharper: nacc both are accepted there in the users:
<smoser> generally speaking i think we prefer '_' .
<rharper> that's strange
<rharper> the ssh_import_id config module only checks _
<smoser> but the user's dictionary kindof fronts "useradd" (or adduser, not sure which).
<rharper> so users module is "helping" out ?
<nacc> smoser: yeah, something feels odd
<smoser> 'normalize_user's is helping out, rharper
<nacc> smoser: well that it does work sometimes
<smoser> err... normalize_user_groups
<smoser> what "does work sometimes" ?
<nacc> smoser: '-' works in users but not at the top level
<nacc> (that is, ssh_import_id is the only thing that works at the top level)
<rharper> for ssh_import_id at least
<powersj> smoser: blackboxsw: what if we ran `yum makecache` with retries so we had the metadata cache on hand
<powersj> https://paste.ubuntu.com/25802800/
<nacc> rharper: smoser: do you want me to file a bug? i don't think it's a big deal, but it feels inconsistent, at least
<smoser> nacc: well, you can file a bug. i think probably the "fix" is to not document 'ssh-import-id' in the users/groups case.
<smoser> but only document the '_' varieties of all of those.
<nacc> smoser: right
<nacc> but do you really want to accept undocumented stuff?
<smoser> dont have a choice. :)
<nacc> heh
<smoser> at least not without a legacy warning period.
<dpb1> right
<nacc> sure
<smoser> powersj: so what is the failure path that we're seeing ?
<powersj> smoser: on my sampling of failures it is generally the "Cannot retrieve metalink for repository: epel/x86_64" or something similar.
<smoser> why would 'makecache' succeed when just doing an nisatll not ? i'd think they'd hit the same resources.
<smoser> and i'd also think we want to change the 'install' to be yum -C install
<powersj> They do, I suggested it as it might be easier to put a retry around that one command, but maybe it doesn't matter
<smoser> other wise we'd still hit the network :)
<smoser> blackboxsw: you started 2 minutes after me
<smoser> (i was 430, you are 431). which i think testing same thing.
<smoser> we can let them both run and have double the chance of success :)
<rharper> gambler
<blackboxsw> smoser: yeah oopsie, yeah wanted to hope for success in either case
<smoser> i think you're ahead of me. or its a dead eheat
<smoser> it took you 90 seconds less to checkout
<blackboxsw> powersj: not a bad idea on yum makecache. maybe we could do that part daily, and just use cache-only during yum commands?
<blackboxsw> --cacheonly I mean
<smoser> i think the cache is local to the container that we're running in
<smoser> o daily doenst help i dont htink. but  yes, if not paired with '--cacheonly' on the install i think it does nothing.
<blackboxsw> yeah, looks like it, we'd have to push /var/cache/yum into the container if we wanted to leverage a daily cache update\
<rharper> mmm, bind mount read-only
<blackboxsw> it'd save time trying to refresh cache every test run
<blackboxsw> and maybe even cut out some of the yum install errors
<blackboxsw> hrm, kinda big 45M	/var/cache/yum/
<smoser> testing http://paste.ubuntu.com/25803004/
<powersj> lol@MAYBE_RELIABLE_YUM_INSTALL
<blackboxsw> not bad at all :)
<blackboxsw> jenkins #430 FAIL: failed to install dependencies with read-dependencies ret=1
<blackboxsw> all hopes rest on the lone survivor... 431
<blackboxsw> smoser: with that paste we may want to re-enable fastest mirror plugin per tools/run-centos then right?
<blackboxsw> so we allow yum to use fastest mirror and retry if makecache fails
<rharper> or use modulo to randomly enable/disable the fast mirror
<rharper> sorta want to just ship a USB disk up there and leave it attached
<powersj> if you are behind a proxy == don't use fastest mirror
<rharper> except when you do want it because it works faster
<powersj> per https://wiki.centos.org/PackageManagement/Yum/FastestMirror
<smoser> blackboxsw, powersj had pointed at some doc that said dont use fastest mirror if you have a proxy
<rharper> right
<smoser> of course, that makes a lot of sense to document, instead of just DTRT!
<rharper> except disabling it hasn't measurably improved things
<rharper> so, that's great but it wasn't the core problem AFAICT
<blackboxsw> hah smoser on docs instead of DTRT
<blackboxsw> file a bug :
<blackboxsw> :)
<rharper> lol
<blackboxsw> how cloud fastest mirror plugin possible look at environment variables like http_proxy :)
<rharper> well, proxies are clearly slower than the fastest mirror
<blackboxsw> nice comments on the blog post smoser, didn't notice we could do that
<blackboxsw> ... comment inlin ethat is
<blackboxsw> inline even
<smoser> the 'comment' button.
<smoser> i woudlnt have gone looking, but you said something like "you all comment on it"
<smoser> and i assumed you meant there was something like that :)
<smoser> Error Downloading Packages:
<smoser>   1:perl-Module-Pluggable-3.90-144.el6.x86_64: Caching enabled but no local cache of /var/cache/yum/x86_64/6/base/packages/perl-Module-Pluggable-3.90-144.el6.x86_64.rpm from base
<smoser> powersj: ^
<rharper> shant we summon yum mirror masters for help here?
<smoser> http://paste.ubuntu.com/25803096/
<smoser> i dont understand what makecache does
<powersj> smoser: your -C option means to use the deb itself from the local cache.. that's not what we want
<powersj> my hope was to wrap around makecache to get the actual repo files as that seems to always be the part that fails and understand if we have a mirror issue or a network issue.
<powersj> and by repo files I mean the package metadata info
<rharper> the problem is that we don't have a way to *pull* in everything
<powersj> my understanding of makecache is similar to apt update
<rharper> first
<smoser> "does not download or update any *headers*  unless it has to to perform the requested action"
<smoser> (emphaisis added)
<rharper> that is, those packages aren't going to be in the cache, so we can't use -C until we have a copy of those packages
<rharper> the makecache is for metadata only
<rharper> and the yum install -C means (don't pull from repos only from yum cached rpms)
<rharper> so we've a chicken/egg issue
<rharper> we need at least one successful run of the install with yum conf keepcached=1 true
<smoser> but i'm missing something clearly
<rharper> then we can yum makecache
<rharper> and then use yum install -C for all further runs
<smoser> because it looks to me like there is *ALWAYS* a chicken/egg there.
<rharper> yum install -C is equivalent to apt install --reinstall (but wihtout downloading)
<rharper> I think we need to create a lambda function mircoservice to host a yum repo for the ci run
<smoser> this is insane.
<rharper> it's not unrealistic to kick it out for now
<rharper> if the repos are unreliable, it's not a helpful test
<blackboxsw> https://jenkins.ubuntu.com/server/job/cloud-init-ci/431/ looks hung
<powersj> yeeeppp
<rharper> don't kill anything yet
<rharper> cloud-test-ubuntu-xenial-snapshot-ojta1rm2ous4xibxcp1hrklcwhutd | FROZEN
<rharper> that's expected, right? the snapshot ?
<powersj> yes
<powersj> should the zfs file be on the nvme...
<rharper> pid 405 is stopped
<rharper> it's a script
<rharper> running
<rharper> python3 -m tests.cloud_tests run --verbose --os-name xenial --test modules/apt_configure_sources_list.yaml --test modules/ntp_servers --test modules/set_password_list --test modules/user_groups --deb cloud-init_*_all.deb
<powersj> that's the integration test
<rharper> two tests are still "running"
<rharper> cloud-test-ubuntu-xenial-image-modification-1l8nht7xdh4kgp0n5aq and cloud-test-ubuntu-xenial-modules-set-password-list-gs123zqh6npu
 * rharper execs into them
<rharper> it completed
<rharper> but the harness isn't seeing the result
<rharper> does it ssh into it? or exec cat the result.json file ?
<rharper> same for both
<rharper> they completed, no errors
<powersj> I was under the assumption jenkins waits for the return code of the process
<rharper> what stops the container ?
<rharper> don't we poll fore result.json in the lxd instance ?
<rharper> jenkins is fine, it' script spawned the above python3 -m tests.cloud_tests run command
<rharper> it ran 4 containers
<rharper> 2 of which are completed and exited, these two are remaining, but idle
<rharper> cloud-init inside has completed successfully
<rharper> but the harness "watching" each of those container runs has somehow given up on it's "exit" condition
<smoser> lets see if it likes me
<smoser>  https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/332666
<powersj> rharper: from the jenkins log, I see 3 tests were launched and collected from. At this point, based no output alone, I would expect it is stuck/hung trying to get result.json
<powersj> The next thing it should be doing is launching an 4th container for the 4th test
<rharper> powersj: right, but where is the code in the cloud_tests where it's attempt to get the reoslt ?
<powersj> collect.py
<rharper> what calls collect ?
<powersj> run_funcs.py
<rharper> so instance.run_script
<rharper> hrm, where do we import pylxd ?
<rharper> I see platform/lxd.py
<powersj> rharper: that should be it
<smoser> blackboxsw: shall we say screw jenkins for now ?
<smoser> wrt to fix-device-path...
 * blackboxsw was just writing up a retrospective document. that CI breakage on our test service has been painful to say the least
<blackboxsw> smoser: I think we need to move on (as we are manually testing this)
<blackboxsw> and now have a little bit better unit test coverage
<blackboxsw> we'll add the manual logs for deploying proposed for this bug fix we are pulling in
<rharper> powersj: well, I don't really see anything here;  we don't have a way to get pylxd to log it's commands ? Like the excute ?
<powersj> rharper: I'm not familiar enough with pylxd
<blackboxsw> smoser: getting closer Cannot retrieve metalink for repository: epel/x86_64. Please verify its path and try again
<blackboxsw> :: failed [1] (1/10). sleeping 5.
<blackboxsw> hrm it's looping. so it looks like it's making it farther on attempt 2
<blackboxsw> nice smoser Complete!
<blackboxsw> Installing deps: python-configobj python-oauthlib python-contextlib2 python-coverage python-httpretty python-six PyYAML python-jsonpatch python-jinja2 python-jsonschema python-setuptools python-requests python-unittest2 python-mock python-nose e2fsprogs iproute net-tools procps rsyslog shadow-utils sudo python-devel python-setuptools make sudo tar python-tox
<smoser> horay!
<blackboxsw> maaan
<blackboxsw> that's ridiculous
<smoser> ridonkulous
<rharper> powersj: I'm thinking that we likely want a timeout on those executes with a retry
<nacc> blackboxsw: fwiw, sudo is listed twice
<rharper> the container still works via the normal lxc client, so something's wonky with pylxd
<blackboxsw> rharper: is it possible we are missing a raise in the runscript case?
<rharper> blackboxsw: there are no raises afaict
<rharper> so of pylxd doesn't raise on it's failure, we just sit
<rharper> still smells like a pylxd issue
<blackboxsw> ok gotcha.
<powersj> rharper: agreed
<blackboxsw> ohh true nacc
<blackboxsw> we can tweak the deps script to use set([pkg_list]) to ensure uniques
<smoser> blackboxsw: yeah, so 'sudo' is in the COMMON and then manually
<nacc> blackboxsw: not sure it matters, but it probably implies there's not a uniq-like operation
<nacc> blackboxsw: yeah
<smoser> yeah
<smoser> wait. how did sudo get in there twice
<rharper> one can never have enough sudo
<nacc> ironically, i just figured out that my git-ubuntu job failure was to not having sudo installed in the LXD image :)
<nacc> synchronicity between squads achieved!
<blackboxsw> heh smoser I think it's probably pulling in deps from packages/pkg-deps.json
<blackboxsw> hahaha nacc #achievementunlocked
<smoser> hm.. yeah
<blackboxsw> smoser officially approved https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/332666
<blackboxsw> can you land and I can rebase and pull it into my branch
<smoser> blackboxsw: http://paste.ubuntu.com/25803766/
<smoser> that'd fuix the sudo twice, which is probably "who cares"
<smoser> so i'm fine to ignore it for now.
<smoser> and so would your 'sort'
<smoser> err... set()
<smoser> landing now
<nacc> smoser: yeah i don't think it's fatal, but given the output, it's nice to be able to parse it and not be surprised :)
<smoser> blackboxsw: its in.
<smoser> blackboxsw: you should rebase on upstream/master and then push a change and lets see if you get a happy face.
<blackboxsw> ok smoser , sorry missed the paste I was thinking set.unions, but that looks just as 'noisy' as the for loop
<blackboxsw> rebasing
<blackboxsw> pushed
<blackboxsw> rebuilding
<blackboxsw> https://jenkins.ubuntu.com/server/job/cloud-init-ci/433/console
<smoser> blackboxsw: land ?
<smoser> i approved.
<blackboxsw> smoser: it just completed, I'm waiting on tox
<blackboxsw> and will push
<smoser> \o/
<blackboxsw> yeah almost
<smoser> then you can do mps for x, y, z
<smoser> nacc: we still do not have a 'bb', right ?
<smoser> ie, no archive open
<nacc> smoser: right
<smoser> :-(
<nacc> smoser: you can open tasks though
<smoser> sure.
<nacc> smoser: they are just against an unopened archive :)
<blackboxsw> smoser: pushed
<nacc> afaik, no name yet anyways
<smoser> yeah. ok. so i guess we just upload new upstream snapshot, blackboxsw to x, y, z
<blackboxsw> ok smoser time to sync on that upstream
<blackboxsw> just want to dot some i's and cross t's
<blackboxsw> I'm in hangout
<smoser> k
<smoser> blackboxsw: uploaded x, z, a
<blackboxsw> can't do x
<blackboxsw> oops
<smoser> tomorrow we need to
<smoser> make a recipe build for artful
<smoser> and /me is pumpkin now
<blackboxsw> thx smoser
<blackboxsw> see ya
<blackboxsw> will start testing x,z
#cloud-init 2017-10-24
<smoser> wow firefox
<smoser>   PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
<smoser> 13751 smoser    20   0 11.566g 8.920g  55180 S  53.8 57.3 195:39.16 firefox
<smoser> and then the system locked up.
<redcavalier> Hi, regarding https://bugs.launchpad.net/cloud-init/+bug/1722992 , I was wondering if there was any centos nightly build incorporating a fix I could test?
<ubot5> Ubuntu bug 1722992 in cloud-init "On the latest centos 7 release, we are unable to resize our instances filesystems" [Medium,Confirmed]
<sedition> smoser: the internet with javascript is a terrible place
<sedition> im sorry for your loss
<powersj> redcavalier: there is a nighlty build on our COPR repo: https://copr.fedorainfracloud.org/coprs/g/cloud-init/cloud-init-dev/
<redcavalier> powersj, yes, ultimately I was asking if it incorporated a tentative fix for that bug or not.
<powersj> as far as a fix... quickly reading the bug, I don't see anything completed yet :)
<redcavalier> powersj, thx I'll wait then.
<blackboxsw> redcavalier: I got stuck for a couple days on another cc_resizefs module issue that we just landed in master today. I hope to look at that particular bug either tomorrow or the next day. we'll link an inprogress branch to that bug and mark it in progress as we tackle it
<redcavalier> blackboxsw, ah, thanks for the clarification, I'm not familiar how bugs are handled here hence my questions.
<blackboxsw> no prob. we should be clear (and I think we are clearing up our process on that a bit too) so yeah when we grab a bug we should mark it "in progress" and attach related branches to it until we can call it fix committed (in master). Once it hits fix committed you'd be able to grab that copr repo mentioned as it builds all the time from master
<blackboxsw> I'll definitely ping you here to to celebrate when we get there :)
<blackboxsw> in the meantime. thx for your patience (and the pings).
<prometheanfire> ckonstanski: I'm not sure how much cloud-init directly deals with udev
<prometheanfire> not much I think
<ckonstanski> So the eudev thing may not be an issue, at least for cloud-init.
<ckonstanski> (I still worry about chef/puppet, but one thing at a time)
<prometheanfire> ckonstanski: eudev and udev should be mostly interchangable, it's why we have a virtual/udev
<prometheanfire> smoser: ckonstanski is looking to possibly work on the gentoo cloud-init stuff, I believe he has two focuses, updating the gentoo piece to use the new network stack, and support systemd
<ckonstanski> smoser: promethianfire: I am filling out the contributor agreement. A project manager is required. Who should I use?
<ckonstanski> Done. Got smoser's full name from the launchpad page.
<smoser> ckonstanski: right. http://cloudinit.readthedocs.io/en/latest/topics/hacking.html
<smoser> you dont need to put an email address. just my name is fine.
<ckonstanski> All done
<smoser> ckonstanski: thanks.
<ckonstanski> This first change is trivial:
<ckonstanski> -    init_cmd = ['service']  # init scripts
<ckonstanski> +    init_cmd = ['rc-service']  # init scripts
<ckonstanski>  
<ckonstanski> But now I have to figure out how to test it. Any docs on running in dev?
<prometheanfire> ckonstanski: run a 9999 version pointing to your fork I think
<smoser> ckonstanski: i'm sorry, but my gentoo knowleged is nil.
<smoser> for ubuntu and centos we have scripts in trunk that build a package out of trunk, and i'd be willing to take contributions for gentoo also
<ckonstanski> But in the broader sense: the only way to run it is to spin up a VM? No nifty way to put the code through its paces directly on my dev workstation? promethianfire's suggestion (which I do understand) implies that teh VM route is the only way to go.
<prometheanfire> I'm not sure about the test harness
<ckonstanski> Is there a particular bug that I can test for while spinning up a VM that would fail under the old code and pass if my fix is good? A particular thing I can try in my cloud.cfg?
<prometheanfire> atm, no, since both rc-service and services are both still available
<prometheanfire> the only thing that'd pick this up are unit tests, if available
<ckonstanski> ok
<ckonstanski> I did notice that "service" is installed on my laptop and it's provided by openrc. This is going away?
<prometheanfire> yes
<prometheanfire> eselect news read all
<prometheanfire> that
<smoser> ckonstanski: well, there are unit tests (run via 'tox')
<smoser> but they're just unit tests.
<prometheanfire> the last message there mentions it going away
<prometheanfire> smoser: yarp
<smoser> there are integration tests that run lxc containers, but they dont' run any gentoo
<smoser> they could, but there would be some work involved to do that.
<prometheanfire> that'd be good
<ckonstanski> A packager and lxc. Got it.
<ckonstanski> Does tox usually run cleanly? Getting 3 errors out of 5.
<prometheanfire> pastebin :P
<prometheanfire> smoser: for systemd, I should pass in --init-system=systemd right?
<prometheanfire> not sure about this systemd.generators thing :P
<ckonstanski> tox failures: https://paste.pound-python.org/show/FkZ6lWLKXJBnUXfVHoSB/
<smoser> prometheanfire: yes. init-system=systemd
<smoser> but i think it will default to that.
<ckonstanski> Actually that sucked. Let me redirect 2>&1
<prometheanfire> smoser: so, what if I want to install two init systems on the same system?
<smoser> the systemd generator turns cloud-init on or off based on if its running in a cloud or not.
<smoser> prometheanfire: well, thats up to you. it "should work", in the snese that files can be installed.
<smoser> but interaction with the system, i'm not exactly sure of
<smoser> on ubuntu up until very recently we did have both upstart and systemd being installed.
<smoser> --init-system=systemd,sysvinit
<prometheanfire> oh, you can use commas
<prometheanfire> I was rerunning setup.py
<ckonstanski> For real this time: https://paste.pound-python.org/show/t5M9bUnXhmZBXqRYz1f4/
<prometheanfire> ckonstanski: can you chmod +x files in sysvinit/gentoo/* ?
<prometheanfire> as a separate thing
<smoser> ckonstanski: well, thats not too bad. its just the ntp tests. limited to one set of tests. basically failing because no support for gentoo
<ckonstanski> add to the list
<ckonstanski> Need to keep notes. Don't want to do all this in one branch.
<prometheanfire> :D
<Odd_Bloke> smoser: rharper: Does the cloud-init failure in the log attached to https://bugs.launchpad.net/cloud-images/+bug/1726818 ring any bells?
<ubot5> Ubuntu bug 1726818 in cloud-images "vagrant artful64 box filesystem too small" [Undecided,New]
<smoser> possibly https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1725067
<ubot5> Ubuntu bug 1725067 in cloud-init (Ubuntu Artful) "cloud-init resizefs fails when booting with root=PARTUUID=" [Medium,Fix committed]
<prometheanfire> it looks like 17.1 supports python 2.6 to 3.6, is that right?
<smoser> prometheanfire: yes
<prometheanfire> ok, well 17.1 packaged then
<rharper> smoser: https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/332722
<smoser> Odd_Bloke: i responded on that bug. have you recrated this ?
<Odd_Bloke> smoser: I haven't, no; been in meetings all morning.
<Odd_Bloke> Will try now.
<Odd_Bloke> smoser: I can reproduce with 'vagrant init ubuntu/artful64;vagrant up;vagrant ssh'.
<prometheanfire> ckonstanski: you submit that?
<prometheanfire> should look something like https://git.launchpad.net/~prometheanfire/cloud-init/commit/?id=b1f5be693cbdc857911c9c36080af471ba0b7287 I think
<smoser> Odd_Bloke: did you run on "physical" host system ?
<Odd_Bloke> smoser: I did that on my laptop.
<Odd_Bloke> (If that's what you're asking?)
<smoser> yeah (versus what i did, running in serverstack guest)
<smoser> can you dmesg | pasetbinit
<smoser> i have http://paste.ubuntu.com/25811245/
<smoser> yeah, never mind. the bug opener does/did too
<ckonstanski> promethianfire: I have not submitted anything yet. The day job calls. Will do it later today unless someone else does it first.
<prometheanfire> ckonstanski: I was about to, but I can wait
<smoser> Odd_Bloke: i'm done with that bug. from my perspective this surely looks like a kernel issue.
<smoser> and i collected logs and such, so i'll leave it there. you can mark /triage the 'cloud-images' task however you would.
<Odd_Bloke> smoser: Ack, thanks.
<smoser> kernel traces at 0.000000 seconds into boot
<smoser> although mine didnt' stactrace until the resize time frame.
<ckonstanski> I was going to create launchpad bugs so I could reference them in my commits. But I'm not seeing how to create a bug in launchpad. I wonder if I have more stuff to set up, like the PGP key. This stuff is what's holding me up from pushing these couple easy changes, and why I need to do it later when I can work through these peripheral issues. Or should I just do the rc-service and chmod +x changes without bugs?
<smoser> ckonstanski: if you want to file bugs via email you probably need a pgp key
<smoser> if you just want to file a bug in UI..
<smoser> https://bugs.launchpad.net/cloud-init/+filebug
<smoser> (that was just https://launchpad.net/cloud-init -> 'Report a bug')
<ckonstanski> That helps. I'll at least make the bugs. I have no such link. But I can reach it direcly via the URL. Makes me think my setup is incomplete. Will work on that more later.
<smoser> you didnt have a link.. odd. on the right side.
<smoser> you're logged in ?
<ckonstanski> Logged in yes, link present no.
<ckonstanski> Now that I'm on the right page, yes. There's a link.
<ckonstanski> my bad
<ckonstanski> I can't give it the proper attention right now. Making mistakes.
<smoser> :)
<blackboxsw> smoser: I nominated the following for xenial/zesty/artful https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1724951 not sure if I can actually add the series bug task myself
<ubot5> Ubuntu bug 1724951 in cloud-init "Ntp schema definition permits empty ntp cloud-config, but code disallows" [Medium,In progress]
<powersj> blackboxsw: approved
<blackboxsw> ahh thanks powersj
<prometheanfire> ckonstanski: once you get a branch pushed with your changes showing up in launchpad it's a button push to submit
#cloud-init 2017-10-25
<ckonstanski> Trying to bust out the two easy issues now. I created a launchpad bug for the rc-service one. How do I assign it to myself? Or do I not do that? (This is where I learn the procedures around launchpad)
<ckonstanski> Pushed commits for issues 1727121 and 1727126
<ckonstanski> A brreakdown in the "hacking" instrucitons. Will catch up tomorrow.
<prometheanfire> cool
<prometheanfire> I made https://code.launchpad.net/~prometheanfire/cloud-init/+git/cloud-init/+merge/332754 , but I'd rather just use yours
<prometheanfire> ckonstanski: all you should need to do is go to your fork/branch and click the merge this button
<ckonstanski> target reference path? master?
<ckonstanski> found it in "hacking" -- master
<ckonstanski> done
<prometheanfire> ckonstanski: cool, link?
<ckonstanski> http://cloudinit.readthedocs.io/en/latest/topics/hacking.html
<prometheanfire> ckonstanski: sorry, to your merge request :P
<ckonstanski> https://code.launchpad.net/~ckonstanski/cloud-init/+git/cloud-init/+merge/332755
<ckonstanski> https://code.launchpad.net/~ckonstanski/cloud-init/+git/cloud-init/+merge/332756
<ckonstanski> 2 different ones
<smoser> ckonstanski: prometheanfire i've seen your bugs and mp... i do want to review for you.
<smoser> just time and such. but we'll try to get some feedback today.
<prometheanfire> smoser: sure, don't need to do mine, do ckonstanski's
<prometheanfire> removed (deleted)
<powersj> smoser: your update looks good, however I am unsure about the variable substitution into the shell script
<powersj> when it generates the xml right now it produces "release={release}
<powersj> triggerwhat={triggerwhat}"
<powersj> which isn't what we want I think
<smoser> hm..
<smoser> powersj: i was trying ot generate the xml locally
<powersj> This is why there was lots of copy and pasting... I always got frustrated with this
<smoser> but dont know how ?
<smoser> if i run './test-jenkins-jobs' locally, it ends up i think trying to post xml
<smoser> (and failing reasonably)
<powersj> the test-jenkins-jobs will run "jenkins-jobs test <dir>"
<smoser> yeah, but lcoally
<powersj> and if it produces XML without any exception, then it means the job at least generated
<powersj> that is all local
<powersj> won't try to push anything or update the jenkins as there are no creds in that repo :)
<smoser> http://paste.ubuntu.com/25817851/
<smoser> that is on master (with jenkins-jobs installed from archive)
<powersj> what is your version of jenkins-job-builder? because I am using pipelines it has to be a certain version
<powersj> yeah the archive version won't work
<powersj> if you look at the readme or travis file you will see I have ot use: jenkins-job-builder==2.0.0.0b2
<smoser> powersj: pip install jenkins-job-builder just got me 1.6.2
<ckonstanski> ckonstanski@sphinkpad:~/jobs/talligent/projects/openbook-v4-ci/jjb $ cat requirements.txt
<ckonstanski> jenkins-job-builder>=2.0.0.0b2
<ckonstanski>  
<ckonstanski> That'll git er
<powersj> smoser: yeah if you add the >= or == with version it'll do it
<smoser> powersj: :-(
<smoser> why is my macro not expanding
<smoser> powersj: http://paste.ubuntu.com/25817997/
<smoser> "yaml is hard"
<powersj> HAHAHAHA
<smoser> started with 'foo.yaml' like rthis
<smoser>  http://paste.ubuntu.com/25818005/
<smoser> and iterated
<powersj> that looks good
<smoser> powersj: i'm not sure if i have the 'artifacts' right though
<smoser> what adds the 'cloud-init/' that you have in cloud-init/integration.yaml
<smoser>       - archive:
<smoser>           artifacts: 'cloud-init/results/**'
<powersj> the git clone
<powersj> clone's into cloud-init
<smoser> got it. thanks.
<powersj> so $WORKSPACE/cloud-init
<powersj> smoser: any reason to not merge?
<powersj> looks good on my end
<smoser> powersj: go for it.
<ckonstanski> Is cloud-init an openstack project? Need to know for a question asked at work.
<powersj> ckonstanski: we are not, smoser could give better history though
<powersj> smoser: jobs deployed, looks good! thank you :D
<smoser> ckonstanski: cloud-init 2.0 kind of started as a openstack project, but that work was shelved, to rather continue development on 1.0
<smoser> powersj: i'm going to hit 'build-now'
<smoser>  https://jenkins.ubuntu.com/server/view/cloud-init/job/cloud-init-integration-proposed-x/
<powersj> smoser: go for it
<prometheanfire> ckonstanski: it's not, just heavilly used by images running on openstack
<smoser> powersj: https://jenkins.ubuntu.com/server/view/cloud-init/job/cloud-init-integration-proposed-x/80/console :-(
<powersj> AssertionError: '0.cloud-init.mypool' not found in ''
<powersj> FAIL: test_ntpq_servers (tests.cloud_tests.testcases.modules.ntp_pools.TestNtpPools)
<smoser> ugh. how the $*(# did that fail
<powersj> love this new output btw... thanks :)
<smoser> while 79 passed
<powersj> modules/ntp_pools
<powersj> uhh hold on
<powersj> it built the deb....
<powersj> something may have changed when I merged your jenkins job... doesn't look like it is doing the pull-lp-source cloud-init xenial-proposed
<smoser> oh? did i foobar that ?
<powersj> oh your builder is wrong
<powersj> review fail on my part
<powersj> see I trust you too much ;)
<smoser> i grabbed the wrong code block
<smoser> phft
<smoser> powersj: mp coming . sorry.
<powersj> no worries, I feel bad for missing it in the review
<smoser> powersj: the start dir of a job
<smoser> isnt it going to be the work dir ?
<smoser> ie, a base, clean dir ?
<powersj> smoser: I don't believe so
<powersj> that's why I have rm everywhere
<smoser> doesnt taht seem scary and silly ?
<smoser> ie, what would be in a directory if it was not empty ?
<smoser> only another jobs data
<smoser> which we are going to remove ?
<powersj> it is scary :)
<powersj> hence the rm -rf *
<smoser> https://stackoverflow.com/questions/24412418/whats-the-working-directory-in-jenkins-after-executing-a-shell-command-box
<smoser> Building remotely on torkoal (metal-amd64) in workspace /var/lib/jenkins/slaves/torkoal/workspace/cloud-init-integration-proposed-x
<smoser> doesnt it seem completely broken ?
<powersj> that seems to be talking about multiple build steps, which may treat it differently I haven't played with that though
<powersj> hence why folks ask about cleaning up the workspace: https://stackoverflow.com/questions/28683914/is-there-any-way-to-cleanup-jenkins-workspace
<smoser> powersj do you get hwat i'm saying ?
<smoser> ah. ok. i get it.
<powersj> I do, but I'm not sure if you are implying that Jenkins should be cleaning it up for us and we should not be doing the rm or something else
<smoser> you can't run a job twice at the same time.
<powersj> that is correct
<smoser> ok.
<smoser> powersj: https://github.com/canonical-server/jenkins-jobs/pull/15
<powersj> smoser: I like what you did there
<smoser> powersj: how quick does it go live ?
<powersj> smoser: it is live now
<powersj> smoser: update jobs every 15mins, I just kicked it though to get it live
<smoser> k
 * smoser build-nows
<smoser> oh fudge
<smoser> :-( https://github.com/canonical-server/jenkins-jobs/pull/16
<powersj> smoser: accepted and jobs updated
<powersj> smoser: all looks good now?
<smoser> yeah. \o/
<powersj> :)
<smoser> now some day, clean  up all the other jobs.
<smoser> x ran, i pushed 'go' on 'z'
<powersj> yeah
<smoser> and then will do on 'a'
 * smoser will try to script somthing like i suggested to grab results and put them on a bug.\
<smoser> blackboxsw: ^
<smoser> blackboxsw: did you manually do all those bug updates ?
<blackboxsw> smoser: yeah I'm wrapping up each test (I was missing some it looked like)
<blackboxsw> jsonschema warning->debug is last
<blackboxsw> then I think we are good
<blackboxsw> lemme push
<blackboxsw> I manually added SRU verification logs to our existing ubuntu-sru/bugs*
<blackboxsw> smoser: pushed b0b83fc0e659f3e30f948ef0b9a63df14054aca8 to ubuntu-sru
<blackboxsw> I've manually separated the verification results content and added it to the bug as I've untagged the verification-needed ->done switches
<blackboxsw> but left the script/results as one file in ubuntu-sru
<blackboxsw> we could separate easily if you wanted so it's easier to add results to bugs as a comment
<blackboxsw> last one to validate here http://pad.lv/#1724354
<smoser> blackboxsw: i'm pretty sure you can do this:
<smoser>  http://paste.ubuntu.com/25818813/
<blackboxsw> +1 smoser
<blackboxsw> ok pushed and included ryan's sru info for https://launchpadlibrarian.net/341226002/1718287.log
<smoser> blackboxsw: fyi https://bugs.launchpad.net/cloud-init/+bug/1727358
<ubot5> Ubuntu bug 1727358 in cloud-init (Ubuntu) "cloud-init is slow to complete init on minimized images" [Undecided,Incomplete]
<blackboxsw> hrm from sru validation run on 17.1?
<smoser> no.
<smoser> from cyphermox reported yesterday.
<smoser> i have to run :-(
<powersj> should get actual cloud-init analyze output
<smoser> it wont help much
<smoser> the 120 seconds is lost in that one line of debug
<powersj> ah
<powersj> so he already did the analyze part ;)
<smoser> i have to run, we have successful integration-proposed-x z a
<powersj> woohoo
<smoser> well, i did
 * powersj should now go read it
<smoser> so we should copy those and attach to the sru bug
<smoser> i can do that later, but have to run now.
<roadmr> hey folks
<roadmr> cloud-init 0.7.5 on ubuntu 14.04 doesn't understand the apt: proxy: value, instead it wants apt_proxy. I asked for help with lxc and they told me to report this for cloud-init. I can file a Launchpad bug if you like.
<roadmr> https://github.com/lxc/lxd/issues/3975
<dpb1> roadmr: file a bug please
<roadmr> dpb1: will do!
<roadmr> dpb1: https://bugs.launchpad.net/cloud-init, right?
<blackboxsw> yeah hrm roadmr I think it's worth a bug, I'm not sure if cloud-init 17.1 would support the same cloud-config keys back in 0.7.5 but it's worth a bug so we discuss and document whether or not we'll backport that cloud-config
<blackboxsw> yeah roadmr
<blackboxsw> https://bugs.launchpad.net/cloud-init/+filebug
<dpb1> blackboxsw: right, we will want to make our stance clear in the bug either way
<roadmr> https://bugs.launchpad.net/cloud-init/+bug/1727527
<ubot5> Ubuntu bug 1727527 in cloud-init "cloud-init 0.7.5 doesn't honor the apt: proxy: setting, needs apt_proxy instead" [Undecided,New]
<roadmr> the alternate solution works fwiw, so it doesn't seem horrible, but it would be nice if it were documented :)
<blackboxsw> thanks for the bug
<roadmr> np :)
#cloud-init 2017-10-26
<smoser> ckonstanski: reviewed your two MP, i'll grab those tomorrow.
<smoser> note my updates to commit messages we'll use those.  I realize they're long windeded for the changes but i like to read git logs (yeah, i'm that nerdy).
<smoser> also, LP: #XXXXX
<smoser> is the way you reference a bug.
<smoser> Thanks!
<ckonstanski> Will do. At Charter se used Issue: #xxxxx. In Openstack it's Closes-Bug: xxxxx. One more to learn.
<ckonstanski> The perfect couple of bugs to learn the ropes.
<powersj> smoser: here is the merge https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/331736
<powersj> blackboxsw: ^
<blackboxsw> https://bugs.launchpad.net/cloud-init/+bug/1725067
<ubot5> Ubuntu bug 1725067 in cloud-init (Ubuntu Artful) "cloud-init resizefs fails when booting with root=PARTUUID=" [Medium,Fix committed]
<blackboxsw> so this is the bug I think we expect foundations to approve
<blackboxsw> hrm smoser rharper https://launchpad.net/cloud-init/+packages shows cloud-init for artful at rev 17.1-25-g17a15f9e-0ubuntu1~17.10.1 yes apt update/apt policy on artful still shows only 17.1-18-gd4f70470-0ubuntu1
<blackboxsw> what 'else' do we wait for before we can see a published package version via apt repos once an SRU has landed?
<nacc> blackboxsw: it's in artful-proposed
<blackboxsw> pending-sru.html still shows the cloud-init package in pending status for artful
<nacc> (per rmadison)
<blackboxsw> nacc right, but once we pass SRU, doesn't it move to artful-updates?
<blackboxsw>  
<nacc> blackboxsw: after 7 days
<blackboxsw> ahhh
<nacc> blackboxsw: depending on the package, there are exceptions, but usally there is a back period to flush out regressiosn
<blackboxsw> was missing "what else needs doing" there. Answer:patience child, good things come to those who wait
<blackboxsw> thx nacc makes sense
<nacc> blackboxsw: http://people.canonical.com/~ubuntu-archive/pending-sru.html
<nacc> blackboxsw: it's pending still, but green on bugs meaning all are verified
<blackboxsw> yeah that is still sitting in artful (all green)
<blackboxsw> yeah
<blackboxsw> didn't know if there was another button/person I needed to push (as the artful bugs themselves have been commented by landscape janitor as uploaded
<nacc> yep
<smoser> blackboxsw and powersj http://paste.ubuntu.com/25825019/
<smoser> that is what i have run/am-running fro nocloud tests. and will post to the bugs.
<smoser> bug
<powersj> smoser: that looks good
 * blackboxsw likes it. I'm reviewing powersj kvm branch 
<blackboxsw> I find myself wanting a make target for running out integration tests in general so I don't have to lookup what we are running from jenkins
<smoser> yeah. something needs easier-ing
<powersj> blackboxsw: so like a --proposed and it just does the right thing?
<blackboxsw> powersj: yeah like that, I just want to document it somewhere in cloud-init src proper.
<powersj> blackboxsw: https://cloudinit.readthedocs.io/en/latest/topics/tests.html
<powersj> that page would be a good place for it
<powersj> Have an example runs
<powersj> can put what we do for nightly testing + proposed testing + ci
<smoser> rharper: so.. looking at cyphermox's image
<rharper> y
<smoser> crazy. if i enable debug logging to the console (change 'WARNING' to 'DEBUG' in the stderr logger in /etc/cloud/cloud.cfg.d/05_logging.cfg)
<smoser> then ENOREPRODUCE
<rharper> hrm
<smoser> but by default, it reproduces ;)
<rharper> doesn't change the logging
<rharper> and if you just remove the import random from util.py in your image
<rharper> what happens ?
<smoser> apparently writing stuff to the console constitues entropy
<smoser> i'll try that.
<cyphermox> smoser: oh, hold on
<rharper> oh, yeah
<rharper> for sure
<rharper> console data
<cyphermox> smoser: if you boot once and the reboot you won't necessarily get the same result
<smoser> i'm not booting.
<smoser> mounting, changin file, unmounting booting.
<cyphermox> fair enough
<rharper> do we have the KCONFIG delta ?
<cyphermox> you'll have to ask kamal for that
<rharper> doesn't the image have the config-X file in /boot ?
<rharper> that's enough then we can diff from some other image
<smoser> removing 'import random' didnt help
<cyphermox> rharper: what are you looking for?
<rharper> delta
<cyphermox> presumably for something in particular
<rharper> not yet
<smoser> # pastebinit /boot/config-4.4.0-1009-kvm
<smoser> http://paste.ubuntu.com/25825393/
<smoser> $ diff -u config-4.4.0-59-generic  config-4.4.0-1009-kvm  | pastebinit -f diff
<smoser> http://paste.ubuntu.com/25825403/
<rharper> no CONFIG_USER_NS ?  aren't the minumals used for containers ?
<blackboxsw> powersj: how'd you setup /srv/citest/nocloud-kvm for integration test runs?
<blackboxsw> I'm trying to run your nocloud-kvm branch
<powersj> mkdir -p /srv/citest/; chown -R jenkins:jenkins /srv/citest
<powersj> something like that...
<blackboxsw> kk thx. just trying to document shortcomings on a fresh artful install
<powersj> basically make sure you own that dir
<powersj> :)
<powersj> thx
<blackboxsw> np
<powersj> artful!
<powersj> did you upgrade?
<blackboxsw> my desktop is artful, laptop1 xenial laptop2 zesty
<blackboxsw> :)
<blackboxsw> want to play w/ all the pretty ponies
<sedition> hi team.
<blackboxsw> hi sedition
<sedition> y'all are always working hard. one of the most active channels i'm in.
<blackboxsw> :) maybe we type too much in IRC :)
<sedition> no such thing. i've been an irc junkie for too long
<blackboxsw> heh
<sedition> i've really been digging cloud-init so i figured i would idle and watch
<blackboxsw> that's nice to hear. File bugs/comments/code etc if you find features you think are needed etc. we have that status meeting Monday where we brain dump what we've been working toward.  You are welcome to raise questions concerns there too if there are things you are itching to discuss with a broader audience
<blackboxsw> reminds me I need to update the topic
* blackboxsw changed the topic of #cloud-init to: is Reviews: http://bit.ly/ci-reviews | Meeting minutes: https://goo.gl/mrHdaj |  Next status meeting: Monday 10/30 16:00 UTC | cloud-init 17.1 released
<sedition> ok great! i can do that.
<blackboxsw> so Monday 16:00 UTC there are a few more eyes on this channel
<blackboxsw> :)
<blackboxsw> should be every 2 weeks
<blackboxsw> grr and I need to publish last meeting logs to the github.io page.
<blkadder> Hi all, I am trying to figure out the appropriate incantation to get cloud-init to run multiple commands via pipes while sudo'd to a user. The following works just fine when done as root: " gpg --list-keys --with-colons | grep sub | awk -F: '{print $5}' | gpg --armor --output /repo/qbis-repo.gpg.pub --export" When I try to invoke it via sudo -u -i user however it simply ignores the entire list of runcmd directives.
<blkadder> sudo -u user -i that is...
<blkadder> I've also tried sudo -u user -i sh -c which works on the command line but not via cloud-init.
<blkadder> Am I relegated to making this a script or something?
<rharper> hrm
<rharper> blkadder: do you have a paste of your runcmd yaml ?
<blkadder> Yes, but I have been fiddling with it endlessly to try to get it to work.
<blkadder> How should I post it?
<blkadder> It's just the one line that is causing problems...
<blkadder> If I comment it out everything else works.
<blkadder> And if I do it all as root (no sudo) it all works...
<rharper> yeah, just runcmd: and the lines that don't work
<rharper> we can generally replace the gpg and stuff with piped echos and greps just to simulate the pipes
<rharper> also your cloud-init.log may be helpful in case there's something else going on
<blkadder>    - sudo -u jenkins "gpg --list-keys --with-colons | grep sub | awk -F: '{print $5}' | gpg --armor --output /repo/qbis-repo.gpg.pub --export"
<blkadder> That was one invocation.
<rharper> k
<blkadder> I have also tried with - sudo -i -u jenkins
<blkadder> And even sudo -i -u jenkins sh -c "...foo..."
<rharper> try this
<rharper> -  [sudo, -u, jenkins, "gpg --list-keys --with-colons | grep sub | awk -F: '{print $5}' | gpg --armor --output /repo/qbis-repo.gpg.pub --export"]
<blkadder> Will do, thanks.
<rharper> that gets put into the shell'fied config in /var/lib/cloud/instance/scripts/*  so you can see the script format there; possible run it by hand in the instance to see if there are other errors
<rharper> root@a1:/var/lib/cloud/instance/scripts# cat runcmd
<rharper> #!/bin/sh
<rharper> 'sudo' '-u' 'jenkins' 'gpg --list-keys --with-colons | grep sub | awk -F: '\''{print $5}'\'' | gpg --armor --output /repo/qbis-repo.gpg.pub --export'
<blkadder> That's interesting...
<blkadder> That invocation creates a shell script?
<rharper> runcmd elements get shellified
<rharper> that was the latest cloud-init; possible the older versions just exec them;
 * rharper reads the docs and code
<blkadder> That worked, thanks.
<blkadder> I am doing my best to read docs and googling before bugging the channel. :-)
<blkadder> My google fu failed on this particular nuance.
<rharper> no
<rharper> I know it changed
<blkadder> Ahh. ok.
<rharper> so the string is execve'd
<rharper> the list is shellified
<blkadder> Oh no it didn't quite work.
<blkadder> The parser works now but...
<blkadder> sudo: gpg --list-keys --with-colons | grep sub | awk -F: '{print $5}' | gpg --armor --output /repo/qbis-repo.gpg.pub --export: command not found
<blkadder> Before it wasn't even parsing the yaml blob (errored in logs)
<rharper> yea, I can see that, lemme play with the escaping
<blackboxsw> asking in channel is fine. it helps us know the probs people are hitting. (and it helps me learn from rharper/smoser)
<blkadder> Ok good deal, just didn't want to bother with noob questions...
<rharper> ok, try this
<rharper> -  [sudo, -u, jenkins, --, "gpg --list-keys --with-colons | grep sub | awk -F: '{print $5}' | gpg --armor --output /repo/qbis-repo.gpg.pub --export"]
<blkadder> Ok, one sec.
<rharper> the -- tells sudo that it doesn't have any more argument parsing to do
<blkadder> Thanks for the info I was wondering...
<blackboxsw> rharper: I also just validated pipes in general in runcmd with this
<blackboxsw>  cat runcmd_pipes.yml
<blackboxsw> #cloud-config
<blackboxsw> runcmd:
<blackboxsw>  - cat /etc/os-release | grep CODE > /tmp/cloud-init-output
<blackboxsw> no quotes etc worked for me (I know blkadder's using single quotes etc on a more complex command).
<rharper> well, the real trick is knowing that the  - [run, my, cmd] gets shellified, which makes | easy
<rharper> otherwise you need the exec'ed command to invoke a shell for you sudo -u jenkins -- sh -c 'do this | eat that | write here'
<blkadder> The pipes work fine, it's the sudo that seems to cause issues...
<blkadder> If I run that same command that I posted previously as root, no problem.
<rharper> right, you're in a shell
<rharper> vs. exec, which expects a binary
<blkadder> Same result rharper...
<rharper> no dice, huh; I don't get that --export command not found
<blkadder> Let me double check my yaml file
<blkadder>   - [sudo, -u, jenkins, --, "gpg --list-keys --with-colons | grep sub | awk -F: '{print $5}' | gpg --armor --output /repo/qbis-repo.gpg.pub --export"]
<rharper> I think I answered my own question;  you're sudo'ing to jekins to dump the keys, and then running the rest of that as non-jenkins
<blkadder> Ahh, I want to run it all as jenkins. :-)
<rharper> ok, that makes it easier
<rharper> we'll have sudo run a shell as jenkins
<blkadder> I tried that.
<blkadder> But you have a smarter way I am sure.
<blkadder> I tried sudo -i -u jenkins sh -c ...
<rharper> the trouble you'll have is that you can't write to /root as jenkins
<blkadder> I don't want to.
<blkadder> Am I missing something?
<blkadder> I am writing to /repo
<blkadder> Which is owned by jenkins.
<rharper> oh, right
<rharper> that was a local error
<rharper> I don't have a jenkins, so I'm subbing something else
<blkadder> Ahh
<rharper> -  [sudo, -u, jenkins, sh, -c, "gpg --list-keys --with-colons | grep sub | awk -F: '{print $5}' | gpg --armor --output /repo/qbis-repo.gpg.pub --export"]
<rharper> I replaced jenkins with ubuntu; but running that gets me gpg running and no complaint about export commnd
<blkadder> Hmm...
<blkadder> Let me give 'er a shot..
<blkadder> And we have key!
<blkadder> Thanks!
<rharper> \o/
<sedition> :)
<rharper> blkadder: thanks for stopping by =)
<blkadder> So on a side note would you be willing to explain the difference in invocation method?
<rharper> yeah
<blkadder> I tried sudo -u -i jenkins sh -c which did not work.
<rharper> you had -i
<rharper> which does the login shell
<blkadder> Right.
<rharper> which I didn't try; but resets the env and other things
<rharper> but, you also didnt use the [, list  ] method
<blkadder> I did not.
<rharper> IIURC
<rharper> so, cloud-init attempted to exec("string")
<blkadder> I have used it elsewhere.
<rharper> I suppose that should work if quoting and all were correct
<blkadder> That's what I am getting hung up on... Works fine on command line but not via cloud-init
<rharper> if you have logs of the failure (/var/log/cloud-init*.log ) I might be able to see what went wrong
<blkadder> Well, I can certainly reproduce if it is helpful.
<blkadder> Not a huge deal since I have a working solution, but just more curious why what works on the command line doesn't work the same...
<rharper> well, I think it was the quoting since your were getting yaml parsing errors
<blkadder> Ack.
<rharper> http://paste.ubuntu.com/25825970/
<rharper> you can try that
<rharper> using the | to let you inline the command as a string
<blkadder> Ahh, cool.
<blkadder> Now to figure out why I am seeing read errors in the log...
<blkadder> Thanks again rharper.
<rharper> np
<blackboxsw> powersj: minor comment on https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/331736
<powersj> blackboxsw: re: tox, correct we pulled that in on a different commit, but that is in master
<powersj> https://github.com/cloud-init/cloud-init/commit/aa024e331f8196855fa8d707a2dd7e26e1deab40
<blackboxsw> ok yeah I didn't recal until I saw it working on my other master checkout
<powersj> blackboxsw: what do you think about the use of /tmp
<blackboxsw> powersj: as mentioned inline. use /var/tmp instead
<blackboxsw> it doesn't get clobbered as frequently
<blackboxsw> by systemd
<blackboxsw> it's what cloud-init does too for anything we write out that needs to be somewhat persistent
<powersj> *sigh*
<powersj> ok :)
<blackboxsw> though I don't think it should really affect these tests, best to get rid of any known intermittent failure conditions as that's been the bane of our existing with CI in the last couple weeks
<blackboxsw> s/in the last/over the last/
<dpb1> why do we need persistence of files in tmp?
<powersj> I feel that this should be equally documented then
<blackboxsw> by *our* I mean you and rharper
<dpb1> past a run that is 1 hour long
<blackboxsw> powersj: yeah want me to add it to readthedocs? like runcmd? or write_files ?
<dpb1> I don't think so
<dpb1> it's not a cloud-init specific thing
<blackboxsw> like a Note: don't write to /tmp because of https://bugs.launchpad.net/cloud-init/+bug/1707222?
<ubot5> Ubuntu bug 1707222 in cloud-init (Ubuntu) "usage of /tmp during boot is not safe due to systemd-tmpfiles-clean" [High,Fix released]
<powersj> dpb1: how else would an end user know about this? I get that maybe it isn't a cloud-init only thing
<dpb1> hm
<dpb1> because of the after= before=
<powersj> as an end user writting user-data I'm not sure I'd ever see that...
<dpb1> ok
<dpb1> guess so
<dpb1> strike my comments
<dpb1> yes, we should put it on readthedocs
<dpb1> and yes, powersj should fix his branch
<blackboxsw> it's likely because cloud-init is so early in the boot process we are hit by this more than anywhere else
<powersj> :) I will
<blackboxsw> also, I think we probably shouldn't have examples that use this
<dpb1> yup
<blackboxsw> like https://cloudinit.readthedocs.io/en/latest/topics/modules.html#runcmd
<dpb1> blackboxsw: new bug would be fair
<blackboxsw>  wget, "http://example.org", -O, /tmp/index.html
<blackboxsw> yeah will add it and can clean up docs
<blackboxsw> add it and own it
<blackboxsw> https://bugs.launchpad.net/cloud-init/+bug/1727876
<ubot5> Ubuntu bug 1727876 in cloud-init "Document in RTD that cloud-init can't write to /tmp due races with systemd LP:1707222" [Medium,Triaged]
<powersj> blackboxsw: thx for review I'll make changes and let you know when they are up
#cloud-init 2017-10-27
 * smoser summons the all knowing python hero
<smoser> pip install --pre bzr-fastimport==0.11.0.final.0
<smoser> Collecting bzr-fastimport==0.11.0.final.0
<smoser>   Could not find a version that satisfies the requirement bzr-fastimport==0.11.0.final.0 (from versions: )
<smoser> No matching distribution found for bzr-fastimport==0.11.0.final.0
<smoser> yet ...
<smoser> https://pypi.python.org/pypi/bzr-fastimport
<smoser> so what vifes ?
<smoser> gives even
<smoser> seems taht there is only a pip record, no data
<nacc> smoser: yeah, usually the pypi page has more info
<smoser> the url there is bad, 404
<nacc> smoser: ah that could cause it to fail to fetch, maybe
<smoser> blackboxsw: http://paste.ubuntu.com/25831034/
 * powersj is running a full nocloud-kvm run on his branch after changing /tmp to /var/tmp and then will push
<powersj> blackboxsw: https://paste.ubuntu.com/25831640/
<powersj> blackboxsw: am I missing a commit? I rebased on master and ran into those?
<blackboxsw> powersj: yeah that should be fixed to be a debug msg
<powersj> blackboxsw: should be fixed as in, already in master?
<blackboxsw> yeah
<blackboxsw> checking rev
<blackboxsw> 41152f10ddbd8681cdac44b408038a4f23ab02df powersj
<blackboxsw> oct 17th
<blackboxsw> stale pyc files in your tree?
<powersj> smoser: "With regard to "not logged in as root by default", that is kind of wrong."
<powersj> only lxd and digital-ocean use a root user as default, aws, gce, azure, kvm cloud-images else assumes some sort of generic user, whether that is ubuntu or otherwise
<powersj> help me understand what you are looking for with that comment
<powersj> blackboxsw: ... realized what I was doing wrong... not using in tree cloud-init, so cloud image has old version of cloud-init
<powersj> sigh my bad
<Sargun> Do I have to do anything to get cloud-init to nudge netplan to reload
<smoser> Sargun: what have youd one that made you think netplan needed nudging ?
<blackboxsw> I thought "it just worked"â¢    cloud-init calls "netplan generate" to put appropriate configs in the right place at the right time.
<rharper> there is at least one bug where netplan bug related
<rharper> https://bugs.launchpad.net/ubuntu/+source/nplan/+bug/1669564
<ubot5> Ubuntu bug 1669564 in nplan (Ubuntu Artful) "udevadm trigger subsystem-match=net doesn't always run rules because of reconfiguration rate-limiting" [Undecided,Triaged]
<Sargun> smoser: Using the following cloudinit: https://gist.github.com/sargun/7152cac024896cb73152fc35bef2fb17
<Sargun> using ec2 user metadata
<Sargun> On Ubuntu 17.04
<smoser> Sargun: yu're hoping to get ipv6 dhcp ?
<smoser> you'd have to put that network config inside the image
<smoser> you can't feed it in from user-data
<smoser> that it reads over the network
<smoser> that said, 17.10 should get you wahat you want there.
<Sargun> Well, I was hoping it'd read it, and realize that it needs to restart.
<Sargun> 17.10 AMI?
<Sargun> Does 17.10 automatically do dhcp6?
<smoser> Sargun: at this point that doesnt work.
<smoser> but on 17.10 it should correclty notice that you have ipv6 enabled
<smoser> and will then do the right thing
<smoser> and that is actually in process of sru to 17.04 also
<Sargun> Tried 17.10
<Sargun> It only configured V6 and didn't do v4
<Sargun> I need dual-stack
<smoser> Sargun: hm.
<smoser> it should definitely do v4
<smoser> can you give more details on taht ?
<smoser> in 17.10
<smoser> blackboxsw: ^
<Sargun> smoser: Just used the some cloud-init.
<Sargun> https://gist.github.com/sargun/3bda5690c7bf573a47e5ab8e16d14425
<rharper> urg
<rharper> needs a dhcp4: true in there too
<blackboxsw> so Sargun you are trying to get dual stack up ipv4/6 on an ec2 instance. The provided network-config makes sense as it should be passed through in cloud-init direct to netplan config
<blackboxsw> the first paste was it right
<Sargun> rharper: That's what I'm putting into user-data.
<Sargun> my expectation is that the cloud-init file in /etc will be written out with dhcp4 and dhcp6.
<rharper> we can't pass network config as user-data; it's too late as smoser said
<rharper> Sargun: it should; but need to check the cloud-init code
<smoser> you cna't do it in user-ata.
<Sargun> Hrm.
<rharper> it's going to read EC2 metadata to see what network config looks like
<Sargun> Is there any other mechanism?
<smoser> but it should do the right thing.
<rharper> it should see that you enabled dhcp6 , but it should also enable dhcp4
<Sargun> Yeah
<blackboxsw> dpkg -l cloud-init on that image to see what rev you have on that instance
<Sargun> ii  cloud-init                                            17.1-18-gd4f70470-0ubuntu1      all                             Init scripts for cloud instances
<blackboxsw> thx
<rharper> if nic_metadata.get('public-ipv4s'):
<rharper>             nic_cfg['subnets'].append({'type': 'dhcp4'})
<blackboxsw> right it shoulce to that for both ipv4 and ipv6
<Sargun> I don't have any public-ipv4s
<Sargun> it should check for both private & public ipv4.
<Sargun> if nic_metadata.get('public-ipv4s') or nic_metadata.get('private-ipv4s'):...
<rharper> network/interfaces/macs/mac/local-ipv4s
<rharper> blackboxsw: ^
<rharper> that's the metadata path to private v4
<blackboxsw> could be, was just checking http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html
<Sargun> yeah
<Sargun> you have both public and private IPv4s on the metadata service
<rharper> but you explicitly don't have a public-v4 enabled ?
<Sargun> yeah
<rharper> gotcha
<blackboxsw> right agreed, I think we probably need to add the internal/local ipv4 check too.
<rharper> it's normally enabled by default IIUC
<Sargun> that'sthe default for a private subnet on VPC wizzard.
<rharper> interesting
<Sargun> EC2 is weird.
<blackboxsw> I thought all aws instances had to have public ipv4 addrs.
<blackboxsw> wow
<Sargun> blackboxsw: not VPC
<rharper> looks like ubuntu-bug time
<blackboxsw> yeah sargun for giggles can you ubuntu-bug cloud-init   and follow the prompts to submit a bug on the terminal CLI?
<blackboxsw> Sargun: rather ^
<blackboxsw> I'll add the check there on local-ipv4 too today and put something up for review/landing so Ec2 can be 'fixed' for dual stack internal ipv4 as well
<blackboxsw> so I meant to say: "Can your run 'ubuntu-bug cloud-init'   and follow the prompts from your instance"
<Sargun> Trying
<blackboxsw> thx
<Sargun> I can't from that instance, because it turns out to need IPv4.
<Sargun> oh
<Sargun> wait.
<Sargun> Can I just file it from launchpad.net?
<smoser> Sargun: you can.
<blackboxsw> for sure
<smoser> please do run 'cloud-init collect'
<smoser> and attach that.
<smoser> but yeah, you're already provided a good amount of information.
<smoser> thanks.
<blackboxsw> https://bugs.launchpad.net/cloud-init/+filebug :)
<Sargun> /usr/bin/cloud-init: error: argument subcommand: invalid choice: 'collect' (choose from 'init', 'modules', 'single', 'dhclient-hook', 'features', 'analyze', 'devel', 'collect-logs')
<Sargun> Are you sure?
<blackboxsw> collect-logs
<blackboxsw> instead of collect
<blackboxsw> it should dump a tarfile in your local directory
<blackboxsw> that can be attached to the bug
<Sargun> https://bugs.launchpad.net/cloud-init/+bug/1728152
<ubot5> Ubuntu bug 1728152 in cloud-init "IPv4 and IPv6 Dual Stack Does Not work when instance is not assigned public IPv4 address" [Undecided,New]
<blackboxsw> superb thanks Sargun
<Sargun> you can't actually single-stack (IPv6 only) on ec2
<Sargun> you always get an IPv4 IP
<blackboxsw> yeah I was under mistaken impression that the ipv4 IP also had to be public
<blackboxsw> because of the defaults in vpc
<blackboxsw> will reproduce a faliure on my vpc account and confirm I can see it too and we'll have something. (it's a quick fix to cloud-init)
<Sargun> yeah
<Sargun> I wonder if I can give this instance a public-ipv4 IP.
<blackboxsw> if you can re-associate ipv4 public, you could sudo rm -rf /var/lib/cloud /var/log/cloud-init.*; sudo reboot and cloud-init could come back up w/ the newly rendered network config
<blackboxsw> but minimally Sargun I'm expecting this should work for you...
<blackboxsw> http://paste.ubuntu.com/25832582/
<blackboxsw> save it on the instance to /tmp/patch1  then cd /usr/lib/python3/dist-packages/cloudinit/sources/; sudo patch -p3 < /tmp/patch1; sudo rm -rf /var/lib/cloud  /var/log/cloud-init*; sudo reboot
<blackboxsw> I believe that should render dhcp4 properly
<blackboxsw> as it'll see your local-ipv4s in metadata
<Sargun> Any idea why my apt source isn't working
<blackboxsw> Sargun: you need apt:\n  sources:\n <yoursourcename>:
<blackboxsw> hrm....
<Sargun> ah
<blackboxsw> Sargun: yeah it's admitedly a really long example described at https://cloudinit.readthedocs.io/en/latest/topics/modules.html#apt-configure
<blackboxsw> it has quite a few options for cloud-config
<smoser> rharper: suck.
<smoser> so if i take a stock artful image
<smoser> then
<smoser> qemu-img create -f qcow2 -b artful-cloud-image.img my.img && sudo mount-image-callback my.img -- mchroot touch /etc/cloud/cloud-init.disabled
<smoser> it hangs on boot.
<smoser> not sure what is hanging.
<smoser> ends with a very unhelipful systemd
<smoser> [  OK  ] Started Snappy daemon.
<smoser> [  OK  ] Started LXD - container startup/shutdown.
<smoser> [  *** ] A start job is running for Wait forâ¦e Configured (1min 19s / no limit)
<blackboxsw> smoser: is that what cyphermox hit?
<Sargun> Maybe I'll patch in JSON support into cloud-init.
<Sargun> and write json-schema for this
<rharper> you can turn on systemd debugging via cmdline
<smoser> blackboxsw: i dont think so. but in debugging his and trying different things i found it.
<rharper> systemd.log_level=debug systemd.log_target=console
<rharper> passed as params
<rharper> smoser: I wonder if there's no default network config in the iamge ?
<rharper> so if you don't run cloud-init you don't get a network config ? but you booted with an interface ?
<rharper> can do drop your -net user ?
<rharper> no nic, no wait ?
<smoser> rharper: http://paste.ubuntu.com/25832717/
<smoser> it doesnt help much
<smoser> i had used debug on systemd before.
<smoser> theres all the noise
<smoser> [***] A start job is running blahblahblah ...<removed important part in ...> more blahblahblah
<smoser> ohw ell.
<smoser> suck
<smoser> later.
<smoser> i have to run.
<rharper> I think the .mounts for cloud migth cause issue
<blackboxsw> ok smoser rharper Sargun just put up a trivial branch on this https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/332954
<blackboxsw> I'm going to try testing it now
 * blackboxsw needs to find that default checkbox for internal only ipv4
<blackboxsw> well smoser now I get to write an SRU-verification test for this bug too using the script that I'm halfway done w/ launch-ec2 :)
<blackboxsw> anyway you guys should be leaving, it's EOW time
<smoser> rharper: http://paste.ubuntu.com/25832788/
<smoser> "no limit" finally gave up
<smoser> systemd-networkd-wait-online.service
<smoser> fun
<rharper> lol
<rharper> I knew it
<rharper> it's configured to wait until at least *one* interface is up
<rharper> we gave it none
<rharper> or we'll have one, but no config
<smoser> ireally have to be gone
<smoser> well, cloud-init iddnt run
<rharper> there may need to be a "baked" in netplan yaml for dhcp on en*
<smoser> so it didnt give it anyting
<rharper> right
<smoser> and there wasnt any nick there.
<rharper> but the old networking service asked if there were network devices to be configured
<smoser> either way, it should have decided "nothing to do"
<rharper> networkd-wait-online says, wait until I have networking
<rharper> which will never happen
<smoser> could you file a bug against cloud-images ?
<rharper> I think there is one, but if not I will
<smoser> yeah. which is wrong.
<rharper> I know
<rharper> I argued with slangasek about it before
<rharper> smoser: https://bugs.launchpad.net/cloud-images/+bug/1728164
<ubot5> Ubuntu bug 1728164 in cloud-images "artful images hang when cloud-init is disabled due to no network config" [Undecided,New]
<blackboxsw> here's where I got that ipv4 required "You cannot disable IPv4 support for your VPC and subnets; this is the default IP addressing system for Amazon VPC and Amazon EC2."   per http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-migrate-ipv6.html
<blackboxsw> I interpreted that a public-ipv4 required... bummer
<blackboxsw> meh our chef example is also bogus as far as apt configuration is concerned https://cloudinit.readthedocs.io/en/latest/topics/examples.html#install-and-run-chef-recipes
<blackboxsw> I'll fix that when  I touch bug: #1727876
<ubot5> bug 1727876 in cloud-init "Document in RTD that cloud-init users shouldn't write to /tmp due races with systemd LP:1707222" [Medium,Triaged] https://launchpad.net/bugs/1727876
#cloud-init 2017-10-28
<cyphermox> smoser: blackboxsw: rharper: I would have guessed that your network device is not virtio, but that shouldn't be a big deal /now/ since E1000 is also enabled in this case, and that looks like it's been detected.
<smoser> cyphermox: well i had actually attached *no* network devices. which clearly should mean "all networking that will be configured is configured".
<cyphermox> smoser: I saw your command-line, but I don't think it behaved quite as expected
<cyphermox> or I don't understand what the kernel does, because it tries to rename an ens3 or eth0 that isn't there.
<smoser> what command line?
<smoser> there is no network device.
<smoser> i blieve there is no built-in network configuration in images any more.
<smoser> yeah, the cloud-images have nothing in /etc/netplan/
<cyphermox> err, that's not "no network device"
<cyphermox> and in your paste: http://paste.ubuntu.com/25832717/
<cyphermox> [    1.848991] e1000 0000:00:03.0 eth0: (PCI:33MHz:32-bit) 52:54:00:12:34:56
<cyphermox> [    1.849784] e1000 0000:00:03.0 eth0: Intel(R) PRO/1000 Network Connection
<cyphermox> [    1.851505] e1000 0000:00:03.0 ens3: renamed from eth0
<cyphermox> despite your line 1 being "xkvm --netdev= ..."
<cyphermox> unless you mean something else
<smoser> yea, so the --netdev= did the right thing and did not add a device itself, but qemu must add one if you dont have a network device
<smoser> and don't add '-nodefaults'
<cyphermox> on cloud images it is fair to expect that cloud-init will write your netplan config, I think
<smoser> either way, there was no configuration in /ec/ so no blocking and waiting for magical configuration to appear should have been done.
<cyphermox> sure
<smoser> cyphermox: yeah, your thought process is sane, except it explicitly did *not* write config.
<smoser> and when it does, it writes it pre-networking
<cyphermox> that's right, you disabled cloud-init
<smoser> so no blockign and waiting makes any sense.
<cyphermox> blocking and waiting is correct there; that is what you *should* do if there is config, to ensure that the device has had time to come up before you try to send packets over it
<smoser> well yeah.
<smoser> but there is no config.
<smoser> so blockin and waiting for config to appear is kind of pointless.
<smoser> or for network devices to appear.
<cyphermox> I agree there is a bug in wait-online though, which should not have waiting in your case with cloud-init disabled, because all the devices are unmanaged (since there is no config)
<cyphermox> that's a bug in wait-online
<cyphermox> https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1728181
<ubot5> Ubuntu bug 1728181 in systemd (Ubuntu) "systemd-networkd-wait-online waits when devices are unmanaged" [Undecided,New]
<smoser> right.
<cyphermox> but I don't think booting a cloud image with no metadata provider is something supported
<cyphermox> smoser: so, what were you trying to achieve?
<smoser> cyphermox: well its actually something we've spent way to much time on
<smoser> but that is explicitly required.
<smoser> cloud-init should disable itself entirely when not in a cloud
<smoser> and it does.
<cyphermox> right
<cyphermox> and that's working as expected
<smoser> but then if you have a network device you wait for 120 seconds for no reason.
<smoser> http://paste.ubuntu.com/25833672/
<smoser> thanks for pointing out that it got a nic
<smoser> the above paste there is the same thing but with no nics at all
<smoser> at least then, systmed-networkd-wait-online does the right thing.
<cyphermox> ok
<cyphermox> well, the real question is: "Is a cloud image booted with no datasource something we should support", and I was told "no", if I understood correctly
<cyphermox> so yes, there's a bug in wait-online, and it's one we need to fix anyway, but it seems like you got a different answer than I did to the question above.
<smoser> well, i've had actual users ask to build an image that would boot in the cloud, but when booted with no cloud would let them log in.
<smoser> they'd put a local user in it with a known password
<smoser> then they'd have their image on cloud and local to debug and  can login
<smoser> (or even no password but a shared ssh key)
<smoser> which makes sense.
<cyphermox> yeah, but then they'd have cloud-init running to create the user, no?
<cyphermox> the ssh key case we need to ignore since it requires network, in which case you do need to wait-online.
<cyphermox> the other one we'll fix once wait-online is fixed (bug above), and I expect we'd SRU that fix
<smoser> well, this would be someone created the imag themselfvs.
<smoser> i just did this locally
<cyphermox> well, it doesn't really matter at this level, that's "just" the wait-online bug
<cyphermox> maybe metoo it?
<smoser> $ qemu-img create -f qcow2 -b artful-cloud-image.img my.img
<smoser> $ sudo ~/bin/backdoor-image my.img -v --user=user1 --password=passw0rd1
<smoser> # backdoor image is http://bazaar.launchpad.net/~smoser/+junk/backdoor-image/view/head:/backdoor-image
<cyphermox> smoser: don't worry about the process, I frankensteined my own image earlier to put in a root password
#cloud-init 2017-10-29
<ckonstanski> I just opened https://bugs.launchpad.net/cloud-init/+bug/1728430
<ubot5> Ubuntu bug 1728430 in cloud-init "make NTP tests work in gentoo" [Undecided,New]
<ckonstanski> Don't know how to assign it to myself. I'm working on it.
