/srv/irclogs.ubuntu.com/2014/09/02/#cloud-init.txt

=== devicenull__ is now known as devicenull
nvucinichi guys, is there any way to find out what can i call in post in phone-home  ?12:15
harmwwhat do you mean? just put in an url and it'll GET (or HEAD) that, perhaps even POST :)12:24
harmwyea, I think it supports POST as well - you should just look in the src though, to make sure12:25
nvucinicyeah, it supports post, i got the data12:26
nvucinicmy question was this12:26
nvucinicexample: http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/891/doc/examples/cloud-config-phone-home.txt12:26
nvucinicwhat can i get except pub_keys and instance id ?12:26
harmwah, you mean which items are available to post12:27
harmwor: keys12:27
harmwwell, you could just look at L9 :P12:28
harmwpost: all12:28
harmwjust to get an idea of all available stuff, have it post everything it knows12:29
nvuciniclet's try all :)12:39
harmw:)12:39
nvucinicnot much, id, hostname and keys12:45
harmwwell, if you need more you could fiddle in some bash script in user-data12:47
harmwa little adventurous, but fun :)12:47
=== smoser` is now known as smoser
smosernvucinic, yeah, i think there isn't much else. i'm open to adding things.13:09
nvucinicsmoser: yeah, i looked at script now :) 13:10
harmwsmoser: PING13:11
smoserharmw, whats up?13:13
harmwuhm, let me think for just a minute...13:13
harmwhm hm, what was it know...13:13
harmw:p13:13
harmwtake a look at the fixes from last week and merge, pretty please :)13:14
smoserdid you get the tests to pass on linux ?13:15
harmwharlowja fixed them13:15
smosersweet13:15
smoseri will merge then. thanks.13:15
harmwawesome13:16
harmwand version bump to a shiny new .tar.gz , so I can submit my fbsd port as well13:16
smoseryou really need that, eh?13:20
harmwif possible, yes13:20
nvucinicsmoser: is there any way where i can check  what can i get from cloud.get ?13:20
harmweven 0.7.5.1 is good enough, ofcourse13:20
nvuciniclike hostname, instance id ... 13:20
smoserharmw, ok. we'll get something this week.13:22
smosernvucinic, let me look.13:22
harmwnice!13:22
nvucinicsmoser: thx :)13:23
smosernvucinic, http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/971/cloudinit/config/cc_phone_home.py13:24
smoserunfortunately, it looks like there isnt a lot. 13:24
smoserlooks like the url can have 'INSTANCE_ID' replaced in it.13:25
smoserand the things youi can post are listed at POST_LIST_ALL13:25
nvucinicsmoser: yeah i replaced instance_id 13:26
nvucinicsmoser: i see that hosname is defined here all_keys['hostname'] = cloud.get_hostname()13:26
nvucinicso cloud.get_hostname is how you get hostname 13:26
nvuciniccloud.get_instance_id is how you get instance id13:27
nvucinicis there cloud.get_xxx with list of things that i can get  ?13:27
harmwprobably not that hard to add some stuff to the cloud object13:27
nvucinicharmw: yeah, but what else can you get with cloud.get 13:28
harmwits not cloud.get_BLA13:29
harmwits get_hostname()13:29
harmwor get_bla()13:29
harmwfunctions inside the cloud object (?)13:29
harmwso look inside for whatever functions are there to call :)13:29
nvucinicharmw: great, where can i get list of functions that i can use?: )13:29
harmwhehe, browsing the sourcode would be a great place to start13:30
harmwI don't think this is listed on readthedocs13:30
nvucinicalso if you look at http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/971/cloudinit/config/cc_phone_home.py13:30
nvucinicit's defined as cloud.get_hostname13:30
harmwdef handle(name, cfg, cloud, log, args):13:31
harmwthere's cloud13:31
harmwso you kinda just want to lookup what/where cc_phone_home is called from13:32
harmwwell handle(), that is13:32
smoserharmw, nvucinic the cloud object is terribly not well defined. 13:33
smosercloudinit/cloud.py13:34
nvucinicyes, i can see that :)13:34
smosernvucinic, so look in ther eand you can see.13:34
smoserand you can get cloud.datasource also13:34
nvucinicsmoser: great, ty13:34
smoserwhich likely has more (not terribly well defined) information.13:34
smoserwe want to get those settled down to a reasonable strucutre that we can assume is available on all clouds13:35
nvucinicsmoser: i would like to have ip adress as argument in phone-home script 13:51
smosernvucinic, you can probably get at it.13:53
smoseris this ec2, nvucinic  ?13:53
nvucinicsmoser: no it's that god awful setup with NoCloud 13:54
smosernvucinic, hten its not there. sorry.13:55
nvucinicsmoser: so custom script would be the right way? 13:55
smoseryeah. 13:56
smoseryou can do a part-handler, or a '#!' 13:56
smoserjust run some code that reads 'ip' output (or does it in python)13:57
nvucinicyeah13:57
=== gondoi is now known as zz_gondoi
=== daxroc__ is now known as daxroc
smoserharmw, would you mind opening a bug for "here is one unresolved issue regarding path resolving of modules (?), which is why there is one DataSource explicitly enabled in the configuration file (and working/being used)."16:21
smoserharmw, merged. please test it.16:27
harmwsmoser: sure thing17:24
harmwand thanks :)17:24
=== praneshp_ is now known as praneshp
=== praneshp_ is now known as praneshp
=== zz_gondoi is now known as gondoi
JayFharlowja: thanks for the comment, I'm going to pick up work on that again today or tomorrow :)19:07
harlowjakk19:08
harlowjacool19:08
harlowjabtw, guys something scott and i just threw together, https://etherpad.openstack.org/p/cloud-init-next19:12
harlowjacomments welcome19:12
harlowjaadditions welcome...19:12
harlowjaharlowja JayF  ^19:12
harlowjaharmw i mean19:12
harlowjaha19:12
harmwharlowja: looks fine to me, though I don't have any higher level additions to share19:16
harlowjak19:16
harmwwell, Windows would be nice ofcourse19:16
harlowjawindows19:17
harlowjaargggg19:17
harlowjalol19:17
harmwlike merging some stuff from cloudbase19:17
harlowjaya, that one, tried that before :-/19:17
harlowjai'm not so looking forward to that19:17
harmwhehe19:17
harmwsmoser: just filed that bug19:26
smoserthanks harmw 19:26
harmwok, lets see if the trunk still works :)19:28
harmwsmoser/harlowja, anyone of you using Designate btw?19:28
harlowjanot me19:29
harmwoh ok19:29
smosernot me19:34
smoserharmw, bug number ?19:34
harmwhttps://bugs.launchpad.net/cloud-init/+bug/1364580 19:34
harlowjahmmm, odd19:39
harlowjacan we get a log from that happening harmw ?19:41
JayFharlowja: is 'work with coreos on their go clone' useful input ;)19:41
harlowjathat sounds right up there with the windows clone19:42
harlowjalol19:42
JayFharlowja: I'm 80% kidding, but I did put a note in there about making it easy to vendor your own versions of cloud-init (which go would do)19:42
harlowjawhat does mean 'vendor your own versions'?19:42
JayFso right now19:42
JayFFor Rackspace OnMetal19:42
JayFevery single one of our images19:42
JayFwe package and deploy a custom cloud-init19:42
harlowjayup, same here19:42
harlowja*although not that custom, like 1 patch19:43
harlowjaor 219:43
JayFThis is kind of a pain in the ass because you have to 'lock' the rpm/yum version in place19:43
JayFwhich may break ON dist-upgrade or may just BREAK dist-upgrade19:43
harlowjaok ,don't do that lock stuff then ;)19:43
harlowjause the vendor provided one ;)19:43
* harlowja which is what we at y! are trying to get to (using the fedora one)19:43
JayFVendor provided one doesn't support a huge stack of what we're doing19:43
harlowjaisn't that just getting the code upstream that does that ? :-P19:44
JayF(which is why I'm here helping upstream our support, so we get out of this... but you and  I both know if it goes upstream now I'm still packaging my own for 5+ yrs)19:44
harlowjahopefully more like 2 years19:44
harlowjahaha19:44
harlowja;)19:44
JayFIt's not :(19:44
* JayF is also working on kernel version support for CentOS 6.5, which is Old Software(tm)19:44
harlowjaoverall yes though i know, avoid the fork hell at all possible19:44
JayFI'd say model that into upstream19:45
harlowja6.5 isn't that old, thats brand new19:45
harlowjawe have 5.8 images lol19:45
JayFcloud-init should assume that a lot of vendors will be packaging cloud-init and make it easier19:45
harlowjasure, so then what part not easy?19:45
JayFharlowja: Yeah, it doesn't seem bad, but even a few years with an older kernel makes huge perf differences.19:45
JayFPython generally doesn't make it easy :)19:45
JayFright now, like I said, we're just building a new RPM for a distro and 'lock'ing it there19:46
harlowjaso more of what u are getting at is the go statically linking model?19:46
JayFbut some supported 'omnibus' style install (like opscode uses for chef w/the ruby "omnibus" stuff)19:46
JayFI'm getting at I want to be able to deploy and run a 'tarball' or single file or something very isolated of cloud-init19:46
harlowjaright, the same as statically linking19:47
JayFwell except statically linking is an implementation of my requirement :)19:47
harlowjasure sure19:47
JayFI'm specifically not saying statically link it because I know that's mostly a nonstarter19:47
harlowjaso then if we remove more of dependencies in http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/requirements.txt 19:48
harlowjawhat about that?19:48
JayFthink about it more this way:19:48
JayFas a vendor providing a cloud server19:48
JayF*service19:48
JayFwe need to be able to iterate quickly on cloud-init just like we can on our cloud19:48
JayFpackaging is somewhat of a difficulty there currently, among a bucket of other things :)19:49
harlowjasure19:49
harlowjabelieve me i know, lol19:49
harlowjabut it still feels like a hammer looking for a nail19:50
JayFthe other thing is maybe a philisophical shift: we want a mechanism to update a *booted* system19:50
JayFi.e. I attach some kind of 'cloud storage' volume, update the metadata service to reflect that, and my cloud init data source sees that, and mounts the volume19:51
harlowjaya, smoser  has thoughts about that as well afaik19:51
JayFour use case gets more interesting because we want cloud-init (and other software) to do a lot of the tasks relegated today to a hypervisor19:51
JayFbecause hypervisors are for people who want to do things easy :P (just kidding, although it feels that way sometimes :D)19:52
harlowjawhat about helping build out something like https://github.com/cloudcadre/giftwrap19:52
harlowjahttps://github.com/cloudcadre/giftwrap#giftwrap19:53
smoserhm.. on a call, will look above later19:53
harlowjaJayF afaik the above does something like what u want19:53
harlowjahttps://github.com/cloudcadre/giftwrap#how-it-works19:53
harlowjaso thats why i wonder if its more of a problem thats outside cloud-inits scope19:55
JayFharlowja: IMO the problem is more on the other side19:55
harlowjahow so?19:55
JayFharlowja: I build the package, install it in my default image, customer upgrades, gets a 'newer' version of cloud-init that doesn't have my patches, and cloud-init re-runs and does $unexpected_things because it doesn't support our data source19:55
JayFor if you do a package manager 'hold' in the distro, you restrict the customer from upgrading entire classes of packages19:55
JayFI don't know how to solve this problem, tbh, but I think something along the lines of chef-omnibus packages have to be the way19:56
harlowjahmmm, ok, sounds like u need smarter customers ;)19:56
harlowjaproblem solved!19:56
harlowjaha19:56
JayFbecause then I never install distro cloud-init in my images, I install cloud-init-omnibus which puts everything in /opt/cloud-init/ and depends on nothing outside that dir (except maybe libc) to run19:56
JayFharlowja: Our entire spiel at Rackspace is Managed [cloud|server|etc]... my job is to help write software to enable that.19:57
JayFharlowja: which means I'm the smarter customer :)19:57
harlowja:)19:57
harlowjai just start to wonder if there is something wrong though, like amazon afaik doesn't need to do this, yahoo has so far gotten along just fine without this kind of isolation thing, so is it really a problem that we have to resolve by isolating all of cloud-init (and its own version of python and dependencies) to say /opt/cloud-init19:59
JayFI'd venture to say I'm feeling this pain right now19:59
JayFbecause we're doing /new things/19:59
JayFnobody before us, afaik, has run a public, software-operated, bare metal API-driven provisioning system20:00
harlowjasure sure20:00
JayFWe're using network configs that are different (vlans on top of bonding)20:00
JayFon top of all that :)20:00
harlowjaontop of nicira? 20:01
harlowja;)20:01
JayFfwiw our first major cloud-init change was to stop excluding disk partitions from consideration when looking for configdrives20:01
JayFharlowja: today, no actually :( $vendor_reasons20:01
harlowjaya, i know about some of that i think ;)20:01
JayFWe20:01
JayFWe've blogged about some of it in various places20:02
harlowjaya, i just start to wonder if this is really a good way to go down20:02
harlowjaall of yahoo typically gets installed in /home/y/ and that really has not turned out so well over the years20:03
JayFhah :)20:03
harlowjato much packages that forget to get managed that someone forked to adjust to fit into /home/y...20:03
JayFhave you seen/used omnibus installed chef?20:03
harlowjanot personally, some other people are20:03
JayFhttps://github.com/opscode/omnibus20:03
JayFas a pattern I don't think this is bad20:03
JayFkinda an uglier fork of the container pattern, really20:04
JayFjust ship a package, with all it's deps shoved into a dir away from the rest of the os20:04
harlowjaya, i'm not sure what yahoo is replacing it with (something chef afaik) 20:04
JayFit's ugly I'm sure when you get to the level of installing entire sub-operating systems in ~y/20:04
harlowjayup20:04
JayFbut your configuration system (be it chef, cloud-init, whatever) being able to install and run regardless of what OS deps are installed20:05
JayFcould be a valuable thing20:05
JayFagain sorta goes to the world view of a tool like cloud-init being 'out of band' similar to a hypervisor... even though obviously it's not :)20:05
harlowjaya, i'd be +2 to cloud-init in the kernel20:05
harlowjalike kvm 20:06
harlowjalol20:06
harlowjai'm sure linus would be fine with that20:06
JayFHAH20:06
JayFbut you get the general worldview I'm looking with?20:06
harlowjaya20:06
harlowjai do20:06
harlowjaalthough i still have mixed feelings about it, knowing past experiences :-P20:06
harmwharlowja: you broke my fbsd code!20:06
JayFthe ability to configure the image to utilize $cool_things in my cloud means I need to be able to update cloud-init rapidly as I build $cool_things20:06
harlowjaharmw i might have, ha, who knows20:06
harmwmy branch was working, but trunk now doesn't20:07
harlowjaJayF ya, sure, thats just a slipper slope of becoming your own distro ;)20:07
harlowjaharmw i blame smoser 20:07
harlowjalol20:07
harmw:P20:07
JayFand while I don't expect that long-term to be a 'fork' as it is right now... I do expect to continue to want to run pretty close to 'master' cloud-init ... especially when talking about versions shipped in old (>1y old) distros20:07
JayFharlowja: heh, we'd not be A distro... we'd be a fork of /every/ distro which is why I'm talking to you guys upstream trying to solve the problem in a better way for everyone than just me doing something on my own20:08
harlowjaisn't that called the 'JayF distro' (with description being a ' a fork of /every/ distro')20:08
harlowjalol20:08
JayFhah20:08
JayFIt's called coroes20:09
harlowjabut better!20:09
JayF*coreos20:09
harlowjalol20:09
harlowjajayfos20:09
JayFand all my 'forks' are containers20:09
JayFdude if I had my own distro20:09
JayFit'd be called Old OS20:09
* JayF runs oldos.org20:09
harlowjalol20:09
harlowjanice20:09
harmwbtw harlowja I put an example in that bugrport20:10
harlowjaharmw that log looks fine :-/20:10
harlowja'Using distro class <class 'cloudinit.distros.freebsd.Distro'' ?20:10
harlowjalooks like its trying the openstack one20:11
harmw2014-09-02 19:47:06,862 - importer.py[DEBUG]: Failed at attempted import of 'freebsd' due to: No module named freebsd20:11
harmw2014-09-02 19:47:06,892 - stages.py[DEBUG]: Using distro class <class 'cloudinit.distros.freebsd.Distro'>20:11
harlowjaya, look at the line after that20:11
harlowjaits basically logging teh attempts20:11
harlowjaya, perhaps i should shutup that log20:12
harmwi posted another one20:12
harmwnah wait, let me post one with specifying the datasource in cloud.cfg20:12
harmwIll do that later on20:13
harlowjakk20:13
harlowjai'm gonna shutup that log that says failed imports20:13
harlowjato confusing20:13
harlowjau aren't the first to say wtf it failed importing20:13
harlowjawhen it really didnt, ha20:13
harmwyea, Ill try and find out why the F fbsd is failing on me right now with trunk code20:13
harlowjakk20:14
harlowjaJayF anyways, lets see what smoser thinks, he's from more of the world of distro-land than i am20:22
harlowjayahoo i know is trying to move forward with stuff like docker and all, so i can see how it might all work out 20:23
JayF I am trying to move forward with stuff like docker and all :) 20:23
JayFbut I have to support whatever reasonable patterns my customer wants20:23
JayFs/customer wants/customers want/20:23
harlowjaya, although sometimes customer not doing the right thinkg ;)20:25
harlowja*thing20:25
JayFthat's where the 'reasonable' part comes in :)20:28
harlowja;)20:29
smoserharmw, i'm being asked for cirros 0.3.3 with mtu fix.20:30
harmwdon't we have that already?20:30
harmwthe fix, that is20:30
harmwor is that to a pending merge?20:31
smoseri forget. i thought maybe i had it with other changes. let me look if we can easily do a 0.3.3 20:32
smoserthat has it.20:32
harmwthere is some code from me (?) in LP to adress this20:32
harmwudhcpc wrapper stuff20:32
harmwsmoser: https://bugs.launchpad.net/cirros/+bug/130195820:35
smoserright20:39
smoser+## currently disabled as it depends on busybox and buildroot20:39
smoser+## version numbers (in the patch paths).20:39
smoser+## note, that without this the cirros-udhcpc is inert, but unoffensive20:39
smoserthat was why it was non-trivial20:39
harmwahyes, you had to upgrade buildroot first20:40
harmwhmgrr, setting the hostname is broken in my fbsd code... it doesn't set what it got from metadata, but instead re-uses `hostname`, or something20:45
harmwah, so datasource.get_hostname() is returning that20:51
harmwbingo: why is it entering this... if self.metadata or 'local-hostname' not in self.metadata:20:57
smoserlater all. i have to run.20:57
gholmso/20:58
harmwharlowja: http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/revision/1000.1.1/cloudinit/sources/__init__.py21:08
harmwI believe that has brutally broken the setting of hostnames21:08
harmwatleasy in my current testing of the fbsd module21:08
harmwsince I just confirmed to have a perfectly fine hostname (host.bla.tld) inside self.metadata['local-hostname']21:09
harmwso please revert that smoser or harlowja :)21:10
harlowjaya, seems like it could21:10
harmwJayF: you broke it :p21:10
harlowjaharmw lets see what JayF thinks, i wonder if that was just a mistake21:11
harmwI'm guessing thats a yes :)21:11
harlowjame to21:11
JayFwait what21:12
harmwoh well, it's always nice to gte the oppertunaty to dive in to the inner workings of c-i :P21:12
harlowja;)21:12
JayFoh man I have a handful of changes like that :(21:12
harmwlets hope your hands are small, shall we :P21:13
JayFOoooh, I see how this one changes the logic21:13
harlowjapretty sure its 'if not self.metadata or 'local-hostname' not in self.metadata: '21:13
harlowjaor supposed to be ^21:13
JayFpep8 screams about that21:13
harlowjaodd21:13
harmwthe old line was fine though, just reverted it locally and *everything* just works now :)21:13
JayFharmw: make pep8 will fail though, I presume :(21:14
* JayF might be too used to being able to trust unit tests21:14
harmwdon't know, I don't have alle the test tools in this instance to verify21:14
harlowjalet me try21:14
harlowjaharmw JayF https://code.launchpad.net/~harlowja/cloud-init/fixer-up/+merge/233126 21:18
harlowjaalso https://code.launchpad.net/~harlowja/cloud-init/better-errors/+merge/23312521:18
harlowjaseems like pep8 doesn't complain about 23312621:21
harlowjaand retains the same old logic21:21
JayFgimme one sec, fixing a network issue21:21
JayFshould we also write a check to catch this instance?21:21
JayFs/check/test/21:21
harlowjaprobably21:22
JayFif you say pep8 works, then +1 to the change21:22
harmwharlowja: 126 is spot on21:23
harlowjaya, (cloud-init)harlowja@getlookcrowd[0]:~/dev/os/cloud-init $ pep8 cloudinit/sources/__init__.py 21:24
harlowja(cloud-init)harlowja@getlookcrowd[0]:~/dev/os/cloud-init $ 21:24
harlowjano errors/complaints21:24
harmwgod, and this only took 2 precious hours of my tonight-time..21:24
JayFharmw: :( I'm sorry man21:24
harmwand you should!21:25
harmw:p21:25
harmwnah, I've seen worse21:25
harmwharlowja: you're merging this? or do you need permission from the supreme uber godlike master wizard?21:26
harlowjaharmw i can merge, although i'll just let smoser to,  where'd he go21:27
harlowjai usually just delegate to him ha21:27
harmw:>21:27
harlowja:)21:27
harmwwell, other than this the fbsd code works btw. Having it set the hostname and adding a user+ssh key wrks just fine21:28
harlowjacool21:29
JayFI can't pull this right now, merge it if you think it's right21:34
JayFsorry :( dealing with local network stuff 21:36
harlowjaya, i'll let smoser pull it in unless its so critical can't wait21:37
harlowjathen i'll try to remember the merge stuff, haha21:37
JayFharmw: we were just talking around here the otehr day about how neat it'd be to have fbsd support :)21:38
JayFharmw: (here=rackspace onmetal)21:38
=== shardy is now known as shardy_afk
=== gondoi is now known as zz_gondoi

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!