=== devicenull__ is now known as devicenull [12:15] hi guys, is there any way to find out what can i call in post in phone-home ? [12:24] what do you mean? just put in an url and it'll GET (or HEAD) that, perhaps even POST :) [12:25] yea, I think it supports POST as well - you should just look in the src though, to make sure [12:26] yeah, it supports post, i got the data [12:26] my question was this [12:26] example: http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/891/doc/examples/cloud-config-phone-home.txt [12:26] what can i get except pub_keys and instance id ? [12:27] ah, you mean which items are available to post [12:27] or: keys [12:28] well, you could just look at L9 :P [12:28] post: all [12:29] just to get an idea of all available stuff, have it post everything it knows [12:39] let's try all :) [12:39] :) [12:45] not much, id, hostname and keys [12:47] well, if you need more you could fiddle in some bash script in user-data [12:47] a little adventurous, but fun :) === smoser` is now known as smoser [13:09] nvucinic, yeah, i think there isn't much else. i'm open to adding things. [13:10] smoser: yeah, i looked at script now :) [13:11] smoser: PING [13:13] harmw, whats up? [13:13] uhm, let me think for just a minute... [13:13] hm hm, what was it know... [13:13] :p [13:14] take a look at the fixes from last week and merge, pretty please :) [13:15] did you get the tests to pass on linux ? [13:15] harlowja fixed them [13:15] sweet [13:15] i will merge then. thanks. [13:16] awesome [13:16] and version bump to a shiny new .tar.gz , so I can submit my fbsd port as well [13:20] you really need that, eh? [13:20] if possible, yes [13:20] smoser: is there any way where i can check what can i get from cloud.get ? [13:20] even 0.7.5.1 is good enough, ofcourse [13:20] like hostname, instance id ... [13:22] harmw, ok. we'll get something this week. [13:22] nvucinic, let me look. [13:22] nice! [13:23] smoser: thx :) [13:24] nvucinic, http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/971/cloudinit/config/cc_phone_home.py [13:24] unfortunately, it looks like there isnt a lot. [13:25] looks like the url can have 'INSTANCE_ID' replaced in it. [13:25] and the things youi can post are listed at POST_LIST_ALL [13:26] smoser: yeah i replaced instance_id [13:26] smoser: i see that hosname is defined here all_keys['hostname'] = cloud.get_hostname() [13:26] so cloud.get_hostname is how you get hostname [13:27] cloud.get_instance_id is how you get instance id [13:27] is there cloud.get_xxx with list of things that i can get ? [13:27] probably not that hard to add some stuff to the cloud object [13:28] harmw: yeah, but what else can you get with cloud.get [13:29] its not cloud.get_BLA [13:29] its get_hostname() [13:29] or get_bla() [13:29] functions inside the cloud object (?) [13:29] so look inside for whatever functions are there to call :) [13:29] harmw: great, where can i get list of functions that i can use?: ) [13:30] hehe, browsing the sourcode would be a great place to start [13:30] I don't think this is listed on readthedocs [13:30] also if you look at http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/971/cloudinit/config/cc_phone_home.py [13:30] it's defined as cloud.get_hostname [13:31] def handle(name, cfg, cloud, log, args): [13:31] there's cloud [13:32] so you kinda just want to lookup what/where cc_phone_home is called from [13:32] well handle(), that is [13:33] harmw, nvucinic the cloud object is terribly not well defined. [13:34] cloudinit/cloud.py [13:34] yes, i can see that :) [13:34] nvucinic, so look in ther eand you can see. [13:34] and you can get cloud.datasource also [13:34] smoser: great, ty [13:34] which likely has more (not terribly well defined) information. [13:35] we want to get those settled down to a reasonable strucutre that we can assume is available on all clouds [13:51] smoser: i would like to have ip adress as argument in phone-home script [13:53] nvucinic, you can probably get at it. [13:53] is this ec2, nvucinic ? [13:54] smoser: no it's that god awful setup with NoCloud [13:55] nvucinic, hten its not there. sorry. [13:55] smoser: so custom script would be the right way? [13:56] yeah. [13:56] you can do a part-handler, or a '#!' [13:57] just run some code that reads 'ip' output (or does it in python) [13:57] yeah === gondoi is now known as zz_gondoi === daxroc__ is now known as daxroc [16:21] harmw, would you mind opening a bug for "here is one unresolved issue regarding path resolving of modules (?), which is why there is one DataSource explicitly enabled in the configuration file (and working/being used)." [16:27] harmw, merged. please test it. [17:24] smoser: sure thing [17:24] and thanks :) === praneshp_ is now known as praneshp === praneshp_ is now known as praneshp === zz_gondoi is now known as gondoi [19:07] harlowja: thanks for the comment, I'm going to pick up work on that again today or tomorrow :) [19:08] kk [19:08] cool [19:12] btw, guys something scott and i just threw together, https://etherpad.openstack.org/p/cloud-init-next [19:12] comments welcome [19:12] additions welcome... [19:12] harlowja JayF ^ [19:12] harmw i mean [19:12] ha [19:16] harlowja: looks fine to me, though I don't have any higher level additions to share [19:16] k [19:16] well, Windows would be nice ofcourse [19:17] windows [19:17] argggg [19:17] lol [19:17] like merging some stuff from cloudbase [19:17] ya, that one, tried that before :-/ [19:17] i'm not so looking forward to that [19:17] hehe [19:26] smoser: just filed that bug [19:26] thanks harmw [19:28] ok, lets see if the trunk still works :) [19:28] smoser/harlowja, anyone of you using Designate btw? [19:29] not me [19:29] oh ok [19:34] not me [19:34] harmw, bug number ? [19:34] https://bugs.launchpad.net/cloud-init/+bug/1364580 [19:39] hmmm, odd [19:41] can we get a log from that happening harmw ? [19:41] harlowja: is 'work with coreos on their go clone' useful input ;) [19:42] that sounds right up there with the windows clone [19:42] lol [19:42] harlowja: I'm 80% kidding, but I did put a note in there about making it easy to vendor your own versions of cloud-init (which go would do) [19:42] what does mean 'vendor your own versions'? [19:42] so right now [19:42] For Rackspace OnMetal [19:42] every single one of our images [19:42] we package and deploy a custom cloud-init [19:42] yup, same here [19:43] *although not that custom, like 1 patch [19:43] or 2 [19:43] This is kind of a pain in the ass because you have to 'lock' the rpm/yum version in place [19:43] which may break ON dist-upgrade or may just BREAK dist-upgrade [19:43] ok ,don't do that lock stuff then ;) [19:43] use the vendor provided one ;) [19:43] * harlowja which is what we at y! are trying to get to (using the fedora one) [19:43] Vendor provided one doesn't support a huge stack of what we're doing [19:44] isn't that just getting the code upstream that does that ? :-P [19:44] (which is why I'm here helping upstream our support, so we get out of this... but you and I both know if it goes upstream now I'm still packaging my own for 5+ yrs) [19:44] hopefully more like 2 years [19:44] haha [19:44] ;) [19:44] It's not :( [19:44] * JayF is also working on kernel version support for CentOS 6.5, which is Old Software(tm) [19:44] overall yes though i know, avoid the fork hell at all possible [19:45] I'd say model that into upstream [19:45] 6.5 isn't that old, thats brand new [19:45] we have 5.8 images lol [19:45] cloud-init should assume that a lot of vendors will be packaging cloud-init and make it easier [19:45] sure, so then what part not easy? [19:45] harlowja: Yeah, it doesn't seem bad, but even a few years with an older kernel makes huge perf differences. [19:45] Python generally doesn't make it easy :) [19:46] right now, like I said, we're just building a new RPM for a distro and 'lock'ing it there [19:46] so more of what u are getting at is the go statically linking model? [19:46] but some supported 'omnibus' style install (like opscode uses for chef w/the ruby "omnibus" stuff) [19:46] I'm getting at I want to be able to deploy and run a 'tarball' or single file or something very isolated of cloud-init [19:47] right, the same as statically linking [19:47] well except statically linking is an implementation of my requirement :) [19:47] sure sure [19:47] I'm specifically not saying statically link it because I know that's mostly a nonstarter [19:48] so then if we remove more of dependencies in http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/requirements.txt [19:48] what about that? [19:48] think about it more this way: [19:48] as a vendor providing a cloud server [19:48] *service [19:48] we need to be able to iterate quickly on cloud-init just like we can on our cloud [19:49] packaging is somewhat of a difficulty there currently, among a bucket of other things :) [19:49] sure [19:49] believe me i know, lol [19:50] but it still feels like a hammer looking for a nail [19:50] the other thing is maybe a philisophical shift: we want a mechanism to update a *booted* system [19:51] i.e. I attach some kind of 'cloud storage' volume, update the metadata service to reflect that, and my cloud init data source sees that, and mounts the volume [19:51] ya, smoser has thoughts about that as well afaik [19:51] our use case gets more interesting because we want cloud-init (and other software) to do a lot of the tasks relegated today to a hypervisor [19:52] because hypervisors are for people who want to do things easy :P (just kidding, although it feels that way sometimes :D) [19:52] what about helping build out something like https://github.com/cloudcadre/giftwrap [19:53] https://github.com/cloudcadre/giftwrap#giftwrap [19:53] hm.. on a call, will look above later [19:53] JayF afaik the above does something like what u want [19:53] https://github.com/cloudcadre/giftwrap#how-it-works [19:55] so thats why i wonder if its more of a problem thats outside cloud-inits scope [19:55] harlowja: IMO the problem is more on the other side [19:55] how so? [19:55] harlowja: I build the package, install it in my default image, customer upgrades, gets a 'newer' version of cloud-init that doesn't have my patches, and cloud-init re-runs and does $unexpected_things because it doesn't support our data source [19:55] or if you do a package manager 'hold' in the distro, you restrict the customer from upgrading entire classes of packages [19:56] I don't know how to solve this problem, tbh, but I think something along the lines of chef-omnibus packages have to be the way [19:56] hmmm, ok, sounds like u need smarter customers ;) [19:56] problem solved! [19:56] ha [19:56] because then I never install distro cloud-init in my images, I install cloud-init-omnibus which puts everything in /opt/cloud-init/ and depends on nothing outside that dir (except maybe libc) to run [19:57] harlowja: Our entire spiel at Rackspace is Managed [cloud|server|etc]... my job is to help write software to enable that. [19:57] harlowja: which means I'm the smarter customer :) [19:57] :) [19:59] i just start to wonder if there is something wrong though, like amazon afaik doesn't need to do this, yahoo has so far gotten along just fine without this kind of isolation thing, so is it really a problem that we have to resolve by isolating all of cloud-init (and its own version of python and dependencies) to say /opt/cloud-init [19:59] I'd venture to say I'm feeling this pain right now [19:59] because we're doing /new things/ [20:00] nobody before us, afaik, has run a public, software-operated, bare metal API-driven provisioning system [20:00] sure sure [20:00] We're using network configs that are different (vlans on top of bonding) [20:00] on top of all that :) [20:01] ontop of nicira? [20:01] ;) [20:01] fwiw our first major cloud-init change was to stop excluding disk partitions from consideration when looking for configdrives [20:01] harlowja: today, no actually :( $vendor_reasons [20:01] ya, i know about some of that i think ;) [20:01] We [20:02] We've blogged about some of it in various places [20:02] ya, i just start to wonder if this is really a good way to go down [20:03] all of yahoo typically gets installed in /home/y/ and that really has not turned out so well over the years [20:03] hah :) [20:03] to much packages that forget to get managed that someone forked to adjust to fit into /home/y... [20:03] have you seen/used omnibus installed chef? [20:03] not personally, some other people are [20:03] https://github.com/opscode/omnibus [20:03] as a pattern I don't think this is bad [20:04] kinda an uglier fork of the container pattern, really [20:04] just ship a package, with all it's deps shoved into a dir away from the rest of the os [20:04] ya, i'm not sure what yahoo is replacing it with (something chef afaik) [20:04] it's ugly I'm sure when you get to the level of installing entire sub-operating systems in ~y/ [20:04] yup [20:05] but your configuration system (be it chef, cloud-init, whatever) being able to install and run regardless of what OS deps are installed [20:05] could be a valuable thing [20:05] again sorta goes to the world view of a tool like cloud-init being 'out of band' similar to a hypervisor... even though obviously it's not :) [20:05] ya, i'd be +2 to cloud-init in the kernel [20:06] like kvm [20:06] lol [20:06] i'm sure linus would be fine with that [20:06] HAH [20:06] but you get the general worldview I'm looking with? [20:06] ya [20:06] i do [20:06] although i still have mixed feelings about it, knowing past experiences :-P [20:06] harlowja: you broke my fbsd code! [20:06] the ability to configure the image to utilize $cool_things in my cloud means I need to be able to update cloud-init rapidly as I build $cool_things [20:06] harmw i might have, ha, who knows [20:07] my branch was working, but trunk now doesn't [20:07] JayF ya, sure, thats just a slipper slope of becoming your own distro ;) [20:07] harmw i blame smoser [20:07] lol [20:07] :P [20:07] and while I don't expect that long-term to be a 'fork' as it is right now... I do expect to continue to want to run pretty close to 'master' cloud-init ... especially when talking about versions shipped in old (>1y old) distros [20:08] harlowja: heh, we'd not be A distro... we'd be a fork of /every/ distro which is why I'm talking to you guys upstream trying to solve the problem in a better way for everyone than just me doing something on my own [20:08] isn't that called the 'JayF distro' (with description being a ' a fork of /every/ distro') [20:08] lol [20:08] hah [20:09] It's called coroes [20:09] but better! [20:09] *coreos [20:09] lol [20:09] jayfos [20:09] and all my 'forks' are containers [20:09] dude if I had my own distro [20:09] it'd be called Old OS [20:09] * JayF runs oldos.org [20:09] lol [20:09] nice [20:10] btw harlowja I put an example in that bugrport [20:10] harmw that log looks fine :-/ [20:10] 'Using distro class looks like its trying the openstack one [20:11] 2014-09-02 19:47:06,862 - importer.py[DEBUG]: Failed at attempted import of 'freebsd' due to: No module named freebsd [20:11] 2014-09-02 19:47:06,892 - stages.py[DEBUG]: Using distro class [20:11] ya, look at the line after that [20:11] its basically logging teh attempts [20:12] ya, perhaps i should shutup that log [20:12] i posted another one [20:12] nah wait, let me post one with specifying the datasource in cloud.cfg [20:13] Ill do that later on [20:13] kk [20:13] i'm gonna shutup that log that says failed imports [20:13] to confusing [20:13] u aren't the first to say wtf it failed importing [20:13] when it really didnt, ha [20:13] yea, Ill try and find out why the F fbsd is failing on me right now with trunk code [20:14] kk [20:22] JayF anyways, lets see what smoser thinks, he's from more of the world of distro-land than i am [20:23] yahoo i know is trying to move forward with stuff like docker and all, so i can see how it might all work out [20:23] I am trying to move forward with stuff like docker and all :) [20:23] but I have to support whatever reasonable patterns my customer wants [20:23] s/customer wants/customers want/ [20:25] ya, although sometimes customer not doing the right thinkg ;) [20:25] *thing [20:28] that's where the 'reasonable' part comes in :) [20:29] ;) [20:30] harmw, i'm being asked for cirros 0.3.3 with mtu fix. [20:30] don't we have that already? [20:30] the fix, that is [20:31] or is that to a pending merge? [20:32] i forget. i thought maybe i had it with other changes. let me look if we can easily do a 0.3.3 [20:32] that has it. [20:32] there is some code from me (?) in LP to adress this [20:32] udhcpc wrapper stuff [20:35] smoser: https://bugs.launchpad.net/cirros/+bug/1301958 [20:39] right [20:39] +## currently disabled as it depends on busybox and buildroot [20:39] +## version numbers (in the patch paths). [20:39] +## note, that without this the cirros-udhcpc is inert, but unoffensive [20:39] that was why it was non-trivial [20:40] ahyes, you had to upgrade buildroot first [20:45] hmgrr, setting the hostname is broken in my fbsd code... it doesn't set what it got from metadata, but instead re-uses `hostname`, or something [20:51] ah, so datasource.get_hostname() is returning that [20:57] bingo: why is it entering this... if self.metadata or 'local-hostname' not in self.metadata: [20:57] later all. i have to run. [20:58] o/ [21:08] harlowja: http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/revision/1000.1.1/cloudinit/sources/__init__.py [21:08] I believe that has brutally broken the setting of hostnames [21:08] atleasy in my current testing of the fbsd module [21:09] since I just confirmed to have a perfectly fine hostname (host.bla.tld) inside self.metadata['local-hostname'] [21:10] so please revert that smoser or harlowja :) [21:10] ya, seems like it could [21:10] JayF: you broke it :p [21:11] harmw lets see what JayF thinks, i wonder if that was just a mistake [21:11] I'm guessing thats a yes :) [21:11] me to [21:12] wait what [21:12] oh well, it's always nice to gte the oppertunaty to dive in to the inner workings of c-i :P [21:12] ;) [21:12] oh man I have a handful of changes like that :( [21:13] lets hope your hands are small, shall we :P [21:13] Ooooh, I see how this one changes the logic [21:13] pretty sure its 'if not self.metadata or 'local-hostname' not in self.metadata: ' [21:13] or supposed to be ^ [21:13] pep8 screams about that [21:13] odd [21:13] the old line was fine though, just reverted it locally and *everything* just works now :) [21:14] harmw: make pep8 will fail though, I presume :( [21:14] * JayF might be too used to being able to trust unit tests [21:14] don't know, I don't have alle the test tools in this instance to verify [21:14] let me try [21:18] harmw JayF https://code.launchpad.net/~harlowja/cloud-init/fixer-up/+merge/233126 [21:18] also https://code.launchpad.net/~harlowja/cloud-init/better-errors/+merge/233125 [21:21] seems like pep8 doesn't complain about 233126 [21:21] and retains the same old logic [21:21] gimme one sec, fixing a network issue [21:21] should we also write a check to catch this instance? [21:21] s/check/test/ [21:22] probably [21:22] if you say pep8 works, then +1 to the change [21:23] harlowja: 126 is spot on [21:24] ya, (cloud-init)harlowja@getlookcrowd[0]:~/dev/os/cloud-init $ pep8 cloudinit/sources/__init__.py [21:24] (cloud-init)harlowja@getlookcrowd[0]:~/dev/os/cloud-init $ [21:24] no errors/complaints [21:24] god, and this only took 2 precious hours of my tonight-time.. [21:24] harmw: :( I'm sorry man [21:25] and you should! [21:25] :p [21:25] nah, I've seen worse [21:26] harlowja: you're merging this? or do you need permission from the supreme uber godlike master wizard? [21:27] harmw i can merge, although i'll just let smoser to, where'd he go [21:27] i usually just delegate to him ha [21:27] :> [21:27] :) [21:28] well, other than this the fbsd code works btw. Having it set the hostname and adding a user+ssh key wrks just fine [21:29] cool [21:34] I can't pull this right now, merge it if you think it's right [21:36] sorry :( dealing with local network stuff [21:37] ya, i'll let smoser pull it in unless its so critical can't wait [21:37] then i'll try to remember the merge stuff, haha [21:38] harmw: we were just talking around here the otehr day about how neat it'd be to have fbsd support :) [21:38] harmw: (here=rackspace onmetal) === shardy is now known as shardy_afk === gondoi is now known as zz_gondoi