=== rangerpbzzzz is now known as rangerpb [15:02] Good morning, I’m having trouble debugging my cloud-config script. I’m on ubuntu 16.04. I edit /var/lib/cloud/instance/user-data.txt and run cloud-init --debug init, but i get the same error message of “Unhandled non-multipart (text/x-not-multipart) userdata: ‘b’{‘…’” every time i run it, regardless of changes to user-data.txt [15:03] tz__, cloud-init does not re-read that file. [15:03] it reads form the datasource. [15:04] can i edit the data source on digital ocean after the VPS has been launched? [15:04] not terribly easily. [15:04] for debugging quick launches i suggest using lxc [15:04] how can i iterate quickly on the cloud-config file then without having to destroy, create a droplet? [15:05] since you already have one up, try this [15:05] sudo lxd init [15:05] ancer its questions [15:05] then as non-root [15:05] s/ancer/answer/ [15:05] then as non-root [15:07] lxc image copy ubuntu-daily:xenial local: --alias xenial --auto-update [15:07] actually, i woudl install zfs first (apt-get install zfs) === shardy is now known as shardy_afk [15:07] then you can use zfs with lxc which is much faster. [15:07] then you launch new instances with [15:07] lxc image copy ubuntu-daily:xenial local: --alias xenial --auto-update [15:07] fudge [15:07] sorr. [15:08] launch a new instance with user data in file 'user-data' like: [15:08] lxc launch ubuntu-daily:xenial my-name "--config=user.user-data=$(cat ./user-data)" [15:09] okay cool, trying it out now [15:09] thanks smoser! [15:11] is there anyway to see cloud init output inline when i launch? [15:11] or whats the best way to see the cloud init output? [15:12] no. actually. [15:12] that sucks. [15:13] at least not easily [15:13] you probably can [15:13] but you have to fiddle with the config options of the container [15:13] lxc used to put console output right in front of you [15:13] but lxc2.0 does not. [15:13] hmmm so how do people build the cloud config the first time? im probably gonna have to edit it 100 times before i get it right [15:13] tz__ but you can easily enough do things like: [15:14] lxc exec NAME tail -f /var/log/cloud-init.log [15:14] tz__, if you're passing cloud-config, the first thing is that it needs to be valid yaml [15:14] so checking that is as simple as: [15:14] python -c 'import yaml; yaml.load(open("your-user-data"))' [15:16] okay, that passes at least [15:18] got “Failed to shellify” then a dumb of all the runcmds [15:20] okay, i think it’s because i have blank lines (trying to split up the runcmds into sensible chunks) [15:23] smoser: thanks for your help, i’m now seeing cloud-init debug information [15:25] tz__, couple things... [15:25] you can do a print() of the loaded yaml to make sure it looks like you epxect [15:25] and also yaml "anchors" are useful for making things readable [15:26] look at http://paste.ubuntu.com/22322345/ [15:29] okay, i’ll check that out [15:29] thanks! === shardy_afk is now known as shardy [18:01] is there such a thing as cloud config snippets? a repo of best installations of, for example, redis? [18:01] node, databases, node app [18:02] sshd_config [18:02] tz__, no. [18:02] sad. === shardy is now known as shardy_afk [18:03] seems like it would be super valuable since cloud configs are cross platform(ish) [18:04] check out what you want on the box, and it generates the cloud config for you [18:04] harlowja, around ? [18:05] sup [18:05] https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+ref/version-info [18:05] actually.. [18:05] let me propose for merge to make it easier for you [18:05] thx :-P [18:06] do we need to start using git rebase more also? [18:06] to avoid crap histories? [18:06] rebase then push --force [18:08] https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/302176 [18:08] harlowja, are you trolling ? [18:08] not really :-P [18:08] thats been my biggest pain ever. [18:09] 2 things that bzr did better than git [18:09] a.) bzr commit -p [18:09] u just gotta know how to use git right ;) [18:09] (this just shows diff in the EDITOR window, so you can write your message and look at the diff) [18:09] http://blog.adamspiers.org/2015/03/24/why-and-how-to-correctly-amend-github-pull-requests/#sec-3 [18:09] b.) uses '--first-parent' by default behaviori for git log [18:09] its the same crap for github PRs that most people don't know how to do :-P [18:10] http://blog.adamspiers.org/2015/03/24/why-and-how-to-correctly-amend-github-pull-requests/ [18:10] using first-parent by default is genious [18:10] genius [18:10] ya, bzr prob did that nicer [18:10] gerrit forces it [18:10] but github doesn't [18:10] it means that people dont have to squash and throw away data [18:10] in order to avoid crap history [18:10] ya, fair [18:10] i almost consider just getting crap history [18:11] and saying "please use --first-parent" [18:11] so there... at that mp, i tested building of rpm. [18:12] cool [18:12] how'd that go [18:12] ha [18:12] http://paste.ubuntu.com/22339840/ [18:13] the way to build is this: [18:13] http://paste.ubuntu.com/22339872/ [18:13] cools [18:13] that does it all in an lxc container, but basic path is there. [18:13] will use that :-P [18:13] right. no one. although maybe sometime soon honestly as we want to get c-i on centos [18:14] larsks, you probably have thoughts on my brpm stuff [18:14] spec looks like thsi: http://paste.ubuntu.com/22339840/ [18:14] so only other thing i'm not sure about is [18:14] say i have a addon that doesn't really belong in cloud-init, but i need to add it into cloud-init [18:15] i can show u this addon if u really care [18:15] how do i make sure that the version number won't mess with the upstream one [18:15] by default, we dont bump the 'Release' as iunderstand thats like "packaging release" in debian and in order to get a new version of that, you kind of had to bump the source release so 'Version' would go up. [18:15] k [18:15] example ? [18:16] a godaddy module addon [18:16] config module [18:16] ah. [18:16] and you have patches/ that you use --patches ? [18:16] this one its not yet a patch, but ya, its equivalent to [18:17] This is where I wish cloud-init using something like stevedore for discovering out-of-tree modules... [18:17] right [18:17] https://gist.github.com/harlowja/e6d79d1de62e0edd821bedf76bfd5a20 [18:17] smoser thats part of that module [18:17] bunch of crap that getting out of puppet (doing it in puppet and half of it in cloud-init was causing stupid issues) [18:17] so fixing that by just moving some of it to a cloud-nit module [18:17] right. [18:17] so how should we do that? [18:18] so stevedore is one option [18:18] you want yours to be bigger than trunk or something. [18:18] the version ? [18:18] is that what we're talking about ? [18:18] well i either need the release to be bigger or smaller :-P [18:18] yeah, larsks i do want to support out-of-tree moduels. [18:18] and datasources. [18:18] oorrr go the path of larsks with stevedore [18:19] i can cook up osmething with stevedore [18:19] we already have enough dynamic loading [18:19] (yet another runtime dep...) [18:19] we dont need to add stevedoor for it. [18:19] stevedore is just a tiny wrapper over entrypoints [18:19] which is a python thing [18:19] and /me cries everytime he runs a stevedore-using executable [18:19] stevedore shouldn't make u cry [18:19] stat stat stat stat stat disk io disk io [18:19] its pretty dinky in all reality [18:20] * larsks cries every time he sees another project re-implement the same basic features. [18:20] stevedore is a light layer over the python entrypoints stuff [18:20] so either use stevedore, or i just use entrypoints like stevedore is doing [18:20] cloud-init right now has its custom thing [18:20] i dont know. [18:20] https://github.com/openstack/stevedore/blob/master/stevedore/extension.py#L147 [18:20] thats really the 'thing' that does this, lol [18:20] i'm very much feeling pain of startup performance right now. [18:20] 'eps = list(pkg_resources.iter_entry_points(namespace))' [18:21] mainly that, ha [18:21] and also feel pain of external dependencies. [18:21] the rest of stevedore is just lue [18:21] *glue [18:21] so i'm in the middle between larsks and smoser [18:21] in that if i could use something like entrypoints [18:21] An entrypoints-based internal implementation would I guess be fine. [18:21] then i can get rid of this complexity :-P [18:22] larsks ya, i can easily make something dinky using entrypoints [18:22] its not really that hard, ha [18:22] https://github.com/openstack/stevedore/blob/master/stevedore/extension.py#L153 is the loop, and the other code is the finding [18:22] not magical crazy stuff, ha [18:22] which is why stevedore is a dinky layer [18:23] Yeah. also, pkg_resources.iter_entry_points may have most of what we need... [18:23] yup [18:25] so 2 thigns i'd like to support out of tree [18:25] a.) config modules [18:25] b.) datasources [18:25] and thats easy enough to "find" them. [18:25] but getting them into the right order is somethign we dont have. no way to declare a dependency or something. [18:25] so at the momemnt a list of datasources and a list of modules at each stage. [18:26] harlowja, so back to your version / thing... [18:26] what were you saying ? [18:26] smoser: well, config modules get an ordering from cloud.cfg. [18:26] does using that version with a git cause issues for you? [18:26] smoser i forget [18:26] larsks, ah. yeah, but not if they're just laid down on disk. [18:26] ie, ideally you just put new modules in some directory and they get found [18:26] and datasources too [18:26] someway of making sure that a godaddy built cloud-init.rpm doesn't use the same stuff (Version...) as a public cloud-init.rpm [18:27] harlowja, we could expose branch [18:27] harlowja: add a suffix to the package name (cloud-init-godaddy) and make it obsolete the other one? [18:27] in the spec [18:27] larsks ah, that might work to [18:27] until we have this stevedore and ... stuff [18:28] then i can get out of this :-P [18:29] smoser: looked briefly at your brpm changes, which seem fine, but there is a weird try/except block at the top of brpm that seems to be a noop (it catches any exception...and raises it) [18:35] larsks, yeah its stupid [18:35] for flake8 [18:35] it complains about module level imports not at top of filel [18:37] harlowja, you know of a nicer way to work around that ? [18:37] not sure [18:38] smoser, I don't see that if I remove the try/except block. no complaints from flake8 (https://asciinema.org/a/bbqseaf84npuoon71hciwvc99) [18:38] (where flake8 == 2.4.1) [18:39] how'd you do that larsks ? [18:40] do which? [18:40] the asciicinima thing [18:40] https://asciinema.org/ [18:40] There is a simple 'asciinema' cli tool [18:41] Looks like you need a ppa for ubuntu. Seems to be packaged for fedora and debian... [18:41] well, its really pep8 . pep8 1.7.0 here does not like it. [18:41] Interesting. I seem to have 1.5.7. I guess I would add a comment there, otherwise everyone who looks at the code is going to ask the same question :) [18:41] yeah [18:41] deal [18:42] hmmm asciinema neat [18:45] if "avoid-pep8-E402-import-not-top-of-file": [18:46] import ... [18:46] committing that [19:16] harlowja, [19:16] https://bugs.launchpad.net/cloud-init/+bug/1603533 [19:16] is fix committed ? [19:22] i think so [19:23] assuming its still carried forward with the git stuff (and not lost) === shardy_afk is now known as shardy [21:46] hey all. i have some changes here locally. [21:47] i'm going to mark a 0.7.7 release soon. tonight or tomorrow. [21:47] and call it released. [21:47] and then we can work towards a quick 0.7.8 iteration on that. [21:47] the git stuff now in i think. [22:03] smoser: http://lists.openstack.org/pipermail/openstack-dev/2016-August/101007.html is something you might find interesting === rangerpb is now known as rangerpbzzzz [22:37] cool [23:29] anyone have a snippet that shows how to pull a repo from github? I have http://pastie.org/private/kpnunfkgktxxse2m4xwsga , but problems ensue