[14:23] <smoser> Ahrotahntee, you can't (unless you are using 'nocloud') provide metadata. the general line is that 'metadata' comes from the cloud provider, and userdata comes from the user.
[14:23] <smoser> but then further, yeah, there is no way to easily reference those things generically from inside a config or environment or anything.
[14:30] <Ahrotahntee> smoser: once the environment is up I have no problems (it's just a json object), I was just really hoping I could bake it into the config
[14:30] <Ahrotahntee> oh well; I'll figure something else
[14:33] <smoser> what is it that you ant to bake in ?
[14:33] <smoser> i'm kinda confused on what your goal is..
[17:01] <ajorg> Good morning. I just filed https://bugs.launchpad.net/cloud-init/+bug/1658734 but I'm happy to cook a patch for it if I can get a hint about how you'd prefer to see it resolved. At the very least what you'd like the config option to be called. I'm thinking a top-level list `dns_redirect_ips: []`, but if there's another top-level section it should live under
[17:01] <ajorg> I'm happy to entertain that notion.
[17:02] <ajorg> Also, apologies for not upstreaming more of our (AWS) patches yet.
[17:02]  * ajorg makes some excuse about being "busy" or something.
[17:05] <ajorg> smoser: looks like the feature was your work? https://git.launchpad.net/cloud-init/commit/cloudinit/util.py?id=1bb67be5
[17:10] <smoser> ajorg, i dont think i unerstand what 'dns_redirect_ips' would do
[17:11] <smoser> (the bug is well written, and shows you've done homework, thank you)
[17:16] <ajorg> Would set the global _DNS_REDIRECT_IP (or otherwise get that into the method) to the provided list, so that we can set it to an empty list to disable the feature.
[17:20] <ajorg> smoser: it would be a slightly opaque way to disable the feature, but it would also add the ability for users to set that list.
[17:22] <smoser> hm..
[17:22] <ajorg> exactly the initial reaction i thought i'd get, which is why i thought i'd ask before proposing a patch
[17:23] <smoser> yeah, and it'd mean that a config  module or somethign woudl have to set the value of that variable
[17:23] <smoser> (and then each time cloud-init ran it would have to set it too).
[17:24] <smoser> ie, that code doesnt have easy access to the full config
[17:24] <ajorg> a bit ugly, for sure. i wasn't sure what you had in mind when you added the global.
[17:24] <smoser> well the global is really just to keep it from being re-done on per call
[17:25] <smoser> so you should only see 3 queries to your dns server, rather than N*3 in
[17:25] <ajorg> ah, right... duh
[17:25] <smoser>  results = [x for x in mynames if is_resolvable(x)]
[17:27] <smoser> hmm.
[17:28] <smoser> id ont have a good s olution of the top of my head.
[17:28] <smoser> multiple things use is_resolvable, and generally its nice to protect against stupid dns hijacking even though it should not really ever be used in a "real cloud"
[17:29] <ajorg> totally agree, just someone ran into this IDS problem
[17:34] <ajorg> I can patch it to just disable the feature, but I'd rather not, now that our repositories are open for anyone to use outside of EC2 (since we made a public docker container of Amazon Linux).
[17:36] <smoser> ids ?
[17:37] <ajorg> intrustion detection system
[17:37] <ajorg> apparently some of them alarm on suspicious DNS queries
[17:37] <ajorg> and these (or at least the random ones?) can get flagged as suspicious
[17:44] <ajorg> smoser: shall i pass in the list, and use the global if the passed parameter is None?
[17:45] <smoser> well, where woul dyou change the callers ?
[17:45] <smoser> in ec2 you probably hit that code in cloudinit/config/cc_apt_configure.py and cloudinit/sources/DataSourceEc2.py
[17:46] <ajorg> all of the callers should at least have access to on-disk config (except _is_resolvable_url, but it can also take that parameter and pass it on)
[17:46] <ajorg> it's also hit when deciding if it can talk to the instance metadata service, sadly
[17:46] <smoser> and cloudinit/sources/DataSourceOpenStack.py even
[17:47] <ajorg> yup
[17:48] <smoser> so here is what i dont like...
[17:48] <smoser> ubuntu makes images available for download on cloud-images.ubuntu.com
[17:49] <smoser> if someone downloads that and uploads a new ami from it it should work well
[17:49] <smoser> but you're telling me unless we changed the default value to not do this searching, then your IDS would flag an instance from that image.
[17:50] <smoser> but if the image is used on some non-cloud, it makes good sense to have that.
[17:50] <smoser> the end result of all of that above is i dont like "fix this by disabling it in specific cases"
[17:50] <ajorg> EC2 doesn't have an IDS that alarms at customers, but some customers have their own.
[17:51] <ajorg> So I'm totally fine with leaving this on by default, but some customers will need to be able to disable it.
[17:51] <ajorg> Right now the only way to disable it is to patch the sources.
[17:53] <ajorg> I'd like to provide a way to disable it from the on-disk config, so we can tell specific customers how to do that (and document it, of course).
[17:54] <smoser> ok. thats more acceptable.
[17:54] <ajorg> cool. sorry it wasn't obvious that's what i wanted.
[17:55] <smoser> so as it is right now, i think we hit that code even if we pass an ip address in
[17:55] <smoser> right ?
[17:55] <ajorg> yeah, that's true
[17:55] <smoser> we could make is_resolvable not do this if it is an ip address.
[17:55] <smoser> which would mean that ec2's search and openstack's search would not do it.
[17:55] <smoser> the  mirror searches still would
[17:56] <ajorg> but making it skip IPs would make cloud-config user-data effective at controlling it
[17:56] <ajorg> do you want a disable_it config or a i_know_my_redirect_ips config?
[17:59] <ajorg> i'm confident either would satisfy my needs
[18:01] <smoser> i think disable it would be better really.
[18:07] <ajorg> k, i'll take another look at the code and try to come up with a proposal
[18:07] <ajorg> might be a while as i'm juggling a few things (as always)
[20:27] <smoser> larsks, https://code.launchpad.net/~larsks/cloud-init/+git/cloud-init/+merge/315276
[20:27] <smoser> you see that just by running 'nosetests' ?
[20:27] <larsks> smoser: I did, yes.
[20:28] <smoser> really odd that i've never seen it.
[20:29] <smoser> tox just runs nosetests
[20:29] <larsks> If you'd like, I can write up a reproducer and add that to the lp issue.
[20:29] <smoser> other than in a virtual env, and with 'python -m nose' but i was under the impression those were the same, and in ubuntu build process we use nosetests too
[20:29] <smoser> you think its more than just a ordering thing ?
[20:30] <smoser> i really do suspect its an issue, i'm just kind of wondering why i've never see it.
[20:30] <larsks> I'm not sure.  Let me see if I can reproduce somewhere other than on my F25 system.
[20:31] <larsks> Might be a couple of days, but I will update the lp issue with the results.
[20:31] <powersj> would be interesting to see your CLI and versions.... https://paste.ubuntu.com/23853943/
[20:31] <larsks> Will do.
[20:32] <smoser> larsks, is that python27 or python3 ?
[20:32] <larsks> That was python3, I think.
[20:57] <smoser> i think probably we should be httpretty.activate decorating the individual test methods in tests/unittests/test_datasource/test_gce.py
[20:57] <smoser> like tests/unittests/test_datasource/test_openstack.py and tests/unittests/test_ec2_util.py:
[20:58] <smoser> rather than the class like is done in tests/unittests/test_datasource/test_gce.py
[20:58] <smoser> Odd_Bloke, do you have thougths on above ?
[20:58] <smoser> much of the httpretty stuff is currently blame'ing to you (just thats why i'm asking... blame in a git way,  not 'blame' in a bad way)
[20:59] <smoser> hm.. and as i look further, 33e1f251 which basically changes to activating on the class.
[21:02] <smoser> yeah, thats what does it... looking at the http pretty code
[21:02] <smoser>  https://github.com/gabrielfalcao/HTTPretty/blob/master/httpretty/core.py#L1169
[21:27] <smoser> larsks, well,, looking at that httpretty code, it sure looks like its trying to decorate individual methods
[21:28] <smoser> so i can't explain how ou'd see this.