[11:29] <dynek_> hey
[11:31] <dynek> anyone using cloud-init to setup redhat atomic host? I'm don't understand where DNS settings are stored on disk when I mention them in network-interfaces
[11:31] <dynek> and on first boot /etc/resolv.conf is not populated
[11:31] <dynek> and finally, it's not possible to use power_state to reboot the machine :/
[14:42] <smoser> rharper, around ?
[14:43] <smoser> looking at https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+ref/nocloud-boottime-improvements seems like a general caching mechanism might be best
[14:45] <rharper> here
[14:45] <rharper> yeah, I started looking at a memoize decorator
[14:46] <rharper> but those typically require arguments as it uses a dictionary to cache previously seen invocations
[14:46] <rharper> the blkid output parsing  is different than typicall caching of the result since it just opens and reads instead of execing blkid again;
[14:49] <smoser> well, blkid unless told otherwise does the same, does it not?
[14:49] <rharper> no
[14:50] <rharper> it refreshes data and re-writes tab each run AFAICT
[14:50] <rharper> even if it does read it; the exec of blkid vs. open/read  is non-trivial
[14:53] <smoser> maybe. but i think that might be over optimization.
[14:53] <rharper> *shrug*; those items were tracked as the highest cost
[14:53] <rharper> during nocloud image boots, and subsequent boots
[14:53] <rharper> I didn't pick them out of the hat
[14:54] <smoser> blkid as non-root here takes 2 thousandths of a second as (0.002)
[14:54] <smoser> and as root in a container 0m0.003s
[14:54] <smoser> as root not in container it does take almost 0.03 and i can see that is expensive.
[14:54] <rharper> I'm almost done with preparing a post with the tools
[14:55] <smoser> but its kind of non-avoidable.
[14:55] <rharper> so, we can reproduce the data
[14:55] <smoser> well, i suggest that generally just being able to cache a 'subp' call
[14:56] <smoser> would make many things easier. and would lose performance-wise to your blkid specific solution by a very small margin
[14:56] <rharper> the memoize  decorator could do that
[14:56] <rharper> I suspect though that there may be some commands where we do expect different answers given the same call
[14:57] <rharper> in which case a general caching of subp is not going to be correct
[14:57] <rharper> I think I'd rather continue to drive changes based on profiling
[14:57] <smoser> right.
[14:58] <smoser> so a 'cachable' option on subp defaulting to False
[14:58] <smoser> and then call subp(['dpkg', '--print-architecture'], cachable=True)
[14:58] <smoser> means "use the cache or fill it"
[15:02] <rharper> as you say for the blkid, it won't help as much;  we make 4 different calls
[15:03] <rharper> I think a cachable subp could produce additional improvements; but I would posit that we keep the blkid tab parsing in-addition
[15:09] <smoser> there is a blkid method in curtin
[15:09] <smoser> which i originall did some thing s like this.
[15:09] <smoser> but the idea woudl easily have a cache added.
[15:10] <rharper> lsblk ?
[15:10] <smoser> no. block.blkid()
[15:10] <smoser> it basically ujust ensures that no cache is read adn the retuns result as dict
[15:12] <rharper> right;  so the other challenge with blkid specifically is that the parameters are mostly just formatting/filtering output;  but for the use of blkid in cloud-init now; they really just want the filtering part (TYPE=X , path=Y) and to return the device name (if it's found)
[15:13] <rharper> I had a stab at adding 'read from cache' to the current find_devs_with() function but it seemed like lots of extra work for no gain since no part of cloud-init was using things like -sUUID to return the UUID value of the match, vs. just the devname
[15:13] <smoser> right. i'd just change all callers to just use a 'blkid()' method and operate on thee dict response.
[15:13] <rharper> yeah
[15:14] <rharper> I forgot the dict part;
[15:14] <rharper> and a wrapper could exist to just match and return the device name like users of find_devs_with expect
[15:14] <smoser> yeah.
[15:19] <smoser> just noticed that 'devs' in blkid in curtin does ntohing.
[15:22] <rharper> yah pylint
[15:22] <rharper> we did *something* with it
[15:22] <rharper> I expect that was a devs filter
[15:31] <smoser> rharper, revno 221 made it do nothing
[15:34] <smoser> i swear udev is supposd to update some blkid cache.
[15:34] <smoser> or run blkid so that it updates the cache
[15:34] <rharper> it doesn;t look like it was used even with change; ie, I don't see any callers passing devs in
[15:35] <nacc> there are some blkid rules in /lib/udev, but nothing is sticking out to me
[15:35] <nacc> it seems like mostly for optical?
[15:36] <nacc> oh for md devices too
[15:46] <smoser> rharper, http://paste.ubuntu.com/23247253/
[15:46] <smoser> that is a cachable_subp that seems like it shoudl generally work
[15:55] <rharper> so, my first though is that my current patches are more focused and have fewer possible side-effects;  it would be nice to see them land and then have more time with cachable subp and blkid (with cache) after? or hold off until each of those can land?
[15:58] <smoser> well, i dont  like specifically caching _IS_CONTAINER when a much more generic solution works.
[15:59] <rharper> surely using generalized subp cache has a wider impact;  it's not clear to me that it's safe to cache all callers of subp at this time
[15:59] <rharper> where as a specific caching of _is_container is straightforward and safe
[16:00] <smoser> of course its not safe to cache all callers of subp
[16:00] <smoser> change the callers that *are* safe to use cachable_subp
[16:01] <rharper> ok
[16:02] <smoser> i'm also kind of worried about the the blkid /dev/sr0 and blkid /dev/sr1 specifically
[16:03] <smoser> as you can blame them... they are basically there to kick that device
[16:03] <smoser>  812f82e7b3bad3f8127face552c76ef974b54661
[16:03] <rharper> that's also for 2.6
[16:03] <rharper> at least the comment says
[16:04] <smoser> right
[16:05] <smoser> rharper, comments on bug 1628337
[16:05] <rharper> just the module order should be needed?
[16:06] <rharper> your pastebin had another change in util
[16:08] <smoser> oh. yeah, just the module order
[16:08] <smoser> https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/307055
[18:42] <harlowja> ok rharper smoser i updated https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/301729
[18:42] <harlowja> with the tiny change there
[18:42] <harlowja> i still would just like 1 simple logging location :-P
[18:42] <smoser> harlowja, i agree with that.
[18:44]  * harlowja just likes the simple things
[18:44] <harlowja> lol
[18:45]  * smoser doesn't even know how messages get to rsyslog anymore
[18:45] <smoser> $ ls -l /dev/log
[18:45] <smoser> lrwxrwxrwx 1 root root 28 Aug 31 12:12 /dev/log -> /run/systemd/journal/dev-log
[18:45] <smoser> this is odd
[18:52] <harlowja> lol
[18:57] <rharper> harlowja: thanks;  I agree with smoser;  however, I do find the change in format between logging and rsyslog in the *same* file cloud-init.log to be jarring;
[18:58] <rharper> if cloud-init wanted to join with syslog, I'd at least like it to have a confirming message format
[18:59] <rharper> I don't yet know how to ensure we get consistent formatting between the two;  if we fallback to syslog, then it's a per-image/distro config mechanism; ie, what the default is for rsyslog in the image; unless we can have cloud-init emit specific format that matches the syslog format
[19:03] <harlowja> meh, i like files
[19:03] <harlowja> files good
[19:03] <harlowja> lol
[21:01] <smoser> harlowja, https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/305882
[21:01] <smoser> ...
[21:01] <smoser> would be nice
[21:02] <harlowja> oh
[21:02] <harlowja> hey
[21:02] <harlowja> i know u
[21:02] <harlowja> yes, one sec
[21:02] <harlowja> reworking cloud.cfg -> cloud.cfg.ubuntu and cloud.cfg.fedora
[21:03] <smoser> :)
[21:03]  * smoser runs now
[21:03] <harlowja> lol
[21:58] <harlowja> smoser nrezinorn https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/307100
[21:58] <harlowja> that also fixes why brpm wasn't working
[21:58] <harlowja> (or part of it)
[22:02] <harlowja> part uno
[22:02] <harlowja> of part 300
[22:02] <harlowja> lol
[22:04] <harlowja> nrezinorn i'm going to take a stab at destroying brpm
[22:04] <harlowja> though i know u love it
[22:04] <harlowja> lol
[22:16] <harlowja> smoser https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+ref/sys-io-errors updated
[22:16] <harlowja> in return can u fix the following
[22:16] <harlowja> https://gist.github.com/harlowja/4f996d19b633862eb3a647d0c6053ac9
[22:16] <harlowja> i shall include 1 goat
[22:17] <harlowja> ^ things that aren't working on rhel