=== cpaelzer_ is now known as cpaelzer [11:29] hey === dynek_ is now known as dynek [11:31] anyone using cloud-init to setup redhat atomic host? I'm don't understand where DNS settings are stored on disk when I mention them in network-interfaces [11:31] and on first boot /etc/resolv.conf is not populated [11:31] and finally, it's not possible to use power_state to reboot the machine :/ [14:42] rharper, around ? [14:43] looking at https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+ref/nocloud-boottime-improvements seems like a general caching mechanism might be best [14:45] here [14:45] yeah, I started looking at a memoize decorator [14:46] but those typically require arguments as it uses a dictionary to cache previously seen invocations [14:46] the blkid output parsing is different than typicall caching of the result since it just opens and reads instead of execing blkid again; [14:49] well, blkid unless told otherwise does the same, does it not? [14:49] no [14:50] it refreshes data and re-writes tab each run AFAICT [14:50] even if it does read it; the exec of blkid vs. open/read is non-trivial [14:53] maybe. but i think that might be over optimization. [14:53] *shrug*; those items were tracked as the highest cost [14:53] during nocloud image boots, and subsequent boots [14:53] I didn't pick them out of the hat [14:54] blkid as non-root here takes 2 thousandths of a second as (0.002) [14:54] and as root in a container 0m0.003s [14:54] as root not in container it does take almost 0.03 and i can see that is expensive. [14:54] I'm almost done with preparing a post with the tools [14:55] but its kind of non-avoidable. [14:55] so, we can reproduce the data [14:55] well, i suggest that generally just being able to cache a 'subp' call [14:56] would make many things easier. and would lose performance-wise to your blkid specific solution by a very small margin [14:56] the memoize decorator could do that [14:56] I suspect though that there may be some commands where we do expect different answers given the same call [14:57] in which case a general caching of subp is not going to be correct [14:57] I think I'd rather continue to drive changes based on profiling [14:57] right. [14:58] so a 'cachable' option on subp defaulting to False [14:58] and then call subp(['dpkg', '--print-architecture'], cachable=True) [14:58] means "use the cache or fill it" [15:02] as you say for the blkid, it won't help as much; we make 4 different calls [15:03] I think a cachable subp could produce additional improvements; but I would posit that we keep the blkid tab parsing in-addition [15:09] there is a blkid method in curtin [15:09] which i originall did some thing s like this. [15:09] but the idea woudl easily have a cache added. [15:10] lsblk ? [15:10] no. block.blkid() [15:10] it basically ujust ensures that no cache is read adn the retuns result as dict [15:12] right; so the other challenge with blkid specifically is that the parameters are mostly just formatting/filtering output; but for the use of blkid in cloud-init now; they really just want the filtering part (TYPE=X , path=Y) and to return the device name (if it's found) [15:13] I had a stab at adding 'read from cache' to the current find_devs_with() function but it seemed like lots of extra work for no gain since no part of cloud-init was using things like -sUUID to return the UUID value of the match, vs. just the devname [15:13] right. i'd just change all callers to just use a 'blkid()' method and operate on thee dict response. [15:13] yeah [15:14] I forgot the dict part; [15:14] and a wrapper could exist to just match and return the device name like users of find_devs_with expect [15:14] yeah. [15:19] just noticed that 'devs' in blkid in curtin does ntohing. [15:22] yah pylint [15:22] we did *something* with it [15:22] I expect that was a devs filter [15:31] rharper, revno 221 made it do nothing [15:34] i swear udev is supposd to update some blkid cache. [15:34] or run blkid so that it updates the cache [15:34] it doesn;t look like it was used even with change; ie, I don't see any callers passing devs in [15:35] there are some blkid rules in /lib/udev, but nothing is sticking out to me [15:35] it seems like mostly for optical? [15:36] oh for md devices too [15:46] rharper, http://paste.ubuntu.com/23247253/ [15:46] that is a cachable_subp that seems like it shoudl generally work [15:55] so, my first though is that my current patches are more focused and have fewer possible side-effects; it would be nice to see them land and then have more time with cachable subp and blkid (with cache) after? or hold off until each of those can land? [15:58] well, i dont like specifically caching _IS_CONTAINER when a much more generic solution works. [15:59] surely using generalized subp cache has a wider impact; it's not clear to me that it's safe to cache all callers of subp at this time [15:59] where as a specific caching of _is_container is straightforward and safe [16:00] of course its not safe to cache all callers of subp [16:00] change the callers that *are* safe to use cachable_subp [16:01] ok [16:02] i'm also kind of worried about the the blkid /dev/sr0 and blkid /dev/sr1 specifically [16:03] as you can blame them... they are basically there to kick that device [16:03] 812f82e7b3bad3f8127face552c76ef974b54661 [16:03] that's also for 2.6 [16:03] at least the comment says [16:04] right [16:05] rharper, comments on bug 1628337 [16:05] just the module order should be needed? [16:06] your pastebin had another change in util [16:08] oh. yeah, just the module order [16:08] https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/307055 === Guest46101 is now known as mgagne === Edgan_ is now known as Edgan [18:42] ok rharper smoser i updated https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/301729 [18:42] with the tiny change there [18:42] i still would just like 1 simple logging location :-P [18:42] harlowja, i agree with that. [18:44] * harlowja just likes the simple things [18:44] lol [18:45] * smoser doesn't even know how messages get to rsyslog anymore [18:45] $ ls -l /dev/log [18:45] lrwxrwxrwx 1 root root 28 Aug 31 12:12 /dev/log -> /run/systemd/journal/dev-log [18:45] this is odd [18:52] lol [18:57] harlowja: thanks; I agree with smoser; however, I do find the change in format between logging and rsyslog in the *same* file cloud-init.log to be jarring; [18:58] if cloud-init wanted to join with syslog, I'd at least like it to have a confirming message format [18:59] I don't yet know how to ensure we get consistent formatting between the two; if we fallback to syslog, then it's a per-image/distro config mechanism; ie, what the default is for rsyslog in the image; unless we can have cloud-init emit specific format that matches the syslog format [19:03] meh, i like files [19:03] files good [19:03] lol [21:01] harlowja, https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/305882 [21:01] ... [21:01] would be nice [21:02] oh [21:02] hey [21:02] i know u [21:02] yes, one sec [21:02] reworking cloud.cfg -> cloud.cfg.ubuntu and cloud.cfg.fedora [21:03] :) [21:03] * smoser runs now [21:03] lol [21:58] smoser nrezinorn https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/307100 [21:58] that also fixes why brpm wasn't working [21:58] (or part of it) [22:02] part uno [22:02] of part 300 [22:02] lol [22:04] nrezinorn i'm going to take a stab at destroying brpm [22:04] though i know u love it [22:04] lol [22:16] smoser https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+ref/sys-io-errors updated [22:16] in return can u fix the following [22:16] https://gist.github.com/harlowja/4f996d19b633862eb3a647d0c6053ac9 [22:16] i shall include 1 goat [22:17] ^ things that aren't working on rhel