/srv/irclogs.ubuntu.com/2016/09/28/#cloud-init.txt

=== cpaelzer_ is now known as cpaelzer
dynek_hey11:29
=== dynek_ is now known as dynek
dynekanyone using cloud-init to setup redhat atomic host? I'm don't understand where DNS settings are stored on disk when I mention them in network-interfaces11:31
dynekand on first boot /etc/resolv.conf is not populated11:31
dynekand finally, it's not possible to use power_state to reboot the machine :/11:31
smoserrharper, around ?14:42
smoserlooking at https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+ref/nocloud-boottime-improvements seems like a general caching mechanism might be best14:43
rharperhere14:45
rharperyeah, I started looking at a memoize decorator14:45
rharperbut those typically require arguments as it uses a dictionary to cache previously seen invocations14:46
rharperthe blkid output parsing  is different than typicall caching of the result since it just opens and reads instead of execing blkid again;14:46
smoserwell, blkid unless told otherwise does the same, does it not?14:49
rharperno14:49
rharperit refreshes data and re-writes tab each run AFAICT14:50
rharpereven if it does read it; the exec of blkid vs. open/read  is non-trivial14:50
smosermaybe. but i think that might be over optimization.14:53
rharper*shrug*; those items were tracked as the highest cost14:53
rharperduring nocloud image boots, and subsequent boots14:53
rharperI didn't pick them out of the hat14:53
smoserblkid as non-root here takes 2 thousandths of a second as (0.002)14:54
smoserand as root in a container 0m0.003s14:54
smoseras root not in container it does take almost 0.03 and i can see that is expensive.14:54
rharperI'm almost done with preparing a post with the tools14:54
smoserbut its kind of non-avoidable.14:55
rharperso, we can reproduce the data14:55
smoserwell, i suggest that generally just being able to cache a 'subp' call14:55
smoserwould make many things easier. and would lose performance-wise to your blkid specific solution by a very small margin14:56
rharperthe memoize  decorator could do that14:56
rharperI suspect though that there may be some commands where we do expect different answers given the same call14:56
rharperin which case a general caching of subp is not going to be correct14:57
rharperI think I'd rather continue to drive changes based on profiling14:57
smoserright.14:57
smoserso a 'cachable' option on subp defaulting to False14:58
smoserand then call subp(['dpkg', '--print-architecture'], cachable=True)14:58
smosermeans "use the cache or fill it"14:58
rharperas you say for the blkid, it won't help as much;  we make 4 different calls15:02
rharperI think a cachable subp could produce additional improvements; but I would posit that we keep the blkid tab parsing in-addition15:03
smoserthere is a blkid method in curtin15:09
smoserwhich i originall did some thing s like this.15:09
smoserbut the idea woudl easily have a cache added.15:09
rharperlsblk ?15:10
smoserno. block.blkid()15:10
smoserit basically ujust ensures that no cache is read adn the retuns result as dict15:10
rharperright;  so the other challenge with blkid specifically is that the parameters are mostly just formatting/filtering output;  but for the use of blkid in cloud-init now; they really just want the filtering part (TYPE=X , path=Y) and to return the device name (if it's found)15:12
rharperI had a stab at adding 'read from cache' to the current find_devs_with() function but it seemed like lots of extra work for no gain since no part of cloud-init was using things like -sUUID to return the UUID value of the match, vs. just the devname15:13
smoserright. i'd just change all callers to just use a 'blkid()' method and operate on thee dict response.15:13
rharperyeah15:13
rharperI forgot the dict part;15:14
rharperand a wrapper could exist to just match and return the device name like users of find_devs_with expect15:14
smoseryeah.15:14
smoserjust noticed that 'devs' in blkid in curtin does ntohing.15:19
rharperyah pylint15:22
rharperwe did *something* with it15:22
rharperI expect that was a devs filter15:22
smoserrharper, revno 221 made it do nothing15:31
smoseri swear udev is supposd to update some blkid cache.15:34
smoseror run blkid so that it updates the cache15:34
rharperit doesn;t look like it was used even with change; ie, I don't see any callers passing devs in15:34
naccthere are some blkid rules in /lib/udev, but nothing is sticking out to me15:35
naccit seems like mostly for optical?15:35
naccoh for md devices too15:36
smoserrharper, http://paste.ubuntu.com/23247253/15:46
smoserthat is a cachable_subp that seems like it shoudl generally work15:46
rharperso, my first though is that my current patches are more focused and have fewer possible side-effects;  it would be nice to see them land and then have more time with cachable subp and blkid (with cache) after? or hold off until each of those can land?15:55
smoserwell, i dont  like specifically caching _IS_CONTAINER when a much more generic solution works.15:58
rharpersurely using generalized subp cache has a wider impact;  it's not clear to me that it's safe to cache all callers of subp at this time15:59
rharperwhere as a specific caching of _is_container is straightforward and safe15:59
smoserof course its not safe to cache all callers of subp16:00
smoserchange the callers that *are* safe to use cachable_subp16:00
rharperok16:01
smoseri'm also kind of worried about the the blkid /dev/sr0 and blkid /dev/sr1 specifically16:02
smoseras you can blame them... they are basically there to kick that device16:03
smoser 812f82e7b3bad3f8127face552c76ef974b5466116:03
rharperthat's also for 2.616:03
rharperat least the comment says16:03
smoserright16:04
smoserrharper, comments on bug 162833716:05
rharperjust the module order should be needed?16:05
rharperyour pastebin had another change in util16:06
smoseroh. yeah, just the module order16:08
smoserhttps://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/30705516:08
=== Guest46101 is now known as mgagne
=== Edgan_ is now known as Edgan
harlowjaok rharper smoser i updated https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/30172918:42
harlowjawith the tiny change there18:42
harlowjai still would just like 1 simple logging location :-P18:42
smoserharlowja, i agree with that.18:42
* harlowja just likes the simple things18:44
harlowjalol18:44
* smoser doesn't even know how messages get to rsyslog anymore18:45
smoser$ ls -l /dev/log18:45
smoserlrwxrwxrwx 1 root root 28 Aug 31 12:12 /dev/log -> /run/systemd/journal/dev-log18:45
smoserthis is odd18:45
harlowjalol18:52
rharperharlowja: thanks;  I agree with smoser;  however, I do find the change in format between logging and rsyslog in the *same* file cloud-init.log to be jarring;18:57
rharperif cloud-init wanted to join with syslog, I'd at least like it to have a confirming message format18:58
rharperI don't yet know how to ensure we get consistent formatting between the two;  if we fallback to syslog, then it's a per-image/distro config mechanism; ie, what the default is for rsyslog in the image; unless we can have cloud-init emit specific format that matches the syslog format18:59
harlowjameh, i like files19:03
harlowjafiles good19:03
harlowjalol19:03
smoserharlowja, https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/30588221:01
smoser...21:01
smoserwould be nice21:01
harlowjaoh21:02
harlowjahey21:02
harlowjai know u21:02
harlowjayes, one sec21:02
harlowjareworking cloud.cfg -> cloud.cfg.ubuntu and cloud.cfg.fedora21:02
smoser:)21:03
* smoser runs now21:03
harlowjalol21:03
harlowjasmoser nrezinorn https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/30710021:58
harlowjathat also fixes why brpm wasn't working21:58
harlowja(or part of it)21:58
harlowjapart uno22:02
harlowjaof part 30022:02
harlowjalol22:02
harlowjanrezinorn i'm going to take a stab at destroying brpm22:04
harlowjathough i know u love it22:04
harlowjalol22:04
harlowjasmoser https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+ref/sys-io-errors updated22:16
harlowjain return can u fix the following22:16
harlowjahttps://gist.github.com/harlowja/4f996d19b633862eb3a647d0c6053ac922:16
harlowjai shall include 1 goat22:16
harlowja^ things that aren't working on rhel22:17

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!