/srv/irclogs.ubuntu.com/2016/09/30/#cloud-init.txt

smoserharlowja, well, i fixed the ones i fixed00:01
harlowjalol00:01
smoserthe 2 i fixed with ip00:02
harlowjathx :)00:02
smoserthey are i'm pretty sure either00:02
smosera.) you have no output of 'ip'00:02
smoseror (i think)00:02
smoserb.) you have not 'ip' in your path00:02
smoserbecause redhat typically does noth ave /sbin in path00:02
smoserbut either way00:02
smoserthe next one i think is just that have dns hijacking going on on that host00:02
harlowjacould be00:02
smosersee is_resolvable00:03
smoserit tries to work around that stuff00:03
smoserbut apparently not enough there.00:03
smoserso...00:03
smoserhelp me get a cent6 / cent7 container00:03
harlowjai only know VMs00:04
harlowjalol00:04
smoserwell, i have a container00:04
harlowjaVM00:04
smoseryou just get it so things work00:04
harlowja:)00:04
harlowjaagreed00:04
harlowjawill get that in a few, templating cloud.cfg00:04
smoserwell https://public.etherpad-mozilla.org/p/cloud-init-centos-unittest00:05
smoserright now a bunch of stuff fails for me00:05
harlowjakk00:05
harlowjapaage still loading00:05
harlowjalol00:06
smoseri have a node up 45.55.168.7700:06
smoserthat you can go to ubuntu@00:06
harlowjak00:06
smoserand lxc is there...00:06
harlowjacubswin?00:06
harlowjalol00:06
smoserssh keys.00:06
harlowjak00:06
smoser10100:06
smoseris cubs win total00:06
smoserkind of like lol00:06
harlowjahaha00:06
harlowjaclose enough00:06
smoserdo you use python-mock from the distro ?00:08
smoseri think its too old.00:08
harlowjai usually run tox which then installs all the things00:09
harlowjaso nothing from distro00:09
smoserhm.00:13
smoserwell, i couldnt run tox00:13
smoserit didn't work either.00:13
harlowjahmmm, did u install tox ?00:13
harlowjahow much did work?00:13
smoserfrom rpm ?00:13
smoserer.. yum00:13
harlowjaor from pip00:13
smoserhttp://paste.ubuntu.com/23253475/00:15
harlowjaah, yes, pip install setuptools --upgrade also00:15
harlowjastupid old busted00:16
harlowjalikely also pip install virtualenv --upgrade to00:16
harlowjai believe virtualenv has the bundled version of setuptools that tox then uses00:16
harlowjabut i forget00:16
harlowja pip install setuptools tox virtualenv --upgrade00:17
smoserstill issues00:17
smoserchecking that.00:17
harlowjak00:17
harlowjaand keep on saying to yourself 'old stuff is more stable'00:17
harlowjalol00:17
smoserso essentially if this works, i'm basically running in tox00:17
harlowjaya00:17
smoserwhich...00:18
smoserprotects me from most things00:18
harlowja*for testing00:18
smoseri'd like to run nosetests with the versions of things that are in the distro00:18
harlowjaprobably need to cap mock version then00:19
harlowjawhich is prob 1.0.1 on epel00:21
harlowja*epel 700:21
harlowjaand 0.8 on epel 6 :-/00:21
harlowjadon't forget keep on saying to yourself 'old stuff is more stable'00:21
harlowjalol00:21
smoserwell, i guess i dont care too much about the mock version00:22
harlowjaya, as long as it works, which it should00:22
smoserright. i'm fine to have newer of that, but if we're using a library we want to ouse the version in the distro.00:23
smoseri like that:00:23
smoser yum install valid-package invalid-package valid-package200:23
harlowjayum is sorta retarded...00:23
smosergoes and installs everything and then somewhere in the log it says 'invalid-package not available'00:23
smoserrather than jsut saying "um... can't do that"00:23
harlowjaya, i think the exit code is also 0 for that case00:23
harlowjafrom what i remember00:23
harlowjalol00:23
harlowjaso u can't detect it in bash scripts00:24
harlowja(at least not via exit codes)00:24
harlowjahopefully dnf fixes that00:26
smoserFAILED (SKIP=28, errors=30, failures=1)00:27
harlowjanice, more than with tox it seems, lol00:27
smoserso thats in my cent6 after pip install and then tox00:27
smosersome recently regressed. :-(00:27
smoserTypeError: decode() takes no keyword arguments00:27
harlowjaah, ya, that one00:28
harlowjadidn't i fix that00:28
harlowjai forget00:28
harlowjalol00:28
smoserhow do you fix that ?00:28
harlowjacent6 and py26 are awesome00:28
harlowjai think u can just not give a keyword argument :-P00:28
harlowjaand just do it positional argument only00:28
harlowjaya, just positional should be fine00:29
harlowjaS.decode([encoding[,errors]]) -> object00:29
harlowjaprob could do encoding=encoding in py27 only or something00:29
smoseroh. right.00:29
harlowja'old stuff is more stable'00:29
smoserok. with taht change, now00:29
harlowjalol00:29
smoserFAILED (SKIP=28, errors=3)00:29
harlowjacool00:30
* powersj thinks we should probably get CI unit tests going again as they fail right now00:30
harlowjawhats the last 300:30
harlowja'old stuff is more stable' 'old stuff is more stable' 'old stuff is more stable'00:30
harlowjalol00:30
smoserhttp://paste.ubuntu.com/23253529/00:31
smoserk. first is easy enough (content)00:31
harlowjaya, the rest are dict comphrenhsions00:33
smoserset comprehension on one of them :)00:33
harlowjaoh ya00:33
harlowjaya, just turn those into set(iterable) and dict(iterable)00:33
harlowjaand that's all those become00:33
harlowjaso not so bad00:34
smoserhttp://paste.ubuntu.com/23253540/00:34
harlowjacool u can even do00:35
harlowjaset(m['uri'] for m in f['apt']['primary'])00:35
harlowjaif u really care00:35
harlowjaeither will be fine00:35
harlowjano need for intermediary list00:35
smoseroh.00:36
smoserhow is that00:36
smoseroh. ididnt know you coudl00:36
smoseri like that00:37
harlowja:-p00:40
harlowjam['uri'] for m in f['apt']['primary'] is a generator00:40
smoseryeah, but only if wrapped in parens00:41
smoser$ python -c 'm for m in (1,2,3)'00:41
smoser  File "<string>", line 100:41
smoser    m for m in (1,2,3)00:41
smoser        ^00:41
smoserSyntaxError: invalid syntax00:41
smoser$ python -c '(m for m in (1,2,3))'00:41
smoserhappy00:41
harlowjaright00:41
harlowjaso u put it in parans00:41
harlowjaha00:41
harlowjaset( )00:41
harlowjalol00:41
smoserit is kinda wierd like that00:42
smoserthat the parens to the function call suffice00:42
smoserthat kind of hurts my brain00:42
harlowjaya, its probably a weirdness in the python syntax that allows for it00:43
smoserhow should i say "what version of centos am i on"00:45
smoserie, i want to know 6 or 600:45
smoseror 700:45
smoseralright. good.00:47
harlowja in /etc/redhat-release i think00:48
harlowjaor lsb_release i think has something00:48
harlowjaor python has some stuff u can use00:49
smoserhttps://public.etherpad-mozilla.org/p/cloud-init-centos-unittest00:49
smoserso... given that patch above, i can run tox in cent6 or cent700:50
smoserthats good, but ideally we'd be able to run 'nosetests' against distro-installed versions (as would be found in a runtime)00:50
harlowjaya, that may require a little more version tweaking00:50
smoserand the 'build' thing mostly worked... at least used to.00:51
smoserso, tahts good. thanks harlowja00:51
harlowjaya, there is a file or 2 that is missing for brpm00:51
harlowjabut i'm hoping with nrezinorn that brpm goes away00:51
smoseryou seem my comments https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/30588200:51
smoseri have to run00:51
harlowjaya00:51
smoserthanks for your help jxharlow00:51
harlowjawhos that00:51
harlowjalol00:51
smoseryou changed your middle name when you moved to godaddy00:52
harlowjaJXMenHarlow00:52
smoseris that because harlowja.coolguy was taken?00:52
smoserbut you could still get harlowjx00:52
harlowjanah, i asked and they just said, meh that's what u got00:52
harlowjai asked about harlowja, and it seemed like alot of work00:52
harlowjaso i just gave up00:52
harlowjalol00:52
harlowjahttps://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+ref/kill-brpm (where brpm goes away)00:53
harlowjait seems to work on cent7 at least, ha00:53
harlowjanrezinorn just wants a spec file, lol00:53
smoserbut i like brpm00:53
* harlowja runs away00:53
harlowjalol00:53
smoseras long as some way we can make an rpm00:54
harlowjaya00:54
* smoser out00:54
smoserlater00:54
=== shardy is now known as shardy_lunch
=== shardy_lunch is now known as shardy
=== rangerpbzzzz is now known as rangerpb
smoserharlowja, https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/30733315:39
smoserwhat do you think about that ?15:39
avshHi, I am using vmware with openstack15:54
avshVirtaul CD-Rom which used for config drive is not removed after Machine is provisioned15:54
avshLeaving admintrative password and other data as clear text in ( DVD-DRIVE config-2 )15:54
avshwhat configuration is need for cloud init to remove the drive when vm is provisioned?15:54
avshcan anyone please help me with my query?15:55
smoseravsh, do you have non-root users who can mount a cdrom ?16:03
avshyes16:03
smoserto my knowledge, cloud-init couldn't on its own rid the system of that drive. you could 'eject /dev/cdrom'16:04
smoserand then it would not be there, but possibly a 'eject -t /dev/cdrom' would pull back in the tray and have it again16:04
smoserand if it is Read-only media, then cloud-init can't write it to blank it16:04
smosercan you give the file that has the password in it ? it'll help me diagnose where to look to see what esle could be done.16:05
avshec2/16:11
avshopenstack/2012-08-10/16:11
avshopenstack/2013-04-04/16:11
avshopenstack/2013-10-17/16:11
avshopenstack/content16:11
avshopenstack/latest/meta_data.json16:11
avshopenstack/latest/user_data16:11
avshopenstack/latest/vendor_data.json16:11
avshThis is the Folder Structure16:11
avshand openstack/latest/meta_data.json has contents like { "admin_pass": password, "random_seed": "*******" }16:11
smoseroh. well, good for you that cloud-init doesn't care what is in there16:20
avshsmoser, let me know if you need any info16:20
smoserit ignores it16:20
smoseras on linux, ssh keys are preferred.16:20
avshbut users can see other sensitive information which is a security concern16:21
avshlike if i install a software on the vm, that software password is an example16:21
smoseravsh, do you have a reason to let users mount that disk ?16:25
smoserwhy not just remove users from the 'cdrom' group16:25
avshwe can do that. we don't see this issue with Openstack + KVM16:26
avshonly with Vmware + Openstack16:26
avshtrying to understand, it is configuration with cloud-init or vmare nova driver16:27
smoserjust because you're not looking in the right place :)16:27
smoseri suspect the same data is availble in http://169.254.169.254/openstack/16:27
smoserand any malicious user already knew that.16:28
avshok16:28
smosercloud-init can route off the that particular address so that only root would be able to get at it16:28
smoserwith16:29
smoser disable_ec2_metadata: true16:29
avshok, let me check on kvm instance with the url you provided16:30
avshsmoser, you made my day16:40
avshyou are correct, I am not seeing at right place with kvm + openstack16:40
avshIt can access all the data with the above url16:40
avshI can rule out vmware opensstack nova drive16:40
smoseravsh, alternatively you can do things in a different way.16:41
smoseryou can use '#include-once' to include other cloud-config things... and make those expiring or one-time-read urls16:42
smoserthe metadata services are not intended to be secure16:42
avshsmoser, I will check on the #include-once, thanks17:00
smoserrharper, around ?17:45
smoseri've 2 things for you... one. do you have a readthedocs.org account (or can you get one).17:45
smoser2.  http://paste.ubuntu.com/23256482/17:45
rharpersmoser: here19:03
rharperI have a rtd account19:03
rharperlemme get on (2)19:03
rharperwhat am I looking at with (2) ?19:04
smoserwhat is rtd account ?19:06
smoseri will share access to cloud-init project19:06
rharperah, I see , a slow ish boot; total time is 11 seconds though19:06
rharperread-the-docs19:06
rharperah, right19:07
smoserrharper, yeah....19:07
smoserum more interesting than that.19:07
rharperraharper19:07
smoserthat was a 2+ minute boot19:07
smoser:)19:07
rharper17:32:37 to 17:32:5619:07
rharperit's not cloud-init log19:07
smoseryeah.19:08
rharperthat's like 20 seconds wallclock19:08
smoserthats what is fun19:08
rharperso something else (look at systemd-analyze blame19:08
rharpercan I haz ssh to ami ?19:08
smoseri think clock is moving backwards19:08
rharperoh, ntp!19:08
smoseryou can... yeah19:08
smoserlet me set access up for you through my bastion19:09
smoseri assume its reproducible on serverstack19:09
smoserbut19:09
rharperok19:09
rharperdmi data /sys/class/dmi/id/product_name returned OpenStack Nova  -- that log is not from AMI on EC2 ..19:10
smoserright19:11
rharperif it's kernel related, it's possible that it could be reproduced in sstack; however, given that the virt layer is going to handle memory differently (booting xen on ec2, vs kvm on openstack) that may mean we won't reproduce the same amount of slowdown;  the kernel but that's reference has to do with SLAB/SLUB config and other changes19:16
smoserrharper, ssh-via proxy-user@10.245.162.60 ubuntu@10.5.0.18519:22
smoserfirst ip is my bastion. i set you up to jump through there.19:22
smosersecond is the system.19:22
smoserssh-via is http://smoser.brickies.net/git/?p=tildabin.git;a=blob;f=ssh-via;19:22
rharperblob_plain is what I want19:23
rharpersmoser: in19:27
rharperSep 30 17:31:33 ubuntu systemd[1]: Time has been changed19:29
rharpersomething *did* reset the clock19:29
rharperSep 30 17:31:33 ubuntu systemd-timesyncd[556]: Synchronized to time server 91.189.89.198:123 (ntp.ubuntu.com).19:30
rharperSep 30 17:30:42 - prior event19:30
rharperSep 30 17:31:33 - time has changed19:31
rharperso, slow clock moved forward by just under a minute19:31
rharperand then, ntp syncs it and moves it another minute forward, Sep 30 17:32:2319:32
rharperbrb19:33
smoserrharper, its wierd though that cloud-init's logging didn't see that.19:37
smoserrsyslog must be doing it ? and it keeping its own clock or something?19:37
rharpersmoser: it's in journctl19:56
rharperthe time change happened async from cloud-init execution19:56
rharpernot quite sure how journctl keeps track of time vs. python logging/rsyslog19:56
rharpersmoser: note the odd delta between the entry timestamp (Sep 30 17:32:37, vs the timestamp collected for the welcome message: at Fri, 30 Sep 2016 17:30:39 +0000)19:58
smoseryeah. its wierd.19:59
smoserand systemd is confused by this19:59
smoseras it *does* say cloud-init took 2 minutes to run19:59
smoserwhen i'm pretty sure watching a wall clock that is not the case.19:59
rharpercorrect20:00
rharperI was going to do the relative time between events in ci and I'm positive it didn't take all that time (rather we've got a clock jump)20:00
rharperI suspect if you uncloud-init data , reboot20:01
rharperit won't be as a long20:01
rharperI wonder why the VM clock is so far off though20:01
rharper2 minute adjustment is pretty large20:01
=== rangerpb is now known as rangerpbzzzz
rharperyou launched me at: Fri, 30 Sep 2016 17:29:59 +000020:06
rharperkernel booted     : Fri, 30 Sep 2016 17:30:30 +000020:06
harlowjasmoser interesting if u haven't seen it20:06
harlowjahttps://cloud.google.com/compute/docs/containers/vm-image/#using_cloud-init20:06
smoserrharper, yeah, i agree.20:08
rharperso, I don't know what to do unless timectl could tell us how much time was adjusted20:09
rharperit seems like it's a systemd bug too20:09
smoserreally sucks20:09
rharpersince analyze didn't update (acknowledge that ntp changed time)20:09
rharperif it knows the timedelta, it could apply that20:09
smoserin cloud-init we could read uptime20:09
rharperto provide true time20:09
rharperdespite clock shift20:10
smosertheres a way to get that i think rather thatn /proc/uptime20:10
smoser(althoguh /proc/uptime is mocked in a container for us, and a kernel interface probably isnt)20:10
rharperright20:11
smoserhttps://github.com/xmonader/linuxsysinfo/blob/master/sysinfo.py20:13
rharperyeah, the procfs is slightly slow in container20:15
rharperthe cloud-final message is always high on blame due to reading sysinfo20:15
rharperit's relatively slow to other modules20:16
smoserrharper, well, we could read via sysinfo.20:19
rharpersyscalls, yeah; but that'd be host data, right ?20:19
rharpersorta like dmesg20:19
smoserprobalby could even read once from /proc/uptime20:19
rharperit's just not right20:19
smoserand then count that offset20:19
smoser:)20:19
rharperright, proc/uptime once and caching that would be useful20:19
smoserand then form then on out ask the sysinfo20:20
rharperbut the issue is the 4 invocations of cloud-init20:20
smoserwell, 4 reads of /proc/uptime is probably "not that bad" in the grand scheme of all the bad things.20:20
rharperbut from exec to exit, we could do sysinfo for time;  but honestly; I think rdtsc is likely faster for absolute cycles20:20
rharpersmoser: it's the hottest thing left on lxd reboots20:20
smoserwell then.20:20
rharperit's not bad at all20:21
smoser:)20:21
rharperwe're at 0.25 second reboot20:21
rharperbut it's still roughly 30% of that in cloud-final message20:21
rharperso, if we could reduce that , then we'd see sub .2 second reboots20:21
rharperrdtsc would be faster than call to proc due to fuse20:22
rharperhrm, the monotonic clock jumps too when time is set; I suppose that's expected... but that sounds wrong20:28
rharpersmoser: so, can we see if /var/lib/systemd/clock exists in the cloud-image ?20:37
rharperah, it doesn't (at least the lxd rootfs doesn;'t have it)20:38
smoseras in is it created at runtime you mean ?20:39
smoserversus already present?20:39
rharperright20:39
rharpertimesyncd uses that as a restore clock to this as soon as it starts (if it exists)20:39
smoseroh.20:40
rharperthat could be a jump but it would have been backwards quite a bit20:40
smoserrestore from what ?20:40
rharpereach sync with ntp, that file is updated with the last good stamp from ntp20:40
rharperhttps://lists.freedesktop.org/archives/systemd-devel/2015-May/031988.html20:40
rharperIt20:40
rharper  implements sNTP and will sync the last known time to disk every time20:40
rharper  it gets an sNTP sync or the system is shut down.20:40
rharperAt boot it uses20:40
rharper  that time to reinitialize the clock, as early as possible, before20:40
rharper  NTP is done. THis will give you monotonic time which should solve20:40
rharper  your probelm.20:40
rharperso, the journal, and other loggers on the system use CLOCK_MONOTONIC, which is susceptable to NTP changes in clock (always forward)20:41
rharperso, AFAICT, this is just a really out of sync clock on the host20:41
smoserwhich *is* susceptable ?20:49
smoseror is not succeptable20:49
rharperis20:50
rharperCLOCK_MONOTONIC_RAW is not20:50
smoserhm.20:51
smoserharlowja, https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/30736320:51
smoserpowersj, that gives us a way (via the gist linked there) to run uni tests in centos6 pretty easily20:52
smoserrharper, raharper is now a maintainer of CloudInit in readthedocs20:58
smoserand fyi, magicalChicken's doc fixes are there now!20:58
smoser(it had stopped building when we moved from bzr)20:59
rharpersmoser: cool, i see it21:00
* smoser has to run21:01
smoserharlowja, if you could look at that... you can merge it if you want21:02
smoserit seems nice.21:02
harlowjakk21:02
harlowjacools21:02
smoserlater.21:02
powersjsmoser, very cool21:03
harlowjawoah21:03
harlowjanice nice21:03
harlowjaall the modules got filled in???21:03
harlowjasweet21:03
harlowjathat only took a couple years to finish, lol21:04
harlowjathx guys! :)21:04
jgrimmmagicalChicken, ^^21:10
jgrimmharlowja, smoser: so in the gce link, it mentions that they've implemented setting UID. Any particular reason that hasn't been implemented in base cloud-init up to now besides priorities?21:14
harlowjai forget21:14
harlowjai thought i remember a patch for that21:14
harlowjalol21:14
jgrimmbug for it. https://bugs.launchpad.net/cloud-init/+bug/139636221:14
jgrimmyeah, i remembered seeing it in the backlog21:15
harlowjaunsure about that one21:15
harlowjawhy is it a diff file21:15
harlowjalol21:15
harlowjadid they not sign the CLA21:15
harlowjaweird21:15
jgrimmwhen i saw it in the gce doc, reminded me that i'd seen it requested before21:15
harlowjaya21:15
powersjsmoser, I setup cloud-init to run unit tests 2x a day now across the architectures. It will git clone master, figure it is better than noting.21:22
powersjI can figure something out for the new centos one too when I get back home21:23
powersjand since it does not respond to merge requests, but just does master, it will email us on failures, hence the mail you probably just got21:24

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!