| smoser | rharper, i'd appreciate your thoughts. | 01:24 |
|---|---|---|
| smoser | https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/324639 | 01:24 |
| smoser | powersj, askb the ubuntu user will be used probalby if he installs from a trunk created rpm | 01:25 |
| smoser | https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/307485 | 01:26 |
| smoser | would fix that | 01:26 |
| askb | smoser, would this fix the original issue with bug filed ? | 01:30 |
| * askb 1821: Operation in progress | 02:12 | |
| === sambetts|afk is now known as sambetts | ||
| smoser | askb, i'm sorry, did you ever get a /var/log/cloud-init.log ? | 12:48 |
| smoser | from the trunk built one would be helpful. | 12:48 |
| askb | smoser, updating the logs now | 13:00 |
| askb | smoser, wondering if the bug you mentioned above is expected to fix the original issue ? | 13:01 |
| === rangerpbzzzz is now known as rangerpb | ||
| smoser | askb, logs where ? | 13:31 |
| smoser | sorry for not following above. | 13:32 |
| askb | smoser, oh sorry! my bad ... have updated the logs on the bug filed here: https://bugs.launchpad.net/cloud-init/+bug/1692424 | 13:45 |
| ubot5 | Ubuntu bug 1692424 in cloud-init "util.py[WARNING]: Failed to disable password for user centos" [Undecided,Incomplete] | 13:45 |
| askb | I noticed earlier today, you mentioned that 307485/324639 may fix ubuntu user issue caused while running the passwd command | 13:48 |
| askb | Was curious if that fixes the issue in question and if I could help with testing the fix on our env | 13:49 |
| rharper | smoser: on the udev-settle MP, I'm still confused about the previous comment on removing blkdev re-readpt, and when I look at the diff on launchpad, line 13 still does util.subp(blkdev_cd) ? what am I missing? | 13:57 |
| rharper | als, this is the link removal; with a build of that cloud-init, I'm now passing the vlan tests that failed before | 13:57 |
| rharper | https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/324675 | 13:58 |
| smoser | rharper, see lsat comment right now | 14:08 |
| smoser | i think you'll understand :) | 14:08 |
| smoser | askb, i think selinux must be stopping cloud-init from invoking 'passwd -l' | 14:09 |
| smoser | see | 14:09 |
| smoser | Stderr: passwd: Libuser error at line: 124 - error creating `/etc/shadow+': Permission denied. | 14:09 |
| rharper | smoser: ack | 14:10 |
| smoser | rharper, do you recal something about selinux and systemd services.... | 14:10 |
| rharper | yes | 14:10 |
| smoser | there was something. maybe larsks ? | 14:10 |
| rharper | when I was debugging some centos7 issues | 14:10 |
| larsks | smoser: what where when? | 14:10 |
| smoser | thanks. | 14:10 |
| rharper | restorecon | 14:10 |
| rharper | failures | 14:10 |
| rharper | IIURC | 14:10 |
| smoser | larsks, https://bugs.launchpad.net/cloud-init/+bug/1692424 | 14:10 |
| ubot5 | Ubuntu bug 1692424 in cloud-init "util.py[WARNING]: Failed to disable password for user centos" [Undecided,Incomplete] | 14:10 |
| * larsks looks | 14:10 | |
| smoser | cloud-init is running passwd -l | 14:10 |
| rharper | if it failed to restore a file in a path, it quit recursing | 14:10 |
| smoser | and failing with permission denied. | 14:11 |
| smoser | i think i joked that the solution was to turn of enforcing :) | 14:11 |
| smoser | and i think larsks took me seriously | 14:11 |
| larsks | smoser: no, that was a "what is the topic of this discussion" as opposed to a "what are you thinking?" | 14:12 |
| sauloaislan | smoser i find this erro in my cloud-init log route info failed | 14:12 |
| larsks | Why do we think this is a restorecon failure? I don't see that in the bug anywhere. | 14:13 |
| rharper | there was an update to the library needed but wasn't released into 7.3 at the time | 14:13 |
| sauloaislan | this test with the ironic.conf network interface = neutron | 14:13 |
| larsks | There was a bug in python-selinux, yeah. | 14:13 |
| smoser | rharper, or larsks could you fill some info in on that bug if that is what you think it is ? i think you're saying there might be a bug in python-selinux ? (but cloud-init doesn't use that for invoking passwd -l to my knowledge) | 14:18 |
| smoser | so i'm kind of confused. | 14:18 |
| rharper | I don't know that your passwd -l is related | 14:18 |
| smoser | sauloaislan, more info needed. paste a cloud-init.log somewhere ? and if its openstack an network_data.json | 14:18 |
| larsks | smoser: *I'm* confused. Looking briefly at the bug I'm not sure where selinux comes up. | 14:18 |
| rharper | I saw log messages on a centos7 boot related to the restorecon | 14:19 |
| smoser | larsks, well, 'selinux' is what i thought could make a process runnign as UID=0 not able to run 'passwd -l' | 14:19 |
| smoser | the other thought was root was mounted ro, but that seemed less likely. | 14:19 |
| larsks | smoser: but I don't see 'passwd' in that bug either. Am I looking in the right one? #1692424 | 14:20 |
| smoser | well. the last cloud-init.log he's posted there. | 14:20 |
| rharper | reading it now | 14:20 |
| smoser | the title of the bug is "Failed to disable password" | 14:21 |
| smoser | which is done with 'passwd -l' | 14:21 |
| smoser | log: https://launchpadlibrarian.net/321286768/cloud-init.log | 14:21 |
| larsks | This is setting information for user "ubuntu"? Is this actually an ubuntu system? | 14:21 |
| smoser | the other errors in that log are a result of askb running with a cloud.cfg from trunk | 14:21 |
| smoser | (as the confusing user) | 14:21 |
| larsks | Anyway, if it is an selinux problem, then there would be corresponding denials logged in /var/log/audit/audit.log. | 14:22 |
| * smoser is surprised that larsks's redhat system doesnt have a ubuntu user :) | 14:22 | |
| rharper | heh, I think this rpm wasn't built right w.r.t centos | 14:22 |
| smoser | it was built from trunk | 14:22 |
| larsks | It would be interested to see those to confirm the problem. | 14:22 |
| rharper | but the centos config in the rpm (/etc/cloud/cloud.cfg) | 14:22 |
| smoser | https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/307485 | 14:22 |
| rharper | need to check that; when I was hand building rpms for centos I got that error (ubuntu user in centos) | 14:23 |
| smoser | so... some history... | 14:23 |
| larsks | It is also possible that if someone mucked about with the filesystem (e.g., file injection via virt-customize or something) that they simply forgot to relabel the filesystem. | 14:23 |
| smoser | askb ran into a bug with 0.7.5 | 14:23 |
| smoser | i said "that is really old...." | 14:23 |
| larsks | So: audit.log or it's something else | 14:23 |
| rharper | larsks: ack; | 14:23 |
| smoser | he went to try something newer by building trunk | 14:23 |
| smoser | and got cloud.cfg from trunk (which isn't really reasonable for redhat) | 14:23 |
| rharper | right | 14:23 |
| rharper | harlowja: had the split config patch | 14:23 |
| rharper | I used that in my branches | 14:24 |
| rharper | we need to pull that back end config/cloud.cfg.<distro> | 14:24 |
| rharper | s/end/in | 14:24 |
| smoser | we should get the merge above so that trunk-built cloud.cfg are good. | 14:24 |
| rharper | yes | 14:24 |
| smoser | see mp above ^ | 14:24 |
| rharper | ack | 14:24 |
| rharper | I used it about a month ago when I was doing initial passthrough testing | 14:24 |
| rharper | I still have some cloud-init rpms i hand built | 14:24 |
| smoser | askb, so, the net of all the above.... can you grab a 'dmesg' and a /var/log/audit/audit.log | 14:25 |
| rharper | let me see if I still have my rebase dir on that | 14:25 |
| sauloaislan | I dont have the cloud-init log but I have neutron-metadata-agent.log http://paste.ubuntu.com/24667212/ | 14:27 |
| smoser | sauloaislan, the log is inside the guest | 14:30 |
| smoser | /var/log/cloud-init.log | 14:30 |
| larsks | smoser: fyi, this is the selinux-python issue: https://bugzilla.redhat.com/show_bug.cgi?id=1406520 | 14:43 |
| ubot5 | bugzilla.redhat.com bug 1406520 in libselinux "calling libselinux python restorecon fails on /var/lib/nfs/rpc_pipefs" [High,Verified] | 14:43 |
| rharper | yes | 14:44 |
| smoser | larsks, thank youi | 14:46 |
| === sambetts is now known as sambetts|afk | ||
| smoser | rharper, https://code.launchpad.net/~xnox/cloud-init/+git/cloud-init/+merge/324010 | 17:23 |
| smoser | can you change your review there | 17:23 |
| smoser | as he updated test cases. | 17:23 |
| smoser | oh. wait. that wasnt your complaint. | 17:23 |
| smoser | or may be it was. | 17:23 |
| smoser | i have the larger set of improvement in my branch | 17:23 |
| smoser | but his does fix an issue for him, at low regression risk and want to get that bug fix in | 17:24 |
| smoser | mine is more invasive | 17:24 |
| smoser | rharper, https://code.launchpad.net/~farcaller/cloud-init/+git/cloud-init/+merge/324273 you could change your NEEDS fixing too | 17:27 |
| paulmey | Hi all, some team here on Azure is testing some changes and has an issue with Ubuntu core not starting cloud-init | 18:00 |
| paulmey | they have modified the way they are generating the provisioning iso, but I don't exactly know how snappy/systemd pick that up in ubuntu core | 18:01 |
| paulmey | Is this a good place for that question? | 18:01 |
| smoser | hey | 18:06 |
| smoser | sorry. | 18:06 |
| smoser | paulmey, yeah. | 18:07 |
| smoser | so... there are some issues still outstanding with getting ubutnu core to r un cloud-init. | 18:07 |
| smoser | rharper, has been fighting that fight and can fill you in. | 18:07 |
| paulmey | :-) | 18:07 |
| smoser | all the changes we think are necessary are in cloud-init currently and i intend to sru that to 16.04 | 18:08 |
| smoser | (starting today) | 18:08 |
| paulmey | yeah I saw a couple of commits in cloud-init in that direction | 18:08 |
| smoser | but there are some changes in the image build process that have to be made (in addition to getting this newr version of cloud-init) | 18:08 |
| paulmey | we have core 16 running on Azure... | 18:08 |
| paulmey | but I'm trying to figure out what change causes it to break now... | 18:09 |
| paulmey | I don't know enough about how snappy and systemd work together to get the system up... | 18:09 |
| paulmey | for some reason, the whole cloud-init.target doesn't get run... | 18:09 |
| paulmey | not sure what that would be conditional on... | 18:10 |
| smoser | its explicitly disabled in the image. | 18:10 |
| smoser | i thikn they touch /etc/cloud/cloud-init.disabled | 18:10 |
| paulmey | oh ok | 18:10 |
| paulmey | nah I checked that | 18:10 |
| rharper | oh *sigh* | 18:10 |
| paulmey | lol | 18:10 |
| smoser | funny, eh. | 18:10 |
| rharper | well, the second is the bug w.r.t detection of snappy | 18:10 |
| rharper | and then we may have other items that come up w.r.t. the network configuration that's baked in that we need to remove that was having some conflicts... | 18:11 |
| rharper | paulmey: depending on which versions of the core.snap are included, we have to bake in a few cloud-init configurations that ensure that the cloud identification is found which will enable cloud-init units to run; | 18:12 |
| paulmey | Odd_Bloke: you bake this image, right? :-) | 18:12 |
| paulmey | https://www.irccloud.com/pastebin/RrttwRMj/ | 18:13 |
| rharper | hrm, well, Odd_Bloke and I worked together getting something working; | 18:13 |
| paulmey | I see a whole bunch of these "/usr/bin/snap[825]: cmd_auto_import.go:198: error: cannot mount /dev/..." errors in the journal... | 18:15 |
| rharper | those should have patched parts which do the right thing; well, then next I would suspect the cloud id bits; typically in that image if cloud-init doesn't run, you can still login to it via serial console interaction; not sure if that's possible in your case though. | 18:15 |
| rharper | noisy warnings | 18:15 |
| rharper | if you can get into the image then getting to see /run/cloud-init/* would be useful to see if we didn't identify the platform | 18:16 |
| paulmey | yeah, I've hacked my way in through the console | 18:16 |
| rharper | the UC16 images run in strict mode | 18:16 |
| rharper | is it possible that your platform doesn't identify itself in the known way ? | 18:17 |
| paulmey | https://www.irccloud.com/pastebin/TiZP9HLY/ | 18:19 |
| paulmey | yep, that;s the tip I needed, I think | 18:19 |
| rharper | paulmey: ok | 18:19 |
| rharper | please ping me if you run into any more of those | 18:19 |
| rharper | I've spent *way* more time that anyone should have | 18:19 |
| rharper | so I hopefully can prevent you from wasting any time | 18:20 |
| paulmey | I'll see if I can find out why ds-identify has rc=1 | 18:20 |
| rharper | it's likely the mounts | 18:20 |
| paulmey | it's the cdrom label... | 18:26 |
| paulmey | in the error case it is labeled DVD_ROM | 18:26 |
| paulmey | has_fs_with_label "rd_rdfe_*" && _RET="$name" && return ${DS_FOUND} | 18:26 |
| paulmey | ok, thanks, I can tell them how to fix it now... | 18:26 |
| rharper | great | 18:31 |
| smoser | wait.. | 18:41 |
| smoser | was there a bug in ds-identify paulmey ? | 18:41 |
| paulmey | not necessarily, that is how you look at it... | 18:41 |
| paulmey | this team of ours is changing the way they generate the iso | 18:41 |
| paulmey | and apparently they were not giving it the same label | 18:42 |
| paulmey | if they can't maintain the same label, I'm sure we'll propose a change to ds-identify | 18:42 |
| paulmey | but it will take time for that to percolate into the image in the gallery as well... | 18:43 |
| smoser | paulmey, cool. | 18:52 |
| smoser | i'd definitely be open to more specific label... or any other way of identifying "You are running on azure" | 18:53 |
| smoser | blackboxsw, i'm running a integratino test on trunk, and then will prepare an upload to artful and x, y, z. so witth that i'll ahve a large list of sru bugs | 19:00 |
| smoser | (horay) | 19:00 |
| smoser | other requests, if you're bored | 19:00 |
| smoser | - need unit test for | 19:00 |
| smoser | https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/324274 | 19:00 |
| smoser | - need further review on smoser mask2cidr | 19:00 |
| smoser | https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/324677 | 19:00 |
| smoser | - need further review on smoser aliyun | 19:00 |
| smoser | https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/324625 | 19:00 |
| powersj | :) | 19:00 |
| smoser | powersj, its ok if i can just push 'go' ? | 19:02 |
| smoser | https://jenkins.ubuntu.com/server/view/Cloud-init/job/cloud-init-integration-a/ | 19:02 |
| smoser | and | 19:02 |
| smoser | https://jenkins.ubuntu.com/server/view/Cloud-init/job/cloud-init-integration-x/ | 19:02 |
| smoser | can i do them at the same time ? | 19:02 |
| powersj | if you run -x it will autokick off y then z then a | 19:02 |
| powersj | if you want results faster than sure just kick off a and x | 19:02 |
| powersj | and a will get run twice, but that's ok | 19:03 |
| powersj | and yes running at the same time is fine | 19:03 |
| smoser | o. cool | 19:04 |
| smoser | i'll just push 'x' then | 19:04 |
| smoser | larsks, just an fyi, after blackboxsw and I both individually broke unit tests on centos6, i asked powersj to get c-i reporting on merge propsals with a run in a centos6 container. so that will hlpefully stop us from breaking you so often. | 19:07 |
| larsks | smoser: also centos7? | 19:08 |
| powersj | I will do both | 19:08 |
| smoser | powersj, ? i think we can do that. centos6 was the big one though as it is different in 2 ways | 19:08 |
| smoser | a.) python2.6 | 19:08 |
| smoser | b.) not ubuntu | 19:08 |
| powersj | We do have daily centos6/7 build and unit test runs | 19:08 |
| powersj | Need to add that to our merge request reviews | 19:09 |
| smoser | right. | 19:09 |
| larsks | powersj: that would be awesome. | 19:09 |
| powersj | and not run as root :) | 19:09 |
| smoser | we should point out.... its not "real" root. | 19:09 |
| smoser | runs in a unpriviledged container | 19:09 |
| smoser | but the container thinks its root | 19:09 |
| larsks | better than nothing! should at least help highlight issues generating config files or path assumptions, etc. | 19:10 |
| blackboxsw | +1 smoser on sru work, I'm adding one more unit tests for ImportError of jsonschema and then my branch is out of the way. Moving on to your review requests right after | 19:25 |
| blackboxsw | I restructured the auto-gen'd docs to include Examples for the schema | 19:25 |
| smoser | \o/ | 19:34 |
| rharper | https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/324702 | 19:36 |
| rharper | that's needed for pass-through equivalency with curtin.net | 19:36 |
| smoser | http://paste.ubuntu.com/24670061/ | 19:53 |
| smoser | blackboxsw, ^ there is our bug list | 19:53 |
| paulmey | smoser: there are a couple of ideas floating around in Azure to change the source of the provisioning info and/or the recognizability (if that's a word) of the Azure environment... | 19:56 |
| paulmey | none of these have solidified enough to take action, but I'll keep my eye on them... | 19:56 |
| smoser | paulmey, yeah. thats good. it really does make sense from a cloud platform perspective to make yourself easily recognizable. | 20:00 |
| blackboxsw | excellent smoser want me to start on that now or reviews. I've just pushed final changes to cc-ntp-schema-validation | 20:00 |
| smoser | blackboxsw, well, i guess the bug list would be good to start on | 20:01 |
| blackboxsw | starting the bug list | 20:01 |
| blackboxsw | https://trello.com/c/VxjOm2E5/109-upload-to-artful-and-srus or something else? | 20:01 |
| smoser | blackboxsw, twiddled all the ticky marks | 20:37 |
| smoser | https://bugs.launchpad.net/cloud-init/+bug/1685935 | 20:37 |
| ubot5 | Ubuntu bug 1685935 in cloud-init "make deb breaks due to missing debuild script" [Medium,Fix committed] | 20:37 |
| smoser | that one i referenced in the changelog but shouldnt have. | 20:37 |
| blackboxsw | roger smoser | 20:54 |
| smoser | i'm heading out. | 20:54 |
| smoser | have a nice weekend all. | 20:54 |
| blackboxsw | have a good one, thanks | 20:54 |
| blackboxsw | smoser: see you tuesday. forgot monday was a holiday | 20:56 |
| === rangerpb is now known as rangerpbzzzz | ||
| paulmey | rharper, smoser : I found a better way to identify Azure and created a tracking bug here: https://bugs.launchpad.net/cloud-init/+bug/1693939 | 22:57 |
| ubot5 | Ubuntu bug 1693939 in cloud-init "Switch Azure detection to use chassis_asset_tag" [Undecided,New] | 22:57 |
| paulmey | have a great weekend all | 22:57 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!