| kalyan | hi all ... i am facing some issue with cloud init on suse.... is anyone available now to help me? thanks in advance ! | 12:04 |
|---|---|---|
| kalyan | i have installed cloud-init version 0.7.6 ... and i am seeing cloud-init-local or config status as unused | 12:04 |
| kalyan | could someone please help/correct me if i missed something..thanks | 12:04 |
| smoser | kalyan: cloud-init-local does need to run. | 12:31 |
| kalyan | smoser: Thanks, but when i tried starting it ..i am seeing the message "failed" | 12:37 |
| kalyan | smoser: just curious.. so it will be in "unused" state? | 12:42 |
| smoser | what is the init system ? | 12:49 |
| kalyan | cloud init 0.7.6 on suse | 12:50 |
| kalyan | suse 11 | 12:50 |
| smoser | kalyan: the init system.. sorry i'm not familiar enough to know | 12:52 |
| smoser | upstart ? systemd? sysvinit ? | 12:52 |
| smoser | you'll have a file on your system: | 12:52 |
| smoser | /etc/cloud/cloud.cfg.d/05_logging.cfg | 12:52 |
| smoser | you proably have a line like | 12:52 |
| smoser | - [ *log_base, *log_syslog ] | 12:53 |
| smoser | in it. | 12:53 |
| smoser | comment that line out | 12:53 |
| smoser | then re-run and pastebin /var/log/cloud-init.log | 12:53 |
| kalyan | ok sure | 12:53 |
| do3meli | hey smoser. do you have some time to look at some code in https://code.launchpad.net/~d-info-e/cloud-init/+git/cloud-init/+merge/340220 ? | 13:08 |
| do3meli | i am struggeling to find a nice way to easy parse a file that can have 2 different layouts. | 13:09 |
| do3meli | while writing tests for this branch i realized that the output of "mount" command can be different - so we have to take care of both layouts | 13:10 |
| smoser | do3meli: sure | 13:19 |
| smoser | do3meli: well i guess the one thing is to have some intermediate object there. | 13:21 |
| smoser | def parse_mounts(): | 13:21 |
| smoser | """Return a representation of mounts on the system." | 13:21 |
| smoser | if util.isFreeBSD(): | 13:21 |
| smoser | return one thing | 13:21 |
| smoser | else | 13:21 |
| smoser | another | 13:21 |
| smoser | then use that thing | 13:22 |
| smoser | but maybe that doesnt help you | 13:22 |
| smoser | unrelated: dont bother passing the LOG thing around . i just didn't really understand python logging when i put that stuff in (handle taking a 'log=' param) | 13:23 |
| smoser | just use LOG.debug or LOG.warning | 13:23 |
| smoser | oh. i see so we do have 'parse_mount' . and that returns a dev path, fstype and mount point. | 13:24 |
| do3meli | ahh did not know that. my pycharm always told me its not so nice to use a variable that has not been passed. | 13:25 |
| do3meli | i may can get rid of get_mount_parse_regex() i am just about to put together a better regex that makes it obsolete | 13:27 |
| do3meli | i have this regex now that matchs both formats: | 13:29 |
| do3meli | https://regex101.com/r/2F6c1k/1 and the other format https://regex101.com/r/T2en7a/1 | 13:30 |
| do3meli | the problem now is that the group id's are differnet. for the first one the fs_type is group id 4 and for the second one its group id 3. | 13:32 |
| smoser | do3meli: you can do names in your regex | 14:29 |
| smoser | and reference them by name | 14:30 |
| smoser | do3meli: (?P<name>...) | 14:30 |
| smoser | https://docs.python.org/3.4/library/re.html | 14:30 |
| rharper | groupdict() FTW | 14:31 |
| do3meli | let me try that. this would make things much easier ;) | 14:45 |
| do3meli | not sure that works in combination with a positive lookahead | 14:50 |
| do3meli | or the other way around: having the if and else clause the same group name does not work. it says "name must be uniqe" | 14:51 |
| do3meli | see this here: https://regex101.com/r/2F6c1k/2 | 14:55 |
| smoser | do3meli: thats neat. i'd nto seen that. | 16:19 |
| do3meli | i think i found a solution in the meanwhile smoser | 16:19 |
| do3meli | refactoring that whole stuff now ;-) | 16:19 |
| smoser | great. thanks. | 16:20 |
| smoser | do3meli we really apprecate your contributions | 16:20 |
| do3meli | thanks :-) | 16:20 |
| do3meli | saw the monday protocol btw. was not able to attend unfortunately. and most likely can't attend the next one too | 16:21 |
| smoser | blackboxsw: https://code.launchpad.net/~kgarloff/cloud-init/+git/cloud-init/+merge/341844 | 16:43 |
| blackboxsw | smoser: just saw you merged it | 16:43 |
| smoser | merged that with | 16:43 |
| blackboxsw | was bringing that up. will respond on dojordan's branch | 16:43 |
| smoser | review-mps -v --project-name=cloud-init --git-user=smoser --merge | 16:43 |
| smoser | but the related bugs did not get marked fix-committed | 16:43 |
| blackboxsw | ahh, something to tackle today. I'll re-run it and make sure the bugs get marked by review-mpos | 16:44 |
| blackboxsw | ahh, something to tackle today. I'll re-run it and make sure the bugs get marked by review-mps | 16:44 |
| blackboxsw | got it. | 16:51 |
| blackboxsw | running now and pushing the regex improvemnt. needed to match \s* at end of line not \s+ | 16:52 |
| blackboxsw | dojordan: welcome, sorry for the delay, I'm posting a minor round of review comment to your azuretimeout branch and testing it today. something landed in trunk today that affects the exception_cb passed to readurl. | 16:59 |
| blackboxsw | .. and sorry for IRC pouncing | 16:59 |
| dojordan | no worries, its why im here! | 16:59 |
| blackboxsw | I'll have a patch suggestion for you shortly to see what you think. unless you beat me to it after you git fetch; git rebase master | 17:00 |
| blackboxsw | basically exception_cb to readurl needs to return True to retry | 17:02 |
| blackboxsw | if it returns False the current exception gets raised | 17:03 |
| dojordan | yup just found that | 17:03 |
| blackboxsw | basically this I'm thinking //paste.ubuntu.com/p/qPd7fTp7rz/ | 17:04 |
| blackboxsw | but wanted to test it | 17:04 |
| dojordan | yup, my thoughts exactly | 17:04 |
| blackboxsw | ok good deal. | 17:04 |
| blackboxsw | I'm running through a deployment of that + tip + your branch now should clear out on your branch then in the next hour | 17:05 |
| dojordan | want me to push that patch? | 17:06 |
| blackboxsw | +1 dojordan it'd save me time | 17:06 |
| blackboxsw | after a git rebase | 17:06 |
| dojordan | sure, just running unit tests first | 17:06 |
| blackboxsw | just to include Kurt's changes. thanks | 17:06 |
| dojordan | hmm, getting a slow test: tests.unittests.test_ec2_util.TestEc2Util.test_userdata_fetch_fail_server_not_found 5.0287s . Not a chance my change of AzureDataSource affects this but just FYI | 17:12 |
| smoser | hm... yeah, i was just seeing that oto. will investigate, dojordan | 17:13 |
| blackboxsw | ahh good find, that tends to mean a missed requests mock | 17:13 |
| smoser | thanks | 17:13 |
| blackboxsw | or a readurl/wait_for_url mis-mocked | 17:13 |
| dojordan | @blackboxsw pushed | 17:16 |
| blackboxsw | +1 I'm testing live now on a xenial instance w/ your patch ++ the minor pastbin | 17:16 |
| dojordan | sounds good | 17:17 |
| blackboxsw | BTW we are going to change the logic of exception_cb passed to readurl in a subsequent branch (post 18.2 release). Now exception_cb will be expected to raise an exception if warranted and ignore if retries are expected. It's more intuitive than this return True|False stuff (as you have to go re-read readurl implementation to see what expected behavior is) | 17:18 |
| blackboxsw | that change will then align more with wait_for_url behavior too | 17:19 |
| blackboxsw | and we'll fix this callsite for readurl + openstack and scaleway datasources | 17:19 |
| blackboxsw | which also use exception-cb | 17:19 |
| dojordan | great, i'll keep my eye out for that change | 17:20 |
| dojordan | thanks for the warning | 17:20 |
| blackboxsw | dojordan: landing it, both failure modes (404 versus service not present work great) thanks for the bump | 17:59 |
| dojordan | great, thanks for the help! | 18:02 |
| smoser | blackboxsw: 2 reviews please. quick-ish. | 18:12 |
| smoser | https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/341995 | 18:12 |
| smoser | https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/341997 | 18:12 |
| smoser | second is (i think) not objectionable in any way | 18:12 |
| smoser | blackboxsw: ping | 18:26 |
| blackboxsw | smoser: +1 on the second and will get through the first. didn't want to report back til finished :/ | 18:26 |
| powersj | blackboxsw: smoser: thoughts on https://bugs.launchpad.net/cloud-init/+bug/1758409 when you have a sec | 18:27 |
| ubot5 | Ubuntu bug 1758409 in cloud-init "integration tests: restructure ssh timeout " [Undecided,New] | 18:27 |
| * blackboxsw removes python-jsonschema deb from my env as it should have failed for me locally | 18:27 | |
| smoser | blackboxsw: i was pinging becauase of https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/341998 | 18:29 |
| smoser | which dojordan pointed out. | 18:30 |
| smoser | if we wanted to touch that like this... or just go for the bigger fix | 18:30 |
| * blackboxsw looks over read_file_or_url implemetation | 18:31 | |
| blackboxsw | dang, ok missed the third caller | 18:31 |
| blackboxsw | hrm. I think we should just go for the bigger fix since we have context at the moment. it shouldn't be that big. OpenStack. Scaleway, Ec2 and Azure. Then we don't have to revist risk in 18.3 SRU validation too | 18:33 |
| blackboxsw | smoser: well, actually, I have to get through your IBMcloud run too and I'd like us to make a bionic cut today including it if we can | 18:34 |
| smoser | blackboxsw: sure. | 18:34 |
| smoser | i can work on b igger fix | 18:34 |
| blackboxsw | so, I wouldn't be able to write up that branch. :/ so don't want to force you to do it | 18:34 |
| smoser | but we can just as well take the small one now. | 18:34 |
| blackboxsw | yeah +1 | 18:34 |
| blackboxsw | will approve | 18:34 |
| smoser | note that https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/341995 | 18:35 |
| smoser | made jenkins catch the failure | 18:35 |
| smoser | blackboxsw: just becausae youre bored... | 18:37 |
| smoser | https://code.launchpad.net/~dojordan/cloud-init/+git/cloud-init/+merge/340546 | 18:37 |
| smoser | that has a comment on it with a bad hash | 18:38 |
| smoser | (ie, click the link) | 18:38 |
| dojordan | @blackboxsw, random question. Any reason why the upstream commit https://git.launchpad.net/cloud-init/commit/?id=2d618e27 isn't showing up? | 18:39 |
| dojordan | haha | 18:39 |
| smoser | :) | 18:39 |
| dojordan | i had that typed out but assumed it was a stale cache or something | 18:39 |
| blackboxsw | interesting.... hmm, checking how the lander got that hash | 18:40 |
| blackboxsw | hrm hold up. | 18:41 |
| blackboxsw | dojordan: smoser ok fixed. I had the lander sitting at pdb | 18:46 |
| blackboxsw | it did all the launchpad work, just didn't git push | 18:46 |
| dojordan | hahaha | 18:46 |
| dojordan | nice | 18:46 |
| === r-daneel_ is now known as r-daneel | ||
| blackboxsw | smoser: minor tweak and then approved https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/341997 | 18:58 |
| smoser | blackboxsw: maybe it wasnt clear to you why you had jsonschema in your tox env. | 19:19 |
| smoser | its in requirements.txt | 19:19 |
| smoser | so it gets put in there by setup.py | 19:19 |
| smoser | so we could either deal with "optional" things in a generic way, which woudl be fine or like this. | 19:20 |
| blackboxsw | smoser: yeah, I just generally had performed a pip remove of it when testing new schema additions, I forgot about that when I put up test_snap and test_ubuntu_advantage | 19:20 |
| smoser | but for the moment thats the only optional | 19:20 |
| smoser | this way C-I will catch it for you. | 19:20 |
| blackboxsw | hence the tests which passed locally, (because or requirements.txt) | 19:20 |
| blackboxsw | yeah | 19:20 |
| blackboxsw | I like that | 19:20 |
| blackboxsw | smoser: and oauthlib right? | 19:21 |
| blackboxsw | as for optional requirements | 19:22 |
| smoser | and pyserial :) | 19:22 |
| blackboxsw | ohh right yep | 19:22 |
| smoser | so we could do something more generic | 19:22 |
| smoser | but the schema is the most invasive | 19:22 |
| smoser | the others are very optional paths | 19:22 |
| blackboxsw | yeah I think your approach in tox -e xenial. it at least asserts we are exercising optional paths without the dependency | 19:24 |
| blackboxsw | we could come up with something a bit more generic if we have more cases I guess. | 19:25 |
| blackboxsw | can you rebase and push https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/341995 so we get the pretty ci green | 19:26 |
| blackboxsw | now that you've landed the prior branch | 19:26 |
| smoser | done | 19:27 |
| smoser | blackboxsw: wow. MAAS comopatibiltiy testing takes 11 minutes now | 19:28 |
| smoser | i swear it wasnt that long before | 19:28 |
| smoser | powersj: ^ | 19:28 |
| smoser | https://jenkins.ubuntu.com/server/job/cloud-init-ci/ | 19:28 |
| smoser | something got slower... but of course i have to data to that. | 19:28 |
| powersj | that's about right | 19:29 |
| powersj | that was half the ci time | 19:29 |
| smoser | well, it looks like its more 2/3 now | 19:29 |
| smoser | but ohw ell | 19:30 |
| powersj | the other times, for reasons unknown to me, seem to have gone down | 19:30 |
| powersj | I know blackboxsw and I found a number of test escapes last fall and fixed them | 19:30 |
| powersj | The build and the integration tests specifically | 19:31 |
| blackboxsw | right, yeah invalid mocks w/ httpretty, but yeah I don't know what else would have been improved | 19:31 |
| powersj | I would like to consider moving the maas compat back out and run it only nightly on trunk | 19:32 |
| smoser | blackboxsw: c-i happy now | 19:47 |
| blackboxsw | then bbsw is happy | 19:47 |
| smoser | and i updated the commit message | 19:47 |
| blackboxsw | landed | 19:51 |
| blackboxsw | IBMcloud next | 19:51 |
| blackboxsw | then chrony/ntp | 19:51 |
| blackboxsw | minor comments on https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/341774ConfigDriveReader | 21:24 |
| blackboxsw | sans the ConfigDriveReader | 21:24 |
| smoser | blackboxsw: https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/342007 | 21:26 |
| smoser | thats not all the way done.. but | 21:26 |
| smoser | blackboxsw: i had thought the .json thing too... | 21:27 |
| smoser | i dont know. | 21:28 |
| blackboxsw | smoser: yeah I just have a hard time looking over 3 item tuples instead of two. take it or leave it not critical by any means | 21:28 |
| smoser | blackboxsw: i really have to go now. | 21:29 |
| smoser | i'd like to have that in too, bjut just out oftime | 21:29 |
| smoser | if you want you can take it and just finish it. | 21:29 |
| smoser | i'm really 30 minutes over :-( | 21:30 |
| blackboxsw | +1 see ya. | 21:30 |
| blackboxsw | have a good weekend | 21:30 |
| smoser | i will pop backc in if you have aMP for upload | 21:30 |
| blackboxsw | +1 will have one | 21:30 |
| smoser | just because that is "easy" | 21:30 |
| smoser | later | 21:30 |
| rharper | hrm, all of the reword of the url reading makes me a little nervous, post 18.2, right ? | 21:35 |
| blackboxsw | rharper: your comment on the ibmcloud branch.... blocker for release or tech-debt we can pickup as a separate branch? | 21:38 |
| blackboxsw | Shouldn't we update util.find_devs_with() to make use of this? or some other refactoring | 21:38 |
| blackboxsw | since we now have to callers to blkid? | 21:38 |
| rharper | not a blocker | 21:38 |
| rharper | I understand the two modes | 21:38 |
| rharper | but it is tech debt IMO | 21:38 |
| blackboxsw | I see your point, lots of commonality with find_devs_with. it seems like a good candidate for consolidation though | 21:38 |
| rharper | find_devs should be implemented with util.blkid() however, util.blkid needs to change to accept the parameters that are in-use by find_devs_with() | 21:39 |
| rharper | so, that'll be a bit of refactoring | 21:39 |
| blackboxsw | agreed | 21:39 |
| blackboxsw | your comment about the reword of url reading is related to this one right? https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/342007 | 21:40 |
| rharper | well, there are like at least two no? the raise exception, and then the follow up ? | 21:41 |
| blackboxsw | yeah we've landed the prior already which was to invest the readurl(exception_cb) logic to continue retries when exception_cb returns True. False == raise | 21:41 |
| rharper | blackboxsw: the invert and then consistent | 21:42 |
| blackboxsw | rharper: right those have both landed, invert and consistent across 4 datasources | 21:42 |
| rharper | =/ | 21:42 |
| rharper | ok | 21:42 |
| blackboxsw | yeah, it was inconsistent for OpenstackDatasource for months. resulting in long retries on 404s | 21:43 |
| rharper | I should have spoke sooner; those sort of changes seemed post 18.2 release, but I guess we'll just need to poke on them | 21:43 |
| rharper | Openstack 404ed in what scenario ? | 21:43 |
| blackboxsw | https://bugs.launchpad.net/cloud-init/+bug/1702160 | 21:44 |
| ubot5 | Ubuntu bug 1702160 in cloud-init "OpenStack datasource should not retry user-data on 404" [Medium,Fix committed] | 21:44 |
| blackboxsw | when no user-data is provided, it'd retry to exhaustion | 21:44 |
| blackboxsw | instead of bailing and continuing | 21:44 |
| rharper | lol | 21:46 |
| rharper | why wouldn't the file be empty ? | 21:46 |
| blackboxsw | and finally smoser's last attempt is to change behavior of exception_cb to actually raise if needed, instead of expecting that readurl will do the raise. only reason I thought about getting that 'in' is because we are already exposed on the exception_cb changed behavior front | 21:46 |
| blackboxsw | so might as well only cross that changed-behavior on 4 clouds once. | 21:47 |
| rharper | y | 21:47 |
| rharper | it all makes sense, just wish it was post 18.2; we have one more release before 18.04 anyhow; but I guess for bionic, at least, sooner is better | 21:47 |
| rharper | it does worry me re: SRU though | 21:47 |
| blackboxsw | +1, yeah we'll have to manually exercise upgrade path and fresh install on Openstack too (we already handle Azure/ec2). Not sure what to do about Scaleway though | 21:49 |
| blackboxsw | I'll give ec2 a run right now with tip before we cut an MP for publish to bionic (just to make sure the system hasn't been killed) | 21:50 |
| rharper | y | 21:50 |
| blackboxsw | just tested on EC2 with tip of smoser's branch. | 23:08 |
| blackboxsw | I'm awaiting a jenkins integration test then I'm merging and proposing a bionic release | 23:08 |
| blackboxsw | https://jenkins.ubuntu.com/server/view/cloud-init/job/cloud-init-ci/922/ | 23:08 |
| blackboxsw | tested on azure earlier today with basically the same content (regarding the exception_cb behavior changes as do-jordan's branch had to account for that) | 23:09 |
| blackboxsw | smoser: it's up for review | 23:29 |
| blackboxsw | https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/342011 | 23:29 |
| blackboxsw | if you wanted to enable IBMCloud in debian cloud.cfg.tmpl go4it | 23:29 |
| blackboxsw | :) | 23:29 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!