| cpaelzer | smoser: already around? | 11:20 |
|---|---|---|
| cpaelzer | smoser: while the former two quesitons are solved by time, I would still ahve another if you are around | 12:18 |
| smoser | cpaelzer, here now. | 13:22 |
| smoser | what sup? | 13:22 |
| cpaelzer | hiho | 13:23 |
| cpaelzer | I'm testing more nd more regarding the cloud-init issue you reported due to my merge | 13:23 |
| cpaelzer | most things are done, I found a few on top thou | 13:23 |
| cpaelzer | working on these, but that was not the question | 13:23 |
| cpaelzer | at some point I'll have to throw it into a adt or ppa to test if it works there as well | 13:23 |
| cpaelzer | since packaging isn't in the repo's itself I wondered what the recommended way is to do so | 13:24 |
| cpaelzer | I mean there are many ways | 13:24 |
| cpaelzer | pull-lp-source - merge a bzr tree in | 13:24 |
| cpaelzer | bzr tree and merge a debian dir into that | 13:24 |
| cpaelzer | or something totally different | 13:24 |
| cpaelzer | I assume you don't do that manually each timne, so I wanted to know which way I should go to test ppa/adt builds | 13:24 |
| cpaelzer | smoser: ^^ | 13:25 |
| smoser | cpaelzer, right. need to test building in ppa. | 13:25 |
| smoser | ./packages/bddeb | 13:26 |
| smoser | it hsa --help, but largely i'd run: | 13:26 |
| smoser | ./packages/bddeb -S | 13:26 |
| smoser | then | 13:26 |
| smoser | sbuild ... | 13:26 |
| cpaelzer | /packages/bddeb -S gievs a .dsc and a orig tarball then? | 13:27 |
| * cpaelzer is reading bddeb ... | 13:27 | |
| smoser | yeah, it uses bzr export to get a .orig and then goes from there. | 13:27 |
| cpaelzer | ok nice, then I can hit these stages once I fixed a corner case of the keyserver issues | 13:28 |
| cpaelzer | smoser: btw I also made sure I port fixes I did this way between curtin/cloud-init as I work on the same code for both | 13:28 |
| cpaelzer | so I hope to end the day with an update to the curtin MP and a new cloud-init MP | 13:28 |
| smoser | ok. great. | 13:29 |
| GivenToCode | hi i have a custom cfg in /etc/cloud/cloud.cfg.d that specifies an fs_setup to mkfs ext4 on ephemeral0 (ec2), however on c3 instances types we get an error in the log that /dev/xvdb is already mounted when trying to mkfs... | 13:50 |
| GivenToCode | this is 0.7.5-0ubuntu1.18 | 13:50 |
| GivenToCode | I would expect fs_setup to run before any mounting, right? | 13:51 |
| GivenToCode | there is an entry for xvdb in fstab, and I've tried to override that by specifying a mounts [ ephemeral0, null ] | 13:52 |
| cpaelzer | GivenToCode: yes it should set up fs_setup things before the mount step of cloud-init | 14:11 |
| cpaelzer | GivenToCode: but I'd wonder if something else could mount it even before | 14:11 |
| cpaelzer | GivenToCode: depending on you influence to this environemnt you could add in your .cfg commant to check the status early on | 14:11 |
| cpaelzer | GivenToCode: like early_commands: checkmounts: 'mount' | 14:12 |
| cpaelzer | and then check that output if it really is mounted at the beginning | 14:13 |
| cpaelzer | cpaelzer: sorry I'm mixing up things here - let me rethink | 14:14 |
| cpaelzer | writing too muhc curtin / cloud-init code at the same time - that would have been the curtin answer | 14:14 |
| cpaelzer | GivenToCode: doc/examples/cloud-config-boot-cmds.txt should be better | 14:15 |
| cpaelzer | GivenToCode: if you prefer an url http://cloudinit.readthedocs.io/en/latest/topics/examples.html#run-commands-on-first-boot | 14:17 |
| cpaelzer | smoser: did the mirrorlist issues (the last one you added the skipif for) happen to you in sbuild as well, or only in the buildd's? | 14:49 |
| smoser | cpaelzer, no. only in a buildd | 14:50 |
| smoser | cpaelzer, but think about it working on centos | 14:50 |
| cpaelzer | smoser: k, thx - setting up a victim ppa then | 14:50 |
| smoser | we want unit tests to work there. | 14:50 |
| cpaelzer | smoser: I already have | 14:50 |
| smoser | ok. | 14:50 |
| smoser | i maeant to tell you this... | 14:50 |
| smoser | there is a 'FilesystemMocking' class | 14:50 |
| smoser | or somethignt | 14:50 |
| cpaelzer | smoser: I already covered that - at least I think so :-) | 14:50 |
| smoser | aht basically mocks all fsreads. | 14:51 |
| smoser | to another directory that you can popoulate | 14:51 |
| cpaelzer | smoser: for this in particular it doesn't need a fs mocking class | 14:51 |
| smoser | ok. thats fine. | 14:51 |
| cpaelzer | the feature of influencing apt environment is very very ubuntu/debian specific | 14:51 |
| cpaelzer | so are the tests | 14:51 |
| cpaelzer | but that can be nicely done with a skipIF to the apt binary | 14:51 |
| cpaelzer | I covered this and more and you can take a look once it is mature enough for a merge proposal | 14:52 |
| cpaelzer | for now I seem to uncover further issues faster than I like and EOD is coming closer ... | 14:52 |
| cpaelzer | :-/ | 14:52 |
| smoser | cpaelzer, i dont know.. | 14:53 |
| smoser | it is in fact debian/ubuntu specific | 14:53 |
| smoser | but that does not mean the unit tests should not run elsewhere. | 14:53 |
| cpaelzer | for the start I'd like to disable things so they work properly | 14:54 |
| cpaelzer | over time one can (r not) commit the time to enable this on more environments | 14:55 |
| cpaelzer | that is what I'd prefer | 14:55 |
| cpaelzer | I feel to postpone too much as I try to get this done - so I wouldn't add a centos env test to the task list right now | 14:55 |
| cpaelzer | but you are right overall - you want a centos dev on cloudinit to see when he breaks things | 14:56 |
| smoser | well, thats fine. but generally speaking the failure i saw was from either a file existing or not existing (or a directory or something) | 14:56 |
| smoser | i'm not really sure | 14:56 |
| smoser | and the fact that that exists or does not exist is somethign that would ideally be covered by the unit test | 14:56 |
| smoser | which should not require the system its running on to be in a given state | 14:56 |
| cpaelzer | ah that one - it is a file delivered by cloud-init itself, and we don't check if the file exists | 14:57 |
| cpaelzer | we check if cloud-init "checks" if the file exists | 14:57 |
| cpaelzer | because that is the code path that it should take and that is what the unit test does | 14:57 |
| cpaelzer | the file doesn't exist on my system either | 14:57 |
| cpaelzer | but the test works | 14:57 |
| cpaelzer | what happens there is the issue in find_apt_mirror_info that I need to debug in the ppa environment, because I can't see locally what is breaking | 14:58 |
| cpaelzer | an issue in that function causes the one you have seen as follow on error | 14:58 |
| smoser | right. there is a build logo from an existing one if you wantd. | 14:58 |
| cpaelzer | I have checked the two logs you linked | 14:58 |
| smoser | yah. ok. | 14:59 |
| cpaelzer | but I need more debug data on it | 14:59 |
| smoser | you're doing well,. sorry to sound like i'm giving you flack | 14:59 |
| cpaelzer | for now I'm unhappy wit hthe gpg key things - I wanted to fix them right | 14:59 |
| smoser | you wrote a bunch of test cases for stuff that wasnt' tested and fixed a bunch of stuff and there was fallout | 14:59 |
| smoser | the first part is +100 | 14:59 |
| smoser | ) | 14:59 |
| cpaelzer | I'm fine and we will make it good :-) | 14:59 |
| cpaelzer | thanks | 14:59 |
| smoser | the second part is admittedly *my* fault and cloud-init fault for accepting your MP before it built in a ppa | 14:59 |
| smoser | :) | 14:59 |
| cpaelzer | so I added a nice fallback in case network is unavailable - but it will use real network if it is available | 14:59 |
| cpaelzer | unfortunately now that is "can" work it complains in sbuild envs for things that are not writable | 15:00 |
| cpaelzer | will get it done ... | 15:00 |
| smoser | please dont depend on presense of a keyserver | 15:00 |
| cpaelzer | just gets a longer and longer task :-) | 15:00 |
| smoser | those things really suck | 15:00 |
| cpaelzer | smoser: thats what I ment - I don't DEPEND anymore, but IF I can reach it properly I do | 15:00 |
| smoser | that will cause arbitrary failures | 15:00 |
| cpaelzer | and if I can't reach it, it has a fallback that works | 15:00 |
| cpaelzer | so the test is happy with and without keyserver | 15:01 |
| cpaelzer | it even works with broken keyservers or random dns names | 15:01 |
| cpaelzer | but IF it works it just is closer to the real thing | 15:01 |
| cpaelzer | which isn't bad for a unit test | 15:01 |
| smoser | i was honestly fairly OK with not unit testing that shell blob. and prefer that to cloud-init's unit tests mucking with my .gpg dir | 15:01 |
| cpaelzer | hehe | 15:02 |
| cpaelzer | it isn't a shell blob anymore | 15:02 |
| cpaelzer | :-) | 15:02 |
| smoser | ok. but i still would really rather it not ever modify my .gpg dir | 15:02 |
| cpaelzer | thatr with the .gpg dir is what I have to tackle next | 15:02 |
| cpaelzer | for sbuild env anyway | 15:02 |
| cpaelzer | smoser: so you tihnk it is ok to throw parts away and mock that side of things? | 15:02 |
| cpaelzer | ok, I can do that - that will throw away some code that cost me some sweat today, but it was good for the learning still | 15:03 |
| cpaelzer | at least we got the shell blob to become python as you wanted as well, thanks to a nice suggestion of rharper and some polishing I did | 15:06 |
| rharper | cpaelzer: nice | 15:07 |
| cpaelzer | I think with the plan to not mess around with .gpg at all in the tests I won't complete it today anyway | 15:07 |
| cpaelzer | smoser: do you mind if it is tomorrow and not today that you get the next MP revision? | 15:07 |
| rharper | yeah, I'd be very dubious of relying on a .gpg dir | 15:07 |
| rharper | rather a tmpdir and export GPG var to point to it | 15:08 |
| rharper | certainly GPG will check ENV for that location | 15:08 |
| cpaelzer | rharper: it has an argument | 15:08 |
| cpaelzer | rharper: just need to find a way to insert that into the subp's | 15:08 |
| cpaelzer | I still want to test the original code and not mock it all | 15:08 |
| rharper | sure, lemme see if I've example of setting env for subp | 15:09 |
| smoser | you can just mock the output of gpg | 15:09 |
| smoser | capture that call really is i think the right way to do it. | 15:09 |
| cpaelzer | I'd mock the gpg calls but then assert on them to make sure they were poked the right way | 15:10 |
| smoser | honestly, mocking the call to that shell blob as i did, we have 2 lines of untested code out of that. | 15:10 |
| smoser | and no mucking with your .gpg | 15:10 |
| cpaelzer | mucking vs mocking :-P | 15:10 |
| smoser | for code that hasnt changed and worked fine for 4 years (except for arguably invalid user input) | 15:10 |
| smoser | basically, i dont want you to spend a bunch of time making sure we get test coverage on that. | 15:11 |
| cpaelzer | ok | 15:11 |
| cpaelzer | I'll try to mock it then and get it working tomorrow | 15:11 |
| rharper | cpaelzer: Popen takes a env= param; that's not exposed in subp IIRC, but one could, then myenv = os.environ_copy(), and myenv[VAR]=VAL , pass env=myenv; | 15:12 |
| cpaelzer | rharper: thanks for checking, but given the requirements - for now - I'd go with smoser and "just" mock some things | 15:12 |
| rharper | yep | 15:12 |
| smoser | subp takes env also. | 15:14 |
| smoser | yeah. | 15:14 |
| rharper | nice | 15:24 |
| GivenToCode | cpaelzer, yes something is mounting everything in fstab before fs_setup, we don't have anything custom running before cloud-init | 15:24 |
| GivenToCode | ok, i think the fix is to remove it from fstab when we bake the image | 15:38 |
| ajorg | smoser: Can I submit GitHub pull requests instead of bzr branches for contributions? | 15:59 |
| smoser | ajorg, wont be github pull requests. but launchpad git pull requests should be shortly. | 16:02 |
| smoser | ajorg, basically upstream code path will still be launchpad but revision control in git | 16:02 |
| ajorg | okay, I'll try to learn bzr for today and maybe leave some of our more interesting patches for when git is available. | 16:02 |
| ajorg | If I did a terrible job of preparing this, it would be good to correct me now as I have several more patches I'll be trying to submit this week: https://code.launchpad.net/~ajorgens/cloud-init/python26/+merge/296575 | 16:08 |
| smoser | ajorg, that looks fine. i'm really sorry about regressing 2.6 function | 16:09 |
| ajorg | Thanks. | 16:09 |
| ajorg | By way of introduction, I'm the cloud-init maintainer at AWS. | 16:09 |
| ajorg | (for the Amazon Linux AMI) | 16:09 |
| ajorg | We have a feature or two, several bugfixes, and a bunch of backward compatibility stuff for our 0.5 fork from before Yum / RPM were supported. | 16:10 |
| ajorg | I'd like to upstream whatever parts are palatable. | 16:11 |
| smoser | we need to get some c-i in place that can test on a 2.6. the thing that makes it hard is that there is actally not even a python2.6 *available* in a supported ubuntu release (12.04 shipped 2.7) | 16:11 |
| ajorg | Anything I can do to get that in place? | 16:11 |
| ajorg | CentOS 6 testing would be prudent, and would cover 2.6 | 16:12 |
| smoser | harlowja_at_home, has mentioned some options with his remote_tox | 16:12 |
| ajorg | cool | 16:12 |
| smoser | but yeah, we need to get this stuff in place. | 16:12 |
| harlowja_at_home | https://github.com/harlowja/remote_tox ifu care | 16:12 |
| smoser | dict comprehension is just so nice | 16:13 |
| smoser | i think thats the right word | 16:13 |
| ajorg | I know! | 16:13 |
| smoser | {a:b for a, b in d.items()} | 16:13 |
| harlowja_at_home | dict((a,b) for a, b in d.items()) ? | 16:13 |
| harlowja_at_home | just use the word 'dict' lol | 16:13 |
| smoser | harlowja_at_home, deal. | 16:13 |
| ajorg | Amazon Linux AMI has been on 2.7 for a few releases, so we won't be a fruitful source of 2.6 patches going forward. | 16:14 |
| ajorg | hmm, I get to learn how to do author attribution in bzr for some of these | 16:15 |
| smoser | ? | 16:16 |
| smoser | author attribution ? | 16:16 |
| ajorg | assuming it's possible, submitting a patch in someone else's name (other authors @amazon.com) | 16:16 |
| ajorg | I know how to do it in git. | 16:17 |
| smoser | ah. well you can commit with '--author=' | 16:25 |
| smoser | that sprobably the thing you want. | 16:25 |
| smoser | then you become 'committer' and author is author | 16:25 |
| ajorg | perfect, very much like git | 16:28 |
| ajorg | thank you | 16:28 |
| smoser | larsks, fyi, i just uploaded a 0.29 tarball with your growpart fix https://launchpad.net/cloud-utils/trunk/0.29 | 17:06 |
| larsks | smoser: thanks for the heads up! | 17:10 |
| harlowja | ok backs | 17:49 |
| larsks | This is a dumb question...under ubuntu, "python3 setup.py install ..." wants to put cloud-init into site-packages, but that dir doesn't exist. What's the right way to install it? | 19:08 |
| larsks | Ah, --prefix=/usr/local seems to be the thing. | 19:11 |
| smoser | yeah, need some --prefix. | 19:12 |
| smoser | ./packages/bddeb though would probably be preferable in most cases | 19:12 |
| larsks | This is just for testing, not packaging...I was just confused by the "site-packages" does not exist error. | 19:12 |
| larsks | Apparently I haven't run setup.py on an ubuntu system before. | 19:13 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!