=== sambetts|afk is now known as sambetts
[15:07] <powersj> wow all the tests failed yesterday O.o
[15:14] <powersj> blackboxsw smoser rharper dpb1 Our open build service project: https://build.opensuse.org/package/show/Cloud:Tools:Next/cloud-init
[15:34] <smoser> powersj, i am now 'smoser' user there.
[15:35] <powersj> smoser: added
[16:32] <rharper> powersj: ok, will join
[16:32] <dpb1> powersj: cool
[16:33] <ajorg> smoser: should all things going through subp be disconnected from tty? (disconnecting from tty is easier than I thought - the main trouble was that I was scared to defy the warnings in the subprocess documentation, no need for a cat process)
[16:33] <ajorg> it's incompatible with capture=True, so I thought about having it be the behavior for capture=False.
[16:43] <rharper> powersj: ok, I'm raharper there
=== sambetts is now known as sambetts|Afk
=== sambetts|Afk is now known as sambetts|afk
[16:45] <powersj> rharper: added
[16:45] <smoser> ajorg, i'd think generally that we would not want a tty.
[16:45] <rharper> tx
[16:45] <smoser> but that seems a scary change.
[16:49] <ajorg> smoser: https://paste.ubuntu.com/24857815/
[16:55] <ajorg> agreed, it's a bit scary. easy to get something wrong
[16:55] <ajorg> i can drop it for now while working on other patches
[16:56] <smoser> do you have a typo there ?
[16:56] <smoser> p.stdout ?
[17:01] <ajorg> haha
[17:01] <ajorg> yes
[17:01] <ajorg> dum de dum
[17:02]  * ajorg runs unit tests like a good boy before re-posting
[17:03]  * ajorg ... and watches tests.unittests.test_util.TestSubp hang because subprocess is hard
[17:03] <ajorg> back to the drawing board
[17:03] <smoser> ajorg, yeah, we really dont want to be "in the middle" if we can avoid it.
[17:04] <ajorg> smoser: there's no way to disconnect from a tty without being in the middle
[17:04] <ajorg> because if you're not, isatty will find your tty
[17:06] <powersj> blackboxsw: looks like the failures of integration tests occurred after your commit was accepted.
[17:07] <powersj> the control file that gets generated looks a little different, before: http://paste.ubuntu.com/24857896/ after: http://paste.ubuntu.com/24857898/
[17:07] <powersj> Looking now for actually differences
[17:09] <ajorg> oh, 'cause I'm neglecting to write the input...
[17:14] <powersj> blackboxsw: to reproduce, checking out master and doing ./package/bddeb -S should be all you need
[17:14] <blackboxsw> powersj: yeah I'm on a yakkety lxc right now, found 3 missing deps that need to be pulled in.
[17:14] <blackboxsw> thanks
[17:14] <blackboxsw> .. and sry
[17:15] <powersj> no worries, I am hoping our new CI pipeline will fix all of this :)
[17:15] <powersj> tox + build + short integration + centos will be nice :D
[17:17]  * powersj goes and review's smoser's branch
[17:19] <smoser> powersj, thanks. i just updated comment.
[17:20] <smoser> https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/325679
[17:20] <smoser> blackboxsw, ^ you too would be nice.
[17:20] <smoser> that 'fastestmirror' thing is pretty useless
[17:20] <smoser> useless/obnoxious/makes a caching-proxy useless
[17:22] <blackboxsw> reviewing now
[17:23] <ajorg> smoser: okay, this works: https://paste.ubuntu.com/24857958/
[17:26] <smoser> ajorg, i guess i'm ok with it if we add an argument 'notty=False'
[17:26] <smoser> and then the yum install can set that to True
[17:26] <smoser> and toherwise takes old apth
[17:29] <ajorg> Sure, we can make this optional. I'm okay with that.
[17:32]  * powersj goes and looks at proposed tests
[17:42]  * ajorg waves at ilianaw
[17:42]  * ilianaw waves back
[17:51] <smoser> well, here is a week attempt at disabling mirrors
[17:51] <smoser> http://paste.ubuntu.com/24858093/
[18:07] <blackboxsw> smoser: https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/325679 minor review comment added.
[18:07] <smoser> blackboxsw, thanks.
[18:11] <smoser> blackboxsw, you want me to drop the 'pairs' ?
[18:13] <blackboxsw> smoser: not critical to drop as it shows what you intend if things get complex in the future (cmd != package_name) so it's ok to leave as is as guidance for me if we have to go that route . It's just unneeded logic at the moment so it always gives me a moment's pause as we don't really need that little complexity currently.
[18:14] <blackboxsw> if it were a lot of complexity/logic that was added for something that we don't currently need I'd say drop it as we are over-engineering the solution, but this is just a couple lines in a script so it's good
[18:15] <blackboxsw> mostly that comment was just so we could have a discussion.... aaaaaaand ✔
[18:15] <blackboxsw> :)
[18:15] <smoser> k
[18:15] <smoser> i updated mp with your file suggestion
[18:15] <blackboxsw> thx
[18:16] <smoser> but i think yours had wierd logic
[18:16] <smoser> so read mine
[18:16] <smoser>  http://paste.ubuntu.com/24858229/
[18:16] <smoser> thats what you had
[18:16] <smoser> you were looping over each file in the list and building the bad_files sllist
[18:16] <smoser> so... lots more work than necessary
[18:17] <smoser> i did this
[18:17] <smoser>  http://paste.ubuntu.com/24858234/
[18:23] <blackboxsw> umm whoa , bad diff. right
[18:23] <blackboxsw> shouldn't have nested that inside the loop , just declare bad_files outside.
[18:23] <blackboxsw> checking followup
[18:24] <blackboxsw> yes yes, thanks
[18:24] <blackboxsw> +1
[18:24] <blackboxsw> that was careless
[18:24] <smoser> k
[18:25] <smoser> click approve and i'll pull that.
[18:31] <smoser> powersj, youcan then change the jenkins jobs to do the ./tools/run-centos
[18:32] <powersj> Awesome! Once we get bddeb going again I can go back to the pipeline and get it going
[18:37] <smoser> powersj, do you knwo why it is failling ?
[18:38] <smoser> i just started looking, i'm confused.
[18:38] <powersj> I posted this a little bit ago: the control file that gets generated looks a little different, before: http://paste.ubuntu.com/24857896/ after: http://paste.ubuntu.com/24857898/
[18:38] <powersj> but that doesn't make any sense
[18:39] <powersj> the error was "E: Please add apropriate interpreter package to Build-Depends, see pybuild(1) for details at /usr/share/perl5/Debian/Debhelper/Buildsystem/pybuild.pm line 192."
[18:53] <smoser> no it does not
[19:02] <blackboxsw> hmm so diff between deps is the new one has has procps and python-contextlib2 list in Build-depends  now.
[19:03] <blackboxsw> I pulled procps down from Depends  into Build-Depends. lemme see where contextlib2 comes from
[19:03] <blackboxsw> from test-requirements.txt so it now gets pulled into the control.in template
[19:04] <smoser> good: http://paste.ubuntu.com/24858565/
[19:04] <smoser> bad: http://paste.ubuntu.com/24858566/
[19:06] <smoser> it is a bug somewhere.
[19:07] <smoser> but it doesnt like the python3 at the end.
[19:19] <blackboxsw> so we need to make sure python dependency is ordered before test-requirements as it was in the original control file maybe?
[19:19] <blackboxsw>                ${python},
[19:19] <blackboxsw>                ${test_requires},
[19:19] <blackboxsw>                ${requires}
[19:20] <smoser> well, moving it in that list fixes it
[19:20] <smoser> "fixes"
[19:20] <blackboxsw> wow strange workaround
[19:21] <smoser> i was experimenting with jinja
[19:40] <smoser> blackboxsw, http://paste.ubuntu.com/24858833/
[19:40] <smoser> that is working for me now.. hacky still, but got trhough it
[19:42] <blackboxsw> smoser: looks fine, so you wanted to make it readable again (with proper indents, one line per dep)
[19:44] <blackboxsw> I've just pushed ci-deps-fixes branch, wanted to add a --ci-deps optional param to read-dependencies to pull in everything ci wants (make, sudi, tar, devscripts etc) It'll also default the python version to the python binary that's calling us  via sys.version_info.
[19:44] <blackboxsw> this way ci can just :   python3 tools/read-dependencies --distro ubuntu --install --ci-deps
[19:45] <blackboxsw> I'm testing it now to make sure it solves the integration test issues on fresh containers
[19:45] <blackboxsw> what I had also done in that branch was pull "python" package version up to the front of Build-Requires
[20:14] <smoser> blackboxsw, --ci-deps ?
[20:15] <rharper> well, bleh.  bonding on rhel7 is frustrating ...   the bonding module loads and creates a bond0 device but ifup on bond0 doesn't configure it;  ifup on the slaves is skipped, so I can't see how the bond is ever going to come online;  further, the docs show the use of networkmanager
[20:15] <blackboxsw> maybe just ci or continous-integration? Was trying to take a step toward your "goal" suggestion a bit.
[20:16] <blackboxsw> given that we want to setup all CI  dependencies on a fresh container in order to run any build/test scripts, the --ci-deps parameter would install system packages for all requirements.txt and test-requirements.txt plus the additional base needs make, tar, sudo, devscripts
[20:17] <blackboxsw> so basically on a fresh container we can now perform the following: http://pastebin.ubuntu.com/24859055/
[20:29] <smoser> blackboxsw, i just didn't know the 'ci'
[20:29] <smoser> it conflicts . someone chose a bad name for 'continuous-integration'
[20:29] <smoser> when clearly 'cloud-init' had owned the namespace of 'ci'
[20:29] <smoser> :)
[20:29] <nacc> ci-ci
[20:30] <blackboxsw> heh. I just reread your goals suggestion.... "test" "build" as options
[20:30] <smoser> https://gist.github.com/smoser/19e65095b342e98fd4d6466e4d4fa1ac
[20:30] <smoser> there i had 'test-distro', 'test-tox', 'build', 'run'
[20:31] <smoser> blackboxsw, so in my hack patch above... i couldnt figure out how to pretty indent. i'm lacking jinja foo
[20:33] <blackboxsw> yeah /me changes it to --test-distro
[20:33] <blackboxsw> those names make sense
[20:36] <smoser> blackboxsw, do you have something like... really close ?
[20:36] <smoser> cause i think what i have in that hack is at least enough to get us building again
[20:36] <blackboxsw> smoser: the branch it pushed it's tested and runs
[20:36] <blackboxsw> was writing up a review request now
[20:38] <smoser> ah. i didnt see you push a branch
[20:39] <smoser> and didnt see at https://code.launchpad.net/%7Ecloud-init-dev/cloud-init/+git/cloud-init/+ref/master/+activereviews
[20:39] <smoser> so i assumethat is coming
[20:41] <smoser> i'd like to grab all the 'Approved' at that link, but want to have c-i functional before i do
[20:41] <blackboxsw> was writing test instructions. will just post as is and work out testing instructions shortly after
[20:42] <smoser> sure
[20:45] <blackboxsw> pushed for review.
[20:45] <blackboxsw> https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/325693
[20:46] <blackboxsw> adding test instructions for a fresh system.
[20:52] <smoser> blackboxsw, tools/run-centos can use --test-distro
[20:52] <smoser> right ?
[20:52] <smoser> although test-distro isn't necessarily build
[20:52] <smoser> but i think that is what we have right now
[20:56] <blackboxsw> smoser: yes run-centos can use it.
[20:56] <blackboxsw> changing that now
[20:56] <blackboxsw> in the branch
[20:57] <blackboxsw> /paste.ubuntu.com/24859315/
[20:57] <blackboxsw> http:///paste.ubuntu.com/24859315/
[20:57] <blackboxsw> almost
[20:57] <blackboxsw> http://paste.ubuntu.com/24859319/
[21:01] <smoser> blackboxsw, 2 minor reponses.
[21:01] <blackboxsw> looking
[21:02] <smoser> i have to run.
[21:02] <smoser> i'm ok if you or rharper want to squash that and push it
[21:02] <smoser> to get us passing tests again
[21:02] <smoser> but i have to run
[21:03] <smoser> i'll check back in in a few hours and get it then if you dont.
[21:03] <smoser> thanks!
[21:13] <blackboxsw> smoser: pushed
[21:13] <blackboxsw> thanks
[21:13] <blackboxsw> testing again now
[22:09] <blackboxsw> powersj: did you install sbuild on the jenkins ci environment?
[22:11] <blackboxsw> I see we are running it, but it's not available in default yakkety lxcs
[22:14] <powersj> Yes I installed it but we're not running yakkety
[22:18] <blackboxsw> found out what I missed, the initial apt update
[23:24] <blackboxsw> ok merged to master
[23:24] <blackboxsw> https://jenkins.ubuntu.com/server/view/Cloud-init/job/cloud-init-integration-y/227/ is running the test
[23:25] <blackboxsw> powersj: I'm also thinking of something along these lines https://jenkins.ubuntu.com/server/view/Cloud-init/job/cloud-init-integration-y/227/ for server-team-ci
[23:25] <blackboxsw> to ensure all deps are installed prior to build runs
[23:28] <blackboxsw> meh I only updated my local maste 227 is bound to fail
[23:29] <blackboxsw> well check back in later. dinner calls
[23:53] <smoser> blackboxsw, good ?
[23:53] <smoser> i see https://jenkins.ubuntu.com/server/view/Cloud-init/job/cloud-init-integration-y/227/
[23:53]  * smoser will check back in in hour or so
[23:56] <powersj> given it built and is running tests we should be good to go