=== rodlogic is now known as Guest97922 === rodlogic is now known as Guest47802 [02:48] thedac, thx for the extra testing on that === rodlogic is now known as Guest50280 === rodlogic is now known as Guest7886 === rodlogic is now known as Guest67605 === rodlogic is now known as Guest18647 === thumper is now known as thumper-afk === frankban|afk is now known as frankban === rodlogic is now known as Guest6082 === thumper-afk is now known as thumper === rodlogic is now known as Guest81321 === rodlogic is now known as Guest87809 === rodlogic is now known as Guest48101 === rodlogic is now known as Guest2307 === rodlogic is now known as Guest28754 === rodlogic is now known as Guest8122 [14:35] cholcombe, are you joining the retrospective? === Makyo|Vacay is now known as Makyo [15:13] has any work been done with lxd and ceph as a backend for cinder? [15:23] LiftedKilt: lxd and ceph? [15:24] marcoceppi: specifically with cinder - I know that ceph and lxd work together insofar as ceph provides a backend for glance images === chuck__ is now known as zul [15:24] but my understanding was that there are some issues with mounting ceph RBD's to lxd containers [15:25] something along the lines of - when lxd tries to migrate a container, it wants to copy the data along with it, and doesn't know to just remount the rbd [15:26] LiftedKilt: I'm not sure of any progress there, sadly [15:27] marcoceppi: hmmm ok. Who would be running point on that integration? Would it be a canonical thing since it's lxd? Or would it be a general openstack thing because it's nova? [15:27] LiftedKilt: It's probably a LXD thing, zul do you know? [15:28] gnuoy, is there an epic doc for the work I'm going to be doing on barbican & aodh? [15:28] LiftedKilt: nova-lxd doesnt support cinder backends yet\ [15:29] zul: is work actively being done on enabling that, or is there a roadmap for the integration? [15:30] LiftedKilt: both...it should be ready soon [15:30] zul: where would be the best place for me to follow any news on that? [15:31] LiftedKilt: https://github.com/lxc/nova-lxd === rodlogic is now known as Guest54704 [15:31] zul: awesome - thanks a lot === rodlogic is now known as Guest38353 === chuck__ is now known as zul === rodlogic is now known as Guest8087 === frankban is now known as frankban|afk [17:34] === rodlogic is now known as Guest99896 [17:43] thedac, Tribaal's https://review.openstack.org/#/c/314557 is clear for takeoff imo (stable charm update) - can you do the review/landing honors? [17:43] beisner: will do [17:45] beisner: Tribaal +2 [18:04] yo beisner [18:04] question for you [18:04] shoot lazyPower [18:04] oh nevermind i click on the reivew and it 404's [18:04] <3 [18:06] -> not sure if that's good or bad [18:06] great in this case [18:06] i dont need to bug you :D [18:06] not more than i already have anyway [18:07] tvansteenburgh: I pip installed bundletester on xenial but don't have a bundletester command in my path. What did I do wrong? [18:07] oh speaking of, do we need a xenial flavor of charmbox soon? [18:07] aisrael: uhhhh === chuck__ is now known as zul [18:09] deej: /wg 8 [18:09] oops [18:09] tvansteenburgh: I found it in ~/.local/bin/bundletester [18:09] openstack-charmers: Is this a known bug, or am I doing something wrong here -> https://bugs.launchpad.net/charms/+source/neutron-openvswitch/+bug/1580271 [18:09] Bug #1580271: hook failed: neutron-plugin-api-relation-changed for neutron-openvswitch:neutron-plugin-api [18:09] which is not in path [18:10] aisrael: that's weird [18:10] Yeah, I've never seen python bits installed there. Usually they go into /usr/local [18:11] aisrael - thats default if you use pip install --user [18:11] lazyPower: I didn't use --user, though [18:11] aisrael - you'll need to add that path to $PATH if you want anything installed via your userdir isolation [18:11] -u instead of -U ? [18:11] weeeeiiirrrddd [18:11] maybe xenial has different behavior? [18:11] no -u [18:12] marcoceppi: if i spin up a demo environment today can you promise not to switch it off until after tomorrow? :) [18:12] and it dumps a stack trace [18:12] beisner: thedac: thanks a lot! [18:12] hi bdx - /var/log/juju/unit-*vswitch* may also be indicative on that [18:13] tvansteenburgh: http://pastebin.ubuntu.com/16350722/ [18:13] kwmonroe: ping [18:14] beisner: yea, its spammed with connection errors for not having rabbit up yet [18:14] aisrael: looking [18:14] that shouldn't depend on rabbits relation or connectivity though [18:15] bdx, ah yes if rmq is grumpy, generally no one else is happy. [18:17] beisner: ok, yeah, I'll get rabbit in check and see what gives .... is xenial/rabbit successfully deployable to your knowledge ... I've been having a fit with it [18:18] bdx, yep xenial rmq is looking good from my perspective [18:19] beisner: awesome. deployed to lxd as well? [18:19] bdx, but i've not thrown it at juju-tip+maas-tip like you are doing right now ;-) [18:20] bdx, ack. this scenario results in a happy rmq: https://github.com/openstack-charmers/openstack-on-lxd [18:20] aisrael: what charm or bundle? i can't repro and i'm on xenial [18:22] tvansteenburgh: cs:~3-bruno/trusty/quobyte-client [18:23] hmm [18:23] beisner: I see, it deploys for me using lxd cloud type too, just not as a lxd on a MAAS deployed node -- just xenial/rabbit deployed to lxd on maas just hangs on "Waiting for rabbitmq app to start" :-( [18:24] tvansteenburgh: Probably unrelated, but I got this running bundletester from ~ [18:24] http://pastebin.ubuntu.com/16350914/ [18:24] perms on the file were root:root. easy fix, but kind of odd [18:24] beisner: trusty/rabbit seems to deploy successfully to lxd on maas though ... I'll see if I can't grub up a bug for this [18:25] aisrael: bt running against that charm, no errors for me [18:25] aisrael: i did install it in a venv thought [18:25] though [18:26] tvansteenburgh: I'll do that and try again [18:26] I bet it's something with this .local wonkiness [18:28] aisrael: your first paste seems like a legit bug, just not sure how to repro [18:29] wjst [18:31] aisrael: do you have make installed? [18:32] tvansteenburgh: Yep. I installed build-essential [18:32] aisrael: what about charm-tools [18:33] tvansteenburgh: negative. installing now [18:33] aisrael: that was the cause of the bug. will fix [18:33] tvansteenburgh: confirmed. thanks! === rodlogic is now known as Guest21410 [18:37] tvansteenburgh, i've found that when using --no-destroy but not using --allow-failure, bt keeps on trucking onto the next test instead of stopping/bailing on the first fail. i was expecting/need it to halt and keep the deployment so we can poke at the model with bots. idears? [18:39] * tvansteenburgh looks [18:42] tvansteenburgh, fwiw, we do have reset: True ... perhaps we can't have both? [18:45] beisner-food: any other options? [18:51] beisner - yeah thats what you're looking for. setting reset to false, and not allow failure. That will cause the test to abort and your bots can aggregate the resulting env [18:52] but you say it continues barreling on? thats weird [18:58] beisner-food: afaict there's no correlation between these options in the code. one shouldn't affect the other. if you use -F by itself it works as expected? [19:20] tvansteenburgh: May have hit another bundletester issue, at the bottom of this: http://pastebin.ubuntu.com/16351794/ [19:20] tvansteenburgh, lazyPower - yah i'm looking for the best of both worlds: recycle the bootstrap node, but bail on fail. /me retries with reset: false ... will holler back [19:22] tvansteenburgh, i'm not setting -F anywhere, just mentioned that so that you know we're not setting it. [19:22] beisner: -F == --allow-failure [19:23] tvansteenburgh, right, i'm not using -F. i want it to fail and halt. [19:23] oh, right [19:24] beisner: so if you don't use --no-destroy, it stops after first failure?? [19:25] tvansteenburgh, not sure actually. i dove into it with what i wanted, which is --no-destroy + reset: True .. and that blew all the way through tests regardless of pass/fail [19:25] tvansteenburgh, right now, testing with --no-destroy + reset: False. [19:30] beisner, openstack-charmers: my primary point of contention with 16.04 deploy is here -> juju bootstrap --config config.yaml localhost lxd [19:30] ooops [19:30] hehe [19:30] aisrael: yeah, i guess we could handle that case better [19:30] tvansteenburgh: apparently, that fail is because all machines are stuck allocating (due to Your quota allows for 0 more running instance(s). You requested at least 1 (InstanceLimitExceeded) [19:30] here: http://paste.ubuntu.com/16351894/ [19:30] aisrael: indeed [19:32] aisrael: filed a bug for that but it likely won't be high priority [19:33] aisrael: https://github.com/juju-solutions/bundletester/issues/45 [19:33] tvansteenburgh: ack, thanks. [19:35] tvansteenburgh, lazyPower - hrm. with --no-destroy and reset: False and the first test rigged to fail for debugging, it didn't bail: http://pastebin.ubuntu.com/16351971/ === rodlogic is now known as Guest29652 [19:35] beisner that seems like a bug :\ [19:36] no, it's per test file [19:36] to bundletester, a test is a file, it does't know anything about the contents of the file [19:36] is there an equiv to `juju test --set-e` in bt? [19:36] oooo [19:36] thats right [19:37] it'll attempt to complete the current TestCase then bail, right tvansteenburgh? [19:37] so the first test file is known-fail (and it fails). but then it proceeds to the 2nd test file. [19:37] when i say TestCase, i mean finish all the test_ methods in that class... [19:37] in this case, it's the 'deployment' that will knowingly fail [19:37] not really the test [19:37] * tvansteenburgh squints [19:37] beisner: can you paste the whole log? [19:38] beisner: I'm subscribing you to the bug I made for openvswitch, is that cool? [19:39] tvansteenburgh, http://pastebin.ubuntu.com/16352034/ [19:39] bdx, yah, please link here too. thx man [19:39] beisner, openstack-charmers -> https://bugs.launchpad.net/charms/+source/neutron-openvswitch/+bug/1580271 [19:39] Bug #1580271: hook failed: neutron-plugin-api-relation-changed for neutron-openvswitch:neutron-plugin-api [19:42] tvansteenburgh, /me retries w/o no-destroy [19:42] beisner: i'm having trouble coming up with any more excuses for why it doesn't work :P [19:43] stub: realize its not your timezone, but curious where I should submit a bug for the https://git.launchpad.net/postgresql-charm ? a default config option makes it fall over on a xenial deploy [19:44] lutostag: i filed mine here https://bugs.launchpad.net/postgresql-charm :) [19:44] tvansteenburgh: thanks, looks reasonable :) [19:50] openstack-charmers: https://bugs.launchpad.net/charms/+source/rabbitmq-server/+bug/1580316 [19:50] Bug #1580316: Waiting for rabbitmq app to start [19:55] tvansteenburgh, lazyPower - chit. so without --no-destroy and with reset: False, i get the same. [19:55] beisner: good, otherwise it's even weirder [19:56] tvansteenburgh, so it looks like if the fail is a deployment failure, it doesn't count [19:56] beisner: oh... [19:56] openstack-charmers -> https://bugs.launchpad.net/charms/+source/ceph-osd/+bug/1580320 [19:56] Bug #1580320: no block devices detected using current configuration [19:57] beisner: still doesn't make sense, exit code is non-zero which should be all that's needed [19:57] tvansteenburgh, http://pastebin.ubuntu.com/16352231/ [19:58] tvansteenburgh, ha, right!? Exit Code: 1 ... call [next test...] is happening though [19:58] yeah [19:58] beisner: humor me an remove the -r and -o switches [19:59] tvansteenburgh, ack. couple mins... [20:01] beisner: i can't repro locally no matter what i try [20:01] beisner: what's in your tests.yaml? [20:07] ahasenack: you around/available for questioning? [20:09] beisner: wait a sec...in that last pastebin you were passing -F [20:10] * tvansteenburgh suddenly realizes the whole thing was a giant troll [20:11] tvansteenburgh, wait. what. [20:11] f'ing -F [20:11] sure enough. who put that there? [20:12] tvansteenburgh, lazyPower - lol, yah a lil unintended !failfast troll for ya. [20:13] tvansteenburgh: what's up? [20:14] sorry for the noise on that folks [20:15] beisner: np [20:16] ahasenack: when someone does `apt install juju-deployer` how does the system know which python to use? does it just use default system python? [20:16] beisner thats brilliant :D [20:16] tvansteenburgh: use for what, to run apt, or to run deployer? [20:16] ahasenack: deployer [20:16] lazyPower, just making sure we're all paying attention ;-) [20:16] tvansteenburgh: deployer package has a dep on certain version(s) of python, [20:17] tvansteenburgh: other than that, the final call is the shebang line [20:17] $ head -n 1 $(which juju-deployer) [20:17] #! /usr/bin/python [20:19] ahasenack: okay. i thought maybe there was more to it on the packaging side but i guess not. thanks [20:20] tvansteenburgh: packaging is really py2 centric [20:21] ahasenack: deployer with juju2 is only going to work with py3 on trusty, b/c the trusty version of py2 doesn't support TLS 1.2 [20:22] ahasenack: so i was trying to figure out if we need to do anything on the packaging side to accommodate that [20:22] tvansteenburgh: and what is switching to tls 1.2 exclusively? [20:22] server-side? [20:22] juju [20:22] I see [20:22] ahasenack: as of beta7 [20:22] you mean the juju api endpoint [20:22] yeah [20:22] looke like #eco will have their hands full ;) [20:23] well i already did the work in deployer and jujuclient [20:23] *looks [20:23] to port to py3? [20:23] yeah [20:23] it's a fork? [20:23] no [20:23] py2 and py3 support [20:23] pysix? [20:23] yeah [20:23] ok, that will need some smarter packaging [20:24] branches are here if you're curious https://bugs.launchpad.net/juju-core/+bug/1576695 [20:24] Bug #1576695: Deployer 0.7.1 on trusty cannot talk to Juju2 because :tlsv1 alert protocol version [20:24] Hi, is this an okay area to ask some questions regarding juju2.0 errors I'm having trying to run on Ubuntu 16.04 on Linux on System z (mainframe)... need some pointers [20:24] ahasenack: okay, well that's why i asked. i don't know anything about packaging [20:24] almost 4k lines [20:24] tvansteenburgh: that level of packaging is a bit over my head [20:25] tvansteenburgh, ok so we failed well without -F. next ?: any way to tune the timeout happening @ self.d.setup(timeout=900) ? [20:25] beisner: yes! [20:25] wee! [20:26] beisner: https://github.com/juju/amulet/issues/119 [20:26] ahasenack: what do you mean "that level" [20:27] ahasenack: doing py2 and py3? [20:27] ahasenack: producing multiple binary packages, one for py2, one for py3 [20:27] ok [20:27] from one source [20:27] well thanks for your moral support anyway :) [20:27] I'm sure there are tons of tutorials out there [20:27] i've been needing to learn about packaging anyway [20:27] it's just not something I have done before [20:27] my time has come [20:28] tvansteenburgh, \o/ cool thanks! [20:28] beisner: sure thing [20:28] ivandov: certainly, ask away [20:29] caveat, i know nothing about Z [20:29] * tvansteenburgh looks around [20:30] Haha, that's okay... I'm likely needing some basic juju help first... I'll muddle through the z issues afterwards [20:31] * magicaltrout checks his hotel room for a spare SystemZ mainframe to test on [20:31] So, I followed the Getting Started devel docs, which worked swimmingly, but now when I try to deploy charms, they fail... I assume because some of the hooks aren't written for z... [20:32] But how would I go about actually seeing the error details that are shown under 'juju status' for a particular machine... It doesn't seem to let me do a 'juju ssh' into it... and 'juju debug-logs' just hangs [20:32] juju debug-log --replay [20:33] yup, that has no output [20:33] hrm [20:33] sounds like a bigger issue [20:34] juju status shows the machine in an error state... and the individual units show a message of "Waiting for agent initialization to finish " [20:34] ivandov: yeah, that's a juju problem, not the charms [20:34] marcoceppi: is juju supposed to work on Z? [20:35] Yes, it was announced as supported with 16.04 LTS GA [20:35] okay [20:36] The list of supported charms is still something i'm looking for, but from what I understood... Juju was supported [20:36] ivandov: can you pastebin the output of `juju status --format yaml` === rodlogic is now known as Guest74084 [20:37] http://pastebin.com/CaggRfWU [20:38] ivandov: did you bootstrap with --upload-tools? [20:39] No, as per the devel documentation... I followed this line "juju bootstrap lxd-test lxd" for bootstrapping [20:41] ivandov: i'm not sure how to fix the "no matching tools available" error in machine 0...marcoceppi, lazyPower? [20:44] ivandov - can you give it another go with --upload-tools? I'm not certain what the arch is for Z series, and we may / may not have published simple streams for those and the version of juju you are using. [20:45] i'm not terribly familiar with that, and i defer to our release team [20:46] z arch is s390x ... okay, do I have to somehow delete the current lxd bootstrapped environment? [20:46] try juju destroy-model default && juju destroy-controller lxd-test [20:47] that should clean up the currently deployed controller + default model. if you created any additional models, it will complain at you that there are active models and they need to be destroyed before you can destroy the controller [20:47] yup, that destroyed it... then create a new controller as documented in the devel docs? https://jujucharms.com/docs/devel/getting-started [20:48] make sure you append --upload-tools [20:48] so just "juju bootstrap lxd-test lxd --upload-tools" ? [20:49] yeah [20:51] ivandov - one additional note. i noticed the workload you deployed was docker. THats going to fail on the lxd provider. It doesn't ship with the proper profile enabled by default, and there are some additional concerns to be worked out there before any docker/layer-docker based workloads will be deployable via lxd [20:52] Okay, bootstrap complete... any specific charm I should try to deploy... I have been unable to find a list of charms compatible with z... I doubt the mediawiki-single bundle would work [20:52] if i can suggest an alternate workload - the elasticsearch charm is a good option [20:53] its java based and shoudl work ootb if z has a jvm and/or works with openjdk [20:53] i'm fairly certain that charm assumes openjdk [20:55] Okay, deployed elsaticsearch... seems like a slightly different error... http://pastebin.com/K9dHywyk [20:55] that ones easy :P [20:55] progress :) looks like you dont have the lxd images imported [20:56] Haha, okay... any pointers on how to get that done? [20:56] something like [20:56] lxd-images import ubuntu trusty amd64 --sync --alias ubuntu-trusty [20:56] i'm looking, but it looks like this got rolled into the provider code now.. its importing the images on bootstrap [20:56] it may only be the xenial image though [20:58] magicaltrout: I assume the arch that you have listed in your command won't work with z [20:59] ivandov correct, sub the arch as required [20:59] yeah that was a copy and paste :) [21:00] i don't have a mainframe in my hotel room ;) [21:00] Haha... come on! Get on that! [21:00] yeah i'll have a word [21:00] It's a great space heater === natefinch is now known as natefinch-afk [21:00] question: can i play tetris on it? [21:01] arosales: tell your boss I would like a system z mainframe in my hotel room to test on please [21:01] magicaltrout: with the lxd-images command... is that a utility I need installed? [21:01] erm [21:02] i may have changed in recent releases [21:03] lxc image import --alias [21:03] according to linuxcontainers.org [21:03] Yup, just found that with a little google-fu... now I need to find an image file that would work with z [21:03] hmm.. [21:04] theres a published list of images somewhere..... [21:04] magicaltrout: come to his talk or bof this afternoon and you can tell him yourself ;-) [21:04] i might just do that [21:05] https://images.linuxcontainers.org/ looks like the list i'm looking for [21:05] that be the one [21:05] i see s390x images in the list [21:05] \o/ [21:05] not one for trusty though [21:06] you might be scuppered :P [21:06] Yup, was just about to say that [21:06] xenial + though [21:06] i dont see one for trusty [21:06] So, I'd need to find a charm that's based off of xenial to get a little further in testing some charm deployments? [21:07] ivandov - well plot thickens. I have a xenial ready charm, that works on juju 2.0 using resources [21:07] do you know if there's an etcd s390x binary or if you're comfortable compiling one? [21:07] not sure... I can definitely poke around... I guess it'd be an interesting practice to try to compile one lol [21:08] 2https://jujucharms.com/q/xenial [21:08] -2 [21:08] thats about the first time in its history the search function has actually been useful [21:08] magicaltrout nicely done [21:09] the ubuntu charm is probably the lowest bandwidth test to see if it works [21:09] jamespage, fyi, one to watch re: bundletester switch (re: juju test deprecation) & amulet via tox/venv: https://review.openstack.org/#/c/314773 [21:09] juju deploy ubuntu --series=xenial [21:10] should I destroy the model & controller again? [21:10] you shouldn't need to, no [21:10] you can destroy that service that's stuck looking for trusty image data [21:10] and, do I need to add an lxd image for s390 before doing that? Or was that done during bootstrapping? [21:11] it should have prepared you a xenial image. lxc list images will tell you what you've got available. [21:12] hmm lxc list images didn't look promising [21:12] http://pastebin.com/eW0T2788 [21:13] mine says the same [21:13] don't trust it [21:13] but you can grab an image anyway it doesn't really matter [21:13] okay, then i'll just try juju deploy ubuntu --series=xenial [21:13] a lot of this has changed since the last time i used it. [21:14] magicaltrout - dude, resources are p. cool [21:14] this is my new fanfare, is resources. offline'ing charms one dependency at a time :D [21:15] i'm not entirely sold on resources, I think you should be able to tie a resource into the charm in the charmstore [21:15] wooo! the ubuntu charm started [21:16] excellent [21:18] Well, thank you magicaltrout and lazyPower for your help... I gotta run to an engagement I'm late for... but... I'll likely be back on tomorrow trying to figure out the next steps to this [21:18] truly appreciated :) [21:18] cool [21:18] have fun [21:19] cheers :) [21:22] yeah i like the idea of resources lazyPower but I'd like to be able to define one in my charm but not have to have a user download and commit the resource [21:22] with store delivery, they dont have to [21:22] i'd like on the first run for a charm to be able to grab a resource from the web [21:22] store what? [21:23] you version the resource w/ the charm, and they ship as a packaged resource. [21:23] so you as the charm author [21:23] charm push ~fishy/foo [21:23] charm publish ~fish/foo --attach foo=bar [21:24] hmm [21:24] i'll have to dig into that === rodlogic is now known as Guest73492 [23:15] magicaltrout: you at the conf? [23:15] magicaltrout: specifically in the keynote apache sessions?