| magicaltrout | whats the release schedule for push to charm store? | 00:00 |
|---|---|---|
| lazyPower | magicaltrout - i propose an idea | 00:03 |
| lazyPower | nvm i just myth=busted myself | 00:03 |
| c0s | ah... nvm | 00:04 |
| lazyPower | o/ c0s | 00:04 |
| c0s | makes me cry in pain | 00:04 |
| c0s | fortunately, I am not using it ;) | 00:04 |
| lazyPower | i have it on good authority you're looking to do some puppet in charm layers? | 00:05 |
| c0s | not exactly | 00:06 |
| c0s | was rather wondering if this is possible, as all this operational knowledge has been codified a number of times either in puppet or chef | 00:06 |
| c0s | at the risk of sounding like a broken mp3 - Bigtop is the case in point ;) | 00:06 |
| magicaltrout | i didn't realise mp3's were written to spinning plastic discs | 00:07 |
| c0s | yeah I know - I sound like a corner drug peddler ;) | 00:07 |
| magicaltrout | thats more like it! | 00:07 |
| c0s | that's a quote from Futurama ;) | 00:07 |
| c0s | about the mp3 | 00:07 |
| c0s | dunno if there are fans of that cartoon here | 00:07 |
| magicaltrout | as I complained to other Apache members today.... I have 6 1/2 hours of tutorials to write for ApacheCon..... I dont have time to watch tv (or cartoons ;) ) | 00:08 |
| magicaltrout | lazyPower is ya man though | 00:09 |
| lazyPower | well, I can help | 00:09 |
| c0s | fortunately for you, that show ended years ago ;) | 00:09 |
| lazyPower | i'm not super familiar with puppet, but there have been some charms submitted in puppet already - although they are an Openstack SDN vendor | 00:09 |
| lazyPower | so the bindings in there may be helpful for us to look at, and extract for use in reactive charming | 00:09 |
| lazyPower | are your bigtop puppet scripts done in heira? | 00:10 |
| c0s | that'd be great, but first this community need to make up their mind how they want to cooperate with Bigtop | 00:10 |
| c0s | we are using hiera for the configuration, yes | 00:10 |
| c0s | but it isn't like "it is done in hiera" | 00:10 |
| lazyPower | yeah, i think we can extract some learnings there | 00:10 |
| lazyPower | i'll reach out to apuimido and see if he's got 30 minutes or so to riff with me on pain points | 00:10 |
| lazyPower | and i'll fire up charms.puppet i guess :) | 00:11 |
| c0s | ideally, the knowledge part should stay the same and just be upstreamed | 00:11 |
| c0s | that's how OSS works, you know ;) | 00:11 |
| magicaltrout | c0s: explain bigtop to me, its like the hadoop reference stack right? so how does that differ to Apache Hadoop or what these charmer guys ship? | 00:11 |
| blahdeblah | +1 if you can get something like hiera-eyaml integrated so that we can have a more sensible way of shunting secrets around the place | 00:11 |
| c0s | pretty cool. I would be happy to help with that | 00:11 |
| c0s | Bigtop is Apache BigData stack | 00:12 |
| c0s | with binaries, deployment, integration testing, and all 9 yards | 00:12 |
| lazyPower | blahdeblah - we need to speak louder about secret management with juju when we're at sprints. it would be nice to even see a password config type so it at the bare minimum renders a password field in the gui | 00:12 |
| c0s | + a framework to develop your own stacks | 00:12 |
| lazyPower | actually scratch that - we dont need to speak louder, we do need to be consistent with getting a bug filed for it and hten beating the drum of that bug# | 00:12 |
| * lazyPower goes to find it right now | 00:13 | |
| c0s | hopefully it help magicaltrout | 00:13 |
| blahdeblah | lazyPower: I don't know anything about the GUI side; I've never used it. I just know that we end up having lots of secrets sitting around in places that are not encrypted at rest, and that's not a great look. | 00:13 |
| lazyPower | yeah | 00:13 |
| lazyPower | i agree | 00:13 |
| lazyPower | https://bugs.launchpad.net/juju-core/+bug/1287661 | 00:14 |
| mup | Bug #1287661: some config options should be considered secret <canonical-is> <config> <juju-core:Triaged> <https://launchpad.net/bugs/1287661> | 00:14 |
| magicaltrout | yup thanks c0s. I've swung by bigtop over the years but left non the wiser! clearly i need to do more reading | 00:15 |
| c0s | sure, no worries | 00:15 |
| c0s | I has developed a lot lately. | 00:15 |
| c0s | we have this cool feature where one can deploy docker-based cluster for development/testing purposes with a single command | 00:15 |
| c0s | and so on ;) | 00:16 |
| * blahdeblah edits that bug report to reference something other than SSL certs, which aren't actually secrets | 00:16 | |
| lazyPower | :D | 00:16 |
| c0s | and being a reference implementation is really becomes a focal point were the standard for the Hadoop eco-system is getting created. | 00:16 |
| c0s | like that www.odpi.org which I am helping to get off the ground now | 00:17 |
| c0s | anyway, switching off - see yall later. | 00:17 |
| lazyPower | cheers o/ | 00:17 |
| lazyPower | thanks for catching up c0s | 00:17 |
| magicaltrout | ah yeah i recall that bit. cool. | 00:17 |
| magicaltrout | Adios c0s | 00:17 |
| c0s | cheers | 00:18 |
| blahdeblah | lazyPower: I think there's also the whole issue of how they're stored, not just how they're displayed. To achieve repeatable deploys, we currently need to keep secrets files hanging around on the deployment server, which is also not desirable. | 00:19 |
| lazyPower | blahdeblah - right and projects like vault exist for this reason | 00:19 |
| blahdeblah | Haven't played with vault myself, but yeah; we need something like that | 00:20 |
| * lazyPower nods | 00:20 | |
| lazyPower | i'm in the exact same boat | 00:20 |
| lazyPower | i keep thinking i'm going to deploy it and poke at it, but haven't found the time yet | 00:20 |
| arosales | magicaltrout: re push marcoceppi is working on getting the latest version into Ubuntu 16.04 as we speak. Post that I think he is going to announce to the list how folks can try the beta | 00:29 |
| marcoceppi | arosales magicaltrout tldr: tomorrow - https://lists.ubuntu.com/archives/juju/2016-March/006910.html | 00:30 |
| arosales | marcoceppi: thanks | 00:36 |
| lazyPower | marcoceppi - considering the store breaks from DVCS then - i guess i should round up the scripts and give this one more go eh? https://youtu.be/rz5DVte0SKg | 00:39 |
| arosales | marcoceppi: does the charm command allow for setting an upstream url for a bundle? | 00:40 |
| marcoceppi | arosales: probably? | 00:40 |
| marcoceppi | my guess is yes | 00:40 |
| arosales | marcoceppi: I may confirm that with urulama|afk | 00:40 |
| arosales | marcoceppi: thanks | 00:40 |
| lazyPower | arosales - output of charm set shows it as true for a charm, unclear on bundle | 00:40 |
| lazyPower | but i would imagine it has the same meta properties as charms | 00:41 |
| lazyPower | marcoceppi - does this mean tomorrow i need to gut the PPA from devel flavor? | 00:41 |
| marcoceppi | lazyPower: devel flavor of? | 00:41 |
| lazyPower | charmbox | 00:41 |
| arosales | ya I was just wondering if someone wanted to fork or contribute to a bundle how would they do so | 00:41 |
| marcoceppi | lazyPower: does it have juju/devel in there? | 00:41 |
| lazyPower | it does, but it also has ppa:marcoceppi | 00:41 |
| marcoceppi | lazyPower: juju/devel will be a higher version than ppa:marcoceppi | 00:42 |
| lazyPower | perfect | 00:42 |
| lazyPower | no rush then | 00:42 |
| marcoceppi | ~beta vs ~rc | 00:42 |
| marcoceppi | then when xenial lands I'll backport into ppa:juju/stable | 00:43 |
| lazyPower | solid, i just dont want stale cruft hanging around as :devel is going to supplant :latest when we release 2.0 | 00:43 |
| lazyPower | and :latest is going to move to a 1.25 tag, frozen in time | 00:43 |
| marcoceppi | lazyPower: ack | 00:47 |
| marcoceppi | lazyPower: going forward, esp for charm-tools, I want to kind of build a release schedule and standardized ppa setup for it | 00:47 |
| lazyPower | i'm for this :+1: | 00:47 |
| marcoceppi | lazyPower: yeah, we're going to fall in line with the core 6 month cycle | 00:49 |
| lazyPower | part of what i'm doing now is trying to bring the docker images in alignment with our other tooling so its on a stable build cycle too, and i want to get better docs around its uses. its a pretty versatile porky little toolbox. Once we get matts lxd image and our new vagrant scripts in alignment with this effort we'll have a nice pipeline for everything | 00:49 |
| lazyPower | all based out of the groundwork we have in the old vagrant image, now in charmbox, moving along to the other projects :) | 00:50 |
| marcoceppi | lazyPower: you going to be around for a code review in like 30 mins? | 00:50 |
| lazyPower | sure | 00:50 |
| lazyPower | i gotta take out the garbage, let me go do that and i'll do CR for ya | 00:50 |
| arosales | cory_fu: kwmonroe: for what ever reason realtime-syslog-analtics works in us-west-2 | 00:56 |
| magicaltrout | magical reasons | 00:56 |
| * thumper likes magic | 00:59 | |
| lazyPower | arosales - just welcomed cloudguru to the honorary containers-contributors team :) | 01:00 |
| lazyPower | \o/ | 01:00 |
| lazyPower | thumper o/ | 01:01 |
| lazyPower | fancy seeing you here | 01:01 |
| thumper | I'm always here | 01:01 |
| thumper | just not always talkative | 01:01 |
| lazyPower | thats the punchline :D | 01:01 |
| mgz | I'd be careful, thumper probably likes trout too | 01:01 |
| thumper | not a big fish fan actually | 01:02 |
| magicaltrout | \o/ | 01:02 |
| thumper | trout is too subtle for me | 01:02 |
| magicaltrout | unless its rainbow | 01:02 |
| magicaltrout | or magical | 01:02 |
| thumper | lazyPower: when are we next going to be at the same sprint? | 01:03 |
| lazyPower | thumper - i have no clue but i'm totes ready for another coffee adventure | 01:03 |
| arosales | nice congrats cloudguru and thanks for the contributions | 01:05 |
| lazyPower | thumper are you coming to the next charmer summit? | 01:05 |
| thumper | when and where? | 01:05 |
| marcoceppi | lazyPower: payloads, what are teh valid values? | 01:05 |
| lazyPower | marcoceppi : 1 sec | 01:05 |
| lazyPower | thumper: late september, looking like pasadena ca | 01:06 |
| * marcoceppi tries to get 93 patched for 2.0 | 01:06 | |
| thumper | hmm... | 01:06 |
| marcoceppi | thumper: pasadena, ca mid-sept | 01:06 |
| thumper | sounds interesting | 01:06 |
| thumper | how mid? | 01:06 |
| thumper | kiwipycon is 9-11 sep | 01:06 |
| marcoceppi | like 12-14 tentatively | 01:06 |
| thumper | \o/ | 01:06 |
| thumper | hmm... | 01:07 |
| thumper | it's possible then | 01:07 |
| lazyPower | marcoceppi type docker type kvm is all that i'm aware of, but i'm pretty sure it was arbitrary | 01:07 |
| thumper | how does one get to pasadena? | 01:07 |
| thumper | lax? | 01:07 |
| marcoceppi | thumper: basically | 01:08 |
| thumper | or sfo? | 01:08 |
| marcoceppi | thumper: lax for sure, for international | 01:08 |
| thumper | is it a drive or flight from LAX? | 01:08 |
| marcoceppi | thumper: drive, 20 mins or so | 01:08 |
| thumper | hmm... | 01:09 |
| marcoceppi | lazyPower: I'll just have kvm and docker for now, I think 2.0.1 will fill the gaps | 01:09 |
| thumper | I'd really like to come... | 01:09 |
| lazyPower | ack | 01:09 |
| thumper | perhaps we could even fix python-django charm to suck less | 01:09 |
| thumper | like rewrite it or something | 01:09 |
| lazyPower | thumper - i hear this thing called layers is all the rage | 01:09 |
| thumper | yeah, I've had no time to play | 01:10 |
| marcoceppi | thumper: I fixed the python-django charm, by making a django layer | 01:10 |
| lazyPower | :) | 01:10 |
| * thumper knows nothing | 01:10 | |
| thumper | marcoceppi: does it use virtual envs? | 01:11 |
| marcoceppi | thumper: if you want it to, yes | 01:11 |
| thumper | marcoceppi: and can you upgrade the django version? | 01:11 |
| marcoceppi | thumper: yes | 01:11 |
| thumper | python 3? | 01:11 |
| marcoceppi | thumper: ootb | 01:11 |
| thumper | sounds like I want that magic | 01:11 |
| marcoceppi | thumper: sounds like you want layers | 01:11 |
| thumper | fuck | 01:11 |
| * marcoceppi makes rainbow magic hand wave | 01:11 | |
| * thumper needs more hours | 01:11 | |
| thumper | what about nginx? | 01:11 |
| thumper | and gunicorn? | 01:12 |
| marcoceppi | thumper: duh, thta's another layer, and they work together, like magic | 01:12 |
| thumper | for django | 01:12 |
| marcoceppi | ootb | 01:12 |
| marcoceppi | thumper: here's a stupid simple example of implementing a django site as layers with nginx | 01:12 |
| thumper | so instead of having multiple charms, we just have one? | 01:12 |
| marcoceppi | thumper: no, we have layers | 01:12 |
| marcoceppi | and you assemble them to build a charm for your django application | 01:13 |
| marcoceppi | but you reuse the operational components that comprise that solution | 01:13 |
| thumper | right, but you assemble it into a charm right? | 01:13 |
| marcoceppi | yes | 01:13 |
| thumper | so instead of python-django, gunicorn, and my subordinate payload charm | 01:13 |
| marcoceppi | you just deploy "your charm" | 01:13 |
| thumper | I add my payload into a layer, and use django layer | 01:13 |
| marcoceppi | and it's django, gunicorn, nginx, as a charm | 01:14 |
| thumper | right | 01:14 |
| thumper | much nicer | 01:14 |
| marcoceppi | very much | 01:14 |
| marcoceppi | the glue code is pretty straight forward to | 01:14 |
| marcoceppi | thumper: http://bazaar.launchpad.net/~ubucon-site-developers/ubucon-site/ubucon-layer/view/head:/reactive/ubucon.py | 01:14 |
| marcoceppi | that's the basically all it takes to get ubucon.org deployed | 01:14 |
| thumper | unfortunately I have so little time, it's negative right now | 01:14 |
| marcoceppi | thumper: I know. I feel it man | 01:15 |
| marcoceppi | but soon, one day, probably - maybe ;) | 01:15 |
| lazyPower | thumper - i had no idea the days we were spending crossover hours hacking on your project, that it was literally the only time you were ever going to have to hang out | 01:15 |
| lazyPower | RIP fun | 01:15 |
| thumper | yeah... | 01:16 |
| lazyPower | hey man, it was fun while it was a thing :D | 01:16 |
| thumper | model migrations won't be fully functional for 2.0\ | 01:16 |
| thumper | and I've just been pulled onto MAAS 2.0 support | 01:16 |
| lazyPower | eventually you'll come back for the layers, thats how we get ya :D | 01:16 |
| lazyPower | until then i'll keep a pot of coffee on for ya and the light on the porch. | 01:16 |
| lazyPower | mind magicaltrout on your way in | 01:17 |
| marcoceppi | lazyPower: just finishing up unit tests | 01:25 |
| lazyPower | solid, still g2g whenever you are | 01:25 |
| marcoceppi | lazyPower: https://github.com/juju/charm-tools/pull/150 | 01:30 |
| marcoceppi | lazyPower: should clear travis in a min | 01:31 |
| lazyPower | Dude, your test skills | 01:32 |
| lazyPower | they have like, level ++++++ | 01:32 |
| marcoceppi | lazyPower: yeah, my copy and paste skills are on the up and up (shamelessly ripped from whoever - cory I think - made the storage tests for metadata yaml) | 01:34 |
| lazyPower | I thought it looked suspiciously efficient | 01:34 |
| lazyPower | +1 LGTM | 01:34 |
| lazyPower | can i click the button? | 01:34 |
| lazyPower | Because i've waited very patiently for this one \o/ and you just made my day :D | 01:34 |
| marcoceppi | lazyPower: go for it | 01:34 |
| marcoceppi | lazyPower: 2.0 is 100% complete. Time to cut a release | 01:35 |
| marcoceppi | lazyPower: you still around? | 02:35 |
| lazyPower | You betchya | 02:35 |
| marcoceppi | lazyPower: check out ppa:juju/devel and update/upgrade ;) | 02:36 |
| lazyPower | on it! | 02:40 |
| === scuttlemonkey is now known as scuttle|afk | ||
| === thumper is now known as thumper-afk | ||
| === scuttle|afk is now known as scuttlemonkey | ||
| === blahdeblah is now known as Killfirefox | ||
| === Killfirefox is now known as blahdeblah | ||
| === beisner- is now known as beisner | ||
| === urulama|afk is now known as urulama | ||
| jamespage | falanx, that's being worked on atm | 07:08 |
| jamespage | zfs on wily was not as accessible | 07:08 |
| jamespage | gnuoy, charm-tools 2.0.0 just broke our gate for charms... | 08:24 |
| gnuoy | ah, that's why your mps failed | 08:24 |
| gnuoy | I did wonder | 08:24 |
| jamespage | gnuoy, basically the fix is to switch from "charm proof" -> "charm-proof" | 08:24 |
| jamespage | gnuoy, I'll raise reviews now... | 08:37 |
| jamespage | gnuoy, as nothing can get past verification atm | 08:37 |
| gnuoy | kk, ta | 08:37 |
| marcoceppi | jamespage gnuoy if it's easier to change the dep, 1.11.2 is still on pypi if you charm-tools==1.11.2 to avoid 2.0 atm | 08:43 |
| jamespage | marcoceppi, meh | 08:45 |
| jamespage | marcoceppi, its the same work either way | 08:45 |
| marcoceppi | ack, again, sorry about that | 08:45 |
| jamespage | gnuoy, generating reviews now - https://review.openstack.org/#/q/status:open+branch:master+topic:charm-tools-2.0 | 08:59 |
| gnuoy | excellent | 09:00 |
| jamespage | gnuoy, same change across all charms - the only one I could not do was lxd - rockstar - you'll need to pull in the same fix to your in-flight review | 09:01 |
| jamespage | marcoceppi, oh great - theblues has a == on requests 2.6.0 | 09:04 |
| jamespage | which is creating problems on OSCI verification, but not in upstream | 09:06 |
| jamespage | that's odd | 09:06 |
| gnuoy | jamespage, is it right that hacluster is not part of our charm collection on github? | 09:26 |
| jamespage | gnuoy, yes | 09:30 |
| gnuoy | ack, ta | 09:30 |
| jamespage | gnuoy, I'm having to pin requests to 2.6.0 in the charm test-requirements.txt to avoid pep8 lint failures in OSCI for now | 09:31 |
| jamespage | we can drop that later - just want to unblock the gate right now | 09:31 |
| gnuoy | ok | 09:31 |
| jamespage | gnuoy, i think the version of pip and pkg_resources that the osci lab uses is different to upstream and to my local xenial install which causes some issues with entry point loading without that | 09:33 |
| jamespage | gnuoy, ah crap - I managed to re-create the reviews... | 09:35 |
| jamespage | gnuoy, I was using a pre-canned commit message - misses the ID's that git review added first time around | 09:35 |
| === thumper-afk is now known as thumper | ||
| jamespage | gnuoy, I'm making alot of load of OSCI but generally the lint failures I saw first time round have gone with the version pin | 09:54 |
| gnuoy | kk | 09:54 |
| jamespage | gnuoy, anything with a +1 post 10:00 in the following list is good to land IMHO - https://review.openstack.org/#/q/status:open+branch:master+topic:charm-tools-2.0 | 10:29 |
| === scuttlemonkey is now known as scuttle|afk | ||
| marcoceppi | jamespage: yeah, that's what this is about | 11:38 |
| jamespage | marcoceppi, the fixed requests version in theblues? | 11:39 |
| marcoceppi | jamespage: yeah, see pm | 11:39 |
| jamespage | gnuoy, time to unblock the gate? | 12:14 |
| jamespage | https://review.openstack.org/#/q/status:open+branch:master+topic:charm-tools-2.0 | 12:14 |
| jamespage | two failed due to amulet problems - but I suspect they have never passed... | 12:14 |
| jamespage | beisner, ^^ | 12:14 |
| gnuoy | jamespage, ack, looking | 12:14 |
| beisner | dosaboy, ack re: lp bugs going directly to fix-released via upstream. aiui, it's a behavior change and by design. ref: | 13:20 |
| beisner | https://review.openstack.org/#/c/248922/ | 13:21 |
| beisner | http://lists.openstack.org/pipermail/openstack-dev/2015-November/080288.html | 13:21 |
| beisner | jamespage, gnuoy - unless i've missed one, the osci tests are all calling tox -e pep8 which i think should give you the same experience anywhere, shouldn't it? | 13:21 |
| jamespage | beisner, well apparently not | 13:22 |
| jamespage | beisner, as the upstream verification passed, but OSCI bailed on the fixed requests version in theblues.... | 13:22 |
| beisner | jamespage, if the tox ini is set to also use system packages, then definitely not. some of them i believe are. | 13:22 |
| jamespage | beisner, openstack add in new pkg-resources and stuff which i suspect we don't | 13:22 |
| beisner | jamespage, ah right. we're just doing whatever the tox.ini and *-requirements.txt files instruct | 13:24 |
| beisner | jamespage, ack re: cinder-backup amulet not passing. can't raise a bug, but added a card for triage. | 13:39 |
| jamespage | beisner, I suspec the tox/pip/pkg-resource environment on the upstream ci is not the same as trusty... | 13:42 |
| beisner | jamespage, right. imho we shouldn't be leaning on site-packages at all on unit/lint tests. | 13:42 |
| jamespage | beisner, tbh its probably not - but depending on which tox/pip you use determines which versions of stuff you get in the venv | 13:43 |
| beisner | jamespage, lovely. so, build a venv, install a specific version of tox into it, then call tox? | 13:44 |
| jamespage | beisner, not quite | 13:44 |
| beisner | jamespage, ack i see you pinned requests. looking at other upstream projects, they pin requests to a range of versions as well. | 13:57 |
| jamespage | beisner, that should be droppable once theblues sorts things out | 13:57 |
| jamespage | beisner, marcoceppi is dealing with that | 13:57 |
| marcoceppi | jamespage: https://github.com/juju/theblues/pull/20 and http://paste.ubuntu.com/15477938/ | 13:58 |
| jamespage | yah gotcha | 13:58 |
| marcoceppi | jamespage: is that enough? | 13:58 |
| jamespage | marcoceppi, I commented generally on your pull request... | 13:59 |
| marcoceppi | jamespage: I missed that, thanks | 13:59 |
| marcoceppi | some balked at the idea of using a range | 13:59 |
| marcoceppi | and this is why I welcome, snappy, our new packaging overlord | 14:00 |
| jamespage | gnuoy, ok for me to +1 workflow those two rollup config changes? | 14:01 |
| jamespage | they are passing pep8 ok now with a rebase... | 14:01 |
| gnuoy | jamespage, yep, please do | 14:02 |
| jamespage | gnuoy, that feels nice... | 14:03 |
| jamespage | I like ditching lots of old code... | 14:03 |
| gnuoy | tinwood, would you mind taking a look at https://code.launchpad.net/~gnuoy/charms/trusty/hacluster/pause-resume/+merge/289911 if/when you have a moment? | 14:50 |
| tinwood | gnuoy, sure. I'll do it this afternoon. | 14:51 |
| gnuoy | ta | 14:51 |
| tinwood | gnuoy, is hacluster not on github then? | 14:52 |
| gnuoy | tinwood, apparently not, I was surprised too | 14:52 |
| tinwood | gnuoy, how strange. Oh well. | 14:53 |
| tinwood | gnuoy, this ceph-mon charm is quite awkward. I think I've got it right now - just doing the amulet tests on my bastion to see if pause/resume actually works! | 14:53 |
| gnuoy | tinwood, hacluster as odd ball too | 14:56 |
| gnuoy | beisner, I accidentally posted a charm recheck full on https://review.openstack.org/#/c/294554/ after already having a successful run and just my luck the second run failed due to a deploy timeout which I assume is unrelated to my change. Do I have to do another charm-recheck-full to get rid of the canonical ci -1 do you think? | 14:57 |
| === JoseeAntonioR is now known as jose | ||
| === cos1 is now known as c0s | ||
| === scuttle|afk is now known as scuttlemonkey | ||
| falanx | Is there a best practice for filesystems on lxd/juju? There are many articles saying zfs is where the party is, but nothing definitive stating what has the best support vs performance. | 15:27 |
| rick_h_ | falanx: yes, running lxd-init on xenial I think will ask about zfs as an option and that will help with the best setup for speed. | 15:29 |
| rick_h_ | falanx: I think jcastro has done some testing/etc and has a sweet setup | 15:30 |
| beisner | hi gnuoy, i'll have a look and holler back shortly | 15:30 |
| gnuoy | beisner, thanks much appreciated | 15:32 |
| jcastro | falanx: I've been pretty happy with lxd/zfs/juju | 15:38 |
| beisner | gnuoy, yes we can ignore the results of that recheck. it was actually a nova-compute failure on xenial due to libvirt/locale bug 1560939. | 15:40 |
| mup | Bug #1560939: libvirt-bin fails to install on a fresh xenial server <amd64> <apport-bug> <ec2-images> <uosci> <xenial> <libvirt (Ubuntu):New> <https://launchpad.net/bugs/1560939> | 15:40 |
| beisner | gnuoy, shall i push that now? | 15:40 |
| falanx | jcastro rick_h_ thanks! we're looking to do that combo + openstack in production asap =) | 15:52 |
| jcastro | falanx: heh I should note that work laptop filesystem and your production openstack architecture filesystem recommendations are probably different. :) | 15:53 |
| beisner | rockstar, ok although tests did pass a day ago, there has been some movement in charm-tools requirements and tests now fail. all of the charms except lxd have been updated to satisfy and work around that issue. your change will need one more patchset. | 16:33 |
| beisner | rockstar, here is an example of what needs to change: https://review.openstack.org/#/c/296312/ ... and that should let us merge this puppy. | 16:33 |
| rockstar | beisner: a re-sync should do it, right? | 16:34 |
| rockstar | Oh, no, no it won't. :) | 16:34 |
| beisner | rockstar, i don't see that lxd has received those fixes at master | 16:34 |
| rockstar | On it. | 16:35 |
| beisner | coolio | 16:35 |
| tinwood | cholcombe, I wonder if I could ask you a question about what pause-health does in the ceph-mon charm? | 16:54 |
| === cos1 is now known as c0s | ||
| balloons | jcastro, is there an up to date guide on using juju2 with lxd? | 17:11 |
| jcastro | balloons: https://jujucharms.com/docs/devel/config-LXD | 17:19 |
| jcastro | balloons: oh, this reminded me I need to PR and fix a few bits there | 17:20 |
| balloons | jcastro, I have some thoughts on that page, lol | 17:21 |
| balloons | but if that is it, I'm not seeing success. It seems like it's missing some bits, and I'm left confused on a few points | 17:22 |
| jcastro | the lxd-images stuff is all out of date | 17:22 |
| jcastro | all you should need to do is `sudo lxd init` | 17:22 |
| jcastro | balloons: to be fair though, all of this was in flux until the other day | 17:23 |
| jcastro | alexisb: do we know if juju still wants "ubuntu-trusty" as an alias for a container or will it just use the automagic remotes LXD provides? | 17:25 |
| balloons | I have to use the specific alias name? | 17:25 |
| pmatulis | balloons, jcastro: guys, that page was heavily refreshed yesterday. i can't remove lxd-images b/c you can't use Xenial Juju machines without it | 17:25 |
| rick_h_ | jcastro: she's out, I think it looks for that specific alias so that it doesn't stomp on the existing lxd use | 17:25 |
| balloons | jcastro, I guess I'd love to see your PR, before I start firing off bugs | 17:25 |
| jcastro | you used to have to, not sure if it still does | 17:25 |
| jcastro | pmatulis: oh ok so we still need that command then? | 17:26 |
| jcastro | I think that's a bug then? The intent was to just use what LXD provides without the user having to know that command. | 17:26 |
| pmatulis | balloons, jcastro: unless the user is smart enough to know they need to create an alias for the Xenial image | 17:26 |
| pmatulis | beta3 is supposed to create the alias automatically | 17:27 |
| jcastro | ah ok | 17:27 |
| jcastro | so it's still in flux then | 17:27 |
| pmatulis | is there anything else with that page that is wrong or can be extended? lemme know or open an issue | 17:27 |
| mgz | rick_h_: we don't care too much if juju2 is in the xenial beta2 image right? | 17:28 |
| jcastro | all we need after that is for bootstrap to work without needing --upload-tools and I'm good to go | 17:29 |
| jcastro | balloons: which part are you stuck on? | 17:29 |
| jcastro | mgz: we're having a hard time getting in also for charm-tools, so we said nope and going to try as soon as beta2 is out. | 17:31 |
| jcastro | all the bugs are filed, we're just waiting in line | 17:31 |
| pmatulis | anyone know why i would not have juju2 sub-command completion on a fresh Xenial openstack instance? | 17:34 |
| pmatulis | scratch that, seems a reboot fixed it | 17:35 |
| jcastro | pmatulis: I am wondering if keeping around the "changes from lxc local provider" in that page just adds length for something most people won't care about | 17:39 |
| balloons | jcastro, I can't juju bootstrap. And the instructions left me questioning if just installing juju got me the lxd provider or not | 17:40 |
| balloons | also, do I need an environments.yaml file? How do I create a controller for lxd? | 17:40 |
| pmatulis | jcastro: yes, that's debatable | 17:40 |
| jcastro | https://jujucharms.com/docs/devel/controllers-creating | 17:40 |
| jcastro | you do not need an environments.yaml | 17:40 |
| pmatulis | balloons: you're not reading the page :D | 17:41 |
| jcastro | after you have it installed all you should need to do is `juju bootstrap blah lxd --upload-tools | 17:41 |
| jcastro | the style of the docs are not condusive to "ok you've finished this page, go to this page next" because there's no pagination | 17:42 |
| jcastro | perhaps once we have the final changes in from the tool we should consider just doing a quick and dirty summary up top that gets a controller fired up | 17:43 |
| jcastro | and then while that is happening the user can read all the details below | 17:43 |
| pmatulis | jcastro: maybe you're looking for a tutorial or yet another getting-started page? | 17:44 |
| pmatulis | jcastro: Juju is complex and it's very difficult to have a running story that fits everyone's purpose | 17:44 |
| jcastro | I can't fix the get-started page until 2.0 is out though | 17:45 |
| jcastro | and it's not difficult, we're just telling people things in the wrong order | 17:45 |
| pmatulis | jcastro: there are plans for a much-needed "architectural overview" page replete with diagrams and core concepts, evern arrows but there is so much to do | 17:46 |
| jcastro | yeah it's just like, the instructions have CLI commands in them | 17:46 |
| jcastro | and then at the end of the section it's like "xenial comes with lxd by default" | 17:46 |
| pmatulis | what's wrong with saying that? | 17:47 |
| jcastro | actually, let me just ask luca if we can start on the 16.04 version of the get-started page | 17:47 |
| jcastro | well, all the commands above that are explaining how to install and configure lxd | 17:47 |
| pmatulis | install on Wily maybe | 17:48 |
| pmatulis | and LXD does not require any configuration, unless you want to use network stuff or ZFS | 17:48 |
| pmatulis | maybe if you give me a specific example | 17:49 |
| jcastro | I'm just saying we can trim most of that entire section out and make firing up the controller more upfront | 17:49 |
| jcastro | like a TLDR version | 17:49 |
| pmatulis | well, these are the definitive/upstream docs for Juju. everything cannot be a TLDR | 17:50 |
| pmatulis | a TLDR/getting-started/quickstart should be separate | 17:51 |
| balloons | well I had to not use lxd-images (that fails), and then starting a container mannually failed. So not really juju's fault I guess. But the bootstrap command didn't recognize lxd as a provider for instance | 17:51 |
| balloons | anyways, are you planning to swap the 'getting started' guide to using lxd for 2.0? | 17:51 |
| pmatulis | balloons: dunno what's going on over there. sounds like LXD itself is not working | 17:52 |
| jcastro | I don't think we can swap it for LXD on get-started, the get-started is supposed to work on all OSes afaict. | 17:53 |
| balloons | pmatulis, right, so I'm not blaming juju for that at all. But as I said the bootstrap command didn't work, and I installed juju-local to maybe fix it -- but the page doesn't mention that | 17:53 |
| balloons | I know with juju2 it's changing, so if nothing else I can say the page is confusing at the moment | 17:54 |
| balloons | and I would like to play with lxd + juju2 locally, however that may be possible | 17:54 |
| jcastro | wait, are you trying to use juju1 with lxd? | 17:54 |
| pmatulis | balloons: yeah, you should be using Juju 2 | 17:56 |
| pmatulis | and don't get me started with the getting-started page :) | 17:58 |
| balloons | jcastro, I wasn't, but again, those pages make things confusing. Also, I can't get juju2 in juju/stable (or course not), but the page mentions it | 17:58 |
| jcastro | yeah | 17:59 |
| jcastro | you're in the middle of an awkward transition | 17:59 |
| jcastro | the page is being written for the future while the commands are still being written | 18:00 |
| jcastro | balloons: all the good information on how to use juju2 is in the release notes and not the docs | 18:02 |
| jcastro | like how to list-clouds, the new credentials stuff, etc. | 18:02 |
| jcastro | balloons: I can walk you through lxd in a hangout after I'm done with this meeting I'm on now | 18:06 |
| magicaltrout | ballons afaik you should be good on the latest beta from the ppa | 18:15 |
| magicaltrout | juju bootstrap dev-lxd lxd --debug is about all you need once you have a lxd trusty image | 18:15 |
| pmatulis | balloons: yep, i concede there is confusion (no juju2 in stable) but, given the resources we have, it was decided to write for the future. this is a devel branch after all | 18:15 |
| pmatulis | (the software and the docs are not released) | 18:16 |
| balloons | pmatulis, yes I agree. Write towards what is landing (and there is the header stating as such) | 18:16 |
| LiftedKilt | 18:38 | |
| === cos1 is now known as c0s | ||
| === redir is now known as redir-lunch | ||
| pmatulis | nothing under the kilt? | 19:17 |
| === freyes_ is now known as freyes | ||
| === bogdanteleaga_ is now known as bogdanteleaga | ||
| === kadams54_ is now known as kadams54 | ||
| === mup_ is now known as mup | ||
| LiftedKilt | pmatulis: it was feeling a bit breezy there for a second, that's for sure | 19:22 |
| === redir-lunch is now known as redir | ||
| cory_fu | c0s: So, the main problem that admcleod- was talking about earlier when he mentioned multiple namenodes is that there is a bug in Juju leadership in the current stable (1.25) where if you spin up and then tear down two NNs, then spin up two more without re-bootstrapping the entire environment, a leader isn't selected. | 20:16 |
| cory_fu | I haven't encountered it, personally, so I can't really give any more detials | 20:17 |
| cory_fu | *details | 20:17 |
| c0s | cory_fu I am a bit confused: how Juju leader election is related to ZKFC leader election for HDFS HA? | 20:17 |
| cory_fu | But from what he was saying, the HDFS HA should be pretty functional | 20:17 |
| cory_fu | c0s: He's using juju leadership to ensure that only one NN tries to do the formatting and other logic that apparently should only happen once. Once it's up and configured to HA, juju leadership is irrelevant | 20:18 |
| c0s | cory_fu: oh, that's the juju way of handling the distributed locking. Got it. | 20:18 |
| cory_fu | Right | 20:18 |
| c0s | cory_fu: by the way: Here's Bigtop puppet recipes on the topic: https://github.com/apache/bigtop/blob/master/bigtop-deploy/puppet/modules/hadoop/manifests/init.pp#L556 and all the way to line 607 | 20:19 |
| cory_fu | admcleod-: ^ for reference when you sign back on | 20:19 |
| c0s | I will add this to the review notes as well. And thanks for the explanation: now I can focus on the HDFS HA part ;) | 20:20 |
| cory_fu | c0s: I haven't looked at the HA branches in much detail recently, so I can't speak to it other than what's been explained during the dailies | 20:20 |
| cory_fu | Cool. Thanks | 20:20 |
| c0s | cory_fu: speaking of solving the distributed lock issue | 20:30 |
| c0s | what if active and standby NNs know about their roles from the beginning? | 20:30 |
| c0s | This way, the file system formatting can only be done on the primary code by simply checking against the namenode role. | 20:30 |
| cory_fu | How would they know their role? | 20:31 |
| c0s | when you deploy the services you can arbitrary chose to be an active and another to be standby | 20:32 |
| c0s | here's how we do it | 20:32 |
| c0s | Here’s the link to Bigtop’s puppet recipes to setup HDFS HA http://is.gd/d0cWel from line 556 all the way to line 607 | 20:32 |
| c0s | scratch that | 20:32 |
| c0s | https://github.com/apache/bigtop/blob/master/bigtop-deploy/puppet/modules/hadoop/manifests/init.pp#L639 | 20:32 |
| c0s | we always pick the 1st node to carry on as the active and to hell with that ;) | 20:33 |
| c0s | this is out choice | 20:33 |
| c0s | because the same recipe code is getting executed on all the namenodes, with such check we guarantee "only one" execution | 20:34 |
| c0s | cause otherwise to expose the internal HDFS concern up a layer to the orchestrator, and that makes the implementation harder, of course | 20:34 |
| cory_fu | So, that's basically what we're doing with the Juju is_leader check. The "lowest numbered unit" would work and is what charms used before is_leader was available, but is_leader is generally considered better (barring the bug we're seeing) | 20:36 |
| cory_fu | In the end, it basically works out to the same thing, and in theory should actually be simpler since it's a simple boolean check (and the Juju controller decides on the leader) | 20:37 |
| cory_fu | Also, the bug we're hitting with is_leader really only affects iterative development use. Normal usage wouldn't be affected by it | 20:37 |
| cory_fu | I feel like there was a stronger reason why we moved away from "lowest unit" for this specific charm, as well, but I can't recall off the top of my head | 20:38 |
| c0s | ok, that makes sense. Thanks cory_fu for the explanation - learning a bit more each hour ;) | 20:39 |
| mux | new blog post about juju posted here: http://blog.emccode.com/2016/03/23/storage-operations-with-juju-charms/ | 21:03 |
| mux | specifically, exploring what's out there for attachable, detachable, persistent storage for charms | 21:04 |
| mux | (spoiler: not much.) | 21:04 |
| c0s | cory_fu: do you know if the bug for the leader election has been logged somewhere? | 21:05 |
| c0s | I have a feeling that it might be not a bug, but rather a limitation of ZK | 21:05 |
| c0s | if you look here https://aphyr.com/posts/291-call-me-maybe-zookeeper all the way to the Recommendations section, it says | 21:06 |
| c0s | "Also keep in mind that linearizable state in Zookeeper (such as leader election) does not guarantee the linearizability of a system which uses ZK. For instance, a cluster which uses ZK for leader election might allow multiple nodes to be the leader simultaneously..." | 21:06 |
| c0s | And that guy knows what he's talking about ;) | 21:06 |
| cory_fu | c0s: So, this doesn't actually have anything to do with ZK, only the Juju is-leader command. As I understand it, is that Juju's is-leader tool is supposed to return True for at least one unit at all times except for a somewhat brief fail-over period when the leader unit goes away. The problem is that Andrew is seeing normal behavior for units 1 and 2, but not 3 and 4 after removing the service entirely and redeploying it | 21:10 |
| cory_fu | Again, for a new service coming up, at least one unit (the first to come up) should see is-leader return True even before running any charm code | 21:10 |
| c0s | oh, I see. Not I think I got it - was mislead a bit by the presence of ZK from HDFS HA ;) | 21:11 |
| cory_fu | I assume there was a bug filed against core, but I'd have to go looking for it | 21:11 |
| c0s | no worries | 21:11 |
| c0s | so, juju doesn't use ZK under the hood, after all? | 21:11 |
| cory_fu | Yeah, there are a couple of orthogonal ideas of "leadership" going on | 21:11 |
| c0s | k, thanks! | 21:12 |
| cory_fu | No, the controller just picks one of the units as the leader and records that in Mongo. With the current single controller, something like ZK isn't needed at all. | 21:12 |
| cory_fu | c0s: Also, someone in #juju-dev could give a much more accurate explanation of how Juju's leadership code works | 21:13 |
| skay_ | my services seem to be running but workload state shows as unknown when I run juju status. | 21:13 |
| cory_fu | But from a charm's perspective, we're supposed to be able to rely on is-leader telling us if we're leader or not, and (almost) always having a leader | 21:13 |
| === skay_ is now known as skay | ||
| c0s | cory_fu: thanks, this really helps! | 21:14 |
| === skay is now known as Guest88295 | ||
| c0s | the fact of the single controller skipped my mind. | 21:15 |
| cory_fu | And of course, we're really only relying on that to ensure that only one unit runs the HA initialization logic. | 21:15 |
| === Guest88295 is now known as skay | ||
| magicaltrout | zhttp://www.oreilly.com/programming/free/how-to-make-mistakes-in-python.csp?imm_mid=0e1f49&cmp=em-prog-free-lp-ostx16_nem4_mistakes_in_python | 21:42 |
| magicaltrout | its like Oreilly knew I was hacking some python badly...... | 21:42 |
| lazyPower | Mayyybe someone knows the answer to this... i need to mock/patch a private method on an object. my google fu is failing me on finding the method to do so, any helpful pythonista's in the crowd? | 23:37 |
| jrwren | lazyPower: don't do that. :] | 23:51 |
| lazyPower | jrwren - just leave the priate method untested? | 23:51 |
| lazyPower | *private | 23:51 |
| mgz | lazyPower: also, you can just do that | 23:52 |
| mgz | private in python is just a convention | 23:52 |
| lazyPower | well, thing is - i need to patch it as its kind of the core dispatch method in the class | 23:52 |
| mgz | but genrally, you only want to test interface points, so not private helpers | 23:52 |
| lazyPower | i guess i could move it elsewhere and just patch it there | 23:52 |
| lazyPower | nebulous questions without context | 23:53 |
| * lazyPower resumes test hacking | 23:53 | |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!