/srv/irclogs.ubuntu.com/2016/03/23/#juju.txt

magicaltroutwhats the release schedule for push to charm store?00:00
lazyPowermagicaltrout - i propose an idea00:03
lazyPowernvm i just myth=busted myself00:03
c0sah... nvm00:04
lazyPowero/ c0s00:04
c0smakes me cry in pain00:04
c0sfortunately, I am not using it ;)00:04
lazyPoweri have it on good authority you're looking to do some puppet in charm layers?00:05
c0snot exactly00:06
c0swas rather wondering if this is possible, as all this operational knowledge has been codified a number of times either in puppet or chef00:06
c0sat the risk of sounding like a broken mp3 - Bigtop is the case in point ;)00:06
magicaltrouti didn't realise mp3's were written to spinning plastic discs00:07
c0syeah I know - I sound like a corner drug peddler ;)00:07
magicaltroutthats more like it!00:07
c0sthat's a quote from Futurama ;)00:07
c0sabout the mp300:07
c0sdunno if there are fans of that cartoon here00:07
magicaltroutas I complained to other Apache members today.... I have 6 1/2 hours of tutorials to write for ApacheCon..... I dont have time to watch tv (or cartoons ;) )00:08
magicaltroutlazyPower is ya man though00:09
lazyPowerwell, I can help00:09
c0sfortunately for you, that show ended years ago ;)00:09
lazyPoweri'm not super familiar with puppet, but there have been some charms submitted in puppet already - although they are an Openstack SDN vendor00:09
lazyPowerso the bindings in there may be helpful for us to look at, and extract for use in reactive charming00:09
lazyPowerare your bigtop puppet scripts done in heira?00:10
c0sthat'd be great, but first this community need to make up their mind how they want to cooperate with Bigtop00:10
c0swe are using hiera for the configuration, yes00:10
c0sbut it isn't like "it is done in hiera"00:10
lazyPoweryeah, i think we can extract some learnings there00:10
lazyPoweri'll reach out to apuimido and see if he's got 30 minutes or so to riff with me on pain points00:10
lazyPowerand i'll fire up charms.puppet i guess :)00:11
c0sideally, the knowledge part should stay the same and just be upstreamed00:11
c0sthat's how OSS works, you know ;)00:11
magicaltroutc0s: explain bigtop to me, its like the hadoop reference stack right? so how does that differ to Apache Hadoop or what these charmer guys ship?00:11
blahdeblah+1 if you can get something like hiera-eyaml integrated so that we can have a more sensible way of shunting secrets around the place00:11
c0spretty cool. I would be happy to help with that00:11
c0sBigtop is Apache BigData stack00:12
c0swith binaries, deployment, integration testing, and all 9 yards00:12
lazyPowerblahdeblah - we need to speak louder about secret management with juju when we're at sprints. it would be nice to even see a password config type so it at the bare minimum renders a password field in the gui00:12
c0s+ a framework to develop your own stacks00:12
lazyPoweractually scratch that - we dont need to speak louder, we do need to be consistent with getting a bug filed for it and hten beating the drum of that bug#00:12
* lazyPower goes to find it right now00:13
c0shopefully it help magicaltrout00:13
blahdeblahlazyPower: I don't know anything about the GUI side; I've never used it.  I just know that we end up having lots of secrets sitting around in places that are not encrypted at rest, and that's not a great look.00:13
lazyPoweryeah00:13
lazyPoweri agree00:13
lazyPowerhttps://bugs.launchpad.net/juju-core/+bug/128766100:14
mupBug #1287661: some config options should be considered secret <canonical-is> <config> <juju-core:Triaged> <https://launchpad.net/bugs/1287661>00:14
magicaltroutyup thanks c0s. I've swung by bigtop over the years but left non the wiser! clearly i need to do more reading00:15
c0ssure, no worries00:15
c0sI has developed a lot lately.00:15
c0swe have this cool feature where one can deploy docker-based cluster for development/testing purposes with a single command00:15
c0sand so on ;)00:16
* blahdeblah edits that bug report to reference something other than SSL certs, which aren't actually secrets00:16
lazyPower:D00:16
c0sand being a reference implementation is really becomes a focal point were the standard for the Hadoop eco-system is getting created.00:16
c0slike that www.odpi.org which I am helping to get off the ground now00:17
c0sanyway, switching off - see yall later.00:17
lazyPowercheers o/00:17
lazyPowerthanks for catching up c0s00:17
magicaltroutah yeah i recall that bit. cool.00:17
magicaltroutAdios c0s00:17
c0scheers00:18
blahdeblahlazyPower: I think there's also the whole issue of how they're stored, not just how they're displayed.  To achieve repeatable deploys, we currently need to keep secrets files hanging around on the deployment server, which is also not desirable.00:19
lazyPowerblahdeblah - right and projects like vault exist for this reason00:19
blahdeblahHaven't played with vault myself, but yeah; we need something like that00:20
* lazyPower nods00:20
lazyPoweri'm in the exact same boat00:20
lazyPoweri keep thinking i'm going to deploy it and poke at it, but haven't found the time yet00:20
arosalesmagicaltrout: re push marcoceppi is working on getting the latest version into Ubuntu 16.04 as we speak. Post that I think he is going to announce to the list how folks can try the beta00:29
marcoceppiarosales magicaltrout tldr: tomorrow - https://lists.ubuntu.com/archives/juju/2016-March/006910.html00:30
arosalesmarcoceppi: thanks00:36
lazyPowermarcoceppi - considering the store breaks from DVCS then - i guess i should round up the scripts and give this one more go eh? https://youtu.be/rz5DVte0SKg00:39
arosalesmarcoceppi: does the charm command allow for setting an upstream url for a bundle?00:40
marcoceppiarosales: probably?00:40
marcoceppimy guess is yes00:40
arosalesmarcoceppi: I may confirm that with urulama|afk00:40
arosalesmarcoceppi: thanks00:40
lazyPowerarosales - output of charm set shows it as true for a charm, unclear on bundle00:40
lazyPowerbut i would imagine it has the same meta properties as charms00:41
lazyPowermarcoceppi - does this mean tomorrow i need to gut the PPA from devel flavor?00:41
marcoceppilazyPower: devel flavor of?00:41
lazyPowercharmbox00:41
arosalesya I was just wondering if someone wanted to fork or contribute to a bundle how would they do so00:41
marcoceppilazyPower: does it have juju/devel in there?00:41
lazyPowerit does, but it also has ppa:marcoceppi00:41
marcoceppilazyPower: juju/devel will be a higher version than ppa:marcoceppi00:42
lazyPowerperfect00:42
lazyPowerno rush then00:42
marcoceppi~beta vs ~rc00:42
marcoceppithen when xenial lands I'll backport into ppa:juju/stable00:43
lazyPowersolid, i just dont want stale cruft hanging around as :devel is going to supplant :latest when we release 2.000:43
lazyPowerand :latest is going to move to a 1.25 tag, frozen in time00:43
marcoceppilazyPower: ack00:47
marcoceppilazyPower: going forward, esp for charm-tools, I want to kind of build a release schedule and standardized ppa setup for it00:47
lazyPoweri'm for this :+1:00:47
marcoceppilazyPower: yeah, we're going to fall in line with the core 6 month cycle00:49
lazyPowerpart of what i'm doing now is trying to bring the docker images in alignment with our other tooling so its on a stable build cycle too, and i want to get better docs around its uses. its a pretty versatile porky little toolbox. Once we get matts lxd image and our new vagrant scripts in alignment with this effort we'll have a nice pipeline for everything00:49
lazyPowerall based out of the groundwork we have in the old vagrant image, now in charmbox, moving along to the other projects :)00:50
marcoceppilazyPower: you going to be around for a code review in like 30 mins?00:50
lazyPowersure00:50
lazyPoweri gotta take out the garbage, let me go do that and i'll do CR for ya00:50
arosalescory_fu: kwmonroe: for what ever reason realtime-syslog-analtics works in us-west-200:56
magicaltroutmagical reasons00:56
* thumper likes magic00:59
lazyPowerarosales - just welcomed cloudguru to the honorary containers-contributors team :)01:00
lazyPower\o/01:00
lazyPowerthumper o/01:01
lazyPowerfancy seeing you here01:01
thumperI'm always here01:01
thumperjust not always talkative01:01
lazyPowerthats the punchline :D01:01
mgzI'd be careful, thumper probably likes trout too01:01
thumpernot a big fish fan actually01:02
magicaltrout\o/01:02
thumpertrout is too subtle for me01:02
magicaltroutunless its rainbow01:02
magicaltroutor magical01:02
thumperlazyPower: when are we next going to be at the same sprint?01:03
lazyPowerthumper - i have no clue but i'm totes ready for another coffee adventure01:03
arosalesnice congrats cloudguru and thanks for the contributions01:05
lazyPowerthumper are you coming to the next charmer summit?01:05
thumperwhen and where?01:05
marcoceppilazyPower: payloads, what are teh valid values?01:05
lazyPowermarcoceppi : 1 sec01:05
lazyPowerthumper: late september, looking like pasadena ca01:06
* marcoceppi tries to get 93 patched for 2.001:06
thumperhmm...01:06
marcoceppithumper: pasadena, ca mid-sept01:06
thumpersounds interesting01:06
thumperhow mid?01:06
thumperkiwipycon is 9-11 sep01:06
marcoceppilike 12-14 tentatively01:06
thumper\o/01:06
thumperhmm...01:07
thumperit's possible then01:07
lazyPowermarcoceppi type docker type kvm is all that i'm aware of, but i'm pretty sure it was arbitrary01:07
thumperhow does one get to pasadena?01:07
thumperlax?01:07
marcoceppithumper: basically01:08
thumperor sfo?01:08
marcoceppithumper: lax for sure, for international01:08
thumperis it a drive or flight from LAX?01:08
marcoceppithumper: drive, 20 mins or so01:08
thumperhmm...01:09
marcoceppilazyPower: I'll just have kvm and docker for now, I think 2.0.1 will fill the gaps01:09
thumperI'd really like to come...01:09
lazyPowerack01:09
thumperperhaps we could even fix python-django charm to suck less01:09
thumperlike rewrite it or something01:09
lazyPowerthumper - i hear this thing called layers is all the rage01:09
thumperyeah, I've had no time to play01:10
marcoceppithumper: I fixed the python-django charm, by making a django layer01:10
lazyPower:)01:10
* thumper knows nothing01:10
thumpermarcoceppi: does it use virtual envs?01:11
marcoceppithumper: if you want it to, yes01:11
thumpermarcoceppi: and can you upgrade the django version?01:11
marcoceppithumper: yes01:11
thumperpython 3?01:11
marcoceppithumper: ootb01:11
thumpersounds like I want that magic01:11
marcoceppithumper: sounds like you want layers01:11
thumperfuck01:11
* marcoceppi makes rainbow magic hand wave01:11
* thumper needs more hours01:11
thumperwhat about nginx?01:11
thumperand gunicorn?01:12
marcoceppithumper: duh, thta's another layer, and they work together, like magic01:12
thumperfor django01:12
marcoceppiootb01:12
marcoceppithumper: here's a stupid simple example of implementing a django site as layers with nginx01:12
thumperso instead of having multiple charms, we just have one?01:12
marcoceppithumper: no, we have layers01:12
marcoceppiand you assemble them to build a charm for your django application01:13
marcoceppibut you reuse the operational components that comprise that solution01:13
thumperright, but you assemble it into a charm right?01:13
marcoceppiyes01:13
thumperso instead of python-django, gunicorn, and my subordinate payload charm01:13
marcoceppiyou just deploy "your charm"01:13
thumperI add my payload into a layer, and use django layer01:13
marcoceppiand it's django, gunicorn, nginx, as a charm01:14
thumperright01:14
thumpermuch nicer01:14
marcoceppivery much01:14
marcoceppithe glue code is pretty straight forward to01:14
marcoceppithumper: http://bazaar.launchpad.net/~ubucon-site-developers/ubucon-site/ubucon-layer/view/head:/reactive/ubucon.py01:14
marcoceppithat's the basically all it takes to get ubucon.org deployed01:14
thumperunfortunately I have so little time, it's negative right now01:14
marcoceppithumper: I know. I feel it man01:15
marcoceppibut soon, one day, probably - maybe ;)01:15
lazyPowerthumper - i had no idea the days we were spending crossover hours hacking on your project, that it was literally the only time you were ever going to have to hang out01:15
lazyPowerRIP fun01:15
thumperyeah...01:16
lazyPowerhey man, it was fun while it was a thing :D01:16
thumpermodel migrations won't be fully functional for 2.0\01:16
thumperand I've just been pulled onto MAAS 2.0 support01:16
lazyPowereventually you'll come back for the layers, thats how we get ya :D01:16
lazyPoweruntil then i'll keep a pot of coffee on for ya and the light on the porch.01:16
lazyPowermind magicaltrout on your way in01:17
marcoceppilazyPower: just finishing up unit tests01:25
lazyPowersolid, still g2g whenever you are01:25
marcoceppilazyPower: https://github.com/juju/charm-tools/pull/15001:30
marcoceppilazyPower: should clear travis in a min01:31
lazyPowerDude, your test skills01:32
lazyPowerthey have like, level ++++++01:32
marcoceppilazyPower: yeah, my copy and paste skills are on the up and up (shamelessly ripped from whoever - cory I think - made the storage tests for metadata yaml)01:34
lazyPowerI thought it looked suspiciously efficient01:34
lazyPower+1 LGTM01:34
lazyPowercan i click the button?01:34
lazyPowerBecause i've waited very patiently for this one \o/ and you just made my day :D01:34
marcoceppilazyPower: go for it01:34
marcoceppilazyPower: 2.0 is 100% complete. Time to cut a release01:35
marcoceppilazyPower: you still around?02:35
lazyPowerYou betchya02:35
marcoceppilazyPower: check out ppa:juju/devel and update/upgrade ;)02:36
lazyPoweron it!02:40
=== scuttlemonkey is now known as scuttle|afk
=== thumper is now known as thumper-afk
=== scuttle|afk is now known as scuttlemonkey
=== blahdeblah is now known as Killfirefox
=== Killfirefox is now known as blahdeblah
=== beisner- is now known as beisner
=== urulama|afk is now known as urulama
jamespagefalanx, that's being worked on atm07:08
jamespagezfs on wily was not as accessible07:08
jamespagegnuoy, charm-tools 2.0.0 just broke our gate for charms...08:24
gnuoyah, that's why your mps failed08:24
gnuoyI did wonder08:24
jamespagegnuoy, basically the fix is to switch from "charm proof" -> "charm-proof"08:24
jamespagegnuoy, I'll raise reviews now...08:37
jamespagegnuoy, as nothing can get past verification atm08:37
gnuoykk, ta08:37
marcoceppijamespage gnuoy if it's easier to change the dep, 1.11.2 is still on pypi if you charm-tools==1.11.2 to avoid 2.0 atm08:43
jamespagemarcoceppi, meh08:45
jamespagemarcoceppi, its the same work either way08:45
marcoceppiack, again, sorry about that08:45
jamespagegnuoy, generating reviews now - https://review.openstack.org/#/q/status:open+branch:master+topic:charm-tools-2.008:59
gnuoyexcellent09:00
jamespagegnuoy, same change across all charms - the only one I could not do was lxd - rockstar - you'll need to pull in the same fix to your in-flight review09:01
jamespagemarcoceppi, oh great - theblues has a == on requests 2.6.009:04
jamespagewhich is creating problems on OSCI verification, but not in upstream09:06
jamespagethat's odd09:06
gnuoyjamespage, is it right that hacluster is not part of our charm collection on github?09:26
jamespagegnuoy, yes09:30
gnuoyack, ta09:30
jamespagegnuoy, I'm having to pin requests to 2.6.0 in the charm test-requirements.txt to avoid pep8 lint failures in OSCI for now09:31
jamespagewe can drop that later - just want to unblock the gate right now09:31
gnuoyok09:31
jamespagegnuoy, i think the version of pip and pkg_resources that the osci lab uses is different to upstream and to my local xenial install which causes some issues with entry point loading without that09:33
jamespagegnuoy, ah crap - I managed to re-create the reviews...09:35
jamespagegnuoy, I was using a pre-canned commit message - misses the ID's that git review added first time around09:35
=== thumper-afk is now known as thumper
jamespagegnuoy, I'm making alot of load of OSCI but generally the lint failures I saw first time round have gone with the version pin09:54
gnuoykk09:54
jamespagegnuoy, anything with a +1 post 10:00 in the following list is good to land IMHO - https://review.openstack.org/#/q/status:open+branch:master+topic:charm-tools-2.010:29
=== scuttlemonkey is now known as scuttle|afk
marcoceppijamespage: yeah, that's what this is about11:38
jamespagemarcoceppi, the fixed requests version in theblues?11:39
marcoceppijamespage: yeah, see pm11:39
jamespagegnuoy, time to unblock the gate?12:14
jamespagehttps://review.openstack.org/#/q/status:open+branch:master+topic:charm-tools-2.012:14
jamespagetwo failed due to amulet problems - but I suspect they have never passed...12:14
jamespagebeisner, ^^12:14
gnuoyjamespage, ack, looking12:14
beisnerdosaboy, ack re: lp bugs going directly to fix-released via upstream.  aiui, it's a behavior change and by design.  ref:13:20
beisnerhttps://review.openstack.org/#/c/248922/13:21
beisnerhttp://lists.openstack.org/pipermail/openstack-dev/2015-November/080288.html13:21
beisnerjamespage, gnuoy -  unless i've missed one, the osci tests are all calling tox -e pep8 which i think should give you the same experience anywhere, shouldn't it?13:21
jamespagebeisner, well apparently not13:22
jamespagebeisner, as the upstream verification passed, but OSCI bailed on the fixed requests version in theblues....13:22
beisnerjamespage, if the tox ini is set to also use system packages, then definitely not.  some of them i believe are.13:22
jamespagebeisner, openstack add in new pkg-resources and stuff which i suspect we don't13:22
beisnerjamespage, ah right.  we're just doing whatever the tox.ini and *-requirements.txt files  instruct13:24
beisnerjamespage, ack re: cinder-backup amulet not passing.  can't raise a bug, but added a card for triage.13:39
jamespagebeisner, I suspec the tox/pip/pkg-resource environment on the upstream ci is not the same as trusty...13:42
beisnerjamespage, right.  imho we shouldn't be leaning on site-packages at all on unit/lint tests.13:42
jamespagebeisner, tbh its probably not - but depending on which tox/pip you use determines which versions of stuff you get in the venv13:43
beisnerjamespage, lovely.  so, build a venv, install a specific version of tox into it, then call tox?13:44
jamespagebeisner, not quite13:44
beisnerjamespage, ack i see you pinned requests.  looking at other upstream projects, they pin requests to a range of versions as well.13:57
jamespagebeisner, that should be droppable once theblues sorts things out13:57
jamespagebeisner, marcoceppi is dealing with that13:57
marcoceppijamespage: https://github.com/juju/theblues/pull/20 and http://paste.ubuntu.com/15477938/13:58
jamespageyah gotcha13:58
marcoceppijamespage: is that enough?13:58
jamespagemarcoceppi, I commented generally on your pull request...13:59
marcoceppijamespage: I missed that, thanks13:59
marcoceppisome balked at the idea of using a range13:59
marcoceppiand this is why I welcome, snappy, our new packaging overlord14:00
jamespagegnuoy, ok for me to +1 workflow those two rollup config changes?14:01
jamespagethey are passing pep8 ok now with a rebase...14:01
gnuoyjamespage, yep, please do14:02
jamespagegnuoy, that feels nice...14:03
jamespageI like ditching lots of old code...14:03
gnuoytinwood, would you mind taking a look at https://code.launchpad.net/~gnuoy/charms/trusty/hacluster/pause-resume/+merge/289911 if/when you have a moment?14:50
tinwoodgnuoy, sure.  I'll do it this afternoon.14:51
gnuoyta14:51
tinwoodgnuoy, is hacluster not on github then?14:52
gnuoytinwood, apparently not, I was surprised too14:52
tinwoodgnuoy, how strange. Oh well.14:53
tinwoodgnuoy, this ceph-mon charm is quite awkward.  I think I've got it right now - just doing the amulet tests on my bastion to see if pause/resume actually works!14:53
gnuoytinwood, hacluster as odd ball too14:56
gnuoybeisner, I accidentally posted a charm recheck full on https://review.openstack.org/#/c/294554/ after already having a successful run and just my luck the second run failed due to a deploy timeout which I assume is unrelated to my change. Do I have to do another charm-recheck-full to get rid of the canonical ci -1 do you think?14:57
=== JoseeAntonioR is now known as jose
=== cos1 is now known as c0s
=== scuttle|afk is now known as scuttlemonkey
falanxIs there a best practice for filesystems on lxd/juju?  There are many articles saying zfs is where the party is, but nothing definitive stating what has the best support vs performance.15:27
rick_h_falanx: yes, running lxd-init on xenial I think will ask about zfs as an option and that will help with the best setup for speed.15:29
rick_h_falanx: I think jcastro has done some testing/etc and has a sweet setup15:30
beisnerhi gnuoy, i'll have a look and holler back shortly15:30
gnuoybeisner, thanks much appreciated15:32
jcastrofalanx: I've been pretty happy with lxd/zfs/juju15:38
beisnergnuoy, yes we can ignore the results of that recheck.  it was actually a nova-compute failure on xenial due to libvirt/locale bug 1560939.15:40
mupBug #1560939: libvirt-bin fails to install on a fresh xenial server <amd64> <apport-bug> <ec2-images> <uosci> <xenial> <libvirt (Ubuntu):New> <https://launchpad.net/bugs/1560939>15:40
beisnergnuoy, shall i push that now?15:40
falanxjcastro rick_h_ thanks!  we're looking to do that combo + openstack in production asap =)15:52
jcastrofalanx: heh I should note that work laptop filesystem and your production openstack architecture filesystem recommendations are probably different. :)15:53
beisnerrockstar, ok although tests did pass a day ago, there has been some movement in charm-tools requirements and tests now fail.  all of the charms except lxd have been updated to satisfy and work around that issue.  your change will need one more patchset.16:33
beisnerrockstar, here is an example of what needs to change:  https://review.openstack.org/#/c/296312/   ... and that should let us merge this puppy.16:33
rockstarbeisner: a re-sync should do it, right?16:34
rockstarOh, no, no it won't. :)16:34
beisnerrockstar, i don't see that lxd has received those fixes at master16:34
rockstarOn it.16:35
beisnercoolio16:35
tinwoodcholcombe, I wonder if I could ask you a question about what pause-health does in the ceph-mon charm?16:54
=== cos1 is now known as c0s
balloonsjcastro, is there an up to date guide on using juju2 with lxd?17:11
jcastroballoons: https://jujucharms.com/docs/devel/config-LXD17:19
jcastroballoons: oh, this reminded me I need to PR and fix a few bits there17:20
balloonsjcastro, I have some thoughts on that page, lol17:21
balloonsbut if that is it, I'm not seeing success. It seems like it's missing some bits, and I'm left confused on a few points17:22
jcastrothe lxd-images stuff is all out of date17:22
jcastroall you should need to do is `sudo lxd init`17:22
jcastroballoons: to be fair though, all of this was in flux until the other day17:23
jcastroalexisb: do we know if juju still wants "ubuntu-trusty" as an alias for a container or will it just use the automagic remotes LXD provides?17:25
balloonsI have to use the specific alias name?17:25
pmatulisballoons, jcastro: guys, that page was heavily refreshed yesterday. i can't remove lxd-images b/c you can't use Xenial Juju machines without it17:25
rick_h_jcastro: she's out, I think it looks for that specific alias so that it doesn't stomp on the existing lxd use17:25
balloonsjcastro, I guess I'd love to see your PR, before I start firing off bugs17:25
jcastroyou used to have to, not sure if it still does17:25
jcastropmatulis: oh ok so we still need that command then?17:26
jcastroI think that's a bug then? The intent was to just use what LXD provides without the user having to know that command.17:26
pmatulisballoons, jcastro: unless the user is smart enough to know they need to create an alias for the Xenial image17:26
pmatulisbeta3 is supposed to create the alias automatically17:27
jcastroah ok17:27
jcastroso it's still in flux then17:27
pmatulisis there anything else with that page that is wrong or can be extended? lemme know or open an issue17:27
mgzrick_h_: we don't care too much if juju2 is in the xenial beta2 image right?17:28
jcastroall we need after that is for bootstrap to work without needing --upload-tools and I'm good to go17:29
jcastroballoons: which part are you stuck on?17:29
jcastromgz: we're having a hard time getting in also for charm-tools, so we said nope and going to try as soon as beta2 is out.17:31
jcastroall the bugs are filed, we're just waiting in line17:31
pmatulisanyone know why i would not have juju2 sub-command completion on a fresh Xenial openstack instance?17:34
pmatulisscratch that, seems a reboot fixed it17:35
jcastropmatulis: I am wondering if keeping around the "changes from lxc local provider" in that page just adds length for something most people won't care about17:39
balloonsjcastro, I can't juju bootstrap. And the instructions left me questioning if just installing juju got me the lxd provider or not17:40
balloonsalso, do I need an environments.yaml file? How do I create a controller for lxd?17:40
pmatulisjcastro: yes, that's debatable17:40
jcastrohttps://jujucharms.com/docs/devel/controllers-creating17:40
jcastroyou do not need an environments.yaml17:40
pmatulisballoons: you're not reading the page :D17:41
jcastroafter you have it installed all you should need to do is `juju bootstrap blah lxd --upload-tools17:41
jcastrothe style of the docs are not condusive to "ok you've finished this page, go to this page next" because there's no pagination17:42
jcastroperhaps once we have the final changes in from the tool we should consider just doing a quick and dirty summary up top that gets a controller fired up17:43
jcastroand then while that is happening the user can read all the details below17:43
pmatulisjcastro: maybe you're looking for a tutorial or yet another getting-started page?17:44
pmatulisjcastro: Juju is complex and it's very difficult to have a running story that fits everyone's purpose17:44
jcastroI can't fix the get-started page until 2.0 is out though17:45
jcastroand it's not difficult, we're just telling people things in the wrong order17:45
pmatulisjcastro: there are plans for a much-needed "architectural overview" page replete with diagrams and core concepts, evern arrows but there is so much to do17:46
jcastroyeah it's just like, the instructions have CLI commands in them17:46
jcastroand then at the end of the section it's like "xenial comes with lxd by default"17:46
pmatuliswhat's wrong with saying that?17:47
jcastroactually, let me just ask luca if we can start on the 16.04 version of the get-started page17:47
jcastrowell, all the commands above that are explaining how to install and configure lxd17:47
pmatulisinstall on Wily maybe17:48
pmatulisand LXD does not require any configuration, unless you want to use network stuff or ZFS17:48
pmatulismaybe if you give me a specific example17:49
jcastroI'm just saying we can trim most of that entire section out and make firing up the controller more upfront17:49
jcastrolike a TLDR version17:49
pmatuliswell, these are the definitive/upstream docs for Juju. everything cannot be a TLDR17:50
pmatulisa TLDR/getting-started/quickstart should be separate17:51
balloonswell I had to not use lxd-images (that fails), and then starting a container mannually failed. So not really juju's fault I guess. But the bootstrap command didn't recognize lxd as a provider for instance17:51
balloonsanyways, are you planning to swap the 'getting started' guide to using lxd for 2.0?17:51
pmatulisballoons: dunno what's going on over there. sounds like LXD itself is not working17:52
jcastroI don't think we can swap it for LXD on get-started, the get-started is supposed to work on all OSes afaict.17:53
balloonspmatulis, right, so I'm not blaming juju for that at all. But as I said the bootstrap command didn't work, and I installed juju-local to maybe fix it -- but the page doesn't mention that17:53
balloonsI know with juju2 it's changing, so if nothing else I can say the page is confusing at the moment17:54
balloonsand I would like to play with lxd + juju2 locally, however that may be possible17:54
jcastrowait, are you trying to use juju1 with lxd?17:54
pmatulisballoons: yeah, you should be using Juju 217:56
pmatulisand don't get me started with the getting-started page :)17:58
balloonsjcastro, I wasn't, but again, those pages make things confusing. Also, I can't get juju2 in juju/stable (or course not), but the page mentions it17:58
jcastroyeah17:59
jcastroyou're in the middle of an awkward transition17:59
jcastrothe page is being written for the future while the commands are still being written18:00
jcastroballoons: all the good information on how to use juju2 is in the release notes and not the docs18:02
jcastrolike how to list-clouds, the new credentials stuff, etc.18:02
jcastroballoons: I can walk you through lxd in a hangout after I'm done with this meeting I'm on now18:06
magicaltroutballons afaik you should be good on the latest beta from the ppa18:15
magicaltroutjuju bootstrap dev-lxd lxd --debug is about all you need once you have a lxd trusty image18:15
pmatulisballoons: yep, i concede there is confusion (no juju2 in stable) but, given the resources we have, it was decided to write for the future. this is a devel branch after all18:15
pmatulis(the software and the docs are not released)18:16
balloonspmatulis, yes I agree. Write towards what is landing (and there is the header stating as such)18:16
LiftedKilt 18:38
=== cos1 is now known as c0s
=== redir is now known as redir-lunch
pmatulisnothing under the kilt?19:17
=== freyes_ is now known as freyes
=== bogdanteleaga_ is now known as bogdanteleaga
=== kadams54_ is now known as kadams54
=== mup_ is now known as mup
LiftedKiltpmatulis: it was feeling a bit breezy there for a second, that's for sure19:22
=== redir-lunch is now known as redir
cory_fuc0s: So, the main problem that admcleod- was talking about earlier when he mentioned multiple namenodes is that there is a bug in Juju leadership in the current stable (1.25) where if you spin up and then tear down two NNs, then spin up two more without re-bootstrapping the entire environment, a leader isn't selected.20:16
cory_fuI haven't encountered it, personally, so I can't really give any more detials20:17
cory_fu*details20:17
c0scory_fu I am a bit confused: how Juju leader election is related to ZKFC leader election for HDFS HA?20:17
cory_fuBut from what he was saying, the HDFS HA should be pretty functional20:17
cory_fuc0s: He's using juju leadership to ensure that only one NN tries to do the formatting and other logic that apparently should only happen once.  Once it's up and configured to HA, juju leadership is irrelevant20:18
c0scory_fu: oh, that's the juju way of handling the distributed locking. Got it.20:18
cory_fuRight20:18
c0scory_fu: by the way: Here's Bigtop puppet recipes on the topic: https://github.com/apache/bigtop/blob/master/bigtop-deploy/puppet/modules/hadoop/manifests/init.pp#L556 and all the way to line 60720:19
cory_fuadmcleod-: ^ for reference when you sign back on20:19
c0sI will add this to the review notes as well. And thanks for the explanation: now I can focus on the HDFS HA part ;)20:20
cory_fuc0s: I haven't looked at the HA branches in much detail recently, so I can't speak to it other than what's been explained during the dailies20:20
cory_fuCool.  Thanks20:20
c0scory_fu: speaking of solving the distributed lock issue20:30
c0swhat if active and standby NNs know about their roles from the beginning?20:30
c0sThis way, the file system formatting can only be done on the primary code by simply checking against the namenode role.20:30
cory_fuHow would they know their role?20:31
c0swhen you deploy the services you can arbitrary chose to be an active and another to be standby20:32
c0shere's how we do it20:32
c0sHere&rsquo;s the link to Bigtop&rsquo;s puppet recipes to setup HDFS HA http://is.gd/d0cWel  from line 556 all the way to line 60720:32
c0sscratch that20:32
c0shttps://github.com/apache/bigtop/blob/master/bigtop-deploy/puppet/modules/hadoop/manifests/init.pp#L63920:32
c0swe always pick the 1st node to carry on as the active and to hell with that ;)20:33
c0sthis is out choice20:33
c0sbecause the same recipe code is getting executed on all the namenodes, with such check we guarantee "only one" execution20:34
c0scause otherwise to expose the internal HDFS concern up a layer to the orchestrator, and that makes the implementation harder, of course20:34
cory_fuSo, that's basically what we're doing with the Juju is_leader check.  The "lowest numbered unit" would work and is what charms used before is_leader was available, but is_leader is generally considered better (barring the bug we're seeing)20:36
cory_fuIn the end, it basically works out to the same thing, and in theory should actually be simpler since it's a simple boolean check (and the Juju controller decides on the leader)20:37
cory_fuAlso, the bug we're hitting with is_leader really only affects iterative development use.  Normal usage wouldn't be affected by it20:37
cory_fuI feel like there was a stronger reason why we moved away from "lowest unit" for this specific charm, as well, but I can't recall off the top of my head20:38
c0sok, that makes sense. Thanks cory_fu for the explanation - learning a bit more each hour ;)20:39
muxnew blog post about juju posted here: http://blog.emccode.com/2016/03/23/storage-operations-with-juju-charms/21:03
muxspecifically, exploring what's out there for attachable, detachable, persistent storage for charms21:04
mux(spoiler: not much.)21:04
c0scory_fu: do you know if the bug for the leader election has been logged somewhere?21:05
c0sI have a feeling that it might be not a bug, but rather a limitation of ZK21:05
c0sif you look here https://aphyr.com/posts/291-call-me-maybe-zookeeper all the way to the Recommendations section, it says21:06
c0s"Also keep in mind that linearizable state in Zookeeper (such as leader election) does not guarantee the linearizability of a system which uses ZK. For instance, a cluster which uses ZK for leader election might allow multiple nodes to be the leader simultaneously..."21:06
c0sAnd that guy knows what he's talking about ;)21:06
cory_fuc0s: So, this doesn't actually have anything to do with ZK, only the Juju is-leader command.  As I understand it, is that Juju's is-leader tool is supposed to return True for at least one unit at all times except for a somewhat brief fail-over period  when the leader unit goes away.  The problem is that Andrew is seeing normal behavior for units 1 and 2, but not 3 and 4 after removing the service entirely and redeploying it21:10
cory_fuAgain, for a new service coming up, at least one unit (the first to come up) should see is-leader return True even before running any charm code21:10
c0soh, I see. Not I think I got it - was mislead a bit by the presence of ZK from HDFS HA ;)21:11
cory_fuI assume there was a bug filed against core, but I'd have to go looking for it21:11
c0sno worries21:11
c0sso, juju doesn't use ZK under the hood, after all?21:11
cory_fuYeah, there are a couple of orthogonal ideas of "leadership" going on21:11
c0sk, thanks!21:12
cory_fuNo, the controller just picks one of the units as the leader and records that in Mongo.  With the current single controller, something like ZK isn't needed at all.21:12
cory_fuc0s: Also, someone in #juju-dev could give a much more accurate explanation of how Juju's leadership code works21:13
skay_my services seem to be running but workload state shows as unknown when I run juju status.21:13
cory_fuBut from a charm's perspective, we're supposed to be able to rely on is-leader telling us if we're leader or not, and (almost) always having a leader21:13
=== skay_ is now known as skay
c0scory_fu: thanks, this really helps!21:14
=== skay is now known as Guest88295
c0sthe fact of the single controller skipped my mind.21:15
cory_fuAnd of course, we're really only relying on that to ensure that only one unit runs the HA initialization logic.21:15
=== Guest88295 is now known as skay
magicaltroutzhttp://www.oreilly.com/programming/free/how-to-make-mistakes-in-python.csp?imm_mid=0e1f49&cmp=em-prog-free-lp-ostx16_nem4_mistakes_in_python21:42
magicaltroutits like Oreilly knew I was hacking some python badly......21:42
lazyPowerMayyybe someone knows the answer to this... i need to mock/patch a private method on an object. my google fu is failing me on finding the method to do so, any helpful pythonista's in the crowd?23:37
jrwrenlazyPower: don't do that. :]23:51
lazyPowerjrwren - just leave the priate method untested?23:51
lazyPower*private23:51
mgzlazyPower: also, you can just do that23:52
mgzprivate in python is just a convention23:52
lazyPowerwell, thing is - i need to patch it as its kind of the core dispatch method in the class23:52
mgzbut genrally, you only want to test interface points, so not private helpers23:52
lazyPoweri guess i could move it elsewhere and just patch it there23:52
lazyPowernebulous questions without context23:53
* lazyPower resumes test hacking23:53

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!