| === blahdeblah_ is now known as blahdeblah | ||
| === frankban|afk is now known as frankban | ||
| === freyes__ is now known as freyes | ||
| === degville_ is now known as degville | ||
| magicaltrout | kjackal: stupid question but seemingly something i've not got in any of my charms... or not that I can find | 11:05 |
|---|---|---|
| magicaltrout | if I set a state in a reactive charm | 11:05 |
| magicaltrout | how do i then unset it when the relation goes away? | 11:05 |
| stub | magicaltrout: Use the newer Endpoint system and it is @when_not('endpoint.foo.joined'). With the older stuff, it is rather difficult. | 11:14 |
| elmaciej | Morning | 11:40 |
| elmaciej | Question about openstack charms - can I mix nova-lxd and nova-kvm hypervisors in one cloud ? it would be really cool | 11:40 |
| elmaciej | from charms in store descriptions I see that it is not possible but maybe someone know a workaround :) | 11:43 |
| TheAbsentOne | Are there any tomcat juju pro's here? I'm looking for the correct way to deploy the tomcat charm and connect (jndi) to a deployed mysql charm. So that I can deploy a war that uses a db from the mysql server. | 11:52 |
| zeestrat | elmaciej: If you mean running kvm hypervisor on one host and lxd hypervisor on another host, then I imagine you should be able to deploy multiple different instances of the nova-compute charm, e.g. one for kvm and one lxd on different hosts. jamespage could probably confirm. | 12:06 |
| elmaciej | zeestrat: ok, so quickly - I prepare to charm configs and deploy juju deploy --config nova-lxd.yaml nova-compute --to 1 and juju deploy --config nova-kvm.yaml nova-compute --to 2 | 12:11 |
| zeestrat | elmaciej: Almost. You need to define the different deployments as different applications in juju. So you'd get something like `juju deploy nova-compute nova-lxd --config nova-lxd.yaml --to 1` where `nova-lxd` is the name of that application/deployment. Likewise with `juju deploy nova-compute nova-kvm --config nova-kvm.yaml --to 2`. Note that that you will then need to setup relations for these individually with their | 12:19 |
| zeestrat | new application name. | 12:19 |
| gsimondon | Is there a recommended production setup for bare metal for canonical kubernetes with juju? | 12:19 |
| elmaciej | zeestrat: that's cool, but honestly when I think about it what are the pros/cons in having nova lxd hypervisor ? just wondering also on this one to deploy manually: https://docs.openstack.org/zun/queens/install/overview.html | 12:21 |
| kjackal | magicaltrout: You could have a look here: https://github.com/juju-solutions/interface-kube-control/blob/master/requires.py#L52 | 12:24 |
| kjackal | gsimondon: what kind of setup would you expect? MaaS I guess would be nice to have so you can put cdk on top | 12:26 |
| zeestrat | elmaciej: Depends on your use case. Are you comparing lxd to kvm? | 12:26 |
| elmaciej | well from end user point of view. it will be still available as an instance in horizon so only things like latency etc will be better on lxd | 12:27 |
| elmaciej | right? | 12:27 |
| gsimondon | kjackal: nothing like MaaS, we have a custom solution for maas | 12:30 |
| gsimondon | kjackal: basically I know I can create a manual cloud by adding machines but when you spin up apps (units) juju tries to create a new machine | 12:30 |
| gsimondon | kjackal: that's the part that's confusing me in this bare metal setup. | 12:31 |
| kjackal | gsimondon: let me understand something, you have MaaS already setup? | 12:31 |
| gsimondon | kjackal: if you are talking about maas.io - no, we are not using that | 12:32 |
| kjackal | ok, so you you want to use the manual provider over machines you already have provided | 12:33 |
| gsimondon | kjackal: yes. | 12:33 |
| kjackal | You need to pre-provision any machines: https://jujucharms.com/docs/2.2/clouds-manual and then... | 12:34 |
| kjackal | deploy the cdk bundle: https://api.jujucharms.com/charmstore/v5/canonical-kubernetes/archive/bundle.yaml | 12:35 |
| kjackal | But you might need to change this bundle a bit | 12:36 |
| gsimondon | kjackal: I did that multiple times when I was evaluating this distribution the first time. Basically my questions would be 1) It sucks that I have to specify where to deploy apps. Is it possible to now do that? 2) If I deploy stuff from machine A, how can I use that machine to deploy apps to it? When you do juju status it will show only two machines that I manually added using add-machine but machine A | 12:36 |
| gsimondon | won't be available | 12:36 |
| gsimondon | kjackal: 3) Is LXD something worth considering for isolation of components in this setup? | 12:37 |
| kjackal | gsimondon: yes, creating lxd containers will work | 12:39 |
| kjackal | so tou your questions: 1) in the manual provider you have the pre-rpovision you machines so you should be explicit on where each app should go 2) I guess you could add the Machine A as a cloud machine with "juju ssh ubuntu@machine-A" | 12:41 |
| kjackal | 3) lxd should work, I know we do this trick in orange boxes when we do not have enough nodes | 12:42 |
| gsimondon | kjackal: Thanks, very helpful. Do you use LXD just for kubernetes master and workers or basically for everything being deployed by juju? | 12:43 |
| kjackal | gsimondon: have a look at how easyrsa is deployed in this bundle https://api.jujucharms.com/charmstore/v5/kubernetes-core/archive/bundle.yaml it says --to: lxd:0 This might work for you | 12:45 |
| kjackal | My suggestion would be not to place workers inside an lxd container, although it should work you may run into trouble when trying to expose a docker container running inside that lxd container | 12:46 |
| gsimondon | Any rule of thumb on what to put in LXD and what to keep out so far? | 12:47 |
| zeestrat | elmaciej: Yeah, so it depends if you and your users want those lxd advantages. Canonical has a nice marketing page for that: https://www.ubuntu.com/containers/lxd | 12:49 |
| kjackal | gsimondon: not rules, but do not put the etcds on lxd containers on the same host machine :) | 12:50 |
| gsimondon | kjackal: yeah, that wouldn't be so smart. :) | 12:51 |
| pekkari | kwmonroe: do you, or do you know any moderator in bigdata mailing list? | 13:17 |
| kjackal | pekkari: I saw your message do you have the charm somewhere so we can see it in action? | 13:22 |
| kjackal | We would only need to deploy zookeeper, right? | 13:22 |
| pekkari | for the testing is enough to deploy zookeeper kjackal | 13:23 |
| pekkari | I was firing the charm in my laptop, so no environment we can share and see unless we get on a hangout and show it to you | 13:23 |
| === Guest89134 is now known as zeus | ||
| kjackal | I think I know what is wrong with it | 13:24 |
| kjackal | you have somewhere: | 13:25 |
| kjackal | @when('zookeeper.started', 'leadership.is_leader', 'zkpeer.changed') | 13:25 |
| kjackal | def check_cluster_changed(zkpeer) | 13:25 |
| pekkari | any suggestion is really appreciated, as I'm running out of ideas | 13:25 |
| kjackal | no, turns out I donot... | 13:27 |
| kjackal | its not what i thought it was, sorry | 13:27 |
| pekkari | sorry kjackal, my irc just crashed | 13:30 |
| pekkari | I'm all ears | 13:30 |
| kjackal | no, turns out I donot know what is wrong with it | 13:30 |
| magicaltrout | http://canacopegdl.com/images/all-ears/all-ears-9.jpg | 13:31 |
| kjackal | magicaltrout: :) | 13:31 |
| kjackal | is github down for you people or it only me? | 13:32 |
| magicaltrout | https://cdn.head-fi.org/a/6941207_thumb.png or maybe this one | 13:32 |
| magicaltrout | just you | 13:32 |
| kjackal | xm... it is possible to get banned from gh :) | 13:32 |
| magicaltrout | with you | 13:33 |
| magicaltrout | anything is possible | 13:33 |
| kjackal | I am afraid so... | 13:36 |
| pekkari | kjackal: I'm starting to think it's platform related, I just tried to use xenial machines and here I can see the hook triggered with no problem: http://dpaste.com/25CB8NJ | 13:36 |
| pekkari | for similarity with the end user I was using trusty | 13:37 |
| kjackal | Is it possible at some point you had a build without the <method_name>(self) without the self? | 13:38 |
| pekkari | no, I don't think so, this very same build I'm deploying now is what I was testing last friday, no modification on the code | 13:39 |
| elmaciej | zeestrat: kjackal: Guys, do you know when the queens go official with juju charms? queens going to be released in 2 days and it bring's the 18.04 support | 14:06 |
| zeestrat | elmaciej: https://docs.openstack.org/charm-guide/latest/release-schedule.html | 14:07 |
| elmaciej | zeestrat: that's cool! thx | 14:08 |
| zeestrat | elmaciej: If it ain't in the OpenStack Charm Guide, then you can ask the folks on the mailing list (https://docs.openstack.org/charm-guide/latest/find-us.html) or in #openstack-charms | 14:10 |
| === frankban is now known as frankban|afk | ||
| pekkari | kjackal: contribution ready, thanks for the help! | 14:21 |
| kjackal | did nothing, we are the ones to thank you | 14:21 |
| elmaciej | last question for today guys, is there a way to set default user and password when adding a machine to the model ? | 15:04 |
| === chat is now known as Guest83894 | ||
| pmatulis | elmaciej, just for add-machine? | 15:09 |
| elmaciej | yes, I need a user to login directly not by ssh only | 15:09 |
| elmaciej | pmatulis: I know that these images are using the cloud-init so I was wondering on two options - rebuild the image and change the cloud.cfg or somehow passing the user data to create user during the creation | 15:19 |
| zeestrat | elmaciej: passing user data already has some support in 2.3.1+. https://bugs.launchpad.net/juju/+bug/1535891 Note the caveats such as not being able to use the users section. Please feel free to add a new bug/request for that. | 15:51 |
| mup | Bug #1535891: Feature request: Custom/user definable cloud-init user-data <cpe-onsite> <juju:Fix Released by hmlanigan> <https://launchpad.net/bugs/1535891> | 15:51 |
| elmaciej | zeestrat: Thanks for that. I'm trying to modify default curtin preseed to add the default user with oneliner | 15:53 |
| === chat is now known as Guest60542 | ||
| bdx | kwmonroe: I have a requirement for hiveserver2 to be hooked up to zookeeper | 19:41 |
| bdx | I find here https://www.cloudera.com/documentation/cdh/5-0-x/CDH5-Installation-Guide/cdh5ig_hiveserver2_configure.html, the configuration that need be mended | 19:41 |
| bdx | kwmonroe: I'm wondering if you have any suggestions toward managing the config xml for the bigtop software | 19:43 |
| bdx | ? | 19:43 |
| kwmonroe | bdx: that's gonna be tricky. the hive-site.xml that bigtop uses is here: https://github.com/apache/bigtop/blob/master/bigtop-deploy/puppet/modules/hadoop_hive/templates/hive-site.xml, but as you can see, none of those properties are available there. that means the charm doesn't have an easy way to set/change/manage them. | 19:57 |
| kwmonroe | bdx: so the first step to do this right would be to open an issue for bigtop to expose those config options: https://issues.apache.org/jira/secure/Dashboard.jspa -- that requires you to have an apache jira account. if you don't have one, it's of course easy to create, or i can open the jira for you. | 19:58 |
| kwmonroe | and then step 2 would be to get the hive charm to support the zookeeper interface. that likely wouldn't be too difficult. we'd do it very much like we do zookeeper's relation for hbase: https://github.com/apache/bigtop/blob/master/bigtop-packages/src/charm/hbase/layer-hbase/lib/charms/layer/bigtop_hbase.py#L30 | 19:59 |
| kwmonroe | and pass in the appropriate overrides to set the values in hive-site.xml. | 19:59 |
| bdx | kwmonroe: "so the first step to do this right would be to open an issue for bigtop to expose those config options" - the config options already seem to exist in hive-site.xml | 20:03 |
| bdx | https://github.com/apache/bigtop/blob/master/bigtop-deploy/puppet/modules/hadoop_hive/templates/hive-site.xml#L39 | 20:03 |
| bdx | ahhh, the concurrency option is missing | 20:04 |
| kwmonroe | bdx: also, line 39 isn't hive.zookeeper.quorum :) read it again | 20:04 |
| bdx | ooooo | 20:05 |
| bdx | good eye | 20:05 |
| bdx | darn | 20:05 |
| bdx | so, cloudera just drops those properties right in there then? | 20:05 |
| kwmonroe | well, hive upstream probably has those options too.. the full hive-site.xml is like 1000 lines. bigtop only uses a subset of those things that *they* support as being changeable from the default. | 20:07 |
| kwmonroe | so we just need to get bigtop to include options related to hive.zookeeper, and then the charms can manage them. | 20:07 |
| kwmonroe | bdx: i need to run my kid to an appt, but should be back on in about an hour to hear you curse the day you ever mentioned expanding hive to support zookeeper. biab :) | 20:08 |
| bdx | :) ok | 20:08 |
| bdx | https://imgur.com/a/f1W3r - something to think about while you are waiting at your appt | 20:09 |
| magicaltrout | my biggest issue with kwmonroe's work.... is it puts hadoop in the hands of people who don't need it but don't know better ;) | 20:18 |
| magicaltrout | shut it all down kwmonroe | 20:18 |
| bdx | hey | 20:23 |
| bdx | coming from someone who has ran hdfs on / for the entirety of all time | 20:25 |
| bdx | magicaltrout: letting me down holmes | 20:26 |
| magicaltrout | haha :) | 20:30 |
| magicaltrout | i'm not really directing it at you bdx, but people still bug me with "I have data, i need to deploy hadoop" and it irks :) | 20:31 |
| magicaltrout | because then they a) don't have data that requires hadoop b) don't have the volume for hadoop c) don't have the budget for hadoop d) don't have a clue how to write stuff to run on hadoop e) don't know how to monitor hadoop | 20:31 |
| magicaltrout | when all they really need is a column store DB at most | 20:31 |
| bdx | I see the same thing | 20:45 |
| bdx | happens across all technologies | 20:45 |
| magicaltrout | true dat | 20:46 |
| magicaltrout | I like the ones in here | 20:46 |
| === magicaltrout is now known as jujuuser | ||
| jujuuser | hello, I'm trying to deploy openstack | 20:47 |
| === jujuuser is now known as magicaltrout | ||
| magicaltrout | oh really, hows that going? | 20:47 |
| === magicaltrout is now known as jujuuser | ||
| bdx | magicaltrout: we are moving a ~10TB dataset through a slew of pipelines mon | 20:47 |
| jujuuser | well I have a 5 server setup, but its asking to deploy stuff via floppy disk | 20:47 |
| === jujuuser is now known as magicaltrout | ||
| bdx | shoot! | 20:47 |
| magicaltrout | really? What are you deploying on | 20:47 |
| === magicaltrout is now known as jujuuser | ||
| jujuuser | Pentium 4 machines I have lying around in my lap | 20:48 |
| bdx | jujuuser: where does it say "floppy disk"? | 20:48 |
| === jujuuser is now known as magicaltrout | ||
| magicaltrout | hehe | 20:49 |
| magicaltrout | thats honestly a real story from last year | 20:49 |
| magicaltrout | he was having issues because his juju powered openstack cluster was trying to boot from floppy disk | 20:49 |
| bdx | no | 20:50 |
| bdx | lol I believe you | 20:51 |
| bdx | jujuuser: make sure the minimum requirements are met all the way around | 20:51 |
| magicaltrout | hehe | 20:52 |
| magicaltrout | I am excited about DruidIO coming to the Apache Foundation | 20:52 |
| magicaltrout | I'm gonna get that charmed up | 20:52 |
| magicaltrout | for high volume timeseries stuff | 20:52 |
| magicaltrout | that'll be cool | 20:52 |
| magicaltrout | i'm also tidying up my apache-drill charm which makes hooking up SQL apps to all these platforms much easier | 20:53 |
| bdx | magicaltrout: apache-drill just hooks up to a nosql database and lets you query it with SQL? | 20:56 |
| bdx | magicaltrout: what happens when you hook it up to hdfs? | 20:57 |
| magicaltrout | yeah its a distributed SQL interface for NOSQL platforms | 20:57 |
| magicaltrout | flat files, hdfs, hive, hbase, kafka, s3 are the most common ones | 20:58 |
| magicaltrout | so on hdfs you can query a variety of different file formats, it'll do stuff like auto schema discovery for json/csv style files, if they follow a naming strategy it can do cross file queries etc | 20:59 |
| magicaltrout | you can convert files to parquet DB's quickly and easily for more effective querying at scale | 20:59 |
| magicaltrout | it'll push down what it can to the underlying query engine and make up the rest in memory unlike a lot of the JDBC interfaces out there | 21:00 |
| magicaltrout | sponsored and largely developed by MapR and a few Dell folk IIRC | 21:00 |
| magicaltrout | also provides ODBC & REST interfaces for other uses | 21:02 |
| magicaltrout | and data federation capabilities so you can do stuff like "select * from mysql left join (select * from hbase)) | 21:02 |
| magicaltrout | it leverages a lot of Apache Calcite which provides the SQL interface, and has a bunch of other adapters to do SQL over X | 21:04 |
| magicaltrout | https://calcite.apache.org/docs/adapter.html | 21:04 |
| bdx | super cool | 21:38 |
| bdx | for anyone that cares to deploy volumes > 1T https://bugs.launchpad.net/juju/+bug/1751909 | 21:43 |
| mup | Bug #1751909: turn up the volume - artificial quotas? <juju:New> <https://launchpad.net/bugs/1751909> | 21:43 |
| kwmonroe | 1T is too much | 21:43 |
| bdx | no, 1T works | 21:43 |
| bdx | but anything over seems to fail | 21:43 |
| bdx | orr ooo | 21:43 |
| bdx | 1000 GB works | 21:43 |
| bdx | possibly 1T woudl fail | 21:43 |
| kwmonroe | here's the thing bdx, when you get past a couple gigs, you really need hdfs. magicaltrout, back me up. | 21:44 |
| bdx | oooh sorry ... I'm crossing wires here, ^ is unrelated to hdfs | 21:44 |
| magicaltrout | when IRC needs a joy emoticon | 21:45 |
| kwmonroe | no bdx, i'm just being hilarious. /me just finished backscroll of how i'm pushing hadoop on people that don't need it. | 21:45 |
| bdx | ahh yes yes | 21:45 |
| bdx | aha | 21:45 |
| kwmonroe | magicaltrout: are you still looking at ranger? | 23:25 |
| magicaltrout | yeah slowly | 23:26 |
| kwmonroe | magicaltrout: would ranger's inclusion in bigtop help you? | 23:27 |
| magicaltrout | absolutely | 23:27 |
| kwmonroe | magicaltrout: please add ranger to https://github.com/apache/bigtop/tree/master/bigtop-packages/src to include it in bigtop. | 23:28 |
| kwmonroe | ;) | 23:28 |
| kwmonroe | i'm chuckling quietly | 23:28 |
| kwmonroe | who am i kidding, i'm totes LOL. get it done magicaltrout. | 23:29 |
| magicaltrout | :'( | 23:29 |
| magicaltrout | sad times | 23:29 |
| magicaltrout | i will get around to doing something with it soon enough, I've got the LDAP bits and piees that need finishing up because they'll work for Saiku as well | 23:34 |
| magicaltrout | and I had the start of a ranger snap | 23:34 |
| magicaltrout | but then I got sidetracked with the day job | 23:34 |
| magicaltrout | but now i'm circling back around to Hadoop -> Drill -> Saiku stuff for JAAS, i have an excuse to look at it again | 23:35 |
| kwmonroe | magicaltrout: i know somebody on the bigtop PMC if you need support. | 23:48 |
| magicaltrout | i heard the Bigtop PMC was just a bunch of jerks | 23:52 |
| magicaltrout | We'll see, i need to get it in somehow, but I also have DruidIO, Janusgraph and some other stuff on my high priority todo list | 23:53 |
| magicaltrout | Not sure where the chips will land with all of that | 23:53 |
| magicaltrout | plus once the contracts are sorted out at JPL I'm supposed to be getting 80+ hrs a week and 20 odd on a separate cancer research project | 23:54 |
| magicaltrout | so I need to resource them and sleep at some point ;) | 23:55 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!
