| === alexisb is now known as alexisb-afk | ||
| === karlthane_ is now known as karlthane | ||
| === natefinch-afk is now known as natefinch | ||
| magicaltrout | anyone around who can provide some interface writing support | 07:43 |
|---|---|---|
| magicaltrout | or are you all in Pasadena? | 07:43 |
| magicaltrout | babbageclunk: fancy explaining some interface stuff that is clearly very simple but my brain can't process? | 09:08 |
| babbageclunk | um, I'll have a go! Not very familiar with those bits myself yet. | 09:09 |
| magicaltrout | lol | 09:09 |
| magicaltrout | well you're my only hope... no pressure | 09:09 |
| * babbageclunk girds loins | 09:09 | |
| magicaltrout | anyway my idea is pretty simple... I want to create a generic JDBC interface because a bunch of services provide JDBC connections and apps like Saiku can consume JDBC connection info | 09:10 |
| babbageclunk | ok | 09:10 |
| magicaltrout | and all jdbc connections are basically the same, username, password, class and url | 09:10 |
| babbageclunk | sure | 09:10 |
| magicaltrout | so... | 09:10 |
| magicaltrout | http://pastebin.com/Ewpg03nM ignore the fact this is mid munging from another interface I wrote | 09:11 |
| magicaltrout | I want to write the provides side of the interface | 09:11 |
| magicaltrout | and my idea is the charm, be it, mysql, drill, postgres etc would provide a connection string and other metadata | 09:11 |
| magicaltrout | but where I'm struggling is in my understanding of charms <-> interfaces | 09:11 |
| magicaltrout | how do I tell my interface to look up a variable from my providing charm? | 09:12 |
| magicaltrout | the only interface stuff I've done so far has been to get unit info etc | 09:12 |
| babbageclunk | Sorry, this is all new to me. So you'd subclass this to make a relation? | 09:14 |
| magicaltrout | hehe, well my idea was (I don't know if this is possible which I guess is half the issue), charm x provides jdbc url, password, username etc | 09:14 |
| magicaltrout | generic jdbc layer can look up those standard variables | 09:14 |
| magicaltrout | sorry s/layer/interface | 09:14 |
| magicaltrout | and pas them to implementing app to inject into a jdbc connection | 09:15 |
| magicaltrout | that way, for a bunch of jdbc providing charms there could be one standard jdbc interface that all apps could consume without hacking around the data provided over the wire in the implementing charm itself | 09:15 |
| babbageclunk | But is there a lifecycle step where an interface can get customised, or is it just looked up directly from some registry? | 09:16 |
| magicaltrout | I don't think (feel free to prove me wrong) it needs much customisation. | 09:17 |
| magicaltrout | so it would just look up stuff from the providing charm | 09:17 |
| magicaltrout | i shall draw a picture! :) | 09:17 |
| * magicaltrout cracks out google drawings | 09:17 | |
| * babbageclunk reads some docs - this is probably a good time to learn about interfaces. | 09:21 | |
| magicaltrout | here you go babbageclunk | 09:21 |
| magicaltrout | https://ibin.co/2uSum2mXgYHo.png | 09:21 |
| magicaltrout | currently most interfaces are specific to an app... interface:mysql etc | 09:21 |
| magicaltrout | but why not make a generic one that passes over specific variables from the providing charm? | 09:21 |
| magicaltrout | that way requiring charm developers don't need to look up loads of metadata or hop through chopping up unit information | 09:22 |
| magicaltrout | because the providing charm (the folks that know what the jdbc stuff should look like) have done it for them | 09:22 |
| magicaltrout | then "any jdbc compatible application" can then grok the incoming charm connection and use the metadata over the wire | 09:23 |
| babbageclunk | That makes sense to me. How do the app-specific interfaces get the username/password/etc info they need to convey to the requires-side of the relation? | 09:23 |
| babbageclunk | Wouldn't yours work in the same way? | 09:24 |
| magicaltrout | hold on I'll show you what I currently do | 09:24 |
| magicaltrout | https://gist.github.com/buggtb/7e04a90922673312ebc6ca62f57fe464 | 09:25 |
| magicaltrout | so thats on the jdbc application side | 09:26 |
| magicaltrout | but this is basically where my knowledge falls down because in the mysql layer for example: https://github.com/johnsca/juju-relation-mysql/blob/master/requires.py | 09:27 |
| magicaltrout | self.host() etc | 09:27 |
| magicaltrout | https://github.com/johnsca/juju-relation-mysql/blob/master/provides.py on the provides side coming from mysql | 09:27 |
| magicaltrout | I dont' see how the host, port etc get set | 09:27 |
| magicaltrout | which confuses the hell out of me :) | 09:28 |
| babbageclunk | Reading... | 09:28 |
| babbageclunk | So they get passed into provide_database - where does that get called? | 09:29 |
| magicaltrout | exactly! :) | 09:29 |
| babbageclunk | D'oh | 09:29 |
| magicaltrout | well | 09:31 |
| babbageclunk | Starting simply, it's easy to see how you could make one charm that could require all of mysql, psql, drill, etc. | 09:31 |
| babbageclunk | And then provide generic jdbc | 09:31 |
| babbageclunk | (Hmm, actually I might be confused about that but it seems possible, right?) | 09:32 |
| magicaltrout | well my Saiku charm for example, could then require jdbc in metadata.yaml | 09:33 |
| magicaltrout | which would then accept connections from jdbc providing charms | 09:33 |
| magicaltrout | so yeah that seems like its possible | 09:33 |
| magicaltrout | and on the consuming/requiring end it should reduce complexity and repetition | 09:34 |
| magicaltrout | and people wonder why i get annoyed with interfaces, cory is like the bloody interface wizard, but it makes it hard for everyone else ;) | 09:35 |
| babbageclunk | Right. Then the question would be could you do that in a layer so that any charm that provides one of them also privides jdbc? | 09:35 |
| magicaltrout | okay to retort, why would I need a layer instead of an interface? | 09:37 |
| babbageclunk | I think I mean an interface layer - is that a thing? | 09:37 |
| magicaltrout | lol | 09:37 |
| magicaltrout | interface or layer. Layers allow you to reuse existing layered charms | 09:38 |
| magicaltrout | charm inheritance | 09:38 |
| magicaltrout | interfaces are the stuff that allow the hooks to fire | 09:38 |
| babbageclunk | Right, but I was reading this: https://jujucharms.com/docs/2.0/developer-layers-interfaces | 09:38 |
| magicaltrout | so the question would be: could you do that in an *interface* so that any charm that provides the interface can support jdbc connections | 09:39 |
| babbageclunk | Or rather, could you make a layer that would provide a generic interface so that the charm just provides a specific one, and includes the interface layer to magically support the generic one. | 09:40 |
| magicaltrout | well you'd just upgrade the charms to provide interface:jdbc and set some stuff | 09:41 |
| babbageclunk | Which was the bit you were asking. Sorry! | 09:41 |
| magicaltrout | I write the Drill charm so my plan is to test it there then if it works, submit some PR's to mysql/postgres etc | 09:41 |
| magicaltrout | I think its something to do with self.conversation() | 09:43 |
| babbageclunk | Well, conversation is the bit that pushes the connection details to the requires side. | 09:44 |
| babbageclunk | I think | 09:44 |
| magicaltrout | https://github.com/johnsca/interface-mapred-slave/blob/master/provides.py here's another one of cory's | 09:44 |
| magicaltrout | conv.get_local etc | 09:44 |
| magicaltrout | sooo | 09:46 |
| magicaltrout | how do I set something on the conversation I guess | 09:49 |
| magicaltrout | or how does it magically appear | 09:49 |
| babbageclunk | That's a weird one though - since it's a map reduce slave even though it's a provides interface it's getting data from the conversation and storing it. I think if you look at the requires side you'll see something equivalent to the provide_database call. | 09:50 |
| magicaltrout | yeah so on the requires side I need a method like | 09:51 |
| magicaltrout | def get_jdbc(): | 09:51 |
| magicaltrout | which does conv.get_remote() | 09:51 |
| magicaltrout | or something | 09:51 |
| babbageclunk | Hang on, just looking at the other DB relations | 09:53 |
| magicaltrout | and thats part of the annoyance on that charm the only get_remote is getting the private-address | 09:53 |
| magicaltrout | ta | 09:53 |
| babbageclunk | Have you got a locally built version of the mysql charm? | 10:05 |
| magicaltrout | i have a zip bundle download | 10:05 |
| magicaltrout | its not reactive so I guess that counts | 10:05 |
| babbageclunk | You should be able to find the place that calls provide_database in the code? | 10:06 |
| magicaltrout | you'll like this | 10:06 |
| magicaltrout | bugg@tom-laptop2:/tmp/charm$ grep -R provide_database . | 10:06 |
| magicaltrout | bugg@tom-laptop2:/tmp/charm$ | 10:06 |
| babbageclunk | typical | 10:06 |
| magicaltrout | lol | 10:07 |
| magicaltrout | told you | 10:07 |
| magicaltrout | this stuff cory writes is more often than not... magic | 10:07 |
| babbageclunk | https://github.com/johnsca/juju-relation-mysql/blob/master/docs/source/provides.rst | 10:07 |
| magicaltrout | ah | 10:08 |
| magicaltrout | so..... | 10:08 |
| magicaltrout | on the provides side in drill I need to decorate some method | 10:08 |
| babbageclunk | If something's using that code to provide mysql, somewhere it's calling provide_database (with all the info you need to make the jdbc conn details). | 10:09 |
| magicaltrout | to detect the connection and set the variables in a provides interface method call | 10:09 |
| babbageclunk | Yeah | 10:09 |
| magicaltrout | woop | 10:09 |
| magicaltrout | babbageclunk: you code grepping genius | 10:09 |
| babbageclunk | Sorry, I'm a bit slow today! | 10:09 |
| magicaltrout | hehe, this stuff really messes with my head, then you think you're on to something and find some weird bash script at the other end etc | 10:10 |
| magicaltrout | I don't write python well at the best of times, finding out how charms interact melts my mind | 10:10 |
| babbageclunk | You're right it's confusing - the config information flow in the relations isn't always from provides to requires. But I think in the case of things like DB connections it will be. | 10:10 |
| magicaltrout | marvelous... with this mystery half solved i'll try and implement something in drill and see what happens | 10:11 |
| magicaltrout | i reckon an interface with, jdbcurl, username, password, class and a place for "extra stuff" to live | 10:11 |
| magicaltrout | would offer enough genericisim hopefully the interface setup allows it to work how I imagine :) | 10:12 |
| babbageclunk | That sounds neat | 10:12 |
| babbageclunk | ok, great! Now I need to work out why DHCP is MIA in my MAAS | 10:13 |
| magicaltrout | lol | 10:13 |
| magicaltrout | unlucky | 10:13 |
| magicaltrout | I did see some stuff when I was messing with virtualbox the other day where DHCP just refused to start if you didn't set a dynamic dhcp address pool | 10:13 |
| magicaltrout | thats about the depth of my MAAS knowledge | 10:13 |
| autonomouse | Hi, I was hoping for a bit of advice, if anyone is available. I have a reactive charm and an old-style charm, and I'd like to pass some information between them. e.g. url, username and apikey from a django service to something that needs to use that service. With the newer reactive charms, I believe I am supposed to set up an interface to do this. Can anyone tell me if the old-style charm can use this interface, and if | 11:24 |
| autonomouse | not, what the best approach is for handling this please? | 11:24 |
| mbruzek | bdx | 16:25 |
| kwmonroe | admcleod: beisner: https://jujucharms.com/hadoop-processing/ it takes 8 machines by default, but you can change num_units on "slave" from 3 to 1 to get it down to 6: https://api.jujucharms.com/charmstore/v5/hadoop-processing/archive/bundle.yaml | 17:05 |
| jose | jcastro: thought of using summit.ubuntu.com for the summit? | 17:42 |
| marcoceppi | jose: no | 17:55 |
| marcoceppi | the schedule is small, lightweight, and constantly in flux | 17:55 |
| jose | gotcha, was available so thought of you | 17:59 |
| kwmonroe | beisner: admcleod: http://paste.ubuntu.com/23155869/ | 19:24 |
| admcleod | kwmonroe: thanks | 19:25 |
| beisner | thx kwmonroe | 19:37 |
| === scuttle` is now known as scuttle|afk | ||
| magicaltrout | nearly missed tomorrows flight..... lost passport.... | 20:47 |
| magicaltrout | lazyPower: good news.... I have a 2nd talk.... also about docker and juju | 20:59 |
| magicaltrout | @jpl that is | 20:59 |
| lazyPower | nice! | 21:01 |
| lazyPower | magicaltrout - when is that? | 21:01 |
| magicaltrout | the week after the summit | 21:01 |
| lazyPower | oh wow, so soooooon :D | 21:01 |
| lazyPower | we'll have to sync up, make sure we're up to snuff on the latest and greatest | 21:02 |
| magicaltrout | yeah well, i'm in town etc.. | 21:02 |
| lazyPower | about that, when do you arrive for the summit? | 21:02 |
| magicaltrout | saturday afternoon | 21:02 |
| lazyPower | nice, i'll be around | 21:02 |
| lazyPower | i should shoot you my digits | 21:02 |
| magicaltrout | i'll be sat in the bar very jetlagged ;) | 21:02 |
| magicaltrout | ooh | 21:02 |
| magicaltrout | thats tomorrow | 21:02 |
| lazyPower | indeed | 21:03 |
| magicaltrout | okay tomorrow afternoon | 21:03 |
| magicaltrout | very jetlagged | 21:03 |
| magicaltrout | i left my passport in the hotel i was in last week | 21:03 |
| lazyPower | so sunday brunch/lunch then depending on when i roll my lazy bones out of bed | 21:03 |
| magicaltrout | can you believe it?! | 21:03 |
| lazyPower | doh | 21:03 |
| magicaltrout | (yes) | 21:03 |
| magicaltrout | need to pick it up on the way to LHR | 21:03 |
| lazyPower | thats a nice distraction eh? | 21:03 |
| magicaltrout | spent all afternoon trying to find it:) | 21:03 |
| magicaltrout | i have an interface question if you have 3 mins? | 21:04 |
| magicaltrout | not technical, theoretical | 21:04 |
| babbageclunk | IT'S A TRAP | 21:18 |
| magicaltrout | i saw you lurking | 21:18 |
| magicaltrout | i was trying to clear up our theory from this morning :P | 21:18 |
| === scuttle|afk is now known as scuttle` | ||
| admcleod | magicaltrout: took me ~20 hours from BCN. i feel a bit weird. | 21:36 |
| magicaltrout | like usual then admcleod ? | 21:41 |
| lazyPower | magicaltrout - sure | 21:44 |
| lazyPower | fire away | 21:44 |
| babbageclunk | I mean, it's an interesting trap. | 21:45 |
| babbageclunk | :) | 21:45 |
| magicaltrout | sorry for the delay lazyPower | 21:56 |
| magicaltrout | okay i have this idea | 21:56 |
| * lazyPower listens intently | 21:56 | |
| magicaltrout | ideally there will be a bunch of apps that can consume JDBC connections | 21:56 |
| lazyPower | for the clarification of kwmonroe - JDBC is just the java database adapter right? their pluggable abstraction? | 21:57 |
| magicaltrout | soooo I had a plan to create a generic JDBC interface, that the providing charm could populate with the stuff required by the consuming application | 21:57 |
| magicaltrout | yeah, any java data app speaks jdbc in some form | 21:57 |
| lazyPower | yep, similar concepts in .net, go on | 21:57 |
| magicaltrout | so all jdbc connections are basically the same just different details, and I'm sick as a consuming app figuring out the various interfaces and stuff you need to know to build 1 per database or data source | 21:58 |
| magicaltrout | so I figured why not have mysql, postgres, drill etc all suppor the the jdbc interface | 21:58 |
| magicaltrout | which clearly is doable | 21:58 |
| magicaltrout | from a consuming standpoint, is it possible and acceptable for 1 charm to consume the same interface from multiple sources with different deets? | 21:59 |
| jrwren | magicaltrout: you are going to be talking at JPL? that is awesome! | 22:01 |
| magicaltrout | my employers suggested it might be a good idea | 22:01 |
| magicaltrout | not sure when a hacker speaking to a bunch of PhD's was a good idea | 22:01 |
| magicaltrout | but we shall see | 22:01 |
| admcleod | magicaltrout: usual would be usual. weird is different weird. | 22:02 |
| magicaltrout | admcleod: a bit more like the i've been up for 24 hours, its 3pm so technically to early to drink but in reality i was drinking at 9am so wtf? | 22:03 |
| magicaltrout | don't worry you can take the piss out of me tomorrow in the same state | 22:03 |
| magicaltrout | after my drive from LAX to the Sheraton | 22:03 |
| admcleod | magicaltrout: yeah, but i didnt really feel jetlagged, had a great sleep, woke up fresh.. but just hit me now | 22:04 |
| admcleod | magicaltrout: also i would never take the piss. | 22:04 |
| magicaltrout | sleep? | 22:07 |
| magicaltrout | you slept on the plane? | 22:07 |
| magicaltrout | or post landing? | 22:07 |
| magicaltrout | I once slept from Manchester to Toronto without waking once, the flight crew were pretty impressed | 22:07 |
| rick_h_ | magicaltrout: you all there already? | 22:07 |
| rick_h_ | magicaltrout: it's called ambien! only way I can sleep on planes | 22:08 |
| magicaltrout | sadly the first homeless guy to talk to me in Tonronto was from Leeds where i'd just left | 22:08 |
| magicaltrout | i fly tomorrow rick_h_ | 22:08 |
| admcleod | magicaltrout: no, landed last night | 22:08 |
| rick_h_ | magicaltrout: ah ok | 22:08 |
| * rick_h_ doesn't leave until sunday | 22:08 | |
| magicaltrout | i need a day or so to get over the lag | 22:08 |
| rick_h_ | yea, always a good thing | 22:09 |
| magicaltrout | admcleod: ah, yeah i know that one | 22:09 |
| jrwren | i usually need a week, and then i go home and need another week to adjust. | 22:09 |
| magicaltrout | although when i went to vancouver for apachecon the day i landed it smacked me so hard i left the hotel bar without paying | 22:09 |
| magicaltrout | had to grovel the next day ;) | 22:09 |
| jrwren | magicaltrout: I look forward to meeting you. | 22:09 |
| magicaltrout | jrwren: good job i'm there for 2 weeks then ;) | 22:09 |
| magicaltrout | yeah jrwren it's gonna be a blast meeting more of the US folks and the expanding charmer team | 22:11 |
| magicaltrout | also, its amazing how much has changed and moved on since Ghent at the start of the year | 22:12 |
| magicaltrout | i blame jcastro for my continued participation in this mess! ;) | 22:12 |
| icey | does anybody know how to install `charms` in addition to `pycharms`, pycharms seems to overwrite charms and trusty won't install charm-tools anymore | 22:14 |
| rick_h_ | magicaltrout: we all blame jcastro :P | 22:14 |
| rick_h_ | icey: pycharms? | 22:15 |
| magicaltrout | aye | 22:15 |
| icey | yes (says cholcombe, he's the asker :) ) | 22:15 |
| magicaltrout | don't blame you | 22:15 |
| rick_h_ | icey: hmm, pycharm search only finds the editor | 22:16 |
| icey | rick_h_: that's the one | 22:16 |
| rick_h_ | icey: so the charm command interferes with the editor install? | 22:16 |
| icey | rick_h_: backwards, apparently the editor interferes with the charm install | 22:17 |
| icey | he cannot install charm | 22:17 |
| rick_h_ | icey: hmm, and he's getting charm on trusty from the stable PPA? | 22:17 |
| rick_h_ | icey: or which PPA I guess? | 22:17 |
| icey | yes | 22:17 |
| icey | rick_h_: | 22:17 |
| rick_h_ | hmm, there's a few python deps like pytest and path.py that are in the PPA to support charm-tools and the like. I wonder if those are interfering | 22:19 |
| rick_h_ | icey: my short answer is try to to setup one of them in a lxc container or the like atm, I'm not sure what conflicts without figuring out where he's getting pycharm from (it's not listed in ubuntu) and so guessing there's conflicting PPA/repos stuff there | 22:19 |
| icey | thanks rick_h_ | 22:20 |
| rick_h_ | sorry | 22:20 |
| lazyPower | magicaltrout - to circle back to your interface idea | 22:24 |
| magicaltrout | i like the lag | 22:24 |
| lazyPower | i think thats a fine idea. JDBC as a generic abstraction would be great, what would be even better, is if jdbc knew how to consume the existing relations, then its a one sided update. I'm wondering if you dont make a jdbc layer, that implements the known relations, and surfaces the details appropriately | 22:25 |
| lazyPower | otherwise, you're facing adding JDBC interface to every [layer|charm] that needs to communicate to jdbc. I'm trying to noodle how i can save you work there | 22:25 |
| lazyPower | but i dont think we have considered this as a use case | 22:25 |
| magicaltrout | well i might upstream mysql and postgresql and hive | 22:26 |
| magicaltrout | its not masses of effort and saves extra layers | 22:26 |
| lazyPower | when you say upstream, you mean just add the - 'interface:jdbc' to those charms right? | 22:26 |
| lazyPower | and whatever additional bits to utilize it in the layer | 22:27 |
| magicaltrout | yeah and a connection method call somewhere | 22:27 |
| lazyPower | right, ok. upstreaming is such an overloaded term anymore :) | 22:27 |
| lazyPower | yeah, that works. I'm +1 to this, as its straight forward, and makes a ton of sense | 22:27 |
| magicaltrout | you know me... trying to stay hip.. ;) | 22:27 |
| magicaltrout | yeah, i started writing a drill interface, but then i realised apart from the values | 22:28 |
| magicaltrout | the keys are identical to most other jdbc interfaces | 22:28 |
| magicaltrout | so the lovely babbageclunk helped me unmuddle my brain this morning and i stood up something basic in drill, but I think generic interface for regular connections, as long as in practice they work | 22:29 |
| magicaltrout | would make life a lot easier for requring chamrer | 22:29 |
| magicaltrout | charmers | 22:29 |
| magicaltrout | saiku, pentaho, spago can certainly implement jdbc requires, so could drill and probably zeppelin somewhere | 22:31 |
| marcoceppi | magicaltrout: yeah, but jdbc is a connection string format, and if you have an app that takes a JDBC connection for MySQL but won't work on cassandra, then a jdbc interface doesn't really help? | 22:40 |
| magicaltrout | okay so lets assume some common sense on the end of the user | 22:41 |
| magicaltrout | cassandra isn't a jdbc compliant database | 22:41 |
| magicaltrout | whilst there are adapters for it | 22:41 |
| magicaltrout | jdbc is a standard and if you can write SQL you can 95% of the interrogate a database | 22:41 |
| magicaltrout | of course there are exceptions | 22:42 |
| magicaltrout | but in the scheme of things and also the charms you guys have available, those exceptions don't really count | 22:42 |
| magicaltrout | also | 22:44 |
| magicaltrout | its not the law to implement it | 22:45 |
| magicaltrout | you can still do what the hell you like :) | 22:45 |
| marcoceppi | if you want to implement a jdbc interface and add it to mysql/maria/percona/postgresql (any others?) go for it :) | 22:45 |
| marcoceppi | but I thin kit'd be work discussing next week | 22:45 |
| magicaltrout | sure, i can always rename my test jdbc interface drillbit and we're back to the same thing | 22:46 |
| magicaltrout | i just think there should be a nice way for people trying to use the stuff provided by the resources instead of trying to find unit names etc because your providing charm requires a list of ZK nodes etc | 22:47 |
| magicaltrout | i happen to be implementing the drill charm as well so it doesn't count as much. but I wouldn't envy someone trying to work out the jdbc url without charmers help | 22:51 |
| magicaltrout | so why not try and genericise it | 22:51 |
| marcoceppi | +1 | 23:18 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!
