| === verterok` is now known as verterok | ||
| === frankban|afk is now known as frankban | ||
| === dpm_ is now known as dpm | ||
| tzob1x | hello!! | 10:50 |
|---|---|---|
| tzob1x | I get this error Services not running that should be: neutron-plugin-openvswitch-agent | 10:53 |
| tzob1x | I cannot start it from terminal | 10:54 |
| tzob1x | maybe you know what should I do? | 10:54 |
| tzob1x | :) | 10:54 |
| === Guest98782 is now known as spammy | ||
| === tinwood_ is now known as tinwood | ||
| cory_fu | jcastro: Hey, I still need you to update the wiki-simple bundle since I don't have push perms for it in the store | 14:50 |
| kjackal | cory_fu: after nodejs we hit: https://jujucharms.com/apache-processing-spark/ | 15:30 |
| kjackal | sorry | 15:30 |
| kjackal | we hit ImportError: cannot import name 'node_dist_dir' | 15:30 |
| cory_fu | Ok | 15:31 |
| cory_fu | stokachu: Can you confirm that you intended https://github.com/battlemidget/charm-layer-ghost to be using https://pypi.python.org/pypi/nodejs? | 15:31 |
| stokachu | cory_fu: never heard of that python module | 15:32 |
| cory_fu | Oooh, I see! | 15:32 |
| mskalka | %n | 15:32 |
| cory_fu | stokachu: Ok, thanks. I see the issue now | 15:32 |
| mskalka | ignore that :X | 15:32 |
| stokachu | cory_fu: :D i think i haven't updated the import path there | 15:32 |
| stokachu | in lib/charms/nodejs | 15:33 |
| cory_fu | Yep | 15:33 |
| stokachu | cory_fu: are you submitting a PR or I can do it right quick | 15:33 |
| cory_fu | stokachu: I'll submit a PR | 15:33 |
| stokachu | cory_fu: awesome thanks man | 15:34 |
| cory_fu | Have some other changes I'd like to suggest | 15:34 |
| stokachu | perfect sounds good | 15:34 |
| === mskalka is now known as mskalka_afk | ||
| === mskalka_afk is now known as mskalka | ||
| mgz | cory_fu: poke about juju plugins specific change | 16:15 |
| mgz | cory_fu: I have proposed <https://github.com/juju/juju/pull/5731> - can you eyeball it and yell if you see any issue? | 16:16 |
| bdx | hey whats up guys? Is ppa:juju/daily no longer being maintained? | 16:21 |
| bdx | guess I should throw that in #juju-dev | 16:21 |
| mgz | bdx: it is... we just have the issue it was designed to be updated whenever we have a version that passes in CI | 16:22 |
| mgz | I will let you finish that line of logic in your head | 16:22 |
| bdx | ha | 16:23 |
| bdx | mgz: ok, thanks | 16:23 |
| stub | cory_fu: I wonder.... if @hook(cb) simply set cb._reactive_phase = 'hook', then the main loop just needs to perform the same loop over the handlers several times (once for each phase) ignoring handlers not in that phase. And @when and friends would need to propogate the attribute of course. | 16:36 |
| cory_fu | stub: That's basically what it does now. The problem is that we want @when + @hook to trigger the when bit, but we don't want @when by itself to trigger, which is hard to do with just a "phase" var | 16:38 |
| cory_fu | Or perhaps I'm not following your suggestion | 16:38 |
| cory_fu | Hrm, yeah, I was distracted when reading it and I think I might have misunderstood | 16:39 |
| * mgz tears and cory_fu | 16:39 | |
| stub | I tend to think an empty when should always trigger (in the 'other' phase) (but would think that a separate issue) | 16:40 |
| mgz | can you take a look at my mp or wave another eco guy at it? ^^ | 16:40 |
| cory_fu | stub: Yeah, I didn't mean an empty @when (as in, @when() with no states). I meant a @when w/o a @hook or @action. I think I have an idea of how to resolve the issue, from your suggestion. | 16:42 |
| cory_fu | stub: Just to be sure I understand, you're suggesting that all cb have a default phase of 'other' and @hook changes it to 'hook' which lets that handler be considered in that phase, while handlers with the default 'other' are not, and the phase checks that are currently done in @when should be dropped | 16:45 |
| cory_fu | Hrm. I just remembered why I didn't do it that way to start with. External handlers can't attach metadata like that. But I think I can make it work since there have been some changes in how external handlers are checked since the phase checking was first added | 16:46 |
| stub | cory_fu: yes. | 16:46 |
| stub | cory_fu: And renaming the 'other' phase to 'main' might be a good idea ;) | 16:47 |
| cory_fu | Certainly | 16:47 |
| cory_fu | stub: Thanks for the help ont his | 16:47 |
| cory_fu | *on this | 16:47 |
| === zul_ is now known as zul | ||
| cory_fu | mgz: The plugin change seems perfectly reasonable to me | 16:50 |
| mgz | cory_fu: thank you! | 16:50 |
| === frankban is now known as frankban|afk | ||
| petevg | I've got a question for anyone knowledgeable: do we have any best practices for unit testing layered charms? | 17:04 |
| petevg | It looks like the Makefile for our layered charms has a "unit_test" entry by default, which attempts to run tox. | 17:04 |
| rick_h_ | petevg: I think the openstack folks would have loads to say on that along with cory_fu and kwmonroe who have the experience on the hadoop charms | 17:05 |
| petevg | I picked cory_fu and kwmonroe's brains this morning. They said that I should start a convo on #juju :-) | 17:05 |
| petevg | The hadoop charms have loads of integration tests and smoke tests, but no real unit tests yet. | 17:06 |
| cory_fu | petevg: I know jamespage has talked about unit tests in the openstack charms, if he's still around, though I understand much of that was focused on pulling the bits that made since to unit test out into separately packaged libraries rather than testing them directly in the charm. | 17:07 |
| petevg | Pulling libs out of the charm doesn't sound like the worst strategy ... | 17:08 |
| cory_fu | He also mentioned that they use a repo structure that is slightly different from what we use on the big data charms in that they have the layer itself nested one directory deep in the repo so that they have an outer layer to keep all the bits that are necessary for the layer (like unit tests) but that don't make sense in the final built charm | 17:08 |
| petevg | Interesting. | 17:09 |
| petevg | Hiding the unit tests somewhere makes sense. | 17:09 |
| petevg | The main thing that I'm trying to figure out how to do is to build up a virtualenv suitable for running unit tests in without duplicating the logic the populates wheelhouse in a layered charm. | 17:09 |
| cory_fu | That also touches on the need we have to be able to nest base layers in subfolders of repos for upstream purposes while still having them be usable during charm build | 17:09 |
| stub | To start with, I think we need a fixture that lets us specify relevant current state, call a handler (with suitable mocks), and inspect the resulting state. So testing handlers is just like testing any other function. | 17:10 |
| cory_fu | petevg: The wheelhouse.txt file is in the same format as a requirements.txt and can be referenced from within a tox.ini | 17:10 |
| petevg | cory_fu: cool. That makes that easy :-) | 17:10 |
| cory_fu | stub: Since reactive decorators don't actually wrap the functions, they can be called (and thus tested) directly already. | 17:11 |
| stub | That was easy then :) | 17:12 |
| cory_fu | Just pass in Mock() objects in place of relation instances. If the handler manually checks states, of course you have to handle that. Also, it doesn't let you verify that your state patterns do what you want | 17:12 |
| cory_fu | I do wish we could come up with a way to render the state graph as a means of verifying it, but it's too flexible as currently impelemented (which is not necessarily a bad thing) | 17:13 |
| petevg | What I'd like to avoid doing is having tests that are so mocked out that they aren't really doing anything that a linter doesn't already do. | 17:13 |
| cory_fu | petevg: +1 | 17:14 |
| cory_fu | That's my concern with testing reactive charms currently. Especially with reactive charms around things like Bigtop, where all of the heavy lifting is done in something like Puppet | 17:14 |
| lazyPower | cory_fu - charms.reactive -p get_states is my new favorite thing ever | 17:15 |
| stub | I still can't see a way around integration tests. Just need to minimize them since they are slow. | 17:16 |
| mbruzek | Why did it take so long to learn about this option? | 17:16 |
| cory_fu | lazyPower, mbruzek: I usually prefer -y (though I completely forgot there was a shortcut and always used the --format=yaml long-hand) | 17:17 |
| cory_fu | -p is pretty nice, too | 17:18 |
| stub | petevg: Can you just reuse the cde to build a wheelhouse from charmtools.build ? | 17:18 |
| lazyPower | http://paste.ubuntu.com/18038388/ for those following along at home and have no idea what -p does to that command :) | 17:18 |
| cory_fu | stub: Definitely still have to have integration tests. But like you said, we'd all prefer to minimize them. | 17:19 |
| petevg | stub: probably. Though it looks like I don't need to do too much more than just point tox at wheelhouse.txt as its requirements.txt. | 17:20 |
| petevg | ... though I guess wheelhouse.txt goes away once you build the charm. And before you build the charm, you are missing any layers that it depends on, so your unit tests will fail on import. | 17:24 |
| petevg | In any case, I think that I have a place to start. Thank you, everyone. | 17:25 |
| stub | petevg: If you need API provided by a dependency layer, you are probably going to need to charm-build your layer and run the tests there | 17:32 |
| petevg | stub: yep. Which is not super unit testy (at least, not Python unit testy). But I'm not sure that there's a great way to work around it :-/ | 17:33 |
| stub | I think it is either that, or mock your dependencies entirely | 17:33 |
| jose | balloons: happy birthday! | 17:33 |
| petevg | stub: yep. The point where one starts mocking out imports is the point where one has perhaps mocked a bit too far, though :-) | 17:33 |
| stub | Are you mocking my idea? | 17:34 |
| petevg | Only a little :-) | 17:34 |
| stub | petevg: ... you could assume your dependencies are in $LAYER_PATH and futz your import path and compile the wheelhouse.txts... | 17:35 |
| petevg | True ... | 17:36 |
| stub | c/compile/combine/ | 17:36 |
| petevg | There's no guarantee that someone will actually have their layer dependencies checked out, though. | 17:36 |
| stub | I need to investigate my idea of embedding dependency layers in git subtrees | 17:36 |
| petevg | that sounds interesting. | 17:37 |
| LiftedKilt | question on the ceph-osd charm - it states that the default option for osd-journal is "no journal device will be used." Does that mean journaling is done per disk on the osd? or is journaling literally disabled | 17:45 |
| cory_fu | stokachu: I'm hitting an issue with ghost now with encoding: http://pastebin.ubuntu.com/18040360/ | 17:46 |
| cory_fu | It looks like my default encoding is ANSI_X3.4-1968 for whatever reason (this is local provider in 1.25) | 17:46 |
| cory_fu | I'm not sure what the right way to handle that is | 17:46 |
| stokachu | hmm | 17:47 |
| cory_fu | (For comparison, on my laptop, the preferred encoding is UTF-8) | 17:47 |
| stokachu | cory_fu: try replacing with check_call('npm install --production', shell=True) | 17:49 |
| stokachu | from subprocess import check_call | 17:49 |
| stokachu | maybe it's that shell library im using | 17:49 |
| cory_fu | Yeah, shell is setting universal_newlines=True which is causing the implicit decode, but I'm not actually sure why the preferred encoding isn't UTF-8 anyway | 17:50 |
| stokachu | yea that is odd | 17:50 |
| === valeech_ is now known as valeech | ||
| cory_fu | But I guess this is just more evidence that implicit conversions are a bad idea | 17:52 |
| cory_fu | WTG subprocess and by extension, shell | 17:52 |
| stokachu | :D | 17:53 |
| stokachu | we should probably switch out that shell module for subprocess | 17:53 |
| stokachu | and just handle the output ourselves | 17:53 |
| stokachu | cory_fu: does charmhelpers have a generic shell run command? | 17:54 |
| cory_fu | Yeah. I always think that it would be nice to have helpers around subprocess, but then every use case ends up needing something different that makes the helpers not work | 17:54 |
| cory_fu | No | 17:54 |
| cory_fu | Because of that ^ | 17:54 |
| cory_fu | :p | 17:54 |
| stokachu | ah makes sense | 17:55 |
| cory_fu | This is a failing of subprocess, though. The implicit decoding done by universal_newlines should accept an explicit encoding type (and shell should also accept and pass that through) | 17:56 |
| stokachu | cory_fu: im ok with just replacing shell with subprocess and working around it that way | 18:03 |
| cory_fu | stokachu: Yep, I'm on it. :) | 18:03 |
| stokachu | cool thanks :D | 18:03 |
| === valeech_ is now known as valeech | ||
| cory_fu | stokachu: Why does ghost hate me? :) http://pastebin.ubuntu.com/18044477/ | 18:40 |
| stokachu | wow | 18:41 |
| stokachu | are you in /srv/app by chance? | 18:41 |
| cory_fu | Nope. Was just watching a fresh deploy on aws | 18:42 |
| stokachu | hmm | 18:42 |
| stokachu | i wonder what's using /srv/app other than our attempt to remove the directory | 18:42 |
| cory_fu | stokachu: Why is ghost set up to deploy onto a storage mount? | 18:43 |
| stokachu | would juju resources be doing anything? | 18:43 |
| stokachu | so nodejs pulls /srv/app from the storage functions | 18:43 |
| cory_fu | It's a storage mount, but you would think it'd be done once the storage-attached hook fires | 18:43 |
| stokachu | marcoceppi: ^ | 18:43 |
| stokachu | cory_fu: https://github.com/battlemidget/juju-layer-node/blob/master/lib/charms/layer/nodejs.py#L18 | 18:44 |
| stokachu | thats where we pull the app path | 18:44 |
| cory_fu | stokachu: I'm still not seeing the benefit of having the node app live on a storage mount. | 18:45 |
| stokachu | cory_fu: that was a marcoceppi change | 18:45 |
| cory_fu | I can see putting app data there, but why the app itself? | 18:45 |
| cory_fu | Ah | 18:45 |
| cory_fu | marcoceppi: Please enlighten me | 18:45 |
| stokachu | cory_fu: came from https://github.com/battlemidget/juju-layer-node/issues/3 | 18:46 |
| cory_fu | stokachu: Ok, lazyPower helped me sort it out | 18:54 |
| cory_fu | It's an issue in the ghost charm | 18:54 |
| stokachu | what was the issue? | 18:55 |
| cory_fu | We shouldn't be calling rmtree on that path; that will try to delete the mount point itself. We should be removing the app within it, if necessary (I'm not sure it's actually necessary) | 18:56 |
| cory_fu | I think that rmtree / makedirs code is a hold-over from before it was a Juju storage mount | 18:57 |
| matthelmke | I'm doing a bootstrap of juju beta 10 and it returns this error: ERROR failed to bootstrap model: no matching tools available | 18:57 |
| matthelmke | Any idea why? | 18:57 |
| cory_fu | matthelmke: What Ubuntu release is the machine from which you are bootstrapping? Xenail? | 18:58 |
| cory_fu | er, Xenial? | 18:58 |
| matthelmke | xenial | 18:58 |
| cory_fu | Hrm. Odd. It should work. Perhaps try adding --upload-tools | 18:58 |
| cory_fu | juju boostrap --upload-tools <cloud> <controller> | 18:58 |
| cory_fu | That shouldn't be necessary, but might work around the issue for you | 18:59 |
| matthelmke | Nope | 18:59 |
| matthelmke | juju bootstrap --upload-tools --config default-series=”xenial” --constraints tags=juju maas-controller maas | 18:59 |
| matthelmke | Creating Juju controller "maas-controller" on maas | 18:59 |
| matthelmke | Bootstrapping model "controller" | 18:59 |
| matthelmke | Starting new instance for initial controller | 18:59 |
| matthelmke | ERROR failed to bootstrap model: no matching tools available | 18:59 |
| cory_fu | matthelmke: I'm afraid I don't know. You could try asking in #juju-dev or pinging one of the core devs in this channel | 19:03 |
| matthelmke | thx | 19:03 |
| cory_fu | rick_h_: Maybe you could point someone out? | 19:04 |
| cory_fu | I get confused by time zones. :p | 19:04 |
| cory_fu | stokachu: I just realized the current ghost charm doesn't react to changes in the 'release' option. Should it re-fetch and reinstall the ghost app when that changes? | 19:07 |
| stokachu | cory_fu: yea it should and we also need to add an upgrade path | 19:07 |
| stokachu | i think the latest ghost is like 0.8.0 | 19:08 |
| stokachu | matthelmke: juju bootstrap casey maas/172.16.0.1 --upload-tools --config image-stream=daily --config enable-os-refresh-update=false --config enable-os-upgrade=false --config bootstrap-timeout=8400 --bootstrap-series=xenial --credential casey | 19:09 |
| stokachu | matthelmke: thats the cmd i run with beta10 and it works fine | 19:10 |
| matthelmke | Weird...this seems to be working. the only difference I see is the order of the flags | 19:10 |
| matthelmke | juju bootstrap --config default-series="xenial" --constraints tags=juju --upload-tools maas-controller maas | 19:10 |
| stokachu | cory_fu: how's the charm coming? | 19:45 |
| cory_fu | stokachu: Working on it. I'm going to have patches for the nodejs, nginx, and ghost layers | 19:45 |
| stokachu | nice! | 19:46 |
| kwmonroe | hey cory_fu, for hadoop-rest, was the plan to still build using layer:hadoop-client, or are we axing that from layer.yaml and putting hadoop-rest explicitly in the metadata.yaml? | 20:25 |
| kwmonroe | i'm finally getting around to hive, but it's been so long i forgot how we're supposed to use rest. | 20:26 |
| cory_fu | Anything using hadoop-rest should not use hadoop-client | 20:27 |
| cory_fu | It should just use the interface directly | 20:29 |
| kwmonroe | jolly good | 20:29 |
| magicaltrout | ....old chap | 20:29 |
| kwmonroe | hey magicaltrout, do you reckon PDI will need hadoop jars/libs to integrate, or do you just point at the namenode url:port? | 20:44 |
| kwmonroe | also, i'm assuming it's just hdfs integration, right? or does pdi do transforms with mapreduce? | 20:45 |
| magicaltrout | both | 20:46 |
| kwmonroe | you can't say "both" when i asked you like 4 questions. | 20:47 |
| magicaltrout | it installs itself inside HDFS, but you use map reduce steps in your transformation that run on the cluster | 20:47 |
| magicaltrout | so for example, http://wiki.pentaho.com/display/BAD/Using+Pentaho+MapReduce+to+Generate+an+Aggregate+Dataset | 20:47 |
| magicaltrout | scroll down | 20:47 |
| magicaltrout | you'll seem some screenshots | 20:47 |
| magicaltrout | you have a Map/Reduce input & Map Reduce output | 20:48 |
| magicaltrout | and you can do whatever you want in between | 20:48 |
| kwmonroe | yeah, neat! | 20:48 |
| magicaltrout | so it will need lib access AFAIK | 20:48 |
| magicaltrout | so it can run on cluster | 20:49 |
| magicaltrout | but i've not installed it myself before and I can't find the docs offhand | 20:49 |
| magicaltrout | so I could be wrong | 20:49 |
| kwmonroe | doubtful. you're magicaltrout! | 20:49 |
| magicaltrout | likely | 20:49 |
| magicaltrout | you mean | 20:49 |
| kwmonroe | yeah, i get those confused all the time | 20:50 |
| magicaltrout | hehe | 20:50 |
| kwmonroe | anyway, good to know. i'm banging around on cory_fu's hadoop-rest bits.. just curious if pdi would be another consumer. i think not, but we'll see. | 20:50 |
| stokachu | cory_fu: im stepping away for a bit, just shoot me a ping when you have those PR's up and ill get to them tonight | 20:56 |
| cory_fu | stokachu: Ok, just about done (finally). | 20:56 |
| stokachu | cory_fu: cool ill check back in about an hour | 20:57 |
| cory_fu | stokachu: Ok, three PRs ready for you whenever you get back | 22:17 |
| cory_fu | https://github.com/battlemidget/charm-layer-ghost/pull/2 | 22:18 |
| cory_fu | https://github.com/battlemidget/juju-layer-node/pull/8 | 22:18 |
| cory_fu | https://github.com/battlemidget/juju-layer-nginx/pull/5 | 22:18 |
| stokachu | cory_fu: nice will take a look | 22:19 |
| stokachu | cory_fu: merged thanks! I'll push a new version of ghost up soon to charmstore | 22:28 |
| cory_fu | Awesome. Thanks | 22:29 |
| cory_fu | stokachu: When you do, can you comment on the issue that it's been pushed, and the new revision? | 22:31 |
| stokachu | cory_fu: yea will do | 22:31 |
| cory_fu | Thanks | 22:31 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!