[15:01] <ctlaugh> can anyone offer some assistance with python-jujuclient?
[15:52] <kirkland> marcoceppi: howdy!  can you give some attention to https://bugs.launchpad.net/charms/+bug/1342847 and https://bugs.launchpad.net/charms/+bug/1342843 ?
[15:52] <_mup_> Bug #1342847: please add transcode-cluster bundle to the charm store <Juju Charms Collection:New> <https://launchpad.net/bugs/1342847>
[15:52] <_mup_> Bug #1342843: please include transcode charm in charm store <Juju Charms Collection:New> <https://launchpad.net/bugs/1342843>
[15:53] <marcoceppi> kirkland: sure thin
[16:00] <tvansteenburgh> anyone seen this and know how to work around? http://pastebin.ubuntu.com/7856024/
[16:11] <marcoceppi> tvansteenburgh: is that local provider in lxc?
[16:22] <kirkland> marcoceppi: thanks;  it would be fantastic if that could land today-ish :-)
[16:22] <tvansteenburgh> marcoceppi: no, local provider in virtualbox
[16:26] <jose> happy sysadmin day!
[16:28] <mbruzek> to you to jose
[16:28] <mbruzek> Happy Friday jose!
[16:28] <jose> thanks!
[16:33] <mbruzek> kirkland the transcode README could be markdown, looks like you duplicated a step step 5 also
[16:33] <mbruzek> kirkland, charm add readme will generate the markdown template that is recommended
[17:14] <kirkland> mbruzek: okay, great -- is that all that's blocking it?
[17:15] <mbruzek> kirkland, no it is just what I found so far
[17:15] <kirkland> mbruzek: Error: add is not a valid subcommand
[17:16] <jose> kirkland: try `juju charm add readme`?
[17:16] <kirkland> http://paste.ubuntu.com/7857501/
[17:22] <kirkland> mbruzek: can you just point me to your favorite README.md in the charm store, and I'll clone that?
[17:23] <mbruzek> kirkland, in meeting just a second.  jose has a few good ones.
[17:23] <mbruzek> jose can you give kirkland a good example
[17:24] <jose> sure, owncloud will do, sec, grabbing the link
[17:25] <kirkland> thx
[17:25] <jose> kirkland: http://paste.ubuntu.com/7857553/ is the one given by 'charm add readme'
[17:26] <jose> also, I quite like the Chamilo one
[17:26] <jose> https://bazaar.launchpad.net/~charmers/charms/precise/chamilo/trunk/view/head:/README.md
[17:52] <sebas5384> hey! o/
[17:52] <sebas5384> :)
[17:53] <sebas5384> there's any way that I can help to get the Drupal charm reviewed ?
[17:53] <sebas5384> :)
[17:54] <sebas5384> we are planning to divulge how to deploy and scale Drupal with using the charm
[17:55] <sebas5384> and if the fact of being a recommended charm is really important, and the nice logo will appear in the charm too hehe
[18:01] <kirkland> mbruzek: jose: okay, readme now markdown;  what's next?
[18:02] <jose> kirkland: which branch should I take a look at?
[18:02] <marcoceppi> sebas5384: it's definitely in the review queue, http://manage.jujucharms.com/tools/review-queue should be looked at shortly!
[18:02]  * jose just joined for the readme stuff :)
[18:02] <sebas5384> thanks for the update marcoceppi !
[19:35] <kirkland> jose: mbruzek: looks like both of you are deploying now?
[19:35]  * jose is
[19:35] <mbruzek> kirkland, I deployed and tried the first config example in the new README
[19:35]  * kirkland is deploying now, too
[19:35] <mbruzek> kirkland, looking at the logs on transcode/0 I see a loop
[19:36] <kirkland> mbruzek: yep, waiting until all worker nodes are done
[19:36] <kirkland> mbruzek: how many nodes/
[19:36] <kirkland> ?
[19:36] <mbruzek> 4 total
[19:36] <mbruzek> http://pastebin.ubuntu.com/7858501/
[19:36] <mbruzek> It does not seem to exit from this repeating pattern
[19:37] <mbruzek> looks like 0 is done.
[19:38] <jose> kirkland: I'll ping you in 5-10 when this is done :)
[19:38] <kirkland> mbruzek: can I see ls -alF /srv?
[19:39] <mbruzek> http://pastebin.ubuntu.com/7858521/
[19:40] <kirkland> mbruzek: uno momento, I'm trying to reproduce it now
[19:41] <mbruzek> kirkland, Where is the output written to?  I didn't see that in the new README.  The old README told me about the web url to hit, but not what the output file would be named.
[19:42] <kirkland> mbruzek: good point;  I'll update that.  it'll be in /srv/, which is actually served out by apache2 on every single node
[19:42] <mbruzek> kirkland, I also copied a video from my phone to the system. that is what 20140408_*.mp4 is
[19:42] <kirkland> mbruzek: so you can just point a browser to any node
[19:42] <kirkland> mbruzek: cool, that should work
[19:42] <mbruzek> kirkland, seems stuck on the first one.
[19:42] <kirkland> mbruzek: which bzr rev of the charm are you on?
[19:42] <kirkland> mbruzek: r8?
[19:43] <mbruzek> kirkland, no let me destroy and boostrap again
[19:43] <kirkland> mbruzek: okay, I'm bootstrapping now
[19:49] <kirkland> mbruzek: deploying....
[19:49] <kirkland> mbruzek: total of 8 nodes
[19:50] <mbruzek> kirkland, I have a total of 5 now
[19:50] <kirkland> mbarnett: k
[19:50] <kirkland> mbruzek: k;  mbarnett: sorry :-)
[19:52] <kirkland> mbruzek: nearly up...
[19:52] <mbruzek> kirkland, ran this command $ juju set transcode input_url=http://download.blender.org/demo/old_demos/diditdoneit.mpg output_size=640x360
[19:53] <kirkland> jose: 25 nodes, is that right?
[19:53] <jose> kirkland: yep! 25 nodes
[19:53] <kirkland> mbruzek: I just ran the same
[19:54] <kirkland> mbruzek: downloading mpg...
[19:55] <kirkland> mbruzek: mine split into 8 parts, and they're each transcoding now
[19:56] <mbruzek> kirkland, I don't see the mount on transcode/0 .  If I run sudo mount | grep srv I get nothing.
[19:56] <kirkland> mbruzek: did you add the relation?
[19:58] <mbruzek> Yes I did.  http://paste.ubuntu.com/7858678/
[19:58] <kirkland> mbruzek: is /srv/ mounted correctly in any of the other transcode/* ?
[19:58] <kirkland> mbruzek: I'm running right now against MAAS, on an Orange Box, and everything is mounted correctly
[19:59] <kirkland> mbruzek: perhaps a problem with NFS traffic in EC2?
[19:59] <jose> EC2 works fine
[19:59] <mbruzek> kirkland, I am running local lxc containers
[19:59] <kirkland> mbruzek: oh, hmm, well, maybe nfs isn't the best shared-fs for lxc?
[20:00] <kirkland> mbruzek: I've never tried nfs mounts inside of lxc
[20:01]  * mbruzek is switching to hp cloud
[20:02] <kirkland> interestingly, I just had 1 out of my 8 nodes not mount NFS correctly
[20:03] <kirkland> 2014-07-25 19:52:48 INFO shared-fs-relation-changed + mount -t nfs -o rsize=8192,wsize=8192 10.14.100.6:/srv/data/transcode /srv
[20:03] <kirkland> 2014-07-25 19:52:48 INFO shared-fs-relation-changed mount.nfs: access denied by server while mounting 10.14.100.6:/srv/data/transcode
[20:03] <kirkland> 2014-07-25 19:52:48 INFO shared-fs-relation-changed + juju-log 'mount failed: /srv'
[20:03] <kirkland> looks like the nfs service wasn't up yet, when I tried to make this relation
[20:05] <mbruzek> kirkland, I saw a similar error in my log (which is no sadly deleted)
[20:05] <mbruzek> kirkland, you have retry logic in the mount though right?
[20:06] <kirkland> mbruzek: I'll need to look again
[20:06] <mbruzek> for try in {1..3}; do
[20:07] <kirkland> mbruzek: okay;  I hand mounted nfs on the one bad node, and did a juju set, and my transcode job finished immediately
[20:07] <kirkland>    juju set transcode input_url=http://download.blender.org/demo/old_demos/diditdoneit.mpg
[20:07] <kirkland> jose: where are you stuck now?
[20:07] <jose> kirkland: check the logs - it says something about invalid data
[20:08] <kirkland> jose: ah, okay, let me try that url
[20:08] <kirkland> jose: tbh, I haven't tried a .divx file yet;  I've successfully done avi, mpg, mp4, mkv, ogg, mov
[20:09] <jose> so there's another thing there - if something fails on the other nodes it's gonna be stuck in a loop
[20:09] <mbruzek> kirkland, does the basename get renamed or does the output have the same extension
[20:09] <kirkland> jose: I suspect the install hook might need to add some additional codec packages
[20:09] <mbruzek> ?
[20:09] <jose> and there's no way for me to touch and cut that loop
[20:09] <kirkland> mbruzek: the output will tack on the resolution, codecs, and an mp4
[20:09] <mbruzek> kirkland, ack
[20:10] <kirkland> mbruzek: so for mine: diditdoneit.mpg_640x360_x264_aac.mp4
[20:10] <kirkland> mbarnett: I'll add that to the readme
[20:10] <kirkland> mbruzek: ^
[20:11] <kirkland> jose: downloading .divx
[20:12] <jose> kirkland: what do you mean?
[20:12] <kirkland> jose: I'm trying to reproduce your failure
[20:12] <jose> oh, ok
[20:12] <kirkland> jose: and my run is currently wgetting your .divx example
[20:13] <kirkland> jose: I'm trying to figure out what additional packages might be necessary to support that
[20:13] <jose> ok, cool!
[20:14] <kirkland> jose: okay, same problem here
[20:16] <kirkland> mbruzek: I upped the retries in the shared-fs relation from 3 to 60
[20:16] <mbruzek> jose and kirkland if the divx codec is not free it would be appropriate to list that in the known limitations.
[20:17] <kirkland> mbruzek: ack;
[20:17]  * mbruzek suspects divx is not free, remembers something about divx in the past
[20:19] <automatemecolema> so I have a unit of a service stuck in a dying state. Anyway to get this guy to just die
[20:20] <mbruzek> automatemecolema, You could juju destroy-machine # with the one it is on
[20:20] <mbruzek> automatemecolema, by chance were you using debug-hooks ?
[20:20] <kirkland> jose: okay -- could you try a different input format?
[20:20] <jose> kirkland: I could, but I'll have to re-deploy - the loop cannot be cut
[20:20] <kirkland> jose: ?
[20:20] <kirkland> jose: sure you can
[20:20] <kirkland> just update the config
[20:21] <kirkland> there's a couple of killalls in there
[20:21] <jose> but the first instance of the config-changed hook would need to be terminated
[20:21] <kirkland> jose: this just worked fine for me:    juju set transcode input_url=http://download.blender.org/demo/old_demos/diditdoneit.mpg
[20:21] <kirkland> jose: hmm
[20:22] <mbruzek> kirkland, jose brings up a good point.  When my local one was stuck I tried different config options and the old loop never exited.
[20:22] <kirkland> jose: okay, I can add a kill switch
[20:22] <jose> would be awesome
[20:22] <kirkland> jose: mbruzek: for now, you can just touch the DONE files
[20:22] <kirkland>                 while [ $(ls ${filename}.part*.${format}.DONE | wc -l) -lt $total_nodes ]; do
[20:23] <automatemecolema> mbruzek, I think we'll just delete the machine, I was just trying to avoid that
[20:23] <jose> kirkland: are you applying your change now? so I can branch and re-deploy
[20:23] <automatemecolema> mbruzek, my consultant was logged into debug-hooks at some point, but not sure when
[20:23] <kirkland> jose: yes, give me a minute
[20:23] <jose> np
[20:23] <mbruzek> automatemecolema, In some cases the debug hooks command will prevent a charm from completely dying
[20:24] <mbruzek> automatemecolema, you can try juju resolved --retry charm/#
[20:24] <kirkland> jose: mbruzek: how would you like the kill switch to work?  touch a file?  add a config option killall=true
[20:24] <automatemecolema> mbruzek, yea I tried that several times
[20:24] <jose> kirkland: probably not a kill switch
[20:24] <jose> but 'if retried for 10m then quit'
[20:25] <kirkland>   kill_all_jobs:
[20:25] <kirkland>     type: boolean
[20:25] <kirkland>     description: "kill switch for terminating all jobs"
[20:25] <kirkland>     default: false
[20:26] <jose> another thingy there
[20:26] <kirkland> that way, you can set it to true, and then back to false, to re-run
[20:26] <mbruzek> kirkland, the loop could check for a stop file of some kind. That would only be written if a new config-changed was trying to do something
[20:26] <jose> mbruzek: would two config-changed's run at the same time?
[20:26] <kirkland> actually, that would work
[20:26] <mbruzek> jose, not sure... marcoceppi would know
[20:26] <jose> kirkland: problem with that boolean option is that config-get values don't change until the new run
[20:27] <jose> mbruzek: I believe not because juju is event based, and one event needs to finish for the other to run
[20:27] <jose> but let's wait for an answer :)
[20:27] <mbruzek> marcoceppi, will a new config-get run if the old one is still looping?
[20:28] <mbruzek> kirkland, this is kind of a big change, but could you fork the process to a script so the config-get exists immediately?
[20:29] <mbruzek> kirkland, once the charm gets all the information it needs, just call another bash script to do the dirty work?  The loops in the new script could check for a stop flag
[20:29] <kirkland> mbruzek: is there an easy way for my charm to install a file?
[20:29] <kirkland> mbruzek: and a here-doc is not what I'm looking for
[20:29] <mbruzek> yes.
[20:30] <mbruzek> copy it from the charm directory
[20:30] <mbruzek> kirkland, I have created a files/ directory within the charm.
[20:30] <kirkland> mbruzek: okay -- I'm game for that
[20:30] <kirkland> mbruzek: can you give the the cp rune?
[20:30] <kirkland> mbruzek: an example of where I'm copying it from?
[20:31] <kirkland> mbruzek: or, rather, how to call it?  it doesn't need to be copied
[20:31] <mbruzek> cp ${CHARM_HOME}/files/file.tar.gz /tmp/
[20:31] <mbruzek> kirkland, you could just put another bash script in the hooks directory or charm root
[20:32] <mbruzek> source run_transcode_loop.sh arg1 arg2 arg3
[20:32] <mbruzek> not source
[20:32] <mbruzek> my mistake
[20:32] <mbruzek> ./run_transcode_loop.sh would be in the charm dir
[20:32] <mbruzek> ./hooks/run_transcode_loop.sh
[20:33] <mbruzek> kirkland, is that what you are asking?
[20:34] <mbruzek> kirkland, ${CHARM_DIR} == cwd in a hook.
[20:34] <kirkland> mbruzek: yeah, I have the change;  testing locally now
[20:35] <mbruzek> kirkland, I have a hp bootstrapped and ready to test next revision
[20:35] <kirkland> mbruzek: ack, let me put this through a quick local maas test
[20:35] <mbruzek> kirkland, yep
[20:35] <kirkland> mbruzek: http://paste.ubuntu.com/7858922/
[20:35] <kirkland> mbruzek: maybe eyeball that for me?
[20:36] <mbruzek> kirkland, I have lots of other stuff to work on in the mean time.
[20:38] <mbruzek> kirkland, quick look over, I would suggest keeping the juju bits in the hook and just calling transcode with all the arguments that you compute in the hook.
[20:39] <mbruzek> kirkland, but as the author feel free to override me on that.
[20:39] <mbruzek> kirkland, I was thinking the transcode script could be juju free but perhaps that is too big of a change here.
[20:41] <kirkland> mbruzek: more like this?  http://paste.ubuntu.com/7858971/
[20:41] <mbruzek> yep
[20:42] <mbruzek> I am sure both will work.
[20:52] <kirkland> mbruzek: so I can just run ./transcode from within config-changed, if transcode is in the same dir as config-changed, right?
[20:53] <mbruzek> kirkland, no I believe the cwd is charm root
[20:53] <mbruzek> I believe the proper way is to type ./hooks/transcode
[20:53] <mbruzek> iirc
[20:55] <kirkland> doh
[20:55] <mbruzek> Did I mislead you Homer?
[20:55] <kirkland> mbruzek: I'm just confused about the cwd
[20:56] <mbruzek> OK
[21:04] <mbruzek> Hey tvansteenburgh do you have a minute?
[21:05] <tvansteenburgh> yep
[21:05] <jose> kirkland: he means that the folder from which the scripts are ran is your charm root
[21:05] <jose> bare in mind that your scripts are in the hooks folder
[21:06] <kirkland> jose: right -- I'm curious about the cwd when a hook runs
[21:06] <tvansteenburgh> mbruzek: yeah what's up?
[21:06] <jose> it's /var/lib/juju/agents/unit-charmname-number/charm/
[21:06] <jose> that is exported as a variable, called CHARM_DIR
[21:06] <jose> so every time you call $CHARM_DIR, it's taking you to that path
[21:06] <mbruzek> I am trying to deploy a bundle with a local reference to the charm do you remember what we decided worked?
[21:07] <tvansteenburgh> mbruzek: set the 'branch' key for the service
[21:07] <tvansteenburgh> (to the local path of the charm)
[21:07] <mbruzek> http://pastebin.ubuntu.com/7859181/
[21:08] <mbruzek> I am getting a lot of key errors
[21:08] <mbruzek> but that is using   charm: /home/ubuntu/charms/trusty/postgresql-psql
[21:08] <mbruzek> Will switch this to branch and try again
[21:12] <mbruzek> tvansteenburgh, We had this problem solved, I thought it was charm: /path/to/charm but I just tested it with branch.
[21:12] <mbruzek> http://pastebin.ubuntu.com/7859194/
[21:12] <mbruzek> tvansteenburgh, Can you have a look and tell me where I am wrong?
[21:13] <mbruzek> same KeyError with branch as with charm
[21:19] <tvansteenburgh> mbruzek: do you still have your local amulet changes
[21:19] <mbruzek> tvansteenburgh, yes I believe so, but that is a good question, how can I verify?
[21:20] <tvansteenburgh> cd to amulet dir, run `git status`
[21:20] <kirkland> fuck
[21:21] <mbruzek> modified:   amulet/deployer.py
[21:21] <kirkland> it's not working anymore
[21:21] <kirkland> http://paste.ubuntu.com/7859278/
[21:21] <mbruzek> kirkland, looking
[21:21] <kirkland> seems the script just dies now inside of that while loop
[21:21] <kirkland> after the sleep
[21:21] <kirkland> disappears from ps
[21:22] <tvansteenburgh> mbruzek: `git diff`
[21:22] <mbruzek> tvansteenburgh, http://pastebin.ubuntu.com/7859281/
[21:23] <tvansteenburgh> ok, those have already been added to upstream
[21:23] <kirkland> transcode/0 is currently doing the download, and that's going, but it seems to exit right after the wget
[21:23] <kirkland> transcode/1-N just all die inside of that while loop
[21:26] <kirkland> I'm at a loss for idea
[21:26] <kirkland> ideas
[21:27] <kirkland> I'm feeling at this point the charm store is more trouble than it's worth
[21:39] <mbruzek> kirkland, I am looking, trying to find what command is causing the hook to exit?
[21:40] <mbruzek> kirkland, could it be another config-changed event is coming by and killing this first transcode?
[21:41] <kirkland> mbruzek: that would be my best shot in the dark
[21:41] <mbruzek> kirkland, what is dying the transcode or the hook?
[22:02] <bloodearnest> is there a way to juju set a value to either a) and empty string or b) back to it's default value?
[22:06] <mbruzek> bloodearnest, does juju set charm key="" not work?
[22:06] <bloodearnest> mbruzek: nope
[22:06] <bloodearnest> nor = "''", = '""' or just =
[22:06] <bloodearnest> this is on 1.20
[22:09] <bloodearnest> I should say, juju set charm setting='""' does work, but it sets the value to ""
[22:11] <mbruzek> bloodearnest, I was not aware that setting to empty string threw an exception
[22:12] <bloodearnest> mbruzek: yeah, that's new with 1.20
[22:13] <mbruzek> http://pastebin.ubuntu.com/7859627/
[22:13] <mbruzek> bloodearnest, You see a stack trace like that?
[22:13] <bloodearnest> mbruzek: I've been seeing quite a few panics, although not always fatal (as in, juju deploy x shows traceback, but the deploy has been kicked off)
[22:14] <bloodearnest> mbruzek: exactly
[22:14] <mbruzek> That should not be that way.  you should ping in #juju-dev
[22:14] <bloodearnest> mbruzek: kk
[22:14] <mbruzek> bloodearnest, and open a bug against juju-core
[22:15] <mbruzek> It should not panic like that
[22:22] <bloodearnest> mbruzek: https://bugs.launchpad.net/juju-core/+bug/1348829
[22:22] <_mup_> Bug #1348829: juju-core client panics with juju set empty string <juju-core:New> <https://launchpad.net/bugs/1348829>