[00:24] SpamapS, subparsers can be useful, but they usually need some programmatic help on the parse. one extreme example is what i did with jitsu watch [00:24] still better than trying to write that parser from scratch however [00:25] perhaps [00:25] I think I've wrangled it almost [00:25] but still can't suppress --help in subparsers [00:29] SpamapS, add_help=False doesn't help? [00:29] so to speak ;) [01:23] Any still around? Is the only way to get the charm revision number with a --> cat hooks/revision <-- ? === jml is now known as jml_ === jml_ is now known as jml === zyga-afk is now known as zyga === al-maisan is now known as almaisan-away [11:44] <_mup_> Bug #1016003 was filed: "juju debug-hooks -h" doesn't say what it does < https://launchpad.net/bugs/1016003 > [12:15] <_mup_> txzookeeper/trunk r48 committed by kapil.foss@gmail.com [12:15] <_mup_> correct unit tests minors [thanks to ben bangert for spotting] === almaisan-away is now known as al-maisan [12:22] I'm working on a charm that deploys code from a bzr branch. Currently, I'm fetching the branch into $PWD, which is /var/lib/juju/units/$UNIT_NAME/charm/. Is this sensible? [12:22] Is there a better practice? [12:26] jml, its sensible, but you can pull it anywhere [12:26] what user is 'install' run as? [12:26] jml all hooks run as root [12:26] huh === bac` is now known as bac [12:43] jcastro, ping [13:01] is everything in the charm directory copied up to the instance indiscriminately? [13:06] hazmat: pong [13:11] jml, yes [13:11] jml, or at least you can add arbitrary stuff, I don't know that e.g. metadata.yaml is copied exactly [13:14] hey folks.. does juju on openstack require swift? [13:15] marrusl, IIUC yes, but you can point it to S3 and it works fine [13:16] as in, it needs object store, but it can use openstack for compute and s3 for object store [13:16] james_w, aha, ok. great, that makes sense. [13:16] but I don't know how auth works there [13:17] I imagine it doesn't store much, but I wonder if that will be a security issue for some. they might feel safer keeping it all inside. [13:27] marrusl, there's also a standalone s3 compatible impl nova [13:28] if their not using swift they can just stand that up by itself on a nova api server [13:28] hazmat, oooh? i.e. nova-objectstore? [13:28] marrusl, its just a dumb s3 impl in nova for compatibility &testing, no replication etc, just stores files in a dir. [13:29] but functional for juju's needs [13:30] hazmat, indeed. we will check it out. thanks! === zyga is now known as zyga-food [14:20] jml: re your question about whether its good practice to store in the charm dir or not.. I think its actually the best practice, because the charm dir gets completely deleted when the service is destroyed. [14:22] SpamapS: I can't make the connection [14:22] SpamapS: If you're installing a package, you don't care that it's in the system directories [14:23] SpamapS: so why is auto-deletion from the charm dir a win if you're installing/running from a branch? [14:23] jml: I mean, if you are, at runtime, storing flag files or downloaded data or something, the charm dir is a good place to do that. [14:23] SpamapS: ah rigght. [14:24] SpamapS: in this context, I'm am bzr branching at install time [14:24] hm [14:24] for that I might put it somewhere else in case I re-deploy onto the box [14:24] since its basically an immutable cash [14:24] cache [14:24] well, it's just apt by another means, no? [14:25] right, and apt is going to cache your debs in /var/cache :) === zyga-food is now known as zyga [14:42] SpamapS, i had to do an increment on txzookeeper latest is 0.9.6.. it looks like the build doesn't like that though [15:06] hazmat: looks like you still are using the debian dir from trunk instead of distro [15:07] hazmat: so you will need to dch -i in trunk, since the recipe uses debupstream === stevanr_ is now known as stevanr [15:21] SpamapS, ack, will check it out post meeting [15:26] Question for the room: I saw a note about augtool in the wordpress charm. Has anyone tried it out? [15:27] http://augeas.net/tour.html === al-maisan is now known as almaisan-away === stevanr_ is now known as stevanr [15:57] mars, re augtool, that's an old note on my part as a todo. but sure, it would be cool to try [16:01] jimbaker, it has potential, a standard interface for config files is a nice idea. [16:02] jimbaker, otherwise every charm writer will use their own way of hacking config files [16:02] mars: My feeling, after using augtool/augeas a few times, is that it is useful when you absolutely must only *edit* a complicated config file. [16:02] mars: its far simpler to use templating and just build the whole file. [16:03] is there an idiom for 'juju-log if I can but otherwise skip'? [16:03] SpamapS, makes sense [16:03] (my subtly hidden question is, why isn't that an option or even the default behaviour of juju-log?) [16:03] jml: || : ? [16:03] mornin [16:03] jimbaker, SpamapS, thanks [16:03] SpamapS: ok. [16:03] jml: when are you failing to log? [16:04] SpamapS: when I'm running scripts manually on the instance to debug stuff [16:04] No JUJU_AGENT_SOCKET/-s option found [16:06] jml: I find it better to use debug-hooks for that [16:06] jml, fwiw, we saw that exact error on the first day of our sprint [16:06] jml: as then you're running it in the appropriate context [16:07] SpamapS: sorry, I figured debug-hooks wasn't ready for folk to use [16:08] no way, its a pretty awesome toy :) [16:08] jml, we resolved it by making sure we used the correct procedure to run debug-hooks [16:08] as debug-hooks --help doesn't actually say what it is, and the documentation on the website says it can't be used for install [16:08] It has proven hard to describe how to use debug-hooks.. I think we need screenshots in the documentation [16:08] jml: OH thats a bug in the docs.. that was fixed [16:10] +1 for debug-hooks docs with screenshots. The text description of the procedure we have written down is difficult to follow without a live byobu terminal in front of you. [16:11] mars: OK, I'll bite. What's the correct procedure? How did you make sure you used it? [16:11] jml, just a sec, I'll pastebin it [16:12] I do think we need to add a --debug flag to deploy which deploys and immediately fires up debug-hooks so you don't miss the install hook, as that is possible if the machine is already running. [16:12] jml, http://pastebin.ubuntu.com/1052882/ [16:13] mars: ta [16:14] hi [16:15] so I have these neat bash hack that pops up an inotify thingy when long running commands finished [16:16] I wish, I wish, I wish I could have a command that ran for as long as the deploy process took. [16:17] fwiw, notes I've made on today's work so far: http://paste.ubuntu.com/1052894/ [16:18] jml: jitsu watch [16:19] SpamapS: oh that's right. I forgot to play with that. [16:19] jml: and when I release juju-jitsu 0.13 later today or tomorrow, it will actually have a --help :) [16:19] http://paste.ubuntu.com/1052901/ [16:20] jml: still needs a lot of work.. some of the commands don't have their own --help .. but its a nice step forward. :) [16:20] jml, jitsu watch could definitely be nice for starting your notification === salgado is now known as salgado-lunch [16:20] SpamapS: will 'jitsu watch mysql' also stop watching if the service has an error in its deployment? [16:20] jimbaker: btw we need to be able to wildcard unit ids [16:21] SpamapS: looks good :) [16:21] SpamapS, i think this really is covered by --num-units [16:21] jimbaker: how so? [16:21] I really need to figure out how I can make my bash hack more readily usable by others. [16:21] jimbaker: I want to deploy, then immediately wait for a state of started. But id is not guaranteed to be 0, because the service name may have been used before. [16:21] SpamapS, let me dig out the example [16:22] SpamapS: do you know how juju is retrieving the node's public ip when using maas? I seem to be getting .localdomain . In MAAS, i am using a blank domain, because I didnt feel like setting up a real domain. resolves in my environment, .localdomain does not. I think this is causing my relations to not work. [16:22] * jml is on libdep-service/29 [16:23] SpamapS, you can do stuff like this, once the watch-ports branch is approved & merged: [16:23] timeout 600s ./sub-commands/watch \ [16:23] mysql --state=started -r "mysql wordpress" --setting=database \ [16:23] wordpress --state=started --open-port=80 [16:24] so that's saying, wait until at least one unit of mysql (--num-units=1 is implied with the branch) is in the started state, and it has a database setting [16:24] likewise also wait until at least one unit of wordpress is in the started state and it has an open port of 80/tcp [16:25] SpamapS, i think that works better than wildcards on service units, really care about services here [16:25] japage: IIRC, its just 'hostname -f' .. but I could be wrong [16:26] jimbaker: ah ok thats good, I didn't realize I could use --state without a unit id [16:26] SpamapS, yeah, it's pretty nice in that way [16:27] SpamapS, you do need to specify --num-units until watch-ports branch lands however [16:27] Spamaps: yep, that seems to be it... hmmm, i wish that didnt happen. Thanks for pointing me in the right direction [16:27] what am I doing wrong? http://paste.ubuntu.com/1052914/ [16:28] docs say "$ jitsu watch mysql # service is deployed" [16:28] SpamapS, here's another nice example, drawn from your unit test spec [16:28] timeout 600s jitsu watch \ [16:28] mysql --state=started -r "mediawiki:db mysql:db" --setting=database \ [16:28] memcached --state=started -r "mediawiki memcached" --setting=host \ [16:28] mediawiki --state=started --open-port=80 [16:28] but status says service is not deployed. [16:29] so it waits until the full stack is deployed and in a steady state, because the appropriate settings have been made and in particular mediawiki has reached the end of its -relation-changed hook and has opened a port; note in this case it ignores exposed or not [16:29] <_mup_> Bug #1016138 was filed: The juju manpage should mention the JUJU_REPOSITORY environment variable < https://launchpad.net/bugs/1016138 > [16:30] japage: I think there's an assumption in MaaS that you will have working DNS [16:32] SpamapS: It provides you with the option of using blank for the dns, which is supposed to use the maas server's dhcp to provide the dns, however i could be wrong about that. [16:37] well your problem is there is no proper domain setup, not that there isnt dns, localdomain is in the hostfile [16:37] not dns [16:38] thus the node will report it but others will not reach it. break down and setup a domain if you need one :) [16:38] imbrandon: yea [16:39] japage: in otherwords its working as intended , you just need to setup a domain if your going to use one [16:40] still having a bit of trouble with 'jitsu watch' [16:40] is 'jitsu watch ' supposed to wait until is deployed? i.e. until at least until after 'install' completes successfully? [16:41] juju is working as intended, im just not sure why hostname -f is returning maas-1.localdomain [16:41] jimbaker: watch-ports is in trunk... I'll wait a day or so before another release unless anybody needs this now [16:41] jml: no, you need --state=started and --num-units=1 [16:42] japage: because localdomain is set properly in the hostfile and there is no other domain setup for the box [16:42] SpamapS: thanks. [16:42] Anybody familiar with argparse want to help me fix 'jitsu sub-command --help' ? I want to override it to pass --help to the subcommand instead of intercepting and printing lame sub-command help [16:42] japage: i was saying dns is working as intended [16:43] (only an issue in trunk.. 0.12 has no help for jitsu) [16:43] jml, when m_3 releases the next version of jitsu, you won't need to specify --num-units if using service unit specifications [16:43] unless you want to do --num-units=2 or whatever [16:43] m_3, thanks for that merge [16:43] SpamapS: what about juju its self, doesnt it do that ? [16:43] jimbaker: 'if using service unit specifications'? [16:43] jml, correct [16:43] jimbaker: sorry, what I meant was I don't understand that clause [16:43] i quickly realized it was an oversight [16:44] jml, not certain what you mean by which *clause* [16:44] imbrandon: juju's sub-commands are all just python modules [16:44] jimbaker: what's a service unit specification? [16:44] imbrandon: so they all just add their sub-parser to the main parser [16:44] ahh [16:44] imbrandon: but I need to essentially say "don't print --help for sub-commands" [16:44] optparse :) heh [16:45] which I'm pretty convinced argparse just won't allow [16:45] perhaps [16:45] SpamapS, i can take a look at --help support [16:46] SpamapS, i assume you just want what we see with juju --help, right, a synopsis for each available subcommand based on the description [16:46] SpamapS: looks like there's a `parser = argparse.ArgumentParser(prog='PROG', add_help=False)` [16:46] from the subparser [16:46] japage: you see what i'm gettin at ? it returns that ( rightfully ) because that is in its hostfile as it should be, that does not garentee its reaschable [16:46] then maybe explicitly add it to the subcommands [16:47] m_3: so unfortunately, all the sub-parsers will share --help with the main command. [16:47] m_3: as in, if you don't add_help .. you get no help. [16:47] why would, for a maas with no domain set (blank, not local, which tells it to use mDNS), cloud-init write an /etc/hosts file with 127.0.1.1 .localdomain ; instead of 127.0.1.1 .localdomain ? [16:47] and once you add it to one command [16:48] you can't add it to any other [16:48] it's like we need a help delegator [16:48] nvmd, i just figured it out, /me stupid question previously... [16:48] m_3: yeah I think we just need to override the --help action with a function that is smarter than me :) [16:48] japage: becase thats how dns is designed, you need to setup a proper domain if you want to use one [16:48] m_3: but one tricky part is, I want it to use the usual --help action if there's no sub-command specified [16:49] right... doesn't juju do this? [16:50] m_3, indeed, that's what it does. looking at jitsu, it's roughly doing something similar [16:50] but clearly not quite there [16:51] SpamapS: imbrandon: Which one of you "owns" the sexy column-on-the-side layout for the docs? [16:51] me [16:51] hey can we get it reviewed and landed by say Monday PST? [16:51] it's too sexy not to show off at Velocity [16:51] its supose to be landed now, but its not building [16:51] need to get in touch with IS [16:51] oh, heh [16:52] probably on a lucid box or something :-P [16:52] as in the cron is broken or something [16:52] imbrandon: try in a lucid chroot, probably some missing sphinx feature [16:52] k [16:52] yea its using sphinx 0.6.3 OLD we;re all on 1.1.3 [16:52] heh [16:53] jcastro: but yea as soon as we figure that out, its landed [16:53] * imbrandon goes to build a chroot [17:00] imbrandon: mk-sbuild ftw :) [17:00] heh [17:00] Last Generated on Jun 20, 2012. Created using Sphinx 0.6.4. [17:00] sudo debootstrap --variant=buildd --arch i386 lucid /mnt/lucid/ [17:00] :) [17:01] imbrandon: schroot is your friend [17:01] one-off chroots are just a waste of time [17:01] likely but i'll keep this one until they upgrade the docs [17:01] box [17:02] Probably won't happen until 12.04.1 is released in August [17:02] yea, so we may run into this again [17:02] hopefully not, but you know [17:02] heh [17:03] brb more mt dew while that builds [17:04] imbrandon: i set up a real domain, called localdomain. (cheating ftw) :-) [17:05] lol [17:05] imbrandon: where does that branch live btw? [17:06] since I have lucid as of 'schroot -c lucid-amd64 -u root' ;) [17:06] SpamapS: what one ? docs ? lp:juju/docs [17:06] imbrandon: oh so docs just isn't building at all right now? [17:06] right [17:06] look its still old from before all our changes [17:06] Exception occurred: [17:06] File "/usr/lib/pymodules/python2.6/sphinx/builders/html.py", line 653, in [17:07] ctx['toctree'] = lambda **kw: self._get_local_toctree(pagename, **kw) [17:07] TypeError: _get_local_toctree() got an unexpected keyword argument 'maxdepth' [17:07] nice [17:07] ok one sec [17:07] heh, we really should make juju.ubuntu.com a charm [17:07] :) [17:07] like.. seriously [17:07] how its on lucid :) [17:08] unless we chroot it on the box, but yea that woudl be awesome , dogfood it [17:08] mouth.where().put(money) [17:08] ok let me fix the maxdepth issue [17:08] imbrandon: it shouldn't be on lucid forever though. :) [17:09] one sec [17:09] ./source/_templates/ubuntu1204/layout.html:{{ toctree(maxdepth=2) }} [17:09] so thats f'ing it up? [17:09] yea [17:09] imbrandon: redirect :) [17:09] SpamapS: just make toctree(maxdepth=2) == toctree [17:09] 2 is the default anyhow [17:12] hmmm, appears my kludgy dns was a red herring, still seem to be having mysql issues creating relations === Leseb_ is now known as Leseb [17:17] m_3: hehe :) [17:17] imbrandon: seriously... I do agree we should be dogfooding that one [17:18] just have the dang thing redirect to ec2 [17:18] SpamapS: fido [17:18] yea i think i;m gonna do that today [17:18] we dont have a sphinx charm anyhow [17:18] that i know of [17:21] You know, the more I think about our need to cryptographically verify upstream software.. the more I think we should just require embedding anything not in the Ubuntu archive. [17:21] Or, an archive that is sufficiently highly available like that one. [17:21] would simplify a lot of charms to just toss tarballs into them [17:21] and make them more robust [17:21] ... the more i hear stuff like that the more i think about jcastro saying "you can do anything in a charm" [17:22] heh [17:22] YOU can do anything in a charm [17:22] But I'm not going to inflict all the crazy brandon stuff on everyone. ;) [17:22] heh well i can do anything in a deb too for that matter :) [17:22] right! [17:22] lol [17:22] SpamapS: right, so like right now the mod_spdy one is worthless because the google archive times out all the time [17:22] and so on [17:22] deb can do anything charms can do better.. debs can do anything charms can do... [17:23] * SpamapS sings a little song [17:23] there's an offline apt thing that would work for that too [17:23] SpamapS: http://s3.assets-online.com.s3.amazonaws.com/files/nginx/nginx-1.3.1.spdy.tar.gz [17:24] :) [17:24] more and more I think charms might benefit from a build step too.. where you could just run a "rebuild" that downloads and unpacks and stuff.. so the install hook isn't doing so much work [17:24] patched and ready, just not had the time to build it into a charm yet [17:24] imbrandon: yeah thats plenty available [17:24] even from outside Amazon :) [17:25] right, none pre-patched tho, thats why i made that tarbal [17:25] SpamapS: yeah, install feels like a beast sometimes [17:25] I try to move as much out of it as possible [17:25] can do the "build" step in config-changed really [17:25] win 17 [17:26] SpamapS, re jitsu --help on trunk, it's just not introspecting things properly. i can fix that [17:26] m_3: did you not see my objections to kees's sbuild merge in the bug? [17:26] m_3: actually I didn't make them in the bug, whoops [17:26] m_3: anyway, we can't be dropping config options [17:26] m_3: will likely break deployed services [17:26] SpamapS: whoops... sorry, no didn't see that [17:26] m_3: It wasn't well communicated [17:27] well we can, we just need a way to get at the old ones [17:27] keeping the old ones promostes crift [17:27] cruft [17:27] m_3: lets just leave this one be, but the policy doc I put out described this as a 'de-facto' rule.. where if something has been around for 30+ days, it can't be dropped [17:27] SpamapS: roger [17:27] imbrandon: cruft can be removed in the next series [17:27] ugh that sounds bad [17:28] Until juju gives us config-set .. cruft must remain [17:28] next series 5+ years [17:28] imbrandon: no, 2 years [17:28] tho we can fix it in quantal [17:28] just that nobody will care ;) [17:28] server where most of these target [17:28] 5 [17:28] imbrandon: *2* [17:28] every 2 years theres a new LTS [17:29] so you expect that everyone will upgrade ? we still have stuff in IS on lucid [17:29] I don't mind keeping the cruft around for the full LTS lifetime as long as we don't keep it forever in the current LTS [17:29] imbrandon: this isn't about everyone upgrading, its about easing development burden [17:29] existing users won't care about the cruft [17:30] I would like to see juju grow a 'deprecated' tag for options [17:30] so deploy will yell loudly [17:30] and set will warn [17:30] yea thus i say we need a way to get at historical config options not promote cruft [17:30] we are making policy based on bugs [17:30] yes we are [17:30] thats life [17:30] if we made policy on perfection, we'd have no policy :) [17:31] welll thats like me adding to the metadata.yaml cuz i could [17:31] the bug being.. ? [17:31] only approved items should be there [17:31] Thats not a bug, thats a fact. :) [17:32] One agreed upon by a pretty short discussion. [17:32] but nothign enforces that , i am just saying instead of a short lived policy lets fix the bug [17:32] its on the TODO [17:32] ok then no need for policy [17:33] Yes there's a need for a policy so we can ease unwanted affect of the bug being fixed [17:33] and how is it on the todo i just sugested it ? [17:33] Basically the policy is saying "Don't do that, because its not going to work at some point" [17:34] no it will work at some point when the bug is fixed, it dont work NOW and coudl breaak things [17:34] There's already a bug somewhere to start warning on unknown fields. [17:34] see the diff [17:34] no no [17:34] i'm on the original thing, not metadata [17:34] <-> see that, we just talked right past eachother [17:34] i just used that for example [17:34] * SpamapS capitulates [17:34] hehe [17:35] yea i think we;re on the same page just [17:35] was on diff subjects [17:35] ok /me goes back to docs [17:36] btw did you change that, can you change that toctree and rebuild to make sure i'm rtight before i commit [17:36] imbrandon: push to some other branch and I can try it [17:36] k [17:36] I tried removing the maxdepth and got something else [17:36] oh [17:36] fun [17:36] File "/usr/lib/pymodules/python2.6/docutils/nodes.py", line 92, in setup_child [17:36] child.parent = self [17:36] AttributeError: 'NoneType' object has no attribute 'parent' [17:37] which looks way nastier [17:37] yrs [17:37] Perhaps we can request an upgrade to 12.04 :) [17:37] bah [17:37] please [17:37] heh [17:37] but will they do it before monday ? heh [17:38] jcastro: jujucharms.com/docs has the current build too btw in a pinch, but thats on hazmat's $$ sooooooo === salgado-lunch is now known as salgado [17:43] SpamapS, yeah. i suspect its on lucid given how old the sphinx their attempting to use is [17:46] definitely lucid [17:46] such is life, unless we embed sphinx in the branch ;) [17:46] so, I've finally got my thing deploying from my charm [17:46] which is great [17:46] but, that would be eeeevil right ;) [17:46] hey so I have been thinking SpamapS [17:46] jcastro: dangerous that [17:47] what does bundling the tarball in the charm accomplish [17:47] https://code.launchpad.net/~jml/libdep-service/juju/+merge/111396 has the MP. I would *really*, *really* appreciate a review from an experienced charmer [17:47] I still have to trust you. [17:47] by which I mean you, SpamapS [17:47] jcastro: makes the deploy/add-unit more predictable [17:47] I can't confirmed that you didn't check the sha either [17:47] oh ok, so you mean purely for "it will work every time" [17:48] jcastro: presumably we will add some crypto verification to the charm store beyond what we have now (https for launchpad). Either way, we're trusting the charmer to provide a valid SHA.. so providing the actual file is the same thing [17:48] jml: I'll look right now, since you asked so nice [17:48] * jcastro nods [17:48] SpamapS: thanks. [17:49] I *do* think we need to build a Packages.gz type of file for the charm store [17:49] which is signed [17:49] SpamapS: I have to leave in the next couple of minutes, so please put your comments on the MP. [17:49] and have the commits to the bzr branches signed too probably [17:49] jml: will do [17:49] SpamapS: thanks! [17:56] The *config-changed* hook will trigger the first time the charm is deployed. Is this functional to anyone? [17:58] yes, thats where most of my install actions come in [17:58] ver little is done in the install hook for me [18:01] surgemcgee: aye, its guaranteed to run, unless install or start fails [18:02] incidentally, the juju tests take a while to run. [18:02] speaking from personal experience, you want to get on that sucker now, or you'll become Launchpad. [18:06] jml: 7 minutes? [18:07] jml: for 98% coverage.. [18:07] jml: also remember that the python code base is done growing. :) [18:07] jml: on an SSD they only take 4 minutes. [18:07] SpamapS: I just ran ./test on my machine w/ an SSD and it's still going [18:07] SpamapS: maybe I'm supposed to run the tests differently [18:07] ./test [18:07] thats all I do [18:07] SpamapS: way more than 7m [18:08] takes at most 7 minutes [18:08] jml: bad java maybe? [18:08] SpamapS: possible [18:08] it taxes zookeeper quite a bit [18:08] jml: also try 'eatmydata ./test' [18:08] SpamapS: I just have whatever data is on the system [18:08] that at least disables all the syncing that zookeeper wants to do [18:08] SpamapS: so, hang on.... [18:08] if the python code base is "done growing" as you say [18:09] then there's zero point in me contributing patches for the bugs I've filed. [18:09] jml: Its not done living, its just not going to get any more feature dev. [18:09]