[00:01] marcoceppi, while awesome i'd still probably recommend either cloudfront or rackfiles/limelight for a true cdn [00:01] cdn need points of presence for global effectiveness [00:02] hazmat: depends on your perspective. Sometimes all you really want is aggressive caching and fanout [00:02] latency is by far the *best* reason for CDN... but by no means the only one [00:05] mpl: Super, let me check [00:05] SpamapS, ec2 hosts don't really give fanout, all a juju node for cdn amounts to is a single host.. thats a sig difference.. it can work.. but its not the same league [00:06] mpl: Very nice [00:06] /me heads out for beer o'clock [00:06] hazmat: Cheersfully! [00:08] <_mup_> juju/go r21 committed by gustavo@niemeyer.net [00:08] <_mup_> Merged jujugo-log by branch Mathieu. [r=rog,niemeyer] [00:08] <_mup_> This branch moves logging onto the log package so that it is [00:08] <_mup_> usable by any of the subpackages, and simplifies/polishes the [00:08] <_mup_> interface further. [00:22] <_mup_> juju/scp-command r419 committed by jim.baker@canonical.com [00:22] <_mup_> Addressed review points [02:46] hazmat: It's not for a CDN, it's a patch for a CDN issue that needs to scale. So it's using hadoop charm with a custom charm I wrote that creates a sort of...CDN proxy...for an issue several customers are having because Cisco has rackcdn.com blacklisted in their enterprise router software [02:49] For those customers we have our templating engines on their site switch out rackcdn.com with another DNS and the servers fetch the file from the CDN, cache it, then serve it up. Bandaid until the blacklist is lifted which could take a while [04:35] <_mup_> juju/scp-command r420 committed by jim.baker@canonical.com [04:35] <_mup_> PEP8/PyFlakes [04:37] <_mup_> juju/scp-command r421 committed by jim.baker@canonical.com [04:37] <_mup_> Updated tab completion [04:47] <_mup_> juju/scp-command r422 committed by jim.baker@canonical.com [04:47] <_mup_> Slim down help output [04:48] <_mup_> juju/scp-command r423 committed by jim.baker@canonical.com [04:48] <_mup_> PEP8 [04:53] <_mup_> juju/trunk r425 committed by jim.baker@canonical.com [04:53] <_mup_> merge scp-command [r=fwereade,hazmat][f=720307] [04:53] <_mup_> Implements juju scp subcommand. [07:20] fwereade_: mornin' === TeTeT_ is now known as TeTeT [08:03] rog: heyhey :) [08:03] fwereade_: :-) [08:04] rog: hmm, you were up disturbingly early ;) [08:04] fwereade_: it's not uncommon - i usually don't make a peep in here until i see some activitity tho' [08:05] rog, ah, yeah, I tend to follow the same sort of approach [08:05] fwereade_: actually 7.20 *is* earlier than usual! had to go out and scrape the ice off the car. [08:05] rog, oof [08:05] not much ice where you are... :) [08:06] rog: well, it's getting pretty cold, it's 15 now :p [08:06] lol [08:06] mind you, you probably don't have working central heating... [08:07] indeed not :) [08:07] actually I just realised I need to pop out for a bit [08:07] bbs :) [08:07] c y [08:49] Mio [08:50] argh, moo, that's better [09:09] hi all [09:28] mpl: hiya [10:31] if anyone who's written trickyish charms is around and awake, please ping me [11:14] scratch my request above [11:52] Morning all! [11:52] heya niemeyer [11:52] moo [11:55] fwereade_, TheMue: Hey folks! [11:58] niemeyer, TheMue: morning! [12:03] hi rog [12:06] yo [12:13] rog, mpl: Hey folks [12:16] Sooo! Juju Charm School today. \o/ [12:20] mainerror: Ohhh, indeed! [12:20] Can't wait. [12:21] niemeyer: almost forgot about that. :P [12:21] what time (UTC) is it again? [12:23] My Charm drives me nuts. This CMS writes its config options into more than one file ... [12:23] 15:00 UTC jrgifford. [12:28] niemeyer: ec2-ec2test pushed. second stage merge request for ec2 now in (https://codereview.appspot.com/5449065/) [12:28] rog, mpl, TheMue: I think we should give a push to make sure all of the packages we're handling run fine on the latest weekly [12:28] rog: Already on re-review the former one [12:28] rog: Already on re-reviewing the former one [12:29] niemeyer: i was wondering if we should let the latest weekly settle a bit before digging in [12:30] What are you guys going to write in Go by the way? [12:30] mainerror: We're running an experiment to run juju on Go [12:30] Oh, nice. :) [12:31] Right now it is using Python, right? [12:31] mainerror: That's right [12:31] niemeyer: Based on weekly? Not on r60? [12:31] TheMue: Yeah, weekly [12:31] niemeyer: just curious? why not using the releases instead of the weeklies? [12:31] TheMue: r60 is too old by now, and given our timeframe, there's not much benefit in sticking to it [12:31] mpl: ^ [12:31] ok [12:32] niemeyer: hmm, imho weekly is currently changing too often. but i'll look. [12:32] I suspect most of the incompatibilities for Go 1 are already in too [12:33] TheMue: i think it's easier to update in stages rather than all at once. we haven't got a large set of users, so it's not that much problem for us currently. [12:33] Oh wow! Go has GTK bindings. [12:34] rog: yep, the backlog of the current weekly and the r60 is already quite large. [12:34] *sigh* [12:34] mainerror: are you speaking of mattn ones? if so, you should know that nsf is also redoing them. [12:34] Indeed. They are not 100% done yet but still a good start. [12:34] TheMue: gofix does a lot, but it's not a panacea [12:35] I see. [12:37] mpl: Did he host it on Github as well? I can't find it. :/ [12:38] mainerror: well, you'd better ask him directly on #go-nuts [12:39] Oh, awesome. :) [12:40] rog: i know, that's what i wrote on G+ too. i would like a r61 before go 1. [12:40] rog: What's the branch name? [12:41] niemeyer: which branch? [12:41] rog: The one you mentioned [12:41] niemeyer: i pointed to the codereview page. [12:41] niemeyer: it's lp:~rogpeppe/juju/go-juju-ec2-regions [12:41] rog: Ok, I feel like I'm missing something [12:41] i think [12:41] the first one is go-juju-initial-ec2 [12:41] rog: It says " juju/ec2: add code to accept an ec2 region in the configuration file" [12:42] rog: But there's a _lot_ in there, completely unrelated to this description [12:42] rog: Is there a pre-req missing? [12:42] niemeyer: yes - i couldn't add it after i'd forgotten to add it first [12:42] niemeyer: i thought you knew about the first one [12:42] rog: The first one was initial-foo, right? [12:43] rog: which we debated a lot on [12:43] rog: I haven't heard on it since? [12:43] niemeyer: https://codereview.appspot.com/5432056/ [12:43] niemeyer: i pushed a new version which is skeleton only [12:43] niemeyer: didn't you get the email? [12:43] rog: Ok, the last comment says "I'm doing that right now." [12:43] rog: Check the review out [12:44] niemeyer: ah, i did another lbox propose and changed the description, and thought you'd see that [12:44] niemeyer: anyway, there it is. [12:44] rog: Understood, thanks. I'll hack a bit on lbox today I think, to implement submit, and also to fix that problem [12:44] rog: It should mail asking for another look [12:45] niemeyer: i think there should be a distinction between uploading, mailing and changing the description [12:45] rog: But let me review your branches first.. [12:45] niemeyer: that would be good, thanks [12:45] mpl: How's your agenda? Do you have something in mind already to work on? [12:46] niemeyer: nope. I haven't really had time to think about it. I'm very open to suggestions for now ;) [12:47] mpl: May I suggest you go over all of the packages for juju, goamz, gozk, and goyaml, and see if there are any incompatibilities with weekly to be fixed? [12:47] mpl: goamz has a bunch of sub-packages as well [12:47] rog: Have you merged the error fixes on goamz that you had pending, or is there some to go still? [12:48] niemeyer: i *think* so. i'll just check [12:49] niemeyer: alright, will do. can't start on it right away though. [12:49] mpl: That's fine, thanks [12:49] mpl: After that, let's see if we come up with a more interesting/long task involving juju logic [12:50] mpl: These branches that rog is working on are the base, so it's a bit hard to split the workload, but with those in, it should be much easier [12:50] niemeyer: I'd like that. if it can involve me learning about cloud stuff, even better. [12:50] mpl: Absolutely [12:50] mpl: EC2 provider is precisely what we're working on [12:51] niemeyer: no worries. I surely don't want to disturb your workflow guys. [12:51] mpl: You're helping, rather than disturbing [12:52] psh, I know you guys have spent more time hand holding me so far than if you have done it yourself. but I'm too ashamed as it's often a necessary process. [12:52] *not too ashamed. [12:53] niemeyer: have you renamed goamz/aws yet? [12:53] anyways, gotta run to a meeting, ttyl. [12:53] niemeyer: that's the last of error-fixes in goamz [12:53] mpl: see ysa [12:53] ya [12:53] rog: Not sure.. checking [12:53] niemeyer: it still needs pushing [12:53] mpl: Cheers! [12:54] rog: Dude.. we're doing something wrong [12:54] niemeyer: we should probably tag r60 versions of the goamz packages so that people can still use r60 with them [12:54] Crap [12:54] ? [12:54] rog: Yeah, precisely [12:55] niemeyer: that's not too hard, right? [12:55] rog: That's exactly what I was talking about [12:55] rog: It's trivial [12:55] rog: But we may have broken people trying to use it [12:55] * niemeyer handles that right away [12:55] rog: Have you pushed the branch? [12:55] niemeyer: which branch? [12:55] rog: goamz/aws [12:55] niemeyer: i've pushed ec2 and s3 [12:55] niemeyer: nope [12:56] niemeyer: i don't think i sent it for review, in fact [12:56] rog: Ok, do you have any pending changes? [12:56] niemeyer: yes, just error fixes [12:56] rog: Ok, leave that with me then [12:56] rog: I'll fix the whole thing [12:57] niemeyer: there are two ErrorMatches that gofix doesn't catch, otherwise it's all gofix [12:57] rog: Cool.. I'll fix that and fix the tagging [13:00] anyone have experiences with go weekly and release in parallel? [13:00] i would need both [13:00] TheMue: i have them in separate directories [13:00] TheMue: and a prefix command to run one in preference [13:01] TheMue: http://paste.ubuntu.com/757007/ [13:01] (excuse the fact it's not bash, but i'm sure you get the idea) [13:02] TheMue: i call it "go-release". as in: go-release goinstall foo; go-release 6g x.go [13:02] etc [13:02] ic, looks good [13:06] niemeyer: gocheck looks like it needs an r60 tag too - it looks like the r59 version fails [13:06] on r60 [13:07] rog: Uh oh [13:07] rog: Can you pin-point which version to tag? [13:07] niemeyer: will do [13:07] rog: Thank you [13:11] niemeyer: ironically, it's revision 60 [13:11] rog: LOL [13:32] just off for lunch and to bake a cake, back in an hour [13:34] rog: Enjoy [13:35] ... and dcc us some cake [13:35] :) [13:43] rog: I _think_ it's all sorted out [13:43] rog: In goamz, specifically [14:17] niemeyer: cool. [14:18] TheMue: and sorry, would've but didn't have enough bananas so the cake is only 62% the size it should've been. so none spare! [14:20] rog: *deeplySigh* so I've got to find surrogate when I'm on x-mas market later with my family [14:25] jimbaker: you know what would be really cool - if you could use scp to copy between units without involving the local machine, e.g. juju scp wordpress/0:/etc/wordpress/config-* wordpress/1:/etc/wordpress [14:27] Charm school soon \o/ [14:34] * uksysadmin is ready for charm school. "How yoooou doin'?" [14:36] niemeyer: before i continue in the same vein, do you think go-juju-initial-ec2 and go-juju-ec2-region are pointing vaguely in the right direction? if not, i'd prefer to fix those rather than carry further on the wrong way. [14:38] good morning everyone! [14:38] uksysadmin: thanks for stopping by, we'll start in ~20 minutes! [14:40] niemeyer: oh cool, just got yr msg [14:49] looking forward to charming my servers [14:52] * mainerror will probably be 10 minutes late [14:53] rog: Yeah, I think initial-ec2 is ready to go in with those settled [14:53] niemeyer: [14:53] niemeyer: cool. i've done those - i'll do the push [14:54] m_3_: around? [14:56] o/ [14:56] niemeyer: done. [14:56] rog: Looking [14:57] rog: patch set is unchanged [14:57] niemeyer: is it lp:gozk or lp:~juju/gozk/zk that you want me to check? also, fyi https://launchpad.net/gozk/zk returns a 404 [14:57] rog: But it was pushed 5 minutes ago [14:58] mpl: gozk/zookeeper.. you've renamed it :-) [14:58] niemeyer: sorry, did the publish messages, did the push to ~rogpeppe..., forgot to do the lbox propose [14:58] rog: lbox propose pushes as well, btw [14:59] niemeyer: yeah but maybe you wanted me to check the whole project, wasn't sure. [14:59] niemeyer: yeah, but i've had it push to the wrong place, so i tend to do it myself to make sure [14:59] jcastro: yo [14:59] hi! [14:59] rog: Cool [15:00] mpl: Ah, no, just gozk/zookeeper is great already [15:00] niemeyer: dammit, i'm too late, the diff is empty [15:01] \m/ [15:01] niemeyer: rog: mind if we kick you out for the charm school? [15:01] jcastro: no probs [15:01] jcastro: Not at all, thanks for pushing this [15:01] but do idle in case there are questions! [15:01] Ok, welcome everyone to the first ever #juju charm school [15:01] we'll give it another minute for the stragglers to arrive [15:01] rog, mpl: => #juju-tmp [15:01] niemeyer: ah yes, silly me, your message pointing to that url (with zk) is from september. I had the rename the other way around in my head, sorry. [15:02] TheMue: => #juju-tmp [15:05] Ok everyone. [15:05] Welcome to the first ever virtual charm school!!! [15:05] I am your emcee, Jorge Castro [15:05] I work on the community team at Canonical on Cloud [15:05] I am joined today by .... [15:05] Hi all, I'm Mark Mims [15:06] I'm here to help answer charm questions [15:06] ok [15:06] so first off, we'd like to see who's here to learn, so everyone who is here for charm school, raise your hand and give us a one line description of yourself [15:06] o/ [15:06] \o [15:07] o/ I love juju [15:07] o/ [15:07] o/ (just a pm having a look) [15:07] o/ === sergio_ is now known as Guest31766 [15:07] o/ [15:08] o/ interested in learning how to deploy cloud environments in (as well as bare metal provisioning of) OpenStack [15:08] oh nice one. :) [15:08] ok so first, before we get into details on charms [15:09] let me arm you with some docs and links, that you can reference [15:09] first off, obviously, we have: https://juju.ubuntu.com/ and https://juju.ubuntu.com/docs [15:09] which is where we centralize all the information on juju [15:09] for news about juju and the latest charms, we put that on http://cloud.ubuntu.com [15:10] for example yesterday someone submitted a status.net charm [15:10] http://cloud.ubuntu.com/2011/12/deploying-status-net-quickly-with-juju/ [15:10] We like to heavily talk about new charms [15:10] because while juju is an amazing tool, it's the charms that make it useful [15:10] in the same way that you might love apt, but you really love the huge repository that comes along with it [15:11] so we're running these sessions to help people write charms for things they care about. [15:11] So, before we get into the anatomy of a charm [15:11] does anyone have any questions so far? [15:11] hopefully you understand enough about juju to grasp the concepts. [15:11] but if you don't, we can get you started on the right road. [15:11] how does one charm a package to use mysql when the current package has depends for a locally installed mysql? [15:12] erm, to use the mysql on another box.... [15:12] Do you consider juju ready for production environments? [15:12] medberry: mark is answering yours now [15:12] fmo: we consider it a tech preview in 11.10 [15:13] medberry: so the key here is to write a relation hook for your service [15:13] Thank you, it's a bit out of scope but is it the same for Orchestra? [15:13] fmo: it's at the stage where you would start to look at it for 12.04ish timeframe deployment. [15:13] medberry: that relation hook would essentially reconfigure your app to point to the remote mysql instance [15:14] nod. m_3, thanks. Is there an example charm (off the top of your head) I should refer to that does this? [15:14] medberry: it does this by exchanging crediential info for a db with the mysql charm [15:14] fmo: Not quite sure on orchestra but I /believe/ it's at the same level, though orchestra bundles some things that are already mature, like squid, etc. [15:14] m_3_: but you would still end up with unused mysql-server running on 'app' node, wouldn't it? i think this is a packaging bug [15:14] thank you :) [15:14] lots of charms connect to mysql this way... mediawiki would be a good one [15:15] fmo, I believe you can consider Orchestra more mature than Juju and ready for deployments. zul may know more. [15:15] m_3_, thanks, I'll look at it (and others) [15:15] ivoks: your relation hook could pass debconf info and dpkg-reconfig the package though [15:15] m_3_: right, thanks [15:15] ivoks: so what you would do depends on the interface exposed by the package itself [15:16] * medberry was thinking specifically of moodle if someone wants the context. [15:16] oh are you working on moodle? [15:16] that would be great! [15:16] * m_3_ grins [15:17] ok, this brings me to the next quick topic, who has written a charm already? and who is planning on working on one? [15:17] * nijaba plans on doing one for limesurvey [15:17] and does anyone have an itch to scratch to work on a certain charm? [15:17] o/ as far as getting a basic something going (like nginx behind haproxy) to try and understand the concepts [15:18] o/ postfix-dovecot [15:18] (as soon as i find time :) [15:18] (same here) [15:18] o/ finishing phpmyadmin and steam [15:19] (steam in this context is being able to quickly deploy a steam server for game clients to connect) [15:19] jcastro, yep, and someone has pointed out in O and P moodle only has a recommends on the server bits (not a depends like in Lucid.) [15:20] o/ I'm working on an alice IRC charm, which is a self hosted web irc client. [15:20] ok, so let's start with looking at the anatomy of a charm [15:20] marcoceppi, I saw that Aleph One was just released fully open source... [15:21] * marcoceppi nods [15:21] but before we start a charm [15:21] you need to map out what you want your charm to do [15:22] https://juju.ubuntu.com/docs/write-charm.html#have-a-plan [15:22] in this document we outline some tips in the things you need to think about when planning out how to write your first charm. [15:22] usually I make a quick mindmap of what I need [15:22] so for example [15:22] status.net [15:23] I know I'll need mysql [15:23] o/ - I'm a juju fanboy. [15:23] * mainerror is also late [15:23] and that the charm would have a relationship with mysql [15:23] /join #juju-tmp [15:23] http://charms.kapilt.com/interfaces/mysql [15:23] http://pad.ubuntu.com/charmschool-2011-12-02 [15:23] I can then see what other programs use mysql [15:24] this is useful because now I can look at other LAMP projects and use that to build my charm [15:24] which is what we want, reuse. [15:24] So you can use the charm browser to look for other services that have relationships with services you care about [15:24] The guy who wrote the status.net charm just derived it from his last LAMP charm [15:25] part time, it was only 2 days from first cut to being accepted into the charm school [15:25] and Marco/Clint are working on some convenience tools to make common tasks easier [15:25] marcoceppi: can you explain about your convenience functions? [15:25] (and then we'll go into this: http://pad.ubuntu.com/charmschool-2011-12-02) [15:26] wrong URL, I mean this: http://pad.ubuntu.com/charmschool-2011-12-02 [15:27] ok we can get back to Marco [15:27] Sure, so I wrote a few functions that I found myself using a lot in bash, primarily ch_get_file which is passed a file url and either a url to it's hash (md5, sha1, etc) or the actual hash itself. From there it'll download/compare file -> hash [15:27] There are a few others, check if a string is a file or a url, check if a string is a valid hash [15:28] You can find them: http://bazaar.launchpad.net/~charmers/charm-tools/trunk/view/head:/helpers/sh/net.sh to use in a charm make sure you have the juju ppa added, install charm-helper-sh, and source /usr/share/charm-helper/sh/net.sh [15:29] marcoceppi: FYI, there is a PPA just for those helpers, ppa:charmers/charm-helpers [15:29] I believe the idea is to grow out charm helpers to include helper functions in all the languages [15:29] <3 SpamapS [15:29] and if I make something I think can be reused, do we just submit it to charm tools? [15:29] marcoceppi: otherwise you'll pull in a new version of juju [15:30] SpamapS: yay! [15:30] they'll be in the distro for 12.04 [15:30] Since we're just getting a bunch of charms started, if you're writing a charm and feel that you can improve the charm tools to make it better for the next person, then we certainly encourage you to do that. [15:30] definitely [15:31] ok, any questions on the tools before m_3_ gets into the meat of a charm? [15:33] ok, moving on .... [15:33] are the tools webscale? [15:33] ;) [15:33] everyone should have this URL open: http://pad.ubuntu.com/charmschool-2011-12-02 [15:33] and we'll review this charm [15:34] Mark will bold the sections of the document he's talking about [15:34] and then when he moves on he'll unbold it and then bold the section he'll be reviewing next. [15:34] I'll give people a sec to join the pad [15:34] pad needs syntax highlighting :P [15:35] ok, so we'll make sure everyone has juju installed and working in a bit... first of all, let's cover some basic concepts of charms with an example [15:36] I pulled a community-contributed charm off the list... just landed yesterday or the day before [15:37] so the first section we have is a basic listing of the charm contents [15:37] notice there're a couple of yaml files, a copyright, and then a set of scripts in the hooks/ directory [15:38] the scripts in the hooks/ directory are all in shell for this example [15:38] but juju takes great pains to be language/tool agnostic [15:39] you can use your fav toolset.. hooks just have to run and exit with return codes similar to a shell script ( 0 is good ) [15:39] so before we dig into the contents of the hooks [15:39] let's see what this charm is / does [15:39] take a look at the metadata [15:40] * m_3_ not a fast etherpad formatter :) [15:40] so it looks like the service described by this charm... [15:40] consumes a db (mysql in this case) [15:40] and presents a website [15:41] ok, that's pretty simple... and has a _lot_ in common with other services [15:41] so juju uses this metadata specification to stitch together different services [15:42] when juju spins a service unit up for this charm [15:42] it can 'relate' other services that match up [15:42] (does this by your command of course... but we'll get to the juju cli in a bit) [15:43] Any insight on where adding Rackspace support lies in the priority list for the juju project? [15:43] We're based in the UK and so UK IPs are a big must at the moment, which means AWS is out as they only have an Irish presence, nothing in the UK itself. [15:44] Otherwise, I'd be very interested in experimenting with juju, it sounds like an awesome project. [15:44] chmac: juju can handle a number of different providers [15:44] including openstack [15:44] so when juju spins up a service unit for this charm [15:44] the first hook it calls is 'install' naturally enough [15:45] please take a sec to look at the install hook in the pa [15:45] pad [15:45] starts off installing ppas and packages [15:46] it's worth noting here that juju charms can be used to install services that install from packages [15:46] but it can also be used to install other ways as well [15:46] you can clone directly from the tip of your source code repo if you'd like [15:46] anything goes [15:47] for charms contributed back to the community please prefer packages when available [15:47] but that's not necessary [15:47] so this particular charm adds ppas/and packages [15:47] then 'ch_get's a file from status.net [15:48] (this is one of marco's tools to download and cryptographically verify payloads in charms) [15:48] anyway... rest of the script is pretty clear [15:48] note the open-port at the bottom [15:49] juju defaults to everything closed off... you have to explicitly open what you want via the 'juju expose' command [15:49] so that's install [15:49] there're start/stop hooks too... these're usually trivial 'service xxx start' [15:50] so if you fail the open-port, there are iptables that prevent it? (firewall rules?) [15:50] correct [15:50] (not exactly iptables always) [15:50] depends on the provider [15:50] ah, aws / openstack port rules, gotcha. [15:50] (and still in development iirc on bare-metal provider) [15:51] so some charms are for services that just install a service and spin it up [15:51] some services don't relate to other services [15:51] and we'd be done if that were the case for status.net [15:51] here, we need a db [15:51] and we could potentially add a reverse-proxy in front of it [15:52] etc etc [15:52] so before we move on to relations... [15:52] any questions so far? [15:52] relations are the real meat of charms, so we'll pause here for questions [15:54] ok, on to relations!! [15:54] now this is the really cool part of juju... ok _a_ really cool part of juju [15:54] so when you look at the config for a service [15:55] you can often bust it up into "the config specific to installing that node" and "the config specific to relating to another service" [15:55] often the relation config info is a nice concise set of info [15:55] well juju uses these boundaries to 'install' as a separate step from 'relate' [15:56] so the basic story here is to [15:56] 1.) spin up status.net [15:56] 2.) spin up mysql [15:56] 3.) relate the two [15:56] 4.) expose status.net to the outside world [15:57] when we relate two services the charm hooks for those services exchange info [15:57] take a look at what's bold in the pad [15:57] that's an example of the status.net charm getting relation-specific config info from the mysql service [15:58] the mysql (db-relation-{joined,changed} hooks create a new db and credentials and then pass it to the service joining) [15:58] so the status.net charm 'relation-get's the mysql db info that it needs [15:58] now we can write this info into the config for status.net... and bam! [15:59] take a minute to browse the db-relation-changed hook [16:00] BTW all hooks are optional... only implement what you need... events will fire for {joined,changed,departed,broken} [16:00] does this 'relation-get' exchange of information make sense to everybody? [16:00] any questions about this [16:01] it's this very separation of config info into 'install' and 'relation'-specific config that allows juju to start handling horizontal scaling quite nicely [16:01] butmore on that later :) [16:02] ok, take a sec to look at the website-relation-changed hook too [16:02] m_3_, compare "relation-get" with "config-get" which occurs later. [16:02] config-get comes from config.yaml? [16:02] it's perdy darned simple [16:03] was just gonna do config-get next [16:03] okeydokey [16:03] so relation-get gives you info from the other side of a relation... another service [16:03] config-get gives you info from a couple of different places: [16:03] defaults are in config.yaml [16:04] I can set them at deploy-time on the command line [16:04] ( juju deploy -elocal --repository ~/charms --config ~/my_status_net.yaml local:statusnet status [16:05] or I can also set them individually via the command line: [16:05] at any time during the life of the service, I can 'juju set statusnet var1=val1 var2=val2' [16:06] there's a config-changed hook that can handle these changes in addition to the install or relation config [16:07] ok, so one more thing about relations and relation-hooks and then we can move on [16:08] when a relation is joined, relation-changed hooks can fire repeatedly [16:08] they do this as long as either side is calling relation-sets [16:08] what this means is that your code has to be able to be called over and over again without barfing your config === jrgifford_webcha is now known as jrgiffordwebchat [16:09] (idempotency for you math geeks out there.... i^2=i) [16:09] it's not hard, but it's good practice to make all of your hooks idempotent [16:10] alright, I could ramble on all day about scaling service and show you example stacks... but let's get on with charm school! [16:10] any questions about the status.net charm before we move on? [16:12] no questions so far? [16:12] there's no way you guys can be that good. :) [16:13] * m_3_ is just that exciting to watch type :) [16:13] jcastro: lol [16:13] What do I do if there is no install_cli.php script for configure the site? [16:13] mainerror: wow, that's a tough one [16:13] m_3_: I thought I'd read somewhere that juju only supports EC2 compatible clouds at the moment. [16:14] m_3_: Does juju support rackspace right now? [16:14] * jrgiffordwebchat has same question as chmac [16:14] m_3_: Cant juju be installed on any server? [16:14] mainerror: I'd write the install scripts myself and have the charm's install hook call my install scripts [16:14] mainerror I had a problem like that, in where I just had to re-write the web-based installer into the charm [16:15] chmac: yes, during the last ODS we even did the meta thing on openstack... [16:15] I see. Thanks. :) [16:15] i.e., we had a bare-metal rack [16:15] chmac: we used juju to deploy openstack itself on the bare metal rack [16:15] m_3_: Wow, awesome, so juju can deploy new Rackspace servers just as it deploys new EC2 servers? [16:15] openstack is not per se the same thing as rackspace. [16:15] m_3_: Great, I'll get into the documentation in greater detail. [16:16] chmac: then we backed out and used another juju environment to spin up a hadoop cluster on that openstack cloud [16:16] chmac: note that juju doesn't support the rackspace-api... just the ec2-compatible-api on openstack [16:17] medberry: thanks [16:18] I've read that juju's not production ready, is that really true, or is it more cautionary? :-) [16:18] chmac: sorry, I don't know if rackspace has a public-facing openstack cloud yet [16:18] chmac: definitely true [16:18] chmac: there are components that need to be made highly available before it's totally production ready [16:18] chmac: it's a tech preview in 11.10 [16:19] m_3_: Really? It's so tempting to use it in production :-) [16:19] so.. it was a tech preview in 11.04/11.10, for 12.04 it'll be production? [16:19] chmac: we're shooting for production-ready in 12.04 (long-term-support) version of ubuntu [16:19] chmac: I would use it in production [16:19] chmac: but certainly we encourage people to run it in dev so that we can get feedback to make it better in 12.04 for an LTS. [16:19] Makes sense [16:19] chmac: this is why we want people banging on it now [16:19] but you've got to take some extra precautions atm to insure service/data integrity [16:20] m_3_: Now that's what I'm talking about, see, I'm usually happy to test earlier than most :-) [16:20] it does spin up EBS-rooted instances in ec2 atm for instance just to be on the safer side [16:20] I'll investigate the openstack api issue with Rackspace, maybe there's an alternative provider in the UK. [16:20] Thanks for the snappy response, gotta run now, being taken shopping! :-) [16:20] * m_3_ totally ignorant [16:20] thanks for coming! [16:20] fun fun! [16:21] any other questions? [16:21] amithkk: I believe you asked if juju can be installed on any server? [16:21] amithkk: sorry missed your question [16:22] did the above discussion asnwer that? [16:24] What is the best practice to create an install script for a service if there is no direct download option? [16:24] mainerror: best is to use apt-get install [16:24] mainerror: next is to download a payload... binary or tarball [16:25] .. then git clone? [16:25] then install accordingly... cli.sh or ./configure && make && make install [16:25] git clone's totally cool... we have several charms that do it already [16:25] (safer to 'git clone -b stable_branch_name') [16:26] m_3_: yep [16:26] What I meant is, what if there is no wget usable direct URL but only a redirect URL? [16:26] mainerror: so in the latter case, you'd need the install hook to install 'build-essential' [16:26] sorry for the slow responce [16:26] or whatever's needed for that build [16:27] mainerror: hmmm... not sure I'm following, but perhaps a ppa for that charm would work? [16:27] I think I know the answer... but clarifying: [16:27] jcastro, m_3_, if I want to populate some content, does that happen after juju deploy? Assume I'm using public charms--should I mod them or just do the population on the host after the fact? [16:27] Ah, right. That would be an option. [16:27] (ie, load web content, etc.) [16:28] I think mainerror means [16:28] medberry: sure, you can pull/install during a relation hook even [16:28] what if I can't wget a tarball directly? [16:28] and it's buried in some website that has redirects, etc. [16:29] Indeed but m_3 actually answered it, I think. [16:29] so you _could_ just stick a tarball in the charm itself... there're some size limitations to that, but I don't remember them off-hand [16:29] As I understood his answer was to create a package for it in a PPA. Maybe I misunderstood. [16:30] m_3_, but that's no longer really a public charm (if it has my content in it). So I've locally forked the charm? [16:30] mainerror: heck, let's go over the specific thing you're trying to do [16:30] medberry: yes, exactly [16:30] medberry: which is ok... anything goes in charms [16:30] thanks all. [16:30] medberry: your choice to make something harder to maintain, and/or private/proprietary [16:30] Uhm, it was more of a general question. Luckily I can wget it directly. [16:30] mainerror, is there an example--some actual issue you can point us at? [16:31] ah. [16:31] medberry: but contrib back to community set of charms in the 'charm store' is different [16:31] nod. [16:31] they get tagged according to all sorts of different criteria [16:32] I wouldn't consider a tarball in a charm best practice [16:32] ideally if it was an open project you could get them to fix that [16:32] prefer packages if available [16:32] or ask them to mirror their tarball releases on launchpad [16:32] or package it in a PPA as a contribution to ubuntu and that project [16:33] marcoceppi has a 'use_upstream' config option on a charm [16:33] yeah, it switches between latest stable tarbal and what's in the repos. [16:33] true => pull from source, otherwise install the package [16:33] but it always defaults to the repo, you have to juju set inorder to get the upstream version [16:34] which is really a beautiful way to handle that from a user-perspective (as long as default is pkgs :) [16:34] but if it's some horrible program that you can't improve and need it for work or something then feel free to get as ugly as you want, but for the public charms in the charm store we're looking for nice clean charms that people can use and adapt. [16:34] ok, we've gone through the charm [16:34] and this ends the official charm school [16:34] however [16:34] we certainly encourage you to hang out [16:34] smoke if you got em [16:35] and feel free to ask us questions [16:35] or let us know what you're working on [16:35] or ask for help or a review [16:35] we hold office hours (see topic) or you can always post to the list [16:35] but the juju team works a ton, and usually are always in here and love to answer questions from users [16:35] so please don't be shy! [16:36] tx guys. [16:36] awesome session! [16:36] Thanks for this session! [16:36] we'll make the logs handy for everyone [16:37] thanks guys - is service maintenance within the scope of juju? e.g. keeping mysql up to date w/ security patches? [16:37] That's a good question actually [16:38] dannf: yes, there's an upgrade action called from the command line 'juju upgrade ' [16:38] and that is implemented with a hook, i presume? [16:38] that you can customize with a dedicated upgrade hook [16:39] sweet [16:39] usual practice is to make 'install' idempotent and then symlink to the upgrade hool [16:39] s/hool/hook/ [16:39] makes sense [16:40] dannf: rolling upgrades can be done, but you have to segment the service a bit... i.e., plan for it... this'll improve over time [16:41] i.e., if you've got a 20-unit service and only wanna upgrade a few of them at a time [16:41] scaling in juju is just cool! but that'll be another charm school :) [16:42] yes, it's actually unfair to juju not mentioning scaling this time [16:42] but when we get to it you'll really start to see the power [16:42] it's a kind of magic [16:42] sorry, had to [16:42] :) [16:43] * m_3_ now has song stuck in head [16:43] * mainerror too [16:48] hazmat niemeyer: y'all can have the channel back... thanks! [16:48] wheee [16:49] feels good to come out of the dark irc corner [16:58] Oh, hey.. #juju is back [16:58] rog: Review sent [17:00] niemeyer: w.r.t. Bootstrap, I said that I would change the test (to test for len(ms)==1) once Bootstrap actually starts a machine, but for the time being i think it's ok. [17:02] rog: It's not.. [17:03] rog: Comment the test out if you can't satisfy it.. a test assertion that asserts something wrong is doing what exactly [17:04] rog: Or, maybe better [17:04] rog: Comment Bootstrap out [17:04] that's a better idea [17:04] rog: Then the test is probably correct [17:04] rog: It should not Destroy, though [17:05] niemeyer: why not? [17:05] rog: Because that's the complement of Bootstrap [17:05] rog: Destroy doesn't work in a real deployment without Bootstrap, I think [17:05] fwereade_: Does it? [17:05] niemeyer: currently, Destroy takes down all machines that were started from the current environment [17:06] niemeyer: which seems ok [17:06] niemeyer, reading context [17:06] rog: Hmm.. fair enough.. that sounds like a good behavior, even if we don't do that in Python [17:06] niemeyer, I don't think destroy-environment is actually an *error* if you don't have an environment [17:07] fwereade_: The question is whether it kills all machines, even if we don't find the data about it in e.g. S3 [17:08] niemeyer, checking [17:09] niemeyer, it gets ec2 machines by reservation, so it should be safe [17:09] fwereade_: Cool, thanks [17:19] niemeyer: i replied with one question about about open vs Open [17:19] rog: and I already replied to it [17:21] niemeyer: i think i'd prefer to wait until the occasion arises, if that's ok [17:21] rog: Sure, no prob [17:22] niemeyer: the previous weekly builds find with all tests passing btw, so it's *something* that's changed recently [17:22] rog: You mean re. http? [17:22] niemeyer: yeah [17:27] niemeyer: https://codereview.appspot.com/5448085/ [17:28] rog: goamz should be entirely fixed, tagged, and shiny already [17:29] niemeyer: i just pulled and it didn't seem to be [17:29] unless... [17:29] ah, rename! [17:30] no [17:30] niemeyer: http://paste.ubuntu.com/757292/ [17:31] it still needs fixing, i think [17:31] (that was done just after i'd removed the aws directory, BTW) [17:32] rog: I think I screwed up pushing to ~niemeyer.. let me see [17:32] niemeyer: renames are awkward - it should lazily evaluate aliases i think [17:32] rog: yeah, I'm sorry [17:33] rog: The rename worked fine, but I forgot the --remember [17:33] rog: Which means the fixup and tagging afterwards went to the wrong location [17:34] rog: Ok, I _think_ everything should be fine now, including the more unknown packages (sdb, etc) [17:34] niemeyer: if i push to lp:goamz/aws, it should remember that, even if it's later changed to be an alias for something else, i reckon [17:34] rog: It doesn't [17:34] i know [17:34] rog: Unless you say so with --remember [17:34] rog: Ah, you're expressing a wish, sorry [17:34] yeah [17:34] rog: Yeah, it makes sense to me too [17:45] niemeyer: PTAL [17:46] rog: e.Destroy has already been mentioned in two other reviews.. please fix it or respond to the point. [17:47] [17:06] rog: Hmm.. fair enough.. that sounds like a good behavior, even if we don't do that in Python [17:47] rog: That's not the same issue [17:47] rog: We should be able to Open without Destroy [17:48] rog: Or do you think we shouldn't? [17:48] niemeyer: i think that the destroys will happen at the end [17:49] niemeyer: so it shouldn't matter [17:49] niemeyer: what do you propose as a solution? [17:49] rog: So if I open the same environment 10 times, you think we should destroy it 10 times [17:49] niemeyer: yeah. [17:49] rog: That's silly isn't it? [17:49] niemeyer: probably :-) [17:49] rog: So let's do something else.. [17:50] niemeyer: your suggestion was to have suite.Bootstrap, right? [17:50] rog: I suppose each environment only has to be destroyed once, and that we don't really have to do through the same object that created it [17:50] niemeyer: so the environment that got bootstrapped gets destroyed [17:51] rog: Does that make sense? [17:51] happy weekends everybody :) [17:51] fwereade_: thanks, have a great one too [17:51] fwereade_: have a good one [17:51] rog: I don't mind to change the original suggestion.. just want to have some conversation going about the topic [17:52] rog: You say Destroy shouldn't depend on Bootstrap.. that's fine by me too [17:52] niemeyer: i think that if we're wanting to test Bootstrap/Destroy with different opens, we can bypass the suite environment handling [17:53] rog: I don't want to test.. it's the opposite.. I don't want the suite to be doing something that makes no sense all the time [17:53] most tests will be against a single environment, i think [17:55] niemeyer: isn't it worth doing something that makes sense most almost all of the time? hmm, i dunno. [17:55] rog: Sorry, I don't get that point.. [17:56] rog: The same environment shouldn't be destroyed twice.. it will fail, and the test will fail because Destroy fails [17:56] anyway, my thought was that we could have a Bootstrap method as well as an Open method. the Bootstrap method marks the env to be destroyed at the end. [17:56] rog: That was my suggestion, that you advocated against by saying Destroy should always be called :-) [17:56] which will happen exactly once for each env created via bootstrap [17:56] rog: Which is fine by me [17:57] niemeyer: that was then :-) [17:57] rog: The only point I'm trying to address is not destroying thing twice [17:57] rog: and it's easy to do that [17:57] niemeyer: yeah, ok i'll do that [17:57] see whether i can do it in 3 minutes! [17:57] rog: Well.. let's do this [17:57] rog: Please merge it as-is.. [17:58] rog: We'll sort it out later [17:58] ok [17:58] it's ok as is? [17:58] rog: yeah [17:59] rog: I'm starting to think we should just Destroy it within each test as necessary and be done with it. Let's see how things show up in practice once we have a few additional tests [17:59] niemeyer: the problem with that is that the env is left around if you do a failed Assert [17:59] rog: defer [17:59] ok [18:00] But let's see.. it's bothering a bit knowing this will blow up but I'm probably concerned too soon [18:01] bugger, conflicts [18:02] Huh.. how come? [18:08] niemeyer: all sorted now. jeeze. i *think* i got all the steps right. [18:09] gotta go, i am required to pack for the weekend. see ya monday. thanks for all the reviews, i love 'em really :-) [18:10] rog: Thanks a lot for everything this week, it was awesome [18:10] rog: and you're very welcome [18:10] niemeyer: and if you did manage to have a look at ec2-region before then i'd be very happy indeed... [18:10] rog: Oh, wait [18:11] rog: I've checked it out, but the proposal includes unrelated changes from the previous branch [18:11] rog: I mentioned that in the codereview itself [18:11] fuck [18:11] which changes? [18:13] niemeyer: i can fix it if you let me know right now, otherwise i'll have to leave it [18:13] rog: See the codereview [18:14] rog: I think it had all the ec2-initial changes in [18:14] niemeyer: which file? [18:14] rog: All of them [18:14] rog: Or most, anyway.. checking out [18:14] niemeyer: i thought ec2-initial was approved, bar these changes [18:14] rog: It is! [18:14] rog: and it's merged even, right? [18:15] yeah [18:15] rog: My point is that this: rog: https://codereview.appspot.com/5449065/ [18:15] rog: Has a ton of unrelated changes coming from other branches [18:15] ah, sorry, i thought you were referring to the review just done [18:15] one mo [18:17] niemeyer: there, is that better? [18:18] niemeyer: it looks like the changes are relevant now [18:18] gotta go [18:19] rog: Superb, thanks! [18:19] rog: Enjoy [23:01] <_mup_> Bug #899433 was filed: YAML errors in charms should be obvious to users < https://launchpad.net/bugs/899433 >