| SpamapS | wow.. sitting here thinking I might be able to hack out some kind of LXC provider. Everything is twisted around twisted. :-P | 00:51 |
|---|---|---|
| m_3 | SpamapS: you can test out an LXC ensemble entirely within a KVM vm.. <grin> | 00:59 |
| m_3 | SpamapS: ssounds fun man | 01:00 |
| SpamapS | m_3: just trying to wrap my head around how to make "deferred" objects for commands that will run locally is kind of weird. I wish it was more straightforward. :p | 01:09 |
| * SpamapS prefers threads and locking to event based and non-blocking. :p | 01:09 | |
| hazmat | SpamapS, coroutines ftw | 01:11 |
| SpamapS | hazmat: indeed :) | 01:11 |
| hazmat | event based isn't the problem per se.. callback oriented programming is not fun | 01:12 |
| SpamapS | It can be.. gearman makes it pretty simple.. but thats because you say "when told to do X, run function Y".. same with libevent .. | 01:12 |
| hazmat | SpamapS, what are you looking into? | 01:12 |
| SpamapS | twisted seems to take it 2 or 3 steps further. :-P | 01:12 |
| hazmat | SpamapS, yeah... ruby 1.9 fibers with libevent are nice | 01:12 |
| SpamapS | hazmat: I'm tired of whining about having LXC .. i want to JFDI | 01:12 |
| hazmat | SpamapS, awesome! ;-) how's it going so far? | 01:13 |
| SpamapS | hazmat: mostly just renaming stuff.. the spots that reference txaws .. I need an equivilent for lxc basically | 01:13 |
| m_3 | SpamapS: libvirt in-between or not? | 01:13 |
| hazmat | SpamapS, yeah.. the dummy provider is probably your best starting point | 01:14 |
| SpamapS | I'd love to use libvirt but at this point I'm not really very well versed in working with virsh or libvirt.. whereas I know all the steps involved for direct LXC containers. | 01:14 |
| SpamapS | hazmat: I hadn't thought of that.. I'll take another stab later tonight w/ the dummy provider.. the EC2 one has so much .. ec2-ness.. ;) | 01:14 |
| SpamapS | Some things that EC2 wraps in an API call we'll have to emulate.. like generating Ids and allocating storage.. but thats not really all that difficult given that the target is just the local machine. | 01:15 |
| hazmat | SpamapS, ic some benefits of libvirt on the desktop.. but mainly that its already got the networking work setup. | 01:15 |
| SpamapS | hazmat: lxc has the networking stuff done too | 01:15 |
| SpamapS | there are basically 3 steps.. pick bridged or not.. lxc-create to create your container image.. lxc-start to boot it. | 01:17 |
| hazmat | SpamapS, it still requires some manual setup of dnsmasq vs. libvirt | 01:17 |
| SpamapS | hazmat: if you've setup libvirt once, it "just works" magically. :) | 01:17 |
| SpamapS | its kind of cheating.. | 01:17 |
| hazmat | SpamapS, anyways.. libvirt definitely has a lot more moving parts, if you want to go straight lxc, that rocks as well | 01:18 |
| SpamapS | hazmat: so I guess lxc doesn't have it directly worked out.. but 'Depends: libvirt-bin' is kind of all you need. | 01:18 |
| SpamapS | hazmat: would love to do it w/ libvirt .. gains a lot of ability to have kvm and qemu | 01:19 |
| SpamapS | anyway, I'll keep pounding on it. :-P | 01:19 |
| hazmat | SpamapS, indeed.. that would be nice with orchestra | 01:19 |
| hazmat | SpamapS, cool.. if you got any questions i'll be around | 01:19 |
| m_3 | SpamapS: libvirt would rock... either way would be great though! | 01:20 |
| SpamapS | hazmat: sweet. :) | 01:20 |
| SpamapS | Yeah I'm just kind of tired of waiting on the 1s latency for every command (even w/ open-tunnel) chatting with Amazon. | 01:20 |
| SpamapS | and not spending $1.25/formula change to run all my tests would be good too. :) | 01:20 |
| * SpamapS disappears | 01:21 | |
| hazmat__ | SpamapS out of curiosity outside of the current dir for hooks, are there any other things you think need working on in the formula dev cycle? leader election perhaps? or..? | 04:20 |
| kim0 | morning everyone | 08:28 |
| hazmat__ | g | 12:35 |
| kim0 | hazmat__: hmm .. I cannot connect to a bootstrap node .. getting → ERROR Can't yet connect to started machine: Invalid SSH key | 13:34 |
| kim0 | was this part of the code changed recently ? | 13:34 |
| hazmat__ | kim0: no its the same | 13:50 |
| hazmat__ | kim0: its possible cloud-init hasn | 13:51 |
| kim0 | hazmat__: yeah .. I viewed the key ssh inserts and it's not mine | 13:51 |
| hazmat__ | dropped the ssh key yet | 13:51 |
| kim0 | any idea where that key comes from | 13:51 |
| hazmat__ | kim0: odd.. its sniffed from your home directory ~/.ssh else its pulled from an explicit path or content specified in environments.yaml | 13:51 |
| kim0 | hmm | 13:52 |
| SpamapS | so, have we all seen this yet? http://www.livestream.com/oreillyconfs/video?clipId=pla_ae0330db-596b-4b7d-ad28-ccd6ec230b7d&utm_source=lslibrary&utm_medium=ui-thumb | 17:06 |
| SpamapS | It appears to me that cast isn't really all that similar to ensemble in its design, and limits its focus to "deployment" instead of "service orchestration" | 17:13 |
| SpamapS | still, we need to start giving these exact kind of lightning talks | 17:13 |
| m_3 | SpamapS: agree | 17:14 |
| hazmat__ | SpamapS: interesting, haven't seen it yet, thanks for the pointer | 17:15 |
| SpamapS | one slide.. they want it to be a 'zookeeper-lite' | 17:15 |
| kim0 | is zk-lite the so called noah .. | 17:27 |
| === koolhead11 is now known as koolhead11|afk | ||
| SpamapS | So, I'm thinking of importing all of the non-ubuntu formulas into principia, but separating them out using a file in principia-tools.. one for "contrib" and one for "principia" .. contrib being formulas that make use of software not in the formula and/or ubuntu itself. | 17:39 |
| SpamapS | Anybody have thoughts? (I'll send this to the ML too) | 17:40 |
| kim0 | +1 for raising visibility .. yesternight I was contemplating formulizing bigcouch .. and was discouraged since it wasn't packaged either duh | 17:41 |
| m_3 | +1 shows examples of how formulas will be actually used in the wild | 17:41 |
| kim0 | Yes | 17:41 |
| SpamapS | kim0: well.. there are .. I dunno.. 1000 things already in the archive that you could write formulas for... | 17:42 |
| SpamapS | Its intriguing that people are submitting more formulas for stuff not in Ubuntu than stuff in Ubuntu. | 17:43 |
| m_3 | SpamapS: some of that's laziness too... less constrained... less debconf | 17:44 |
| sidnei | was about to say that | 17:44 |
| m_3 | path of least resistance | 17:44 |
| SpamapS | The constraints are there to protect users though. | 17:45 |
| m_3 | it's important we put out as many as we can off the packages... to show how | 17:45 |
| m_3 | and, most importantly, for them to clone / cut-paste | 17:45 |
| SpamapS | And truthfully.. packaging makes it easier to write a formula. :) | 17:46 |
| m_3 | SpamapS: so contrib's critical, but we should always prefer packages when possible | 17:46 |
| m_3 | SpamapS: harder maintainer's job | 17:46 |
| SpamapS | Well I'm not so sure what we prefer will matter all that much.. | 17:46 |
| m_3 | SpamapS: ha | 17:46 |
| m_3 | SpamapS: yes | 17:46 |
| SpamapS | This is a non issue for Ubuntu.. they don't offer anything that can't be in the archive. | 17:47 |
| m_3 | SpamapS: easy to find (search) examples that can be forked | 17:47 |
| SpamapS | But this is sort of like the flash installer.. | 17:47 |
| SpamapS | Its a formula tha downloads what you want. | 17:47 |
| SpamapS | I wonder.. maybe we should make it a policy that if you download the software from somewhere else, you must be able to cryptographically verify it. | 17:48 |
| m_3 | SpamapS: oooh, hotbutton analogy | 17:48 |
| SpamapS | Like, we'll be signing formulas.. so drop the accepted key in there, and gpg --verify the tarball. | 17:48 |
| m_3 | wow, that's pretty limiting | 17:49 |
| m_3 | as a user, I'd wanna pull from my repo... crypto be damned | 17:50 |
| SpamapS | I'm talking only about principia | 17:50 |
| SpamapS | as a user, write a formula that rm -rf's / | 17:50 |
| SpamapS | I don't care | 17:50 |
| m_3 | SpamapS: oh, right... sorry | 17:50 |
| SpamapS | But for distributing to others.. | 17:51 |
| SpamapS | I'm suggesting that we should hav a social contract, like Debian and Ubuntu, that tells users what they can expect from the default ensemble formula namespaces. | 17:51 |
| m_3 | SpamapS: is it too much of a hassle to implement later? | 17:54 |
| m_3 | SpamapS: i.e., anything goes for now while we're getting the word out? | 17:54 |
| SpamapS | Anything goes, until somebody's database winds up in a russian hacker's hands. ;) | 17:55 |
| m_3 | SpamapS: although trust/integrity is important... don't have an answer | 17:55 |
| m_3 | SpamapS: right | 17:55 |
| SpamapS | This isn't that hard. | 17:55 |
| SpamapS | Its not like we're asking everybody to have two signatures from our keyring. | 17:55 |
| m_3 | SpamapS: worth trying... see what it does to constraints in contrib | 17:55 |
| SpamapS | Just verify the tarball. | 17:56 |
| SpamapS | Honestly if its not verifiable.. then its not suitable for common distribution. | 17:56 |
| SpamapS | We can make some tools for making it easy too. | 17:56 |
| _mup_ | ensemble/expose-ec2-provider r260 committed by jim.baker@canonical.com | 17:57 |
| _mup_ | Initial commit | 17:57 |
| SpamapS | Of course.. I tend to call those tools.. dpkg and debhelper.. ;) | 17:57 |
| m_3 | SpamapS: yup | 17:57 |
| m_3 | SpamapS: there are plans for an open formula exchange? | 17:58 |
| SpamapS | m_3: yes, it should end up working just like PPA's | 18:10 |
| _mup_ | txzookeeper/session-event-handling r48 committed by kapil.foss@gmail.com | 18:20 |
| _mup_ | add a test for expired session events, address some review comments [1-3] | 18:20 |
| === daker is now known as daker_ | ||
| kim0 | niemeyer: SpamapS hazmat jimbaker .. I'm starting to push Ensemble outside our usual circle of readers, by getting the word out through guest articles on other high traffic sites. Infoq.com have sent me a preliminary approval, so I'm starting with them. Here is the very crude article I have today (stealing great content from SpamapS :) http://j.mp/lVUEuZ .. It's crude at the moment (just done writing it) .. but your feedback (direct edits or comment | 18:46 |
| kim0 | Also it might be a good idea to add a section about Ensemble's software architecture ... if any of the devs is interested in adding that part | 18:47 |
| kim0 | v0ld3m0rt: hi there | 18:47 |
| v0ld3m0rt | hi.. | 18:49 |
| kim0 | v0ld3m0rt: first time to land in #ubuntu-ensemble ? | 18:50 |
| v0ld3m0rt | yep | 18:50 |
| kim0 | Then you're up for the official welcome | 18:50 |
| * kim0 prepares a Royal ceremony :) | 18:50 | |
| v0ld3m0rt | thank u... | 18:50 |
| v0ld3m0rt | lolz.. | 18:50 |
| kim0 | hehe | 18:50 |
| v0ld3m0rt | i wud be starting on ensemble.. | 18:51 |
| kim0 | v0ld3m0rt: how did you find out about ensemble | 18:51 |
| kim0 | great news! | 18:51 |
| v0ld3m0rt | i have been a inquisitive developer.. | 18:51 |
| v0ld3m0rt | i like to explore mailing lists and forums.. | 18:51 |
| kim0 | good find! | 18:51 |
| v0ld3m0rt | so thats where i heard abt ensemble.. | 18:52 |
| kim0 | v0ld3m0rt: so have you already played with it .. read the docs ..etc yet ? | 18:52 |
| v0ld3m0rt | i hv just started. | 18:53 |
| kim0 | v0ld3m0rt: check out https://ensemble.ubuntu.com/docs/ then :) | 18:53 |
| v0ld3m0rt | oh..thankyou.. | 18:53 |
| v0ld3m0rt | i was looking for that.. | 18:54 |
| v0ld3m0rt | though i started with code.. | 18:54 |
| v0ld3m0rt | just to get feel of it.. | 18:54 |
| kim0 | yeah | 18:54 |
| kim0 | v0ld3m0rt: feel free to grab me anytime to help with anything .. enjoy hacking | 18:55 |
| v0ld3m0rt | yeah sure....thank u very much.... | 18:55 |
| v0ld3m0rt | u guys are really friendly.. | 18:55 |
| v0ld3m0rt | :) | 18:55 |
| kim0 | hehe :) | 18:55 |
| SpamapS | hazmat: so, this special AMI that we're using .. I need to duplicate that for the LXC thing I'm working on | 18:56 |
| SpamapS | hazmat: any tips on where to find the bits that do that? | 18:57 |
| SpamapS | hazmat: also, since the copyright MP doesn't change any code.. can you just merge it, or do you want to wait for a 2nd +1 ? | 19:15 |
| SpamapS | hazmat: err, the txzookeeper merge I mean | 19:15 |
| SpamapS | ahh.. bin/ensemble-make-image | 19:20 |
| * SpamapS wonders if he can just run that | 19:20 | |
| SpamapS | dur.. no | 19:21 |
| m_3 | SpamapS: ubuntu-vm-builder and then copy dpkg selections over from the AMI? | 19:24 |
| m_3 | SpamapS: AMI's gonna have all the /etc/init/cloud-init metadata deps anyways | 19:24 |
| SpamapS | lxc has a simpler shell script based template that I'm using.. | 19:31 |
| SpamapS | just finishing the cloud-init bits. | 19:31 |
| hazmat | SpamapS, i'll just merge it | 19:32 |
| hazmat | SpamapS, the bits are in a couple of pieces for the install. the ec2 providers launch setups a cloud config that will effectively recreate it (on a vanilla image) | 19:33 |
| hazmat | SpamapS, common packages defined in ensemble/providers/common.py which gets formatted by ensemble/providers/ec2/utils format_cloud_ini ... the actual commands run to setup the agents are passed to format_cloud_ini by ensemble/providers/ec2/launch.py and differ depending on if its a bootstrap node vs. service unit node. | 19:35 |
| hazmat | SpamapS, the bin/ensemble-make-image script is used to create the images with the packages preinstalled | 19:36 |
| hazmat | the images haven't been updated yet to default to the ppa | 19:36 |
| SpamapS | hazmat: So what I'm doing is setting an environment variable 'ENSEMBLE_LXC_PROVIDER_CONFIG' which is a script the lxc template will run to configure the agent startup, since I"m not entirely sure how to make cloud-init read a local file. | 19:37 |
| SpamapS | Since everything else seems to be the same, I'm doing all of that in the initial "bootstrap" phase which is basically like making an AMI. :) | 19:38 |
| hazmat | SpamapS, SpamapS, http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/examples/seed/README | 19:42 |
| hazmat | SpamapS, smoser pointed that out to me a few weeks ago | 19:42 |
| hazmat | as docs on using the on disk data for cloud-init | 19:42 |
| SpamapS | Yeah | 19:43 |
| SpamapS | So the format_cloud_ini call is in the wrong place | 19:43 |
| SpamapS | should be in common | 19:43 |
| SpamapS | s/call/definition | 19:43 |
| hazmat | SpamapS, agreed | 19:43 |
| SpamapS | Oh man, I hate when I try to work on Ensemble and end up working on Ensmeble | 19:45 |
| SpamapS | if [ -n "$ENSMEBLE_LXC_PROVIDER_CONFIG" ] ; then | 19:45 |
| SpamapS | $ENSMEBLE_LXC_PROVIDER_CONFIG $rootfs | 19:45 |
| SpamapS | consistent mispellur | 19:45 |
| m_3 | SpamapS: my fingers kept doing ensemple for the longest time | 19:47 |
| SpamapS | hazmat: seems like a lot of the stuff in ec2 should be common | 19:56 |
| hazmat | SpamapS, you mean from ec2.utils or other stuff? | 19:57 |
| SpamapS | hazmat: there's really nothing EC2 about ec2.get_machine_variables | 19:57 |
| SpamapS | hazmat: except the curl to get the instance-id .. but.. hrm. | 19:57 |
| SpamapS | maybe not | 19:58 |
| hazmat | SpamapS, yeah.. that does look pretty generically useful | 19:59 |
| hazmat | get machine options has some ec2 specifics, but the variables are meant for cloud init formatting | 19:59 |
| SpamapS | hazmat: the curl to get the instance-id seems a bit of a hack .. we should be able to just pass that in. | 19:59 |
| hazmat | SpamapS, user-data is passed in the api that returns the instance id | 20:00 |
| SpamapS | Ok, I'm going to go try having a sip of 5 hour energy and see if I can actually think this through. :-P | 20:01 |
| * SpamapS suddenly feels he has lost most cognitive function | 20:01 | |
| * hazmat recommends coffee | 20:03 | |
| hazmat | "It is by caffeine alone I set my mind in motion. It is by the beans of Java that thoughts acquire speed, the hands acquire shaking, the shaking becomes a warning. It is by caffeine alone I set my mind in motion." | 20:04 |
| m_3 | hazmat: nice | 20:05 |
| hazmat | classic dune derivation | 20:05 |
| _mup_ | ensemble/expose-ec2-provider r261 committed by jim.baker@canonical.com | 20:08 |
| _mup_ | Fix open_port/close_port spike | 20:08 |
| SpamapS | hazmat: there's a *ton* of useful generic code in ec2/launch.py | 23:20 |
| * SpamapS 's been refactoring on En-Sem-Ble... alll the live long daaaay | 23:21 | |
| SpamapS | most of it just copies right into common.py without modification... just nice little generic routines for setting up a machine to run the agents and such | 23:21 |
| hazmat | SpamapS, nice | 23:22 |
| SpamapS | Could not find the main class: org.apache.zookeeper.server.quorum.QuorumPeerMain. Program will exit. | 23:45 |
| SpamapS | Hrm.. how do I run the test suite? | 23:45 |
| ajmitch | SpamapS: how are you going at getting this into debian? | 23:47 |
| SpamapS | ajmitch: Not much traction thus far. | 23:48 |
| ajmitch | is the hold up sponsorship or getting it ready? | 23:48 |
| SpamapS | ajmitch: sponsorship, but truthfully, I think i need to re-submit txzookeeper with the license clarified. | 23:48 |
| * ajmitch can probably help out with some sponsorship at least | 23:49 | |
| SpamapS | ajmitch: given the ubuntu-specific nature of ensemble today.. its not 100% critical it go into Debian. It currently hard-codes an AMI that is Ubuntu.. so it can't even drive a Debian EC2 node. | 23:49 |
| SpamapS | ajmitch: much appreciated! :) | 23:49 |
| ajmitch | being less ubuntu-specific in the future would be nice :) | 23:50 |
| SpamapS | But I figure getting ensemble, the client, onto as many platforms as possible, is a good idea. :) | 23:50 |
| SpamapS | ajmitch: thats up to the community. We're.. kind of focused on that platform. :) | 23:50 |
| * SpamapS points to the channel name.. and domain name.. and funding party.. | 23:50 | |
| ajmitch | I can't imagine why :P | 23:50 |
| SpamapS | ajmitch: honestly it should be easily doable. Especially if other platforms adopt cloud-init. | 23:51 |
| SpamapS | Scary as it sounds, ensemble should be pretty easily portable to the Amazon AMI's | 23:51 |
| ajmitch | I'm still a bit fuzzy on the details of how the bits fit together | 23:52 |
| SpamapS | ajmitch: We need more pictures | 23:52 |
| ajmitch | such as whether it relies on ec2/similar cloud platforms | 23:53 |
| SpamapS | Right now, the only "provider" is EC2 | 23:54 |
| SpamapS | which actually means Eucalyptus and OpenStack should work | 23:54 |
| SpamapS | I'm working on a local-only LXC provider so I can develop while disconnected and not sit waiting for Amazon constantly. :) | 23:54 |
| ajmitch | my 'development environment' at work is simply using virtualbox on a desktop with plenty of RAM :) | 23:55 |
| SpamapS | This is the typical engineer idea.. "I can't possibly wait 1 second for every command.. so I'll spend 40 hours writing a solution." | 23:55 |
| SpamapS | ajmitch: Yeah, LXC should alleviate some of that RAM requirement.. shared VFS cache.. shared heap. | 23:55 |
| ajmitch | I'd probably need to grab that backported kernel to do so, running on lucid | 23:56 |
| SpamapS | chroot + network isolation basically | 23:56 |
| SpamapS | Yeah, you would. :P | 23:56 |
| SpamapS | But I hear that it works quite well. | 23:56 |
| ajmitch | a bit worrying to see that zookeeper will be orphaned in debian | 23:58 |
| SpamapS | Thats a tempest in a teapot. | 23:59 |
| ajmitch | probably | 23:59 |
| SpamapS | We'll be taking on maintenance of it if nobody else wants it more. | 23:59 |
| * ajmitch always thinks of zookeepr, which is rather different :) | 23:59 | |
| SpamapS | James Page, one of our guys who has been active w/ the debian java team, has drafted an email and we should be on our way to taking it on soon. Plus I'm actively seeking DD status. | 23:59 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!