| === dendro-afk is now known as dendrobates | ||
| tsarles | Im interested in ubuntu cloud, but it seems almost as if there isn't basic information availible about how it works. I imagine UEC takes the place of say VMWare? But what about storage clustering, does it do that sort of stuff too, or do you need a SAN? is there anywhere i can read up on the basics? | 01:55 |
|---|---|---|
| tsarles | is there a v-motion sort of feature for high availability? | 01:56 |
| theTrav | tsarles: check out walrus | 02:47 |
| theTrav | I'm super new to cloud as well, but it seems like walrus takes care of that side | 02:47 |
| theTrav | ok, so firing up elastic fox to try to connect to my cloud, I'm unable to add the cluster controller | 03:04 |
| theTrav | looking in cloud-error.log I've got NoAcceptingPipelineException | 03:04 |
| theTrav | do I need to set up / start some sort of service on the cloud side to handle the WS interface? | 03:04 |
| theTrav | hmm, on another angle that's possibly related, I am trying to run these things on a network that already has a DHCP server that I have no access to | 03:21 |
| theTrav | from what I can tell (euca-describe-instances) it's trying to do something with 179.19.1.2 however nothing on the network has that IP address | 03:22 |
| theTrav | I notice the CC has 179.19.1.1 on its interface | 03:22 |
| theTrav | so I'm guessing that's something to do with the --addressing private that the instructions told me to use | 03:22 |
| areay | hi all... i'm having trouble creating a volume with euca-create-volume... it returns "creating", but after i run euca-describe-volumes see "Failed" the first time, then "Deleted" | 03:52 |
| areay | I get this each time in /var/log/eucalyptus/cloud-error.log: 03:53:54 [BlockStorage:pool-10-thread-2] ERROR com.eucalyptus.util.EucalyptusCloudException: Could not export AoE device /dev/vg-EnZbTA../lv-hkAQIA.. iface: eth0 pid: 3181 returnValue: | 03:56 |
| areay | nm, i noticed the error said eth0; i've configured the network using eth1.. i changed it in the web interface under configuration and it works perfectly now :) | 03:59 |
| theTrav | so if I'm running my cloud behind a proxy, is there any way for the "store" tab on the web interface to work? | 04:02 |
| theTrav | like for downloading images etc? | 04:02 |
| theTrav | I should be able to ssh into a VM instance running within the cloud from outside the cloud shouldn't I | 05:47 |
| theTrav | it seems I can ping it from the cluster controller but not from another computer on the same network | 05:47 |
| flaccid | theTrav: configure security groups correctly for the instance | 09:49 |
| === Kiall|AFK is now known as kiall | ||
| === TeTeT_ is now known as TeTeT | ||
| === dendrobates is now known as dendro-afk | ||
| === dendro-afk is now known as dendrobates | ||
| === dendrobates is now known as dendro-afk | ||
| === dendro-afk is now known as dendrobates | ||
| kim0 | smoser: I just pushed a post on the ec2 ebs migration tool we talked about http://foss-boss.blogspot.com/2011/01/hack-on-ubuntu-cloud-utils.html | 13:42 |
| kim0 | superxgl: Hi .. did you just email ubuntu-cloud mailing list asking to join ? | 14:07 |
| kim0 | superxgl: You can join using https://lists.ubuntu.com/mailman/listinfo/Ubuntu-cloud | 14:08 |
| === dendrobates is now known as dendro-afk | ||
| === dendro-afk is now known as dendrobates | ||
| gtaylor | cool, and ubuntu-cloud channel | 15:20 |
| kim0 | gtaylor: hey :) | 15:22 |
| gtaylor | I've been hanging out in ubuntu-server, for lack of a better idea | 15:22 |
| kim0 | gtaylor: i.e. you didn't know this existed ? | 15:22 |
| gtaylor | kim0: Correct | 15:22 |
| gsa1 | hi people | 15:23 |
| kim0 | hmm .. ok .. I need to spam at more places then :) | 15:23 |
| kim0 | gsa1: howdy | 15:23 |
| gtaylor | kim0: Which is bad, because I've been building a fairly complicated piece of software on top of Ubuntu's EC2 image :) | 15:23 |
| gsa1 | kim0: Hi Ahmed | 15:23 |
| kim0 | gtaylor: Yeah, someone like you ought to have heard about this channel | 15:23 |
| kim0 | gsa1: hey there | 15:24 |
| kim0 | gsa1: did we meet before ? | 15:24 |
| gtaylor | kim0: A link here would be helpful: https://help.ubuntu.com/community/EC2StartersGuide | 15:24 |
| gsa1 | kim0: Today I sent you a mail about contributions and so (with Pharo :) ) | 15:24 |
| kim0 | gsa1: a ha .. Welcome German | 15:25 |
| kim0 | gtaylor: Are you working on the guide itself ?! | 15:25 |
| gtaylor | kim0: No, just suggesting that as a place to add an IRC link | 15:25 |
| gsa1 | Thanks! | 15:25 |
| kim0 | ah got it | 15:25 |
| kim0 | gsa1: ping .. I'm trying to pm you | 15:32 |
| gsa1 | kim0: sorry, just was responding your mail | 15:33 |
| kim0 | no problemo | 15:33 |
| === gtaylor_ is now known as gtaylor | ||
| === andreserl is now known as RoAkSoAx | ||
| === nwl2110 is now known as sark2110 | ||
| === sark2110 is now known as sark | ||
| kim0 | gtaylor: Hi there | 18:04 |
| kim0 | You mentioned you were working on a major project involving Ubuntu and the cloud | 18:04 |
| gtaylor | kim0: Hello | 18:04 |
| gtaylor | kim0: Yes, an open source distributed encoding system. | 18:04 |
| kim0 | I'm interested to learn more about that ? | 18:04 |
| gtaylor | kim0: Cloud-init is/was a critical part of that. | 18:04 |
| kim0 | I'm even more interested now :) | 18:05 |
| kim0 | encoding as in video encoding? | 18:05 |
| gtaylor | kim0: Video encoding is the first target, but there's no reason it couldn't do audio support with the addition of a class for that. | 18:06 |
| gtaylor | kim0: http://duointeractive.github.com/media-nommer/ | 18:06 |
| gtaylor | It's "working" already, but it's in the early going and I'm in doc writing and testing mode. | 18:07 |
| kim0 | gtaylor: so encoding a "movie" works (at least sometimes) ? | 18:08 |
| gtaylor | kim0: Sure | 18:08 |
| kim0 | huh | 18:08 |
| kim0 | that sounds awesome! | 18:08 |
| gtaylor | kim0: It's just ffmpeg | 18:08 |
| kim0 | I know .. but all the magic is in the glue :) | 18:08 |
| gtaylor | kim0: Though you can use other encoders if you wanted. | 18:08 |
| gtaylor | Yeah, it's been a really interesting project | 18:09 |
| gtaylor | It's basically re-implementing something like encoding.com | 18:09 |
| gtaylor | (without having to pay their markups) | 18:09 |
| kim0 | can I implement various workload handlers (mogrify for images, pdf2ps for pdf ...) ? | 18:10 |
| gtaylor | kim0: Yep, they're called "Nommers" :) | 18:10 |
| gtaylor | To give you an idea, here's the FFmpeg nommer: https://github.com/duointeractive/media-nommer/blob/master/media_nommer/ec2nommerd/nommers/ec2_ffmpeg/nommer.py | 18:11 |
| gtaylor | Most of that is just assembling command line options. It has very little front-facing complexity there. | 18:11 |
| * kim0 reading | 18:11 | |
| gtaylor | Base class does the file-like object shuffling: https://github.com/duointeractive/media-nommer/blob/master/media_nommer/ec2nommerd/nommers/base_nommer.py | 18:12 |
| kim0 | I was about to start a similar project | 18:14 |
| kim0 | was gonna call it cloud-crunch | 18:14 |
| gtaylor | It's quite the undertaking | 18:14 |
| kim0 | it's quite cool :) | 18:14 |
| gtaylor | It'd be easier if I had a bunch of local hardware, I'd just use something like celery. But we don't want to deal with our own hardware anymore. | 18:14 |
| gtaylor | And Amazon AWS is really starting to kick its way into awesomeness | 18:15 |
| kim0 | any reason why you'd want to use celery over SQS | 18:16 |
| gtaylor | We've had great success with it in the past. It's a great way to distribute tasks and track task state over an arbitrarily sized pool of workers. And it's fast and Python :) | 18:17 |
| gtaylor | But in this usage case, we were bent on using EC2 to have a very large amount of scalability. We also didn't want the master daemon and the encoder daemons talking directly, which ruled out celery. | 18:17 |
| gtaylor | Didn't want to have to open firewall holes on either side. | 18:18 |
| gtaylor | Didn't want to tunnel or VPN. | 18:18 |
| gtaylor | So Amazon SQS + SimpleDB it is :) | 18:18 |
| kim0 | why couldn't you run your master daemon with celery on ec2 as well ? | 18:19 |
| === dendrobates is now known as dendro-afk | ||
| gtaylor | Our web app wouldn't be able to query the master daemon without opening a hole on the EC2 side | 18:19 |
| gtaylor | Right now, I can run the master daemon on our existing hardware and our web apps can send things to media-nommer's JSON web API | 18:20 |
| gtaylor | Plus we didn't want $15/month to go down the hole for us and our users just for running a daemon that doesn't even come close to utilizing a full EC2 instance (even a micro) | 18:20 |
| === dendro-afk is now known as dendrobates | ||
| kim0 | ah makes sense | 18:21 |
| gtaylor | Makes things slightly more complicated, but it's also a _lot_ more failure tolerant now. | 18:21 |
| gtaylor | If your master daemon goes down, any current jobs are still on Amazon and are still encoded. Work will continue until the queue is emptied, and after that the EC2 instances will terminate themselves automatically after a period of inactivity. | 18:22 |
| kim0 | cronjob ? | 18:22 |
| gtaylor | for what? | 18:22 |
| kim0 | auto-terminating they self :) | 18:22 |
| kim0 | thy* | 18:22 |
| gtaylor | Each EC2 instance runs a daemon that handles picking jobs off the queue, and "contemplating further existence" | 18:23 |
| kim0 | a ha | 18:23 |
| gtaylor | So the daemon will run a configurable number of encoding processes at a time, and only persist up until a configurable point of idle time. | 18:23 |
| gtaylor | I have them defaulting to 15 minutes of activity results in termination right now, IIRC. | 18:23 |
| kim0 | sounds great | 18:24 |
| kim0 | gtaylor: you're building this system for an online commercial website right | 18:25 |
| gtaylor | kim0: We've got two clients that need it, both of them very different. | 18:25 |
| kim0 | gtaylor: I'm interested why did you choose ubuntu as a platform :) did you consider others | 18:26 |
| gtaylor | kim0: We use it extensively in our work, and I really liked cloud-init. I didn't want any users of media-nommer having to set up their own AMI if they don't want to. Configuration right now involves setting three config values and starting a daemon, that's all. | 18:27 |
| gtaylor | We created an AMI from Ubuntu's EC2 ones, customized it very lightly, and gave it a way to update itself and stat encoding on boot. Users can use our public AMI without modification because of cloud-init. | 18:27 |
| kim0 | I wonder you had to create a custom AMI .. why not everything thru cloud-init | 18:28 |
| gtaylor | My cloud-init chops aren't that great yet. | 18:28 |
| gtaylor | I sure would love it, and it's not a difficult thing to add down the line if a cloud-init rockstar came along and did so, I just didn't have the time budgeted to figure it out. Made a weak attempt but failed. | 18:29 |
| kim0 | what things would cloud-init need to do in that case | 18:30 |
| gtaylor | install supervisor or an upstart job for the master process, automatically use a virtualenv and pip to update media-nommer if restarted, and automatically set the correct AWS credentials on instantiation (already does that). I also mounted /tmp as ephemeral storage, and needed to chmod 777 /tmp on each reboot, for some reason. | 18:31 |
| gtaylor | Nothing overly complex, but the order in which things run is important. | 18:31 |
| kim0 | yeah .. you know with cloud-init you can just run an arbitrary shell script | 18:32 |
| gtaylor | yep | 18:32 |
| kim0 | gtaylor: Would you mind if I turn that chat into an interview article with yourself | 18:33 |
| gtaylor | kim0: Not at all, go for it | 18:33 |
| kim0 | sounds good! | 18:34 |
| gtaylor | kim0: What site are you with? | 18:34 |
| kim0 | gtaylor: can u pm me your email so I can share a google doc with you | 18:34 |
| kim0 | gtaylor: I'm working with canonical caring for the ubuntu cloud community | 18:35 |
| gtaylor | Very cool, I keep an eye on the Ubuntu Cloud Blog | 18:35 |
| kim0 | Might wanna follow planet ubuntu cloud | 18:36 |
| kim0 | http://cloud.ubuntu.com/planet/ | 18:36 |
| gtaylor | Ah, I missed that one | 18:36 |
| kim0 | gtaylor: thanks for all the info man .. I'll share some doc with you, and perhaps asking for some missing pieces | 18:40 |
| gtaylor | kim0: Sure, feel free to email or ping me on here if there's anything else. | 18:40 |
| kim0 | gtaylor: actually, I have a last question. Since you were contracted for this work, why did you decide to open source it | 18:40 |
| gtaylor | kim0: I work for a very small contracting company with some great clients. Our philosophy is we're working on a general purpose tool that we don't anticipate to directly profit from, it might as well be open sourced so others may use it, and hopefully contribute back to make it better for all. | 18:42 |
| gtaylor | We've got a decent public portfolio in the works on GitHub: https://github.com/duointeractive | 18:42 |
| kim0 | does look good indeed :) | 18:43 |
| kim0 | congrats | 18:43 |
| gtaylor | Thanks, it's a pleasure to be able to blend hobby and work with open source software. | 18:44 |
| kim0 | yep, you're lucky hehe | 18:44 |
| gtaylor | Stark contrast to the corporate environment I came from. | 18:45 |
| kim0 | Are you saying corp devs hate their code :) | 18:45 |
| gtaylor | Well, my former employer was starkly opposed to distributing or using open source. Which is a bummer for someone who really gets his jollies from writing and improving open source software :) | 18:46 |
| gtaylor | "If it's Open Source, it must be buggy and of inferior quality!" | 18:46 |
| gtaylor | "Instead, we'll pay some contractors to re-invent the wheel for thousands of dollars. Even though we could just use something that already exists." | 18:47 |
| kim0 | hehhe :( | 18:47 |
| kim0 | sigh .. I've seen a couple of those | 18:47 |
| gtaylor | I get the impression that many other experience similar problems with their management. But it does sound like things are slowly improving. | 18:47 |
| gtaylor | The funny thing is/was, one of the managers was a complete Apple fanboy. Never mind the fact that his Mac has tons of great software with roots or involvement in open source. | 18:48 |
| gtaylor | CUPS, gcc, their X.org port, bash, python, ruby, lots more | 18:49 |
| kim0 | gtaylor: but if it runs on Steves boxes, it all becomes shiny man | 18:53 |
| gtaylor | heh, yes indeed | 18:53 |
| kim0 | hehe :) | 18:53 |
| === gtaylor is now known as gtaylor_afk | ||
| === gtaylor_afk is now known as gtaylor | ||
| === kiall is now known as kiall|AFK | ||
| adamsweet | hi, I have a bit of a noob question... | 20:22 |
| adamsweet | I have an Ubuntu instance on EC2 | 20:22 |
| adamsweet | I don't like running with as the ubuntu user so I added my own user, but I can't use my EC2 keypair with the new user after copying the authorized_keys from the ubuntu user's homedir to my users .ssh/authorized_keys file and setting the perms | 20:24 |
| adamsweet | nor can I use my own regular SSH key which is on my laptop | 20:24 |
| adamsweet | just get Permission denied (publickey) in each case | 20:25 |
| adamsweet | can anybody tell me what is stopping it working, I can't find anything in /etc/sshd_config which is different from any other Ubuntu/Debian install | 20:25 |
| kim0 | adamsweet: shooting in the dark .. but could you add your new user to the same groups as user ubuntu (id -a ubuntu) ? | 20:26 |
| adamsweet | kim0: group memberships for my user and the ubuntu user are the same except for the primary group, but I can try adding my user to the ubuntu groups as a shot in the dark | 20:28 |
| adamsweet | I did also enable password logins but I still get the above error when attempting to SSH as my own user without specifying a keyfile | 20:29 |
| kim0 | hmm | 20:30 |
| adamsweet | kim0, no adding user to ubuntu group didn't help | 20:30 |
| adamsweet | :) | 20:30 |
| kim0 | adamsweet: can you try visudo and adding the new user | 20:30 |
| adamsweet | I suspect I might misunderstand what the .pem file is and how it is used maybe | 20:31 |
| kim0 | adamsweet: yeah! | 20:31 |
| kim0 | I hope you didn't just add that | 20:31 |
| adamsweet | new user is has already been set up for sudo | 20:31 |
| kim0 | are you with putty on windows | 20:32 |
| adamsweet | no | 20:32 |
| adamsweet | gnome-terminal on Ubuntu | 20:33 |
| kim0 | what is the pem you mention | 20:33 |
| adamsweet | I'm a competent Linux user but a bit of a cloud novice | 20:33 |
| adamsweet | my keypair generated in the ec2 interface | 20:33 |
| kim0 | could you su to the new user. And paste the output of ls -ld ~/.ssh ; ls -l ~/.ssh ; cat ~/.ssh/authorized_keys | 20:35 |
| adamsweet | there's obviously a bit of config somewhere which says 'Don't let anyone log in using SSH unless it's the ubuntu user and they have they this keyfile' | 20:38 |
| adamsweet | drwx------ 2 adam adam 4096 2011-01-19 20:08 /home/adam/.ssh | 20:39 |
| adamsweet | total 0 | 20:39 |
| adamsweet | -rw------- 1 adam adam 0 2011-01-19 20:08 authorized_keys | 20:39 |
| kim0 | um, well paste doesn't really paste right here :) next time please use paste.ubuntu.com | 20:39 |
| kim0 | really mean* | 20:39 |
| adamsweet | ahh ok | 20:40 |
| adamsweet | but I kinda halfway fixed the problem | 20:40 |
| kim0 | cool .. what did you find | 20:41 |
| adamsweet | the ownerships on the authorized_keys file were root:root so I fixed that, must have been a leftover from some messing around I did while trying to figure it out | 20:42 |
| adamsweet | but my own personal SSH key still wouldn't work so I catted ubuntu's authorized_keys file into my own and now I can SSH in using the amazon generated keypair | 20:42 |
| adamsweet | so I'm halfway there | 20:43 |
| kim0 | aha | 20:43 |
| adamsweet | the ideal scenario is that individual users would be able to SSH in using their own keys, but I guess that's only a step away from here | 20:43 |
| kim0 | it should work | 20:43 |
| kim0 | adamsweet: I mean if it works with one key .. it should work with the other | 20:44 |
| adamsweet | thanks, sometimes somebody else's brain cycles expose mistakes that I overlooked | 20:44 |
| kim0 | yeah, it's always the simple stuff first :) | 20:44 |
| adamsweet | aha, and I'm in using my own key | 20:45 |
| kim0 | woohoo | 20:45 |
| adamsweet | forgot to restart ssh :( | 20:45 |
| adamsweet | am rusty :) | 20:45 |
| adamsweet | and stupid | 20:46 |
| kim0 | do you actually need to do that for adding a new user! | 20:46 |
| adamsweet | thanks man | 20:46 |
| adamsweet | no but I was testing ssh-copy-id | 20:46 |
| kim0 | you enabled password logins ? | 20:46 |
| adamsweet | I can turn them off now | 20:46 |
| kim0 | yeah | 20:46 |
| adamsweet | done | 20:47 |
| adamsweet | well now I know what I was doing wrong in the first place (authorized_keys ownerships) I can just paste my public key into place | 20:47 |
| adamsweet | thanks :) | 20:47 |
| kim0 | yipee | 20:48 |
| kim0 | np | 20:48 |
| adamsweet | I thought there was some config somewhere which was refusing logins without the ec2 generated key but couldn't find it | 20:49 |
| adamsweet | note to self: check the simple things first | 20:49 |
| kim0 | yeah hehe | 20:51 |
| === dendrobates is now known as dendro-afk | ||
| === dendro-afk is now known as dendrobates | ||
| theTrav | ok, so I've found SQS on amazon | 22:41 |
| theTrav | I'm wondering if the ubuntu enterprise cloud has an equivalent | 22:41 |
| flaccid | theTrav: don't think so | 22:53 |
| === dendrobates is now known as dendro-afk | ||
| === dendro-afk is now known as dendrobates | ||
| theTrav | flaccid: thanks | 23:30 |
| theTrav | from reading the wiki page it looks like it's a bit to laggy anyway (2-10 seconds per message wtf?) | 23:30 |
| === dendrobates is now known as dendro-afk | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!