#ubuntu-cloud 2011-06-20
<kim0> Morning fellas
<fulc> Hello, can anyone help me with UEC?
<flaccid> hello, how long is a piece of string?
<flaccid> you may also like to see the topic.
<flaccid> fulc ^^ don't ask to ask, just ask.
<fulc> alright :)
<fulc> i want to install and enable xen on my server machines that dont support hardware virtualization, and UEC comes with KVM by default
<fulc> just to make it clear, I have 3 machines (1 front-end and 2 nodes) and using UEC for private cloud computing
<fulc> If I'm at the wrong place, please tell me where to go =]
<flaccid> you may have more luck with the eucalyptus doc or #eucalyptus
<fulc> alright, thanks =]
<ubuntucloud582> hi
<ubuntucloud582> sorry im new on ubuntu .. the project is awsome but wat is a  cloud ...ouffff!
<flaccid> haha
<flaccid> someone said oufffff
<fulc> ugh..no luck at #eucalyptus
<flaccid> i wouldn't expect it
<flaccid> try googling xen with eucalyptus..
<flaccid> i prefer cloudstack or openstack, but if you want to use euca thats totally your choice
<fulc> well, not really mine...but my boss' ^_^
<fulc> i'm an intern and they gave me these old machines to install eucalyptus on
<fulc> although, if these cloudstack and openstack are better (more compatible and easy to use), i might be allowed to try them out
<flaccid> if its commercial, you'll need to get euca support i.e. paid support because its so buggy and difficult (imho)
<fulc> uhm
<fulc> i dont know what you mean by commercial :)
<fulc> they just want to use the cloud for internal testing of some software
<fulc> so, which one do you think i should scope? cloudstack or openstack? and what is the difference?
<flaccid> well sorry, i mean like as in professional. i wouldn't use eucalyptus without their support because of how it is
<flaccid> you don't need a private cloud for testing of software though, you can just use virtualizer like virtualbox or vmware
<flaccid> for other private clouds, i'd check out both. UEC will be switching to one of these soon anyway
<fulc> that's exactly what i'm thinking, but they said they want a cloud for this
<flaccid> though there is no reason why you just don't use aws ec2 for your needs.
<flaccid> well Amazon EC2 is the biggest cloud in the world
<flaccid> *public cloud sorry
<smoser> flaccid, fulc Ubuntu's "UEC" is changing its name in 11.10 to "Ubuntu Cloud" .  Previously, that was "Ubuntu Enterprise Cloud".  The 'E' was confusing both under the assumption that Enterprise == "I have to pay for it" and "E" == "Eucalyptus"
<smoser> in 11.10, Ubuntu's default solution when installing cloud will be openstack.
<fulc> great, now i just have to wait 4 months for it :D
<smoser> well, you can try alpha1, but i wont tell you that it going to be a easy experience
<smoser> or that you should judge ubuntu on that.
<smoser> if you need osmething for testing, you really should give a quick look to UEC of 11.04 installed from ISO. it may well "just work" for your needs.
<fulc> i'll probably manage to get this euca thing working...been struggling with it for 2 weeks now
<flaccid> smoser: yeah, thus my comment.
<smoser> i have zero experience with cloudstack, so i am not able to comment there.
<flaccid> there is no reason why you can't just try out both openstack and cloudstack without ubuntu
<flaccid> ubuntu doesn't need to hold your hand there :)
<smoser> ah, regarding xen, that is definitelyi untested path on ubuntu.
<flaccid> personally, i mean if its going to be called ubuntu cloud and its openstack well its not a cloud by ubuntu is it
<fulc> oh well, i installed ubuntu because it came with everything already installed =]
<flaccid> the name change also doesn't explain the major switch of cloud software either does it? :)
<fulc> the UEC as a whole I mean
<fulc> but these are some old machines with no hardware virtualization...and kvm doesnt support the other one :P
<flaccid> decent cloud software is not hard to install and has docs on how to install
<flaccid> well ubuntu doesn't support it but paravirtual with xen is indeed possible with euca, http://open.eucalyptus.com/forum/eucalyptus-xen-or-kvm
<flaccid> support is dependent on the kernel i.e. you may need a xen specific kernel if not compiled in
<flaccid> for euca its more of an option, but for ubuntu its not supported
<flaccid> ec2 and the rackspace cloud both use xen
<fulc> is this amazon ec2 free?
<flaccid> it can be with the aws free tier
<fulc> let me google it up =]
<fulc> erm, i dont understand this...pay as you go computing...are they offering me a cloud?
<fulc> i would just like some software to run on my own machines =)
<flaccid> there is #aws for aws and you basically have the most popular cloud options
<flaccid> ec2 is a public cloud
<flaccid> err *most popular private cloud options
<fulc> right...
<fulc> so i need this aws thing instead of ec2 =]
<flaccid> aws=amazon web services which offers the ec2 cloud as a product
<smoser> amazon does not provide any solution for "private cloud"
<smoser> only public cloud
<fulc> oh i see
<fulc> thanks for the help guys
<iamcloud> hello
<Kyle__> How do you find the log of an individual vm?  From eucalyptus/UEC's point of view, not the vms.
<obino> Kyle__: which kind of logs are you looking for?
<Kyle__> obino: Ones that will show me why a vm didn't start.
<obino> you may want to look into the nc.log
<obino> find the NC that tries to run the instance and look in the nc.log on that machine
<obino> you can trace it from the cc.log (it will tell you the IP address of the NC trying to run the instance)
<Kyle__> OK
<Kyle__> Will it be in there by bucket name or image id?
<obino> Kyle__: instance ID
#ubuntu-cloud 2011-06-21
<SpamapS> anybody have a good guide for using lxc with libvirt?
<koolhead11> hi kim0
<kim0> koolhead11: hey :)
<koolhead11> howdy?
<kim0> koolhead11: very well .. how about yourself
<koolhead11> all well thanks!!
<kim0> Awesome
<koolhead11> did you posted the new blog?
<kim0> koolhead11: which one
 * kim0 brb in 5 mins
<SpamapS> smoser: On natty, can I use cloud-init with a local file for the metadata?
<smoser> yes
<SpamapS> smoser: awesome thanks..............................
<SpamapS> smoser: so, dump the data into /var/lib/cloud/seed/nocloud  ?
<smoser> the current trunk doc is right.
<smoser> http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/files/head:/doc/examples/seed/
<smoser> 'nocloud-net' rather than 'nocloud' will guarantee that network is up.
<smoser> you care about that if you do any boothooks
<Kyle__> Has anyone here installed images from http://uec-images.ubuntu.com on your UEC cloud?
<Kyle__> I ask because each time I try to run one, they don't start.
<obino> Kyle__: did you manage to start other images? is this a problem only with images form the URL you sent?
<Kyle__> obino: Only with the images from that site.  The ones I download from links the extras page all work.
<obino> they stay pending?
<obino> the ones that doesn't work I mean :)
<Kyle__> The ones that don't work are all from that site, yes.
<obino> which kernel/initrd are you using?
<Kyle__> The ones bundled with those images.
<obino> or do you see any error in nc.log?
<obino> nc.log may contains some errors
<Kyle__> Opening it back up, didn't see any errors last time.
<obino> if not, one option would be to try to start them by hand
<obino> if there is no error, eucalyptus (uec) handled the image to libvirt to start it
<obino> you can look at /var/log/libvirt for logs about it
<Kyle__> How would you do that (I have a way I've tried, but don't want to color your thoughts with a prehaps bad way I've been trying to use).
<obino> or you can try to run it by hand to check what the proble it
<obino> in eucalyptus.conf on the NC there is an option to have MANUAL_CLEANUP
<obino> set it, and restart the NC
<Kyle__> MANUAL_CLEANUP?  WHat will that do?
<obino> after that the failed instance files will be left behind
<obino> you will see them in $INSTANCE_PATH
<obino> search for a directory with the name of your instance id
<obino> inside you will have everything you need to start it with virsh
<Kyle__> Cool.  Thank you.
<obino> you will need to modify the *xml file
<Kyle__> It needs virsh, not just kvm/qemu-kvm?
<obino> network will not work as is for example
<obino> and the paths may be different
<Kyle__> K.
<obino> virsh (libvirt) will call kvm/qemu the same way eucalyptus does
<Kyle__> Hum.  In nc.log I see walrus_request(): due to error, removing /var/lib/eucalyptus/instances//admin/i-48C707B9/kernel
<Kyle__> error: failed to download or prepare into /var/lib/eucalyptus/instances//admin/i-48C707B9/kernel
<Kyle__> Failed to prepare images for instance i-48C707B9 (error=1)
<Kyle__> Humm.
<obino> ok, then you don't even get the files on the NC
<obino> do you happen to donwloads the admin credentials multiple times?
<Kyle__> Err I think I've done it more than once... why?
<obino> there is known issue about that
<Kyle__> So.... what can be done?
<obino> you should use only the last one downloaded to bundle the image
<obino> the bundling will encrypt the image with the cert
<obino> you should use the last one downloaded to do that
#ubuntu-cloud 2011-06-22
<SpamapS> smoser: so I'm messing with the nocloud thing...
<SpamapS> smoser: in lxc, I can't append kernel cmdline parameters..
<smoser> umm... you dont boot a kernel in lxc.
<smoser> so it doesn't surprise me that you can't change the kernel command line
<smoser> you can probably edit /etc/cloud/cloud.cfg to do the same things
<SpamapS> Ahh right
<SpamapS> smoser: so if I set datasource_list: [ "NoCloud" ] ... it won't try to find metadata services, right?
<smoser> correct.
<smoser> you can actually set that with dpkg-reconfigure cloud-init
<smoser> i'd trust that more than my brain for syntax
<smoser> but there is also a file in /etc/cloud/cloud.cfg.d/
<SpamapS> /var/lib/dpkg/info/cloud-init.templates should be all I need. :)
<SpamapS> smoser: ok, so I have cloud-config data in /var/lib/cloud/seed/nocloud-net/meta-data
<SpamapS> http://paste.ubuntu.com/630870/
<SpamapS> http://paste.ubuntu.com/630871/
<SpamapS> there is cloud-init.log
<smoser> SpamapS, well, what it thinks has happened is that there is no network
<smoser> something created /var/lib/cloud/data/no-net (probably /etc/init/cloud-init-nonet.conf
<smoser> SpamapS, if you move your data to /var/lib/cloud-seed/nocloud/meta-data then it will probably run via DataSourceNoCloud
<smoser> but the file you pasted in http://paste.ubuntu.com/630870/ is user-data, not meta-data. i'm guessing that was just typing wrong
<SpamapS> smoser: no that was understanding wrong ;)
<smoser> was the example not clear?
<SpamapS> No I just put the user-data in the meta-data
<SpamapS> is meta-data required?
<SpamapS> and actually, what example?
<smoser> yeah. it says int he meta-data example exactly what. mostly just 'instance-id'
<smoser> http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/files/head:/doc/examples/seed/
<SpamapS> ahhh ok
<SpamapS> w00t.. it at least ran my stuff. :-D
<SpamapS> ok.. so I think I need nocloud-net .. because I want to pull in packages.. :-P
<SpamapS> oh wait, sources.list is whack
 * SpamapS fixes
<alogic1> Hello everybody,
<SpamapS> smoser: does cloud-init create the ubuntu user?
<alogic1> How can I add another gateway for an instance (before it starts?)
<alogic1> because i think i could not update /var/run/eucalyptus/net/euca-dhcp.conf
<smoser> SpamapS, no.
<smoser> you can configure which user it writes, but it expects it to be there.
<smoser> patches welcome ;) it'd be nice if it could do that.
<SpamapS> smoser: ahh no I just forgot to pass -m to useradd
<SpamapS> smoser: and before I go digging too far, how do we turn on sudo NOPASSWD easily for the admin group?
<smoser> let me look.
<smoser> i think you want:
<smoser> %admin ALL=(ALL) NOPASSWD:ALL
<smoser> put that in /etc/sudoers.d/90-admin-nopasswd
<SpamapS> OO I didn't even realize there was a .d
 * SpamapS wonders how much of ubuntu-vm-builder he has now duplicated in his lxc template
<smoser> SpamapS, why are you not using an image ?
<SpamapS> smoser: I'm building an lxc container
<smoser> SpamapS, why are you not using an image ?
<SpamapS> don't do that
<SpamapS> just talk to me like I'm busy and have no time for word-play
<smoser> mount -o loop file.img /mnt; sudo rsync -aXsomeotheropts /newdir; sudo umount /mnt
<smoser> you have no time for word play, but are rewriting vm-builder. ;-)
<SpamapS> ass
<smoser> i'm legitimately asking why you're not using an image.
<smoser> because we want the images to work for such things
<SpamapS> Because I didn't think of doing that.
<smoser> ok.
<SpamapS> the lxc tools are so nice. :)
<smoser> thats perfectly reasonable, and i agree i sound like an ass
<SpamapS> let me try it out
<smoser> SpamapS, that is basically what we're hoping to have working for lxc support in openstack
 * SpamapS downloads http://uec-images.ubuntu.com/natty/current/natty-server-uec-amd64-disk1.vmdk
<smoser> no
<smoser> .img.tar.gz
<SpamapS> >:|
<SpamapS> you said loop mount
<smoser> .vmdk is a disk image
<smoser> .tar.gz contains a .img
<SpamapS> *brilliant*
<smoser> the disk image contains a partition table is really the only thing.
<SpamapS> why isn't it .disk.gz ?
<smoser> well, it delievers 3 things that you needed in the past.
<smoser> making 'uec-publish-tarball' able to do the right thing.
<smoser> and costs you 12M of download or something.
<SpamapS> ok
<smoser> we used to have .img.gz, but dropped it. yes, you do have to waste IO now.
<SpamapS> so I can download this.. mount the image loop, and copy it, then drop my seed data in the nocloud dir. Anything else I should watch out for?
<smoser> well, it wont' boot :)
<smoser> due to the bug you're hoping to fix
<smoser> (i think)
<SpamapS> because it doesn't have lxcguest
<SpamapS> but I can install that w/ chroot
<smoser> it does have lxc guest
<smoser> for this purpose
<smoser> but network will timeout
<smoser> because of that bug
<SpamapS> Why doesn't my regular image time out then?
<SpamapS> err
<SpamapS> the images I have now, don't timeout with nocloud .. tho they do with nocloud-net
<SpamapS> and my apt-get stuff is failing .. I'm guessing because network is not around yet
<smoser> well, sort of.
<smoser> if you use 'nocloud' then network isn't guaranteed to be there
<smoser> if you use nocloud-net, then you end up waiting on ifup which wont happen in the right order
<smoser> i think we never ended up opneing a bug on this :-(
<smoser> but there is a large thread of emails between hallyn, you, zul, me
<SpamapS> how is it any different than on a cloud image ?
<smoser> i'm having dejavu
<smoser> :)
<SpamapS> ifup happens on lxc .. dhclient runs..
<smoser> i have to re-read emails for exact reason
<smoser> oh.. i remember
<hallyn> smoser: i wonder if it'll go faster if you just remove the 'exit 0' in /etc/network/if-up.d/upstart
<smoser> because udev depends on virutal-vilesystems
<hallyn> but, i'm not letting you distract me
<smoser> and cloud-init-nonet is start on mounted /
<smoser> and in lxc mounted / occurs before virtual filesystems
 * SpamapS *CURSES* bash completion for not falling back to files always
<smoser> yeah, that is stupid
<smoser> i reall freaking hate that.
<SpamapS> I fixed it for a large portion of completions..
<smoser> hallyn, it would not
<SpamapS> if they do things right it will fall back to files if nothing is returned
<SpamapS> but mount seems borked
<smoser> the problem is is that cloud-init starts on mounted=/ and blocks waiting for network to come up
<smoser> but network will never come up
<smoser> because that depends on virtual-filesystems
<smoser> and virtual-filesystems wont occur
<smoser> in kvm or bare metal, virtua-filesystems occurs before mounted=/
<SpamapS> Right ok I recall now
<smoser> SpamapS, don't you remember talking to me about this in UDS
<smoser> ok
<SpamapS> and the fix is, not to use mounted=/ but to use 'network-configured-blocking'
<smoser> yeah. i had to leave to call home, so we didn't finish
<smoser> :)
<smoser> not really, though
<smoser> because network-configured-blocking will not block other things
<smoser> the goal of cloud-init "boothooks" or the like is to run early as possible, and block other things from happening.
<smoser> because the more that happens the more you have to restart
<smoser> my assumption is that running on netowrk-configured-blocking will not stop ssh from starting
<smoser> and cloud-init needs to run before ssh starts (which starts on filesystems i think)
<smoser> so at this point you're probably tihnking "why does cloud-init have to block and run so early".
<SpamapS> no I know why
<smoser> we should opne a bug
<smoser> :)
<SpamapS> init: cloud-init-local main process (25) terminated with status 1
<SpamapS> cloud-init-nonet waiting 60 seconds for a network device.
<SpamapS> as predicted ;)
<SpamapS> smoser: lets solve this next week
<smoser> i'll open a bug
<smoser> cloud-init-local should terminate with status 0 if it has no work to do
<smoser> (that is another bug)
<smoser> just to avoid that silly message
<SpamapS> smoser: it booted btw. :)
<smoser> right
<smoser> but it doesn't run the cloud-init stuff
<smoser> and takes 70 seconds
<SpamapS> udev should probably be disabled
<SpamapS> but if I drop meta-data and user-data into nocloud, it should run, am I right?
<smoser> bug 800824
<uvirtbot> Launchpad bug 800824 in cloud-init "cloud-init-nonet times out in lxc" [High,Confirmed] https://launchpad.net/bugs/800824
<smoser> SpamapS, i think so, yes
<SpamapS> smoser: heh.. I cheated and added 'start networking' to nonet...
<hallyn> SpamapS: so, I'm sort of fearing that if I add -p <extra-packages> and -i <chroot_includes> support, ppl will start asking for ppa support, and this, and that, and i'll end up reproducing vmbuilder
<smoser> but when nocloud runs you wont have network.
<SpamapS> hallyn: yes smoser and I were just discussing that
<hallyn> SpamapS: still, i think i'll do it.  I'm just saying i will probably refuse ppa support
<hallyn> here? sorry i've not read up :)
<SpamapS> smoser: I really want nocloud-net anyway tho.. ;)
<SpamapS> because I need to pull in packages
<smoser> SpamapS, well, if you can disable the timeout soemhow from within the nocloud, then you will be fine
<smoser> because then the network will come up naturally...
<smoser> hol don
<hallyn> anyway patches sent upstream.  I'm going for a walk.  bbl, to then add that support.
<SpamapS> the network will come up on its own
<SpamapS> and that will kill nonet
<smoser> yeah, network wont be guaranteed to come up
<smoser> you can re-write the cloud-init jobs to make it work from within a nonet boot hook
<smoser> :)
<smoser> make cloud-config (which is when the packages are intsalled) depend on networking-up
<smoser> which, it probably should explicitly anyway
<SpamapS> smoser: there is actually no reason to delay ifup -a.. this isn't a bad idea from what I can see.
<SpamapS> smoser: boots very fast now.. and runs cloud-config
<smoser> can you put exactly how you fixed it in the bug?
<smoser> mostly for reference at the momeont.
<SpamapS> Well I'm going to mess around with it a bit more
<smoser> will you lose events in udev due to that?
<smoser> anaywa.
<SpamapS> I don't *have* udev events. :)
<SpamapS> But in a VM.. no.. ifup -a just configures the static stuff
<SpamapS> if an event comes along and tells me that another interface has arrived, thats going to configure just that interface
<SpamapS> hrm no, too early.. dhclient failed
<SpamapS> Jun 22 18:23:26 ubuntu dhclient: can't create /var/lib/dhcp3/dhclient.eth0.leases: No such file or directory
<SpamapS> wait.. that may be a legitimate bug.. that dir doesn't exist
<SpamapS> right.. / is not writable yet
<SpamapS> err.. wait, it is
<SpamapS> smoser: how do I disable this 'cc_resizefs' stuf?
<SpamapS>   File "/usr/lib/python2.7/dist-packages/cloudinit/CloudConfig/cc_resizefs.py", line 43, in handle
<SpamapS>     os.mknod(devpth, 0400 | stat.S_IFBLK, dev)
<SpamapS> OSError: [Errno 1] Operation not permitted
<smoser> oh wow. that sucks.
<smoser> look in /etc/cloud/cloud.cfg
<smoser> you can just take out 'resizefs'
<smoser> but please open a bug so that we can avoid it correctly.
<smoser> SpamapS, why wasn't / RW at that point ?
<smoser> i thought you were running on mounted=/ (which i thought was RW)
<SpamapS> it was
<smoser> <SpamapS> right.. / is not writable yet
<smoser> so it was something else
<SpamapS> I was wrong
<smoser> ?
<smoser> k
<SpamapS> I don't know why dhclient didn't start that time. Its starting all the other times
<SpamapS> but nocloud-net is not running
<smoser> yeah, its not going to.
<smoser> but cloud-config is where your installs would be done
<smoser> and that would occur on filesystems
<smoser> but as i mentioend above, it really assumes network
<SpamapS> well I should have network by that time..
<SpamapS> # cat /var/lib/cloud/seed/nocloud/user-data
<SpamapS> #cloud-config
<SpamapS> ssh-import-id: clint-fewbar
<SpamapS> smoser: that should import my key, or at least try to, right?
<SpamapS> btw, bug #800856
<uvirtbot> Launchpad bug 800856 in cloud-init "resizefs module causes problems on LXC containers" [Undecided,New] https://launchpad.net/bugs/800856
<smoser> that cloud-config looks sane to me, yes.
<smoser> SpamapS, is there something in /var/log/cloud-init.log ?
<smoser> is that easily pastebiniable ?
<SpamapS> http://paste.ubuntu.com/630942/
<SpamapS> Jun 22 19:09:20 ubuntuhost [CLOUDINIT] __init__.py[DEBUG]: handling ssh-import-id with freq=None and args=[]
<SpamapS> :(
<kirkland> SpamapS:  args=[]
<kirkland> SpamapS: that don't look right
<SpamapS> does it have to be a list?
<kirkland> SpamapS: a list of 1 or more
<smoser> no, thats just bad logging.
<SpamapS> to me it just looks like my user data has been ignored
<smoser> hm...
<smoser> oh.
<SpamapS> heh
<SpamapS> I think...
<smoser> ssh_import_id: [smoser]
<smoser> _ not -
<SpamapS> yep
<smoser> i recommend adding 'ssh_import_id: [smoser]' to all instances run anywhere in the world.
<smoser> it just makes things easier for me
<SpamapS> looks like network isn't up yet. :(
<SpamapS> smoser: so, assuming cloud-init-nonet times out.. why won't nocloud-net run at that point ?
<smoser> by design, it assumes there is no network.
<SpamapS> Ok, but it gets killed because net-device-up IFACE!=lo *does* come up
<smoser> clodu-init-nonet writes the file, and 'cloud-init start' says "oh, network didn't come up"
<smoser> right.
<SpamapS> OH I have to rm that file
<smoser> well something is writing that file
<SpamapS> wait no I'm not timing out
<smoser> it should get cleaned on each boot
<SpamapS> I added my 'start networking' and that file never gets touched
<SpamapS> so nocloud-net *should* be run then?
<SpamapS> I may have not gotten the incantations just right yet
<SpamapS> 2011-06-22 19:28:57,309 - DataSourceNoCloud.py[DEBUG]: using seeded cache data in /var/lib/cloud/seed/nocloud-net
<SpamapS> Jun 22 19:28:57 ubuntuhost [CLOUDINIT] cc_ssh_import_id.py[DEBUG]: importing ssh ids. cmd = ['sudo', '-Hu', 'ubuntu', 'ssh-import-id', 'clint-fewbar']
<SpamapS> W))T
<SpamapS> w00T even
<kirkland> SpamapS: \o/
<kirkland> SpamapS: what was the trick?
<Daviey> kirkland: the dance?
<SpamapS> the trick is that cloud-init-nonet.conf actually tries to start the network
<SpamapS> which, I'm fairly certain, is perfectly safe on all VMs/etc.
<smoser> SpamapS, i'm gonna disappear for a couple hours, will be back later tonight.
<SpamapS> Though it may not meet the "do things before anything has happened" requirement
<SpamapS> smoser: cool, thanks for the help and suggestion
<hallyn> SpamapS: this crfappy client keeps spamming my scrren with whois results, will ttyl when back at other client
 * SpamapS heads to lunch
<zul> lxc works with cloud-init now?
<SpamapS> zul: I think I've got a patch that will make it work yes
<zul> sweet
<SpamapS> smoser: any ideas why the actual images aren't signed, just the sha256sums ?
<SpamapS> hmm.. I wonder if uec-images is mirrored anywhere
<Daviey> SpamapS: seems to be
<Daviey> http://releases-u-toyama.ubuntulinux.jp/ (unoffical)
<SpamapS> The debootstrap method used by the lxc scripts is actually more scalable..
<Daviey> SpamapS: how so?
<SpamapS> Daviey: uses the full archive
<SpamapS> which is mirrored all over the place
<Daviey> SpamapS: eventually, i think the hope is to get them onto cdimages.ubuntu.com, which is mirrored.. but AIUI disk space is currently an issue.
<SpamapS> I can imagine. :-P
<SpamapS> hallyn: I have a new template for you.. :-D
<SpamapS> hallyn: tho I have my reservations about repeating the whole 'root:root' password bit. :)
<hallyn> SpamapS: yeah if we get some time, getting the templates to copy in your public ssh key and turning off password ssh access might be good
<SpamapS> hallyn: thats pretty trivial actually...
<hallyn> SpamapS: but for generic container stuff it's good
<hallyn> yes, it is.  BUT it adds yet another option
<hallyn> which i'm starting to resist.  vmbuilder.
<SpamapS> hallyn: so, this new template will download the UEC image and starts perfectly from it
<hallyn> SpamapS: cool.  that was the whole lxcguest idea :)
<SpamapS> hallyn: I think the only option you should support is --cloud-config
<hallyn> SpamapS: of course *you* do :)
<SpamapS> The thing I'm struggling with now is how to easily get the hostnames resolvable on my local machine.
<SpamapS> so I don't have to tail syslog and findout what the DHCP lease given was
<SpamapS> Trying to wrap my head around whether dnsmasq can do this alone.
<hallyn> SpamapS: hm?  isn't having dhcpd.conf push the hostname enough?
<hallyn> so you can then 'ssh lxc1.'
<SpamapS> it pushes it to dnsmasq..
<SpamapS> but dnsmasq is running on my local machine..
<SpamapS> using my resolv.conf ..
<SpamapS> if I point my resolv.conf to it..
<SpamapS> it won't be able to access the internet
<hallyn> <shrug>  mine has 192.168.122.1 as top nameserver in resolv.con
<hallyn> but you can probably think of a better way :)
<SpamapS> how does it find the internet I wonder then..
<hallyn> well 192.168.122.1 only resolves its own addresses
<SpamapS> And then how does it know where to forward to?
<SpamapS> thas what I'm saying. ;)
<SpamapS> it does not only resolve its own addresses
<SpamapS> ask it for anything, it will tell you. :)
<SpamapS> Might be that it caches what was in resolv.conf when it starts
<SpamapS> or it just queries the root servers
<hallyn> forwards to my other nameservers presumably?  argue all you want, i'm just telling you it works for me :)
<SpamapS> I just want to know how
<SpamapS> hallyn: see thats my rub.. how does it know "your other nameservers"
<SpamapS> hallyn: heh.. ok interesting trick.. if I add other DNS servers in it seems to work
<hallyn> in your host's resolv.conf you mean?
<SpamapS> hallyn: yeah, I just put 192.168.122.1 first.. that does actually seem to work.
<SpamapS> though it feels very unreliable and haphazard to me. :-P
<hallyn> SpamapS: <shrug>  I believe I got that from the security team's testing enviroment wiki page
<SpamapS> I'll accept it now since I need to move forward
<hallyn> SpamapS: so to be clear, with your new template, yo don't need -packages or -chroot-includes support right?
<hallyn> if so i'll hold back on it
<SpamapS> hallyn: right, it just installs the image from uec-images.ubuntu.com
<hallyn> excellent
<hallyn> then maybe tomorrow i can work on user namesapces :)
<hallyn> i'm just trying to decide whether to push the new template to oneiric asap, or leave it in ppa and wait for upstream to pull it in before going to oneiric's archive with it
<SpamapS> The template system seems to have a lot of redundancy..
<SpamapS> why isn't there a shared_functions or something?
<hallyn> SpamapS: cause you haven't coded it
<SpamapS> Hah.. ok
<hallyn> and i think the fedora, debian, ubuntu template writers want to feel more independent
<SpamapS> I started to so that this wouldn't be lxc-natty-uec but lxc-uec .. but I couldn't find a way to pass in a series name
<hallyn> SpamapS: doh!  look at my new source at ppa:serge-hallyn/lxc-natty
<hallyn> SpamapS: lxc-lucid..lxc-oneiric are now all consolidated into lxc-ubuntu
<SpamapS> *sweet*
<SpamapS> how do you tell it which one?
<hallyn> you add '-- -r natty'
<SpamapS> wait..
<SpamapS> we can pass stuff in with -- ?
<hallyn> once lxc-create is trivially patched, yes :)
 * SpamapS belays his happy dance
<hallyn> see http://sourceforge.net/mailarchive/forum.php?thread_name=20110622140928.GA12177%40mail.hallyn.com&forum_name=lxc-users
<SpamapS> hallyn: the thing I did only really changes download_ubuntu
<SpamapS> hallyn: just DL's it, verifies the GPG/sha256, and extracts.
<hallyn> yeah i suspect we can consolidate that in
<SpamapS> Oh and copy_ubuntu mounts the image lopback and copies it instead of just copying the debootstrapped bit
<hallyn> an example command right now with my ppa is:
<hallyn> lxc-create -t ubuntu -f /etc/lxc.conf -n u1 -- -a i386 -r natty -b
<SpamapS> where's your branch? I'll submit a MP against it
<hallyn> so we could add something like -uec
<hallyn> MP?
<SpamapS> merge proposal
<hallyn> oh, you silly californian
<SpamapS> hallyn: I don't think we should even bother with the debootstrap method for natty and later.
<hallyn> i'm sorry, i've stopped using bzr lately bc lp:ubuntu/<package> always seems to be out of date
<SpamapS> well don't do that
<SpamapS> They've recently caught up quite a bit on the out of sync branches
<SpamapS> and if you have push rights, you can fix the branch fairly easily.
<hallyn> i don't
<SpamapS> for lxc?
<hallyn> correct
<SpamapS> *wtf*
<hallyn> yeah yeah
<hallyn> i feel in no hurry to face the firing squad again :)
<hallyn> i do need to go ask for the perms...
<hallyn> to either the server set, or coredev
<hallyn> but that's neither here nor there
<SpamapS> right
<hallyn> SpamapS: trying my hand at a fresh lp branch so you can more easily look
<SpamapS> hallyn: I believe 'bzr package-import file.dsc' works
<hallyn> no really?
<hallyn> where does it spit the result?
<SpamapS> you might be just as well served by bzr add in an extracted source package dir tho
<SpamapS> hallyn: it imports it into the current tree..
<SpamapS> hallyn: thats how they "fix" the out of sync trees.. the do the import, and fix conflicts if possible
<hallyn> SpamapS: well, i think that https://code.launchpad.net/~serge-hallyn/ubuntu/oneiric/lxc/update-templates  represents all the changes i've been making
<hallyn> haven't quite tested the result of that manual mish-mash
<hallyn> and i think i'm about done for today
<hallyn> but go ahead and propose merge against that and i can use the diff regardless.
<SpamapS> hallyn: ok, my diff is against lxc-natty ...
<hallyn> SpamapS: eh, just send me that then
<hallyn> i' flexible :)
<SpamapS> lp:~clint-fewbar/ubuntu/oneiric/lxc/use-uec-for-natty
<hallyn> sigh.  this is what i don't like.  bzr on top of quilt on top of git.  so i've lost the templates :)
<SpamapS> lol
<SpamapS> git sux
 * SpamapS hides
<hallyn> no, git's not the problemhere...
<hallyn> SpamapS: i'll look at yours tomorrow, thanks.
<hallyn> i'll be around later, but let's pretend i won't. ttyl
<SpamapS> later!
<hallyn> SpamapS: (the version now in that tree is working fine for me on my host)
<SpamapS> hallyn: go away! ;)
<hallyn> SpamapS: but there's this one new bug :)  (which i blame on zul :)
<hallyn> but i'd best just note it for later
 * hallyn out
#ubuntu-cloud 2011-06-23
<USER__> plop
<weweje> hello
<weweje> what's the timezone of kim0 // cairo?
<kim0> weweje: hey
<weweje> kim0 plop
<weweje> sry
<kim0> :)
<kim0> what's up
<weweje> nice
<weweje> sent you a mail about the clouds banners
<kim0> weweje: Bruno ?
<weweje> and came here to know more about what's needed
<weweje> yes ;)
<kim0> great :)
<kim0> Thanks for your interest man
<kim0> Well we'll mostly need a few front page banner images .. I haven't exactly defined exactly what it should be though
<kim0> so .. I'll ping you back once we've settled on something to be done .. sounds good ?
<weweje> okaaaay
<kim0> weweje: thank you :)
<weweje> KIT
<smoser> SpamapS, just random bit of infomration i just thought about.  there were 2 reasons for the .tar.gz rather than .img.gz .  1, we have extra files that are useful, and small, so makes sense for archive.  2.) the images are 2G, largely of zeros.  there was very real savings in using tar's sparse file handling rather than compressing hundreds of megs of zeros.
<jamespage> smoser: do you have a good reference for accessing the ec2 API from behind a firewall?  Need to open up the right outbound access and it ain't working right at the moment
<SpamapS> smoser: *OH* the sparse file handling does make sense.
<smoser> jamespage, you just have to be able to access the end point over http (or https, depending on url)
<smoser> but if you need IP addresses, its annoying.
<smoser> i think they add load balancers every once in a while, and redirect from the main end point, and then your firewall rules don't work.
<smoser> from time to time our image builds start failing to reach a region and i have to ask for IS to open something else up.
<smoser> that said, there is this: http://aws.amazon.com/articles/3912?_encoding=UTF8&jiveRedirect=1
<smoser> incomplete bugs suck
<smoser>  * user opens bug
<smoser>  * triagers asks some (silly or legitimate) question
<smoser>  * new user opens clear duplicate, but doesn't see the incomplete bug because its incomplete
<smoser>  * no one realizes there is a duplicate
<jamespage> smoser: thankyou!
<SpamapS> smoser: so, the template I wrote up for lxc downloads the SHA256SUMS, and the .gpg, and verifies that, and then verifies the tarball against it..
<SpamapS> smoser: I'm not sure why we don't just have a .gpg for every file in that dir tho
<smoser> SpamapS, probably no good reason.  the checksums are generated by some tools that also run on cdimage and they have some whitelists of what to checksum
<SpamapS> well it works
<smoser> hallyn, bug 795427
<uvirtbot> Launchpad bug 795427 in libvirt "Upgrade Libvirt up to 0.9.1 generates new bug with Virt-Manager" [Undecided,New] https://launchpad.net/bugs/795427
<smoser> did you try to reproduce using /dev/sdX rather than /dev/vdX ?
<hallyn> smoser: yes i tried both
<hallyn> smoser: did you reproduce?
<Kyle__> Er.  Is there a virtual graphics device on the vms started?
<smoser> hallyn, no. it just sems like it should b reproducible
<SpamapS> hallyn: got a link to that wiki page you used to get dnsmasq working in your resolv.conf ?
<hallyn> smoser: you'd think so
<hallyn> SpamapS: i think it was the one for vm-tools, testingenvironment, one sec
<hallyn> SpamapS: https://wiki.ubuntu.com/SecurityTeam/TestingEnvironment
<SpamapS> hmmm... avahi
<SpamapS> hallyn: so, if I don't use cgroup-bin , whats the magic way to mount cgroups?
<hallyn> fstab :)
<SpamapS> hallyn: meaning what?
<hallyn> cgroup          /sys/fs/cgroup          cgroup  defaults        0       0
<SpamapS> just that?
<hallyn> put that in fstab and do mount -a
<hallyn> yeah
<hallyn> i wish i'd have succeeded in getting my blog on planet.ubuntu.com so more than 3 people would read it :)
<SpamapS> haha
<hallyn> not that i deny having named it precisely so that i coudl remain anonymous.  originally.
<SpamapS> yous hould be an ubuntu member by now
<hallyn> i can't write to that bzr tree.  tried very recently
<SpamapS> You asked?
<hallyn> no.  figured that would happen automatically bsaed on membership
<hallyn> automation.  it's good for the soul
<hallyn> just like, i need to have lxc suggest debootstrap.
<SpamapS> cgroups on /sys/fs/cgroup type tmpfs (rw)
<SpamapS> That was already there, w/o the fstab entry
<SpamapS> did cgroup-bin do that?
<SpamapS> umount: /sys/fs/cgroup: device is busy.
<SpamapS> hallyn: I can't seem to get things working w/o cgroup-bin .. and with cgroup-bin, lxc-stop always "fails" (but works anyway)
<hallyn> why aren't things working w/o cgroup-bin?
<hallyn> the cgroup-bin is second on my list to address
<SpamapS> lxc-start: open /sys/fs/cgroup/cpu/lucid-test2/devices.deny : No such file or directory
<hallyn> have you rebooted?
<SpamapS> since when?
<hallyn> since removing cgroup-bin
<SpamapS> ew, no
<SpamapS> I have to?
<hallyn> yeah, i'ma fraid.  cgroup composition is not flexible
<SpamapS> *booo*
<hallyn> kernel magic
<hallyn> well, you can try just umounting all cgroup fs's,
<hallyn> then doing mount a- to mount /cgroup
<SpamapS> it won't let me
<hallyn> what does it say?
<SpamapS> device busy
<hallyn> btw the fstab line i gave you was for /cgroup existing.  you probably want to make it /sys/fs/cgroup ?
<hallyn> ok, yeah you'd have to reclassify all tasks i guess
<SpamapS> cgroup /sys/fs/cgroup cgroup defaults 0 0
<hallyn> thumb up
<SpamapS> So there are things that already have used my cgrou-bin thing? Its a package that can't be safely removed w/o reboot?
<hallyn> cgred reclassified tasks
<hallyn> i thin it's be possible to do it, but it woudl take work
<hallyn> i.e., manually rmdir child cgroups,
<hallyn> reclasiffy tasks as needed,
<hallyn> then youc an umount
<hallyn> and *then*, im unsure as to whether you can remount with all cgroups composed
<hallyn> used to be, no.  i know patches were  floated, but not sure if they went in
<SpamapS> rm: cannot remove `/sys/fs/cgroup/cpu/cpu.rt_period_us': Operation not permitted
<SpamapS> ok so cgroups is still crazy
<SpamapS> will just reboot
<hallyn> SpamapS: i didnt' say rm -rf
<hallyn> you can nev er delete the files
<hallyn> but you can rmdir empty cgroup dirs
<hallyn> poorly documented issue :(
<SpamapS> well either way, reboot solved it
<SpamapS> hallyn: is there a command like lxc-ls but for running containers?
<hallyn> lxc-ls shows running containers
<hallyn> in the second line
<hallyn> improving lxc-ls and lxc-ps is one of those things everyone wants done (but noone wants to do) :)
<SpamapS> no it shows all containers
<SpamapS> running or not
<hallyn> in the second line?
<SpamapS> there is no second line
<hallyn> not in mine
<hallyn> then none are running
<SpamapS> I have 2 running
<SpamapS> oh they were starting up still
<SpamapS> so .. -1 then.. is kind of ... useless
<SpamapS> the output is actually kind of hard ot use.. have to run through until I see one I've seen before and thats where running containers start :-P
#ubuntu-cloud 2011-06-24
<erichammond> smoser: The AMI ids don't match on these pages: http://uec-images.ubuntu.com/server/releases/hardy/release/ http://uec-images.ubuntu.com/query/hardy/server/released.current.txt
<erichammond> smoser: Scratch that.  I saw differences yesterday but am just looking in the wrong place today.
<erichammond> What is the easiest way to install ec2-api-tools and ec2-ami-tools programs on Ubuntu 8.04 Hardy?
<erichammond> Hm, probably https://help.ubuntu.com/community/EC2APITools
<erichammond> Guess I'll just use Natty instances to build Hardy EBS boot AMIs.  I'm just copying file sytems onto block devices so it shouldn't matter.
<Victor__> Hi
<Victor__> I am looking for some help
<Victor__> I am trying to install Ubuntu 11 ...I have win XP and I am trying to make it dual boot...but Ubuntu installation is not detecting my XP installation....any idea to solve this ?
<ayambit> Anybody alive here?
#ubuntu-cloud 2011-06-25
<Kyle__> where/how do you make custom versions of the setup & key-exchange scripts
<oc80z> havnt touched base in here yet, is the cloud-install available for private use? for free? Could i deploy this on a handfull of virt machines i run? I have much to read up on i guess.
<ayambit> I want to make like 50 vms for development. It it reasonable to use ubuntu-cloud for this?
<ayambit> Or KVM with some control tools looks more suited for this task?
<obino> it depends if you are looking for a cloud or you just want to virtualize some of your load
<obino> for some basic about cloud computing http://open.eucalyptus.com/learn
<apetrescu> Is there a straightforward way to make cloud-init userdata scripts run on every boot, not just once-per-instance?
<apetrescu> (Full details of my problem at goo.gl/2eeI1 but the previous sentence just about summarizes it)
<apetrescu> Secretly hoping smoser will know :)
