#ubuntu-cloud 2011-04-25
<trackerx90> hi there
<trackerx90> i have problem with UEC setup
<trackerx90> my node can't detect cluster
<trackerx90> why? what wrong with me
<rp_> kim0, hey
<rp_> cloud-init does that allow you to specify the versions of the packages as well?
<rp_> i can't find good documentatiokn on it
<SpamapS> hallyn: actually, ping here. ;)
<hallyn> hm?
<SpamapS> smoser: you too.. want to chat about lxc / mountall / cloud-init :)
<hallyn> sigh, ping me when smoser answers, i wanna rmemeber where i kept the selinux testsuite
<smoser> here now.
<smoser> SpamapS, we can chat, but i donht have a ton of time to talk
<SpamapS> smoser: is there a good time?
<smoser> no.
<smoser> so now is fine
<SpamapS> Doh. Ok.
<SpamapS> So I think I understand why you want to block mountall from moving on after virtual-filesystems ..
<SpamapS> so that the network isn't brought up...
<SpamapS> am I right?
<smoser> no.
<hallyn> SLAM
<SpamapS> da da da, let the boys be boys! SLAM!
 * SpamapS channels Onyx
<smoser> cloud-init takes input from a user (user-data in ec2, alternatively, can take it from filesystem or ISO transport for ovf)
<smoser> cloud-init-local searches for data that was provided without needing a network
<smoser> cloud-init searches for data that would come from a network source
<smoser> so, cloud-init needs network
<smoser> that make sense?
<SpamapS> ack
<smoser> cloud-init uses this data from a user to modify the fileystem (or other things)
<smoser> the basic goal is to allow everything (or as close as possible) that you could do via rebundle to be done via user-data
<smoser> in order to do certain things, say write an apache config, you have to make sure that apache isn't going to start while you're in the middle
<hallyn> so do you or do you not want to block certain upstart jobs
<hallyn> hm
<smoser> i want to block system boot
<smoser> until i'm done
<hallyn> what does apache wait on?
<smoser> probably filesystems
<smoser> but it doesn't matter
<smoser> ssh starts on filesystems.
<hallyn> yes it does
<smoser> no
<SpamapS> smoser: you probably then just need to have a "wait" job that encompasses the 'starting' events of everything you want to stop.
<smoser> i want to block as much of the system as i possibly can
<hallyn> yes, and SpamapS and i are trying to figure out how to do that
<smoser> so when cloud-init runs, i want everythign to stop.
<smoser> SpamapS, i dont know what i want to stop
<SpamapS> so 'start on starting' and block *EVERYTHING*
<smoser> that depends upon what the user is doing
<hallyn> i think you want to stop filesystem and rc
<SpamapS> as hallyn found, you can 'start on starting' :)
<hallyn> but then again, maybe this just isn't feasible
<SpamapS> or maybe that was jhunt I forget
<smoser> i can't start on starting
<SpamapS> hallyn: filesystem is not "stoppable"
<smoser> because at starting there is not neceissarily a network
<hallyn> the events after all arent' clearly defined
<hallyn> maybe we should move to #upstart?
<smoser> sure.
<smoser> basic goal:
<hallyn> smoser: and for what you want, it doesn't suffice to just make your changes and then reboot, right?
<smoser>  * run as early as possible in boot
<SpamapS> smoser: you can put a gate in your job that whitelists the things you want to let start.
<smoser>  * stop all other things from happening until you're done
<smoser> hallyn, rebooting is not desireable.
<smoser> SpamapS, and then that whitelist will start to break when other people modify things.
<hallyn> smoser: is there a clearly defined list of services you want to be able to intercept?
<smoser> hallyn, no.
<smoser> "all things"
<SpamapS> smoser: well we can define an environment variable that people can use to be auto whitelisted..
<SpamapS> smoser: or you can even add your job *after* the network comes up
<hallyn> smoser: SpamapS: how about just blocking everything and manually bringing up the network?
<SpamapS> hallyn: need /var/run but yeah thats doable.
<hallyn> the upstart jobs doing that are not exctly rocket science
<smoser> hallyn, its not really doable.
<smoser> how am i to know when the network is up?
<smoser> i need udev
<smoser> that needs virutal-filesystems
<smoser> i need to go arbitrarily loading modules and ifuping
<smoser> blah
<SpamapS> Ok so that does sound complicated.
<hallyn> well you have the advantage of being on less varying virtual hardware i suppose
<smoser> there is a well designed (that might be arguable) system that does that stuff.
<hallyn> we coudl discuss changing mountall?
<hallyn> SpamapS: do you grok mountall's code?
<SpamapS> Simpler is to just check for the state and if the state you want has been reached, block
<SpamapS> hallyn: yes
<SpamapS> but I'm not really sure why mountall would solve anything
<hallyn> ok, so can we block on mounted MOUNTPOINT=/, but teach mountall to keep doing only virtual-filesystems?
<SpamapS> hallyn: no
<hallyn> why not?
<smoser> because mountall should figure out that it can mount /dev/pts , and should do that before /. then that woudl solve the problem.
<smoser> or it just needs smarter ordering
<SpamapS> Because that event is *guaranteed* to block mountall until it completes
<SpamapS> its a sync point
<smoser> it *does* have specific ordering rules it knows about.
<hallyn> so come up with a new event
<smoser> it knows that it can mount /dev without having / as rw, for example
<hallyn> SpamapS: explain to me where mountall gets its list of virtual filesystems
<hallyn> it's not /etc/fstab.  it's not /lib/init/fstab
<SpamapS> hallyn: haha .. from the place where thar be dragons ;)
<hallyn> it appears to waste time spawning jobs for already-mounted filesystems
<hallyn> SpamapS: let me rephrase
<hallyn> SpamapS: there is in fact NOTHING for mountall to do.  It only thinks there is
<hallyn> (in the lxc case)
<smoser> so... the mountall solution seems the easiest at the moment. but it probably solves a specific problem in a specific way.
<hallyn> for that matter, we coudl simply have cloud-init fire off virtual-filesystems once mounted=/ is done.  it'd be technically wrong, but practically not.
<SpamapS> I think you're reading too much into mountall's influence on the boot process.
<smoser> no we're not.
<hallyn> no,
<hallyn> i'm reading the output of 'mountall --debug'  :)
<hallyn> it thinks it wants to mount 9 virtual filesystems, when in fact all are already mounted
<SpamapS> Thats a sure fire way to get fooled into a race tho..
<SpamapS> just because it happens in this order now doesn't mean it will continue to happen that way.
<SpamapS> how does that matter at all in this cloud-init case?!
<smoser> SpamapS, there is already hard coded things such as that in mountall
<hallyn> SpamapS: faking virtual-filesystems i was just mentioning to make a point :)
<smoser> it wouldn't to me seem all that insane that mountall would try to mount all virtual filesystems before mounting / rw
<smoser> basically asusming that all virtual filesystems can be mounted immediately
<hallyn> SpamapS: wha ti'm saying is that mountall could be smarter and faster about the virtual-filesystems to help us out
<hallyn> well, or starting off by checking that they are already mounted,
<SpamapS> Ok.. well I feel that there are two very different things and I don't understand the focus of the converstaion. cloud-init needs to have the system state a bit frozen so it can reliably change things.. and virtual-filesystems isn't as meaningful as it could be.
<hallyn> and not tryign ot be smarter than the sysadmin.  If I take them ou tof /lib/init/fstab, then I mean it.
<SpamapS> I think adding a different hook point to mountall is totally doable to solve the cloud-init issue..
<SpamapS> changing virtual-filesystems seems a bit off that mark though
<smoser> how would you have a different hook point?
<SpamapS> (even if its a good idea)
<SpamapS> smoser: emit another event that is waited on.
<SpamapS> one that is less accidental than "mounted MOUNTPOINT=/"
<SpamapS> deliberately emit something at the exact point in the mountall process that achieves your goal of being able to change things without having them pulled out from under you.
<hallyn> SpamapS: sounds good
<hallyn> smoser: ^ that's basically exactly what you want
<SpamapS> but there may be other things happening in parallel at that point and those need addressing.
<hallyn> though i can see mountall authors complaining
<hallyn> (is htat sjr?)
<SpamapS> yeah its Keybuk
<smoser> right.
<SpamapS> and jhunt maintains it too
<smoser> that seems like a specific solution to a specific problem
<hallyn> SpamapS: but again, i would argue that saying 'wait on X and Y' where X and Y both are issued by mountall, should be interpreted as 'the later of the two'
<hallyn> SpamapS: that would keep mountall from having to change for cloud-init's case
<SpamapS> hallyn: heh.. and that, sir, is upstart 2.0
<hallyn> smoser: yup, but a workable one
<hallyn> SpamapS: how far down the pike is that?
<smoser> i dont think its really workable
<SpamapS> hallyn: and we can do that in a wait job now.. I just haven't figured out what X and Y are yet.
<smoser> i tihnk its an ugly hack
<hallyn> smoser: that's because you are difficult and contrarian :)
<hallyn> SpamapS: x == mounted MOUNTPOINT=/, y == virtual-filesystems
<SpamapS> hallyn: upstart 2.0 is an unknown for us. When keybuk has time.
<smoser> i am. i agree.
<SpamapS> hallyn: except virtual-filesystems is not a blocker.. so I don't think it does what you want.. hence the need for a new hook point
<smoser> but i would suggest that so is keybuk.
<hallyn> smoser: that's the 'difficult' part of you talking :)
<SpamapS> I think its fair to ask for a 'mounted-virtual-filesystems' hook point.
<SpamapS> its a fairly unique point in the filesystem lifecycle that would be helpful for other bits of the boot as well.
<SpamapS> I wonder tho
<SpamapS> would start on stopping networking work too? Thats basically where ifup -a has exitted.
<SpamapS> or do you need to go before ifup -a ?
<smoser> can't go before.
<smoser> well, not for data that comes from a network
<smoser> cloud-init-local does that.
<smoser> but, yeah, via the non-network inputs, you can seed network information.
<SpamapS> smoser: so start on stopping networking is after 'ifup -a' has exitted..
<SpamapS> smoser: unfortunately that won't block rc.. but.. we can make that block too
<SpamapS> meh
<SpamapS> I need a whiteboard and some coffee
<hallyn> SpamapS: btw, there is going to have to be (maybe there already is?) a way to get info on which events are emitted blocking/nonblocking.
<hallyn> asking SpamapS on irc is not going to be scalable :)
<SpamapS> hallyn: the official ones are documented well
<SpamapS>        The  mounted  event  is  generated  by  the mountall(8) daemon after it has mounted a filesystem.  mountall(8) will wait for all services
<SpamapS>        started by this event to be running, all tasks started by this event to have finished and all jobs stopped by this event  to  be  stopped
<SpamapS>        before continuing with other filesystems.
<SpamapS> also the table in 'man upstart-events' is quite clear
<SpamapS> Type, S == signal (not waited on) H == hook point (waited on)
<SpamapS>        The  virtual-filesystems  event  is  generated by the mountall(8) daemon after it has mounted all virtual filesystems listed in fstab(5).
<SpamapS>        mountall(8) emits this event as an informational signal, services and tasks started or stopped by this event will do so in parallel  with
<SpamapS>        other activity.
<SpamapS> So.. really.. I hate to say "RTFM" .. but.. RTFM. ;)
<hallyn> SpamapS: the FM is lying though
<hallyn> i've said it 3 times now, so will wait to see if you know what i'm saying
<SpamapS> which part is lying there?
<hallyn> fing fstab
<SpamapS> Ok so thats a bug.
<hallyn> oh, is it?
<hallyn> i thought it was undocumented but intended black magic
<SpamapS> in the man page yes
<hallyn> ah
<hallyn> ok
<hallyn> in any case i should read the upstart man page :)
<SpamapS> AFAIK, mountall was thrown together while Keybuk was frantically working on boot speed
<hallyn> and upstart?
<hallyn> :)
<zul> hey now play nice
<hallyn> SpamapS: but so can you explain to me what it *is* doing?
<hallyn> zul: that wasn't meant as a slight.
<SpamapS> hallyn: you're not going to be satisfied by any explanation. Its a hack. period.
<hallyn> that's fine.  but a hack to do what?
<hallyn> it just guesses at the virtuals so as not to have to read fstab beforehand?
<zul> hallyn, i know...i was just teasing
<hallyn> zul: just trying to make sure noone comes by and challenges me to do better :)
<SpamapS> hallyn: its a hack to make booting a non blocking independent thread
<SpamapS> hallyn: the guessing, I suspect, is more about not having all the fs's listed in fstab
<hallyn> SpamapS: i thought that was the point of /lib/init/fstab
<hallyn> SpamapS: so wait, did we decide that wait on stopped networking and started rc will work?
<SpamapS> err no
<SpamapS> I was thinking about it
<SpamapS> here's one issue..  rc starts on net-device-up IFACE=lo .. which is *before* stopped networking
<SpamapS> so you need to have a  'starting rc RUNLEVEL=[2345]' too
<SpamapS> might work better to have two separate jobs that do that, and both wait on cloud-init-whatever to finish
<mattbillenstein> hi all
<mattbillenstein> question â when a user-data script runs, where is the output logged?
<mattbillenstein> I've found the script in /lib/cloud/data/scripts/part-000
<mattbillenstein> but I can't seem to find any logs
<mattbillenstein> hmm, just found it on the console...
<aka_> yo
<aka_> somebody here ?
<aka_> D:
<aka_> $this = NOTCOOL
<aka_> ;
<smoser> mattbillenstein, you will have to redirect the output of your script somewhere other than console if you want it there.
<smoser> in natty's cloud-init there is cloud-config syntax for doing that.
<smoser> in anything else, you'll have to handle that in whatever language the script is in
<smoser> ie:
<smoser> #!/bin/sh
<smoser> {
<smoser>  # my script here
<smoser> }
<smoser> # oops
<smoser> } 2>&1 | tee /var/log/my.log
<mattbillenstein> hmm
<mattbillenstein> you can do that in bash?
<smoser> yeah.
<smoser> { ... }
<smoser> does redirection
<smoser> or.. command grouping
<smoser> then the 2>&1 does the redirection
<smoser> or, if you don't care about it going to the console then just do:
<smoser> exec > /var/log/my.log 2>&1
<smoser> at the beginning
<mattbillenstein> sweet
<mattbillenstein> I'll try it out, thanks for the tip!
<aka_> yo
<spicyWith> Hi, I'm running ubuntu 10.04 and would like to upgrade it to the latest distribution. What's the recommended way to do this?
<erichammond> spicyWith: See if this applies to your situation: http://ubuntu-smoser.blogspot.com/2011/02/migrating-to-pv-grub-kernels-for-kernel.html
<spicyWith> erichammond: I was looking at that, and to be honest, I'm not sure. How can I check?
#ubuntu-cloud 2011-04-26
<erichammond> spicyWith (departed): Rather than address me (I'm in and out) spell out the whole question so that anybody on the channel can respond and provide support.
<hallyn> kim0: I need to go run errands, but new updated spice packages are at ppa:serge-hallyn/spice2.  If you have time, please give them a shot!
<hallyn> kim0: all right, this time for real :)  when it's done building, this version of kvm-spice is working for me (from local sbuild)
#ubuntu-cloud 2011-04-27
<rephormat> Good morning everyone.
<rephormat> Has anyone used the UEC Beginners Guide by CSS Corp Open Source Services?
<hallyn_afk> smoser: any chance I could get you to test out myproposed fix for bug 576501?  (Since I think I"ve heard that you use your own private version to build ec2 images)
<uvirtbot> Launchpad bug 576501 in vm-builder "vmbuilder doesn't recognize --ec2 flag" [High,In progress] https://launchpad.net/bugs/576501
<Kyle__> Are there any issues so far with running UEC on 11.04(beta I know).
<smoser> Kyle__, there are no know issues to really mention.
<smoser> the release will occur thursday
 * Kyle__ nods
<Kyle__> I thought so, just wanted to make sure.
<Kyle__> 11 is much happer on mac minis than 10, hence wanting 11.
<smoser> just figured i'd mention this, i just found out about it, and it rocks
<smoser> http://aws.typepad.com/aws/2011/04/aws-management-console-bookmarking.html
<smoser> http://uec-images.ubuntu.com/releases/lucid/release/ uses it now
<erichammond> smoser: sweet!
<erichammond> smoser: I've added run AMI links to the table at the top of http://Alestic.com (which already takes its recent Ubuntu AMI ids from your query API (with 10 minute cache for performance)
<erichammond> I started by opening the AWS console in the current window, but then noticed that Amazon breaks the Back button on the browser, which is a worse sin than opening a link in a new window.
<erichammond> smoser: it would be cool to have a redirect URL for the current released AMI given the required parameters.
<erichammond> For example, http://uec-images.ubuntu.com/aws-console/launch-ami?region=ap-northeast-1&arch=32&root=ebs
<erichammond> could redirect to: https://console.aws.amazon.com/ec2/home?region=ap-northeast-1#launchAmi=ami-5c0fa45d
<erichammond> er, &release=lucid
<smoser> erichammond, yeah. that' dbe nice.  i have no dynamic stuff ther eright now.
<erichammond> smoser: hm, you could use the Apache .asis module and write a bunch of files like /aws-console/launch-ami/lucid/ap-northeast-1/32-bit/ebs/index.asis
<erichammond> Add "index.asis" as a directory index option and drop the redirect HTTP headers into the file.
<erichammond> All static and fast.
<erichammond> http://httpd.apache.org/docs/current/mod/mod_asis.html
<erichammond> http://httpd.apache.org/docs/current/mod/mod_dir.html
#ubuntu-cloud 2011-04-28
<koolhead11> hi all
#ubuntu-cloud 2011-04-29
<flaccid> just released, Ubuntu 11.04 AMIs + RightScale MCIs: http://support.rightscale.com/21-Community/RightScale_OSS#Ubuntu_MultiCloud_Images
<mkrieger> hey folksâwe're seeing periodic freeze-ups of about 3 seconds on our ec2 (m2.4xlarge) instances, running 10.04 with the 2.6.32-314-ec2 kernel. we've tried moving it to an entirely different box (with new EBS drives) with no luckâsame problems result. has anyone seen something similar? During that time, no IO to the network happens, which means all the Postgres queries time out; sar shows that no IO to any of the EBS drives
<mkrieger> occurs during those seconds
<chetan-> anyone notice issues after a dist-upgrade on 10.04 recently?
<chetan-> my ec2 boxes are becoming unresponsive
<RoyK> is it currently possible to setup a sort of "mesh" of servers, as it is with vmware or hyper-v, without a central box (and thus single point of failure) in front?
<erichammond> mkrieger:Are any EBS snapshots initiated or in progress at the time of the freeze?
#ubuntu-cloud 2011-04-30
<mkrieger> erichammond: great q; no snapshots happen on that box (we snapshot from a replica)
<nOStahl> hi guys, I'd like to setup ubuntu cloud tonight, I have three IBM Thinkcentre towers
<nOStahl> they do not support VT thoughso im told I need to use xen?
<nOStahl> anyone have a guide for setting this up on 11.04 server
<RoyK> hi all. is it currently possible to setup a sort of "mesh" of servers, as it is with vmware or hyper-v, without a central box  (and thus single point of failure) in front?
#ubuntu-cloud 2012-04-23
<Lionthinker> question, does anyone have problems with U1 on windows?
<jcastro> jussi: I guess not
<jussi> jcastro: yeah, doesnt look like it
<jussi> hang on a sec
<jcastro> <3
<jussi> right, looks like kim0 is the owner.
<jussi> he still around?
<jcastro> he is long gone
<aoeui> anyone else getting this on precise? W: Failed to fetch bzip2:/var/lib/apt/lists/partial/us-east-1.ec2.archive.ubuntu.com_ubuntu_dists_precise_universe_source_Sources  Hash Sum mismatch
<clubberx> Hi, need to build a production AWS server ASAP as problems an existing one is having problems. If I use the latest Beta images will these upgrade fully to GA release on Thursday? or am I better to wait with gritted teeth and use the GA image.
<clubberx> Probably will not actually be in production until post Thursday but needs building up..
* Pici changed the topic of #ubuntu-cloud to: This channel has been consolidated with #ubuntu-server
