[05:03] <trackerx90> hi there
[05:03] <trackerx90> i have problem with UEC setup
[05:03] <trackerx90> my node can't detect cluster
[05:03] <trackerx90> why? what wrong with me
[16:07] <rp_> kim0, hey
[16:14] <rp_> cloud-init does that allow you to specify the versions of the packages as well?
[16:14] <rp_> i can't find good documentatiokn on it
[16:37] <SpamapS> hallyn: actually, ping here. ;)
[16:37] <hallyn> hm?
[16:37] <SpamapS> smoser: you too.. want to chat about lxc / mountall / cloud-init :)
[16:38] <hallyn> sigh, ping me when smoser answers, i wanna rmemeber where i kept the selinux testsuite
[16:39] <smoser> here now.
[16:39] <smoser> SpamapS, we can chat, but i donht have a ton of time to talk
[16:40] <SpamapS> smoser: is there a good time?
[16:40] <smoser> no.
[16:40] <smoser> so now is fine
[16:40] <SpamapS> Doh. Ok.
[16:41] <SpamapS> So I think I understand why you want to block mountall from moving on after virtual-filesystems ..
[16:41] <SpamapS> so that the network isn't brought up...
[16:41] <SpamapS> am I right?
[16:42] <smoser> no.
[16:42] <hallyn> SLAM
[16:42] <SpamapS> da da da, let the boys be boys! SLAM!
[16:42]  * SpamapS channels Onyx
[16:43] <smoser> cloud-init takes input from a user (user-data in ec2, alternatively, can take it from filesystem or ISO transport for ovf)
[16:43] <smoser> cloud-init-local searches for data that was provided without needing a network
[16:44] <smoser> cloud-init searches for data that would come from a network source
[16:44] <smoser> so, cloud-init needs network
[16:44] <smoser> that make sense?
[16:44] <SpamapS> ack
[16:44] <smoser> cloud-init uses this data from a user to modify the fileystem (or other things)
[16:45] <smoser> the basic goal is to allow everything (or as close as possible) that you could do via rebundle to be done via user-data
[16:45] <smoser> in order to do certain things, say write an apache config, you have to make sure that apache isn't going to start while you're in the middle
[16:45] <hallyn> so do you or do you not want to block certain upstart jobs
[16:45] <hallyn> hm
[16:45] <smoser> i want to block system boot
[16:45] <smoser> until i'm done
[16:46] <hallyn> what does apache wait on?
[16:46] <smoser> probably filesystems
[16:46] <smoser> but it doesn't matter
[16:46] <smoser> ssh starts on filesystems.
[16:46] <hallyn> yes it does
[16:46] <smoser> no
[16:46] <SpamapS> smoser: you probably then just need to have a "wait" job that encompasses the 'starting' events of everything you want to stop.
[16:46] <smoser> i want to block as much of the system as i possibly can
[16:46] <hallyn> yes, and SpamapS and i are trying to figure out how to do that
[16:46] <smoser> so when cloud-init runs, i want everythign to stop.
[16:47] <smoser> SpamapS, i dont know what i want to stop
[16:47] <SpamapS> so 'start on starting' and block *EVERYTHING*
[16:47] <smoser> that depends upon what the user is doing
[16:47] <hallyn> i think you want to stop filesystem and rc
[16:47] <SpamapS> as hallyn found, you can 'start on starting' :)
[16:47] <hallyn> but then again, maybe this just isn't feasible
[16:47] <SpamapS> or maybe that was jhunt I forget
[16:47] <smoser> i can't start on starting
[16:47] <SpamapS> hallyn: filesystem is not "stoppable"
[16:47] <smoser> because at starting there is not neceissarily a network
[16:47] <hallyn> the events after all arent' clearly defined
[16:48] <hallyn> maybe we should move to #upstart?
[16:48] <smoser> sure.
[16:48] <smoser> basic goal:
[16:48] <hallyn> smoser: and for what you want, it doesn't suffice to just make your changes and then reboot, right?
[16:48] <smoser>  * run as early as possible in boot
[16:48] <SpamapS> smoser: you can put a gate in your job that whitelists the things you want to let start.
[16:48] <smoser>  * stop all other things from happening until you're done
[16:49] <smoser> hallyn, rebooting is not desireable.
[16:49] <smoser> SpamapS, and then that whitelist will start to break when other people modify things.
[16:49] <hallyn> smoser: is there a clearly defined list of services you want to be able to intercept?
[16:49] <smoser> hallyn, no.
[16:49] <smoser> "all things"
[16:50] <SpamapS> smoser: well we can define an environment variable that people can use to be auto whitelisted..
[16:50] <SpamapS> smoser: or you can even add your job *after* the network comes up
[16:50] <hallyn> smoser: SpamapS: how about just blocking everything and manually bringing up the network?
[16:50] <SpamapS> hallyn: need /var/run but yeah thats doable.
[16:50] <hallyn> the upstart jobs doing that are not exctly rocket science
[16:50] <smoser> hallyn, its not really doable.
[16:51] <smoser> how am i to know when the network is up?
[16:51] <smoser> i need udev
[16:51] <smoser> that needs virutal-filesystems
[16:51] <smoser> i need to go arbitrarily loading modules and ifuping
[16:51] <smoser> blah
[16:51] <SpamapS> Ok so that does sound complicated.
[16:51] <hallyn> well you have the advantage of being on less varying virtual hardware i suppose
[16:51] <smoser> there is a well designed (that might be arguable) system that does that stuff.
[16:52] <hallyn> we coudl discuss changing mountall?
[16:52] <hallyn> SpamapS: do you grok mountall's code?
[16:52] <SpamapS> Simpler is to just check for the state and if the state you want has been reached, block
[16:52] <SpamapS> hallyn: yes
[16:52] <SpamapS> but I'm not really sure why mountall would solve anything
[16:52] <hallyn> ok, so can we block on mounted MOUNTPOINT=/, but teach mountall to keep doing only virtual-filesystems?
[16:53] <SpamapS> hallyn: no
[16:53] <hallyn> why not?
[16:53] <smoser> because mountall should figure out that it can mount /dev/pts , and should do that before /. then that woudl solve the problem.
[16:53] <smoser> or it just needs smarter ordering
[16:53] <SpamapS> Because that event is *guaranteed* to block mountall until it completes
[16:53] <SpamapS> its a sync point
[16:53] <smoser> it *does* have specific ordering rules it knows about.
[16:53] <hallyn> so come up with a new event
[16:53] <smoser> it knows that it can mount /dev without having / as rw, for example
[16:53] <hallyn> SpamapS: explain to me where mountall gets its list of virtual filesystems
[16:53] <hallyn> it's not /etc/fstab.  it's not /lib/init/fstab
[16:54] <SpamapS> hallyn: haha .. from the place where thar be dragons ;)
[16:54] <hallyn> it appears to waste time spawning jobs for already-mounted filesystems
[16:54] <hallyn> SpamapS: let me rephrase
[16:54] <hallyn> SpamapS: there is in fact NOTHING for mountall to do.  It only thinks there is
[16:54] <hallyn> (in the lxc case)
[16:54] <smoser> so... the mountall solution seems the easiest at the moment. but it probably solves a specific problem in a specific way.
[16:54] <hallyn> for that matter, we coudl simply have cloud-init fire off virtual-filesystems once mounted=/ is done.  it'd be technically wrong, but practically not.
[16:55] <SpamapS> I think you're reading too much into mountall's influence on the boot process.
[16:55] <smoser> no we're not.
[16:55] <hallyn> no,
[16:55] <hallyn> i'm reading the output of 'mountall --debug'  :)
[16:55] <hallyn> it thinks it wants to mount 9 virtual filesystems, when in fact all are already mounted
[16:55] <SpamapS> Thats a sure fire way to get fooled into a race tho..
[16:55] <SpamapS> just because it happens in this order now doesn't mean it will continue to happen that way.
[16:56] <SpamapS> how does that matter at all in this cloud-init case?!
[16:56] <smoser> SpamapS, there is already hard coded things such as that in mountall
[16:56] <hallyn> SpamapS: faking virtual-filesystems i was just mentioning to make a point :)
[16:56] <smoser> it wouldn't to me seem all that insane that mountall would try to mount all virtual filesystems before mounting / rw
[16:56] <smoser> basically asusming that all virtual filesystems can be mounted immediately
[16:56] <hallyn> SpamapS: wha ti'm saying is that mountall could be smarter and faster about the virtual-filesystems to help us out
[16:58] <hallyn> well, or starting off by checking that they are already mounted,
[16:58] <SpamapS> Ok.. well I feel that there are two very different things and I don't understand the focus of the converstaion. cloud-init needs to have the system state a bit frozen so it can reliably change things.. and virtual-filesystems isn't as meaningful as it could be.
[16:58] <hallyn> and not tryign ot be smarter than the sysadmin.  If I take them ou tof /lib/init/fstab, then I mean it.
[17:00] <SpamapS> I think adding a different hook point to mountall is totally doable to solve the cloud-init issue..
[17:00] <SpamapS> changing virtual-filesystems seems a bit off that mark though
[17:00] <smoser> how would you have a different hook point?
[17:00] <SpamapS> (even if its a good idea)
[17:01] <SpamapS> smoser: emit another event that is waited on.
[17:01] <SpamapS> one that is less accidental than "mounted MOUNTPOINT=/"
[17:02] <SpamapS> deliberately emit something at the exact point in the mountall process that achieves your goal of being able to change things without having them pulled out from under you.
[17:02] <hallyn> SpamapS: sounds good
[17:02] <hallyn> smoser: ^ that's basically exactly what you want
[17:02] <SpamapS> but there may be other things happening in parallel at that point and those need addressing.
[17:02] <hallyn> though i can see mountall authors complaining
[17:02] <hallyn> (is htat sjr?)
[17:02] <SpamapS> yeah its Keybuk
[17:02] <smoser> right.
[17:03] <SpamapS> and jhunt maintains it too
[17:03] <smoser> that seems like a specific solution to a specific problem
[17:03] <hallyn> SpamapS: but again, i would argue that saying 'wait on X and Y' where X and Y both are issued by mountall, should be interpreted as 'the later of the two'
[17:03] <hallyn> SpamapS: that would keep mountall from having to change for cloud-init's case
[17:03] <SpamapS> hallyn: heh.. and that, sir, is upstart 2.0
[17:03] <hallyn> smoser: yup, but a workable one
[17:03] <hallyn> SpamapS: how far down the pike is that?
[17:03] <smoser> i dont think its really workable
[17:04] <SpamapS> hallyn: and we can do that in a wait job now.. I just haven't figured out what X and Y are yet.
[17:04] <smoser> i tihnk its an ugly hack
[17:04] <hallyn> smoser: that's because you are difficult and contrarian :)
[17:04] <hallyn> SpamapS: x == mounted MOUNTPOINT=/, y == virtual-filesystems
[17:04] <SpamapS> hallyn: upstart 2.0 is an unknown for us. When keybuk has time.
[17:04] <smoser> i am. i agree.
[17:05] <SpamapS> hallyn: except virtual-filesystems is not a blocker.. so I don't think it does what you want.. hence the need for a new hook point
[17:05] <smoser> but i would suggest that so is keybuk.
[17:05] <hallyn> smoser: that's the 'difficult' part of you talking :)
[17:06] <SpamapS> I think its fair to ask for a 'mounted-virtual-filesystems' hook point.
[17:06] <SpamapS> its a fairly unique point in the filesystem lifecycle that would be helpful for other bits of the boot as well.
[17:07] <SpamapS> I wonder tho
[17:07] <SpamapS> would start on stopping networking work too? Thats basically where ifup -a has exitted.
[17:07] <SpamapS> or do you need to go before ifup -a ?
[17:08] <smoser> can't go before.
[17:08] <smoser> well, not for data that comes from a network
[17:08] <smoser> cloud-init-local does that.
[17:09] <smoser> but, yeah, via the non-network inputs, you can seed network information.
[17:10] <SpamapS> smoser: so start on stopping networking is after 'ifup -a' has exitted..
[17:11] <SpamapS> smoser: unfortunately that won't block rc.. but.. we can make that block too
[17:11] <SpamapS> meh
[17:11] <SpamapS> I need a whiteboard and some coffee
[17:26] <hallyn> SpamapS: btw, there is going to have to be (maybe there already is?) a way to get info on which events are emitted blocking/nonblocking.
[17:26] <hallyn> asking SpamapS on irc is not going to be scalable :)
[17:28] <SpamapS> hallyn: the official ones are documented well
[17:28] <SpamapS>        The  mounted  event  is  generated  by  the mountall(8) daemon after it has mounted a filesystem.  mountall(8) will wait for all services
[17:28] <SpamapS>        started by this event to be running, all tasks started by this event to have finished and all jobs stopped by this event  to  be  stopped
[17:28] <SpamapS>        before continuing with other filesystems.
[17:28] <SpamapS> also the table in 'man upstart-events' is quite clear
[17:29] <SpamapS> Type, S == signal (not waited on) H == hook point (waited on)
[17:31] <SpamapS>        The  virtual-filesystems  event  is  generated by the mountall(8) daemon after it has mounted all virtual filesystems listed in fstab(5).
[17:31] <SpamapS>        mountall(8) emits this event as an informational signal, services and tasks started or stopped by this event will do so in parallel  with
[17:31] <SpamapS>        other activity.
[17:31] <SpamapS> So.. really.. I hate to say "RTFM" .. but.. RTFM. ;)
[17:31] <hallyn> SpamapS: the FM is lying though
[17:31] <hallyn> i've said it 3 times now, so will wait to see if you know what i'm saying
[17:32] <SpamapS> which part is lying there?
[17:32] <hallyn> fing fstab
[17:32] <SpamapS> Ok so thats a bug.
[17:32] <hallyn> oh, is it?
[17:33] <hallyn> i thought it was undocumented but intended black magic
[17:33] <SpamapS> in the man page yes
[17:33] <hallyn> ah
[17:33] <hallyn> ok
[17:33] <hallyn> in any case i should read the upstart man page :)
[17:33] <SpamapS> AFAIK, mountall was thrown together while Keybuk was frantically working on boot speed
[17:33] <hallyn> and upstart?
[17:33] <hallyn> :)
[17:34] <zul> hey now play nice
[17:34] <hallyn> SpamapS: but so can you explain to me what it *is* doing?
[17:34] <hallyn> zul: that wasn't meant as a slight.
[17:34] <SpamapS> hallyn: you're not going to be satisfied by any explanation. Its a hack. period.
[17:34] <hallyn> that's fine.  but a hack to do what?
[17:35] <hallyn> it just guesses at the virtuals so as not to have to read fstab beforehand?
[17:35] <zul> hallyn, i know...i was just teasing
[17:36] <hallyn> zul: just trying to make sure noone comes by and challenges me to do better :)
[17:37] <SpamapS> hallyn: its a hack to make booting a non blocking independent thread
[17:37] <SpamapS> hallyn: the guessing, I suspect, is more about not having all the fs's listed in fstab
[17:40] <hallyn> SpamapS: i thought that was the point of /lib/init/fstab
[17:41] <hallyn> SpamapS: so wait, did we decide that wait on stopped networking and started rc will work?
[17:42] <SpamapS> err no
[17:42] <SpamapS> I was thinking about it
[17:42] <SpamapS> here's one issue..  rc starts on net-device-up IFACE=lo .. which is *before* stopped networking
[17:44] <SpamapS> so you need to have a  'starting rc RUNLEVEL=[2345]' too
[17:48] <SpamapS> might work better to have two separate jobs that do that, and both wait on cloud-init-whatever to finish
[18:39] <mattbillenstein> hi all
[18:40] <mattbillenstein> question — when a user-data script runs, where is the output logged?
[18:41] <mattbillenstein> I've found the script in /lib/cloud/data/scripts/part-000
[18:41] <mattbillenstein> but I can't seem to find any logs
[18:44] <mattbillenstein> hmm, just found it on the console...
[19:44] <aka_> yo
[19:45] <aka_> somebody here ?
[19:56] <aka_> D:
[19:57] <aka_> $this = NOTCOOL
[19:57] <aka_> ;
[21:50] <smoser> mattbillenstein, you will have to redirect the output of your script somewhere other than console if you want it there.
[21:50] <smoser> in natty's cloud-init there is cloud-config syntax for doing that.
[21:50] <smoser> in anything else, you'll have to handle that in whatever language the script is in
[21:50] <smoser> ie:
[21:50] <smoser> #!/bin/sh
[21:50] <smoser> {
[21:50] <smoser>  # my script here
[21:50] <smoser> }
[21:50] <smoser> # oops
[21:51] <smoser> } 2>&1 | tee /var/log/my.log
[21:52] <mattbillenstein> hmm
[21:52] <mattbillenstein> you can do that in bash?
[21:54] <smoser> yeah.
[21:54] <smoser> { ... }
[21:54] <smoser> does redirection
[21:54] <smoser> or.. command grouping
[21:54] <smoser> then the 2>&1 does the redirection
[21:54] <smoser> or, if you don't care about it going to the console then just do:
[21:54] <smoser> exec > /var/log/my.log 2>&1
[21:54] <smoser> at the beginning
[21:55] <mattbillenstein> sweet
[21:55] <mattbillenstein> I'll try it out, thanks for the tip!
[22:53] <aka_> yo
[23:02] <spicyWith> Hi, I'm running ubuntu 10.04 and would like to upgrade it to the latest distribution. What's the recommended way to do this?
[23:14] <erichammond> spicyWith: See if this applies to your situation: http://ubuntu-smoser.blogspot.com/2011/02/migrating-to-pv-grub-kernels-for-kernel.html
[23:16] <spicyWith> erichammond: I was looking at that, and to be honest, I'm not sure. How can I check?