[00:40] last hope, oh my! [00:43] heh [00:43] ok semantic question, on ubuntu mountall.conf takes care of mounting root fs rw, other fs, etc, [00:44] yes [00:44] but, could it be assumed, that by definition, starting /sbin/init, means the rootfs is already mounted, possibly read-only? [00:45] yes [00:45] i mean either a monolithic kernel, or an initrd's pivot_root, or say lxc container starting... [00:45] /* Sanity check, the root filesystem should be already mounted */ [00:45] root = find_mount ("/"); [00:45] if (! root->mounted) { [00:45] nih_fatal ("%s", _("root filesystem isn't mounted")); [00:45] exit (EXIT_ERROR); [00:45] } [00:45] even mountall assumes that :p [00:45] so "start on startup" implies certain parts of FHS are available, yes? [00:45] edgecase: it implies that the root filesystem is read-only [00:46] we tend to assume that means you have /bin, /sbin, /lib and /etc [00:46] but not /usr or /var [00:46] right [00:46] definitely not /usr, /var, /boot, /home, /proc, /sys, /dev, etc. [00:46] although with Upstart, you can assume /proc and /sys are mounted [00:47] i'm just pondering the sysctl.conf race problem [00:47] and with recent kernels, /dev should be a devtmpfs [00:47] sysctl.conf race problem? [00:47] race of setting things vs network interfaces coming up. in Karmic for example [00:48] there's a couple launchpad bug IDs [00:48] i drew out an elaborate map of the issues, but let me try and focus on one aspect [00:50] ok, with sysctl parameters, there are some that affect parts of the kernel which is only compiled in (not a module) [00:50] but with many, it could be a module or compiled in, so there's no *right* place for sysctl.conf to be loaded [00:52] right [00:53] this is why I keep trying to eradicate that silly directory [00:53] but people keep insisting I put it back :p [00:53] i wonder if the whole sysctl paradigm doesn't work so well [00:54] I tend to think that, since we have the kernel source, changing a sysctl default is best done by changing the default in the kernel [00:54] otherwise you have races like "the sysctl doesn't get set until after ssh is running and there's a security hole for a while" [00:54] i'm thinking of network configs here, default isn't enough [00:54] the system needs to be configurable [00:55] sure [00:55] but you can do one off-things like that with udev [00:55] there's a couple real problems with sysctl tho, [00:56] SUBSYSTEM=="module", KERNEL=="i915", RUN+="echo 4 > /proc/sys/..." [00:56] a NIC module could be compiled in, loaded by udev, or by the initrd... hard to put it's settings in one place had have it affect all 3 [00:57] you should still get module events for them [00:57] will that read sysctl.conf ? [00:57] you only don't get module events for built-ins with no parameters [00:57] no [00:57] hence the sticky issue [00:57] and reading sysctl.conf 800 times during boot is out of the question, before you ask ;) [00:58] workaround for the parameters i need is to set in /etc/network/interfaces and avoid the issue [00:58] but i'm also interested in the bigger picture of upstart [00:59] maybe just agree that sysctl isn't good for things not compiled-in [01:00] I don't think there's a picture of sysctl in the upstart painting yet [01:00] it's probably in the big white space off to the right with a doodle of someone's dog in it [01:01] sorry, have to run now [01:02] it's 1am here [01:02] yeah, that's ok, i can workaround for the network stuff i need [01:03] feel free to mail suggestions for how this could work to the ML [01:03] upstart is very much still under development [01:03] so lots of things can be improved [01:03] d'oh! [01:03] be around tomorrow? [01:04] yes, though earlier! [01:04] nite [01:04] ok [01:04] thx [01:15] hmm hostname.conf, nothing depends on it, [07:38] hello! I'm having some problems with upstart in karmic server 64bit. It won't start postgresql. I tried http://superuser.com/questions/98702/how-to-make-postgresql-start-at-boot-time-in-ubuntu but it didn't work. can I remove upstart? I'm afraid to test it because the machine is at a remote location and if I mess it up I won't be able to connect to it anymore [07:38] I can start postgresql using /etc/init.d/postgresql-8.4 manually after login. There are links in /etc/rc*.d to postgresql init script but for some reason it isn't started automatically after reboot [08:21] nothing? [08:53] hi. I got a program that I want to "always be running" as it opens a http listener and waits for input "forever" [08:53] I'm not entirely sure if I should be doing an upstart script for it. [09:54] hi everybody .... I am having serious problems with upstart on ubuntu 910 64 bit .... it would not start CUPS at all .... [09:54] CUPS is properly configrued in /etc/rcX.d and in /etc/init.d .... [10:07] ct529: I have the same problem with upstart on Karmic 64bit. On my machine it won't start postgresql server. I tried http://superuser.com/questions/98702/how-to-make-postgresql-start-at-boot-time-in-ubuntu without luck [10:09] ct529: as a workaround you can use rc.local ... not perfect but at least it works... [10:11] NoReflex: how? [10:11] NoReflex: yes, I have the same problem with mysql and postgresql [10:12] NoReflex: what do you see when you run the command runlevel? [10:13] ct529: what I did was to remove the links from /etc/rc*.d for postgres (sudo update-rc.d -f postgresql-8.4 remove) and add the start command to rc.local [10:13] NoReflex: did you add it manually? [10:13] NoReflex: could you pleas elet me know what do you see when you run the command "runlevel"? [10:14] ct529: runlevel gives N 2 [10:14] NoReflex: so it works on your machine [14:05] elpargo, are you asking if you should write upstart script instead of sysVinit script? [14:17] hello.. Is there some documentation on howto debug the bootup sequence? I'm getting bitten by bind9 trying to start (and failing) before the interfaces with ip-addresses that it is configured to listen-on / listen-on-v6 are brought up. [14:19] (this is happening on ubuntu karmic. someone else has reported it as bug: https://bugs.launchpad.net/ubuntu/+source/bind9/+bug/510587 ) [14:21] kimju: you should ask on #ubuntu about that [14:21] as far as I'm aware, bind9 in ubuntu is still using a sysvinit script and hasn't been converted to an upstart config file [14:22] Keybuk, ok, sorry. I just asked before you joined the channel if there is some documentation on how to debug the bootup sequence, as I'm getting bitten by that bug. [14:23] boot with --verbose on the kernel command-line [14:34] Keybuk, top o' the morning to ya! [14:35] hmm hostname.conf, nothing depends on it, [14:35] so a race is possible with anything that uses hostname/utsname [14:37] edgecase: hey [14:37] ah yes, it's still St Patrick's Month [14:37] i will now continue the barrage! [14:37] edgecase: can you think of anything that cares about the hostname? :p [14:37] syslog? [14:38] it doesn't really depend on it [14:38] it queries it for every log [14:38] ah ok [14:38] so if the hostname changes, it works just fine [14:38] aside, is there anything to graph the dependencies visually? [14:39] would make investigation easier [14:39] for rsyslog, if you log to remote host, might get unknown hostname for a bit? [14:40] no, but there's no reason it shouldn't be easy to write something that makes dotty output [14:40] ok will look at dotty [14:40] for rsyslog, if you log to remote host, you have to wait for the network to come up first :p [14:40] and on the typical laptop system, that won't happen until a user logs in and unlocks the WPA key :p [14:40] you noticed my nick is "edgecase" right? [14:41] i tend to ignore "typical" cases :P [14:41] :) [14:41] anyhow i'll admit hostname is a trivial case [14:42] i notice mountall.conf emits many variations of filesystems [14:42] is there a formal definition of those, that would allow splitting them up? [14:42] "emits filesystem", does that mean root fs is read-write? [14:43] appologies if that info is published already [14:50] hmm, having /var not a mandatory part of root fs, and yet having /var/run mounted on top, seems problematic [14:51] I question the original unix paradigm of "map everything to a file" for daemon .pid files, esp. if they are going to just map back to ram(disk) [14:52] just use unix socket or even abstract namespace one [14:54] edgecase: did you do "man 7 filesystem" ? [14:54] problematic> not really [14:54] it requires some juggling [14:54] but it works [14:54] we have /var/run before and after /var is mounted [14:54] in fact, even as /var is mounting, /var/run is still present [14:54] ok, juggling is acceptable if you don't drop the ball [14:55] *shrug* [14:55] if the boot sequence drops any ball, you can't boot [14:55] badness++ [14:56] why? [14:56] just write things to be reliable [14:57] i'll just assume "even as /var is mounting, /var/run is still present" is relable, moving on... [14:57] yes [14:57] I could teach a course on Theory of Atomic Operations in Operating Systems 101 :p [14:57] I think about them a lot [14:57] the details: [14:57] oh thank god [14:58] we always have a /var/run directory on the root filesystem, under the /var directory [14:58] we mount /var/run there on the first pass [14:58] then when it's time to mount /var, we actually mount it somewhere else (/dev/.var I think :P) [14:58] is there a mount --under option or something? [14:58] we bind mount the current /var/run to /dev/.var/run [14:58] so now it's available at both /var/run and /dev/.var/run [14:59] we then move the /dev/.var mount to /var [14:59] all atomic? [14:59] each of those is atomic yes [14:59] since a mount() operation holds the VFS lock [14:59] ooh, snap! [14:59] mountinfo will show /var/run mounted twice [14:59] with different parent mounts [15:00] if you unmount /var, for whatever reason, /var/run is still mounted under it [15:00] i find no mountinfo or man 7 filesystem [15:00] edgecase: are you on lucid? [15:00] karmic [15:00] i guess i need to be bleeding edge to get bleeding edge [15:00] ah, I think I added that manpage in lucid [15:00] i'll get lucid on a box shortly [15:01] so i can discuss current state of things [15:01] mountinfo => /proc/self/mountinfo [15:01] ah /proc/mounts on steroids [15:02] ok so, what provides /proc and /sys, upstart, before running any scripts? [15:02] I don't quite understand your question [15:02] you asked me what kind of trees apples, oranges and buses grow on :p [15:03] earlier you said assume /proc and /sys when running upstart [15:03] yes [15:03] so what mounts them? [15:03] init [15:03] before running any "start on startup" jobs? [15:03] yup [15:03] ok just getting my bearings [15:03] Upstart uses /proc/self/fd to run shell scripts [15:04] so it needs /proc mounted [15:04] and since it's probably going to need /sys at some point soon, I figured I'd mount that too [15:04] /dev gets mounted by the kernel these days [15:04] is there a formal definition of state when "startup" event fires? [15:04] ie "you get root /proc and /sys" [15:05] and "but don't assume /var and /usr etc" [15:05] no written down one I guess [15:05] use the source luke [15:05] the root filesystem is read-only [15:05] you have /proc and /sys [15:05] you probably have /dev [15:05] reading source takes time tho [15:05] you have /bin, /sbin, /lib and /etc [15:05] but nothing else filesystem wise [15:06] aside, i'm thinking mountall should be broken up, it's a syncronisation point, which defeats some parallelism [15:06] it isn't a synchronisation point [15:06] that's kinda the idea [15:06] make a dummy job with the same syncronisation properties for dumb legacy jobs, [15:06] it replaces the shell script nightmares we had before that were [15:07] but let smart jobs pick off say just root fs being rw [15:07] they can [15:07] on mounted MOUNTPOINT=/ [15:07] mountall emits events? [15:07] oh vfs emits mount events i'm guessing [15:08] no, mountall emits them [15:08] back to upstart? [15:08] yes [15:08] like filesystem for a start ;) [15:08] ok, then back to all the emits in mountall.conf... [15:09] any definition what they mean? [15:09] emits virtual-filesystems [15:09] emits local-filesystems [15:09] emits remote-filesystems [15:09] emits all-swaps [15:09] emits filesystem [15:09] emits mounting [15:09] emits mounted [15:09] (from lucid) [15:09] for any of those, "man 7 EVENT-NAME" [15:09] (in lucid) [15:09] ok will look there [15:09] I've tried to be reasonable well behaved about writing man pages for events [15:09] that's why i say sync point, they all get emitted in parallel [15:09] they don't [15:09] but only after *all* of mountall is done [15:09] no they don't [15:10] they get emitted at completely different times [15:10] ok so it's a declaration, "may emit..." [15:10] yes [15:10] right-o [15:10] did you read "man 5 init" ? :) [15:10] emits EVENT... [15:10] All processes on the system are free to emit their own events by [15:10] using the initctl(8) tool, or by communicating directly with the [15:10] init(8) daemon. [15:10] This stanza allows a job to document in its job configuration [15:10] what events it emits itself, and may be useful for graphing pos‐ [15:10] i see now that i'm way to pessimistic coming into this [15:10] sible transitions. [15:11] emits is basically there to make writing that graphing tool you were asking about possible ;) [15:11] documentation only [15:11] ok things look in order then [15:12] i'm thinking then that just the .conf files can be optimized, to depend only on what they *really* need, [15:12] to enable some more unusual use-cases than "user logs in then unlocks WLAN key..." [15:12] yes, certainly [15:12] like, network booting etc [15:12] the difficulty comes in adapting to users [15:12] like we could write the job to actually depend on /usr rather than filesystem [15:13] exactly [15:13] but that wouldn't work if a user decided to put a different filesystem for /usr/bin and for /usr/lib [15:13] FHS doesn't allow that does it? === elpargo_ is now known as elpargo [15:13] yes it does [15:13] FHS only says directories you shouldn't put on different filesystems [15:13] ok well on that topic, [15:13] things like /usr and /var are free-for-alls [15:13] windows has "reparse points"... [15:13] it's pretty common for example to have /var/lib or /var/spool on different drives [15:14] true [15:14] with unix, a (empty) directory *could* be a mount point, no way to know [15:14] you can read fstab, and hope it's in sync with reality [15:15] reparse points => those are basically crazy symlinks iirc? [15:15] takes place of fstab entries [15:15] you get file dir symlink, and now reparse point fs object [15:15] so, it's like an empty dir, because it has a name, [15:15] actually it might not even be in fstab [15:15] e.g. if you use LABEL to mean mountpoint [15:15] or if it's a removable device under /media [15:15] but it's a type explicityly marking it as a mount point [15:16] ok then that's an arguement for having reparse points in linux [15:16] although 75% of the unix world will issue a death warrant to you [15:16] for saying it [15:16] the cases that are in fstab could be handled [15:17] maybe that's a starting point [15:19] but it's not /usr i'm interested in... but rather starting things that need root fs read-only and /var/run available, starting those asap [15:19] anyhow i see that upstart is thankfully ahead of my expectations, i need to do some more reading now [15:19] thanks for the good work! [15:30] local-filesystems should be ok for those on Ubuntu [15:30] ah [15:30] sorry [15:30] virtual-filesystems [15:30] virtual-filesystems means you have root read-only, and all the things like /var/run and stuff [15:30] we use it to start things like udev, dbus, etc. [15:46] k thx [17:33] oh hi/bye edgecase === elpargo_ is now known as elpargo === elpargo_ is now known as elpargo [20:56] sadmac: did you ever look at the display manager selection thing more? [20:56] i mean http://rstrode.fedorapeople.org/quest_to_find_a_dm.txt === elpargo_ is now known as elpargo