=== JanC_ is now known as JanC [13:23] Why does upstart not print the Process ID of an application launched from a script section ? [13:23] What is the advantage of this way ? [13:42] it does [13:43] w00t ? [13:44] Script part can have tens of commands and those are not printed. Only if there is problems like this one /proc/self/fd/6: /etc/pulse/pulseaudio_pre-start.sh: line 53: amixer: not found pulseaudio start/running, process 599. /proc/self/fd is where it runs script part of the .conf files [13:45] process 599 is the script itself [13:45] and the shared process group of all of those tens of commands [13:46] so it does not print then. [13:51] well, what is the solution way if I need that sort of information, Keybuk ? [13:52] I have just read your mail and I am still not sure the next version (that I do not know when will be released) is useful for my purpose. [13:52] and for sure I do not have time to wait for that. [13:53] audit kernel stuff I thought can be a hackaround for now, but let me know, I am newbie in this area. [13:53] I don't know what your purposes are [13:53] well [13:54] (so it's hard to me to answer questions where you state "for my purpose") [13:54] I need the configuration file name and pid of the application that was run by upstart. [13:54] as it happens quite perfectly with 'exec' entries in the /etc/init/*.conf files. [13:55] but it does give you that, it gives you the pid of the script [13:55] which is all that was run by Upstart ;-) [13:56] nope [13:57] what do you mean "nope" [13:57] I'm pretty sure only one of the two of us *wrote* Upstart, so knows what he's talking about :p [13:57] hwclock:329,sh:328,init:1 [13:57] for example this is the procedure, init -> sh (upstart) -> hwclock [13:58] right, Upstart ran "sh" [13:58] I need the pid of the hwclock and the configuration file name which started it. [13:58] so status in that example will give you the pid of sh [13:58] not of hwclock [13:58] ok, that's *NOT* what you said above [13:58] you said "id of the application that was run by upstart" [13:58] upstart ran sh [13:58] no [13:58] what sh did is entirely up to sh [13:58] are you disputing what you typed? [13:58] because I just copy & pasted it [13:59] okay, you do not still understand it :) [13:59] let me give you an example: [13:59] no, I understand far more than you [13:59] you're just not explaining yourself [13:59] and probably you don't actually know what you want [13:59] and don't understand how processes work on UNIX [14:00] do not insult me, please. [14:00] I believe what you want is: [14:00] script [14:00] foo & [14:00] bar & [14:00] baz & [14:00] end script [14:00] as said, WAIT [14:00] to give you the pids of foo, bar and baz ? [14:00] I am gonna give you an example, patient pls... [14:01] I need to boot the device up. [14:01] is that not what you want? [14:01] seems you cannot be patient :) [14:01] no [14:01] I'm busy [14:02] I'm answering you rather than working [14:02] because I'm nice ;-) [14:02] but I don't have much time [14:02] work now and ask when I provide info ? [14:02] and to avoid the guesswork and wasted time till that ? [14:02] sure, will you hang around here for, say, 6 hours? [14:03] please do not be arrogant for hours, if I may ask... [14:03] I'm not being arrogant, I am, right now, using my lunch hour to try and help you [14:03] and you're not answering my question [14:04] because I still cannot ? [14:04] why? you can answer questions? [14:04] http://pastebin.com/GjeJiy7n [14:05] 14th line for instance. [14:05] that's a shell script [14:05] I would like to get this configuration file back and hwclock pid [14:05] hwclock has no pid at the point that script has finished [14:05] because the shell runs hwclock, waits for it to finish, and reaps it [14:05] weird it has. [14:05] every process has pid [14:05] tell you what [14:05] why not read what I typed? [14:06] AT THE POINT THAT SCRIPT HAS FINISHED [14:06] even tho. the same sometimes than the parent process (setuid e.g.) [14:06] the shell does something like: [14:06] fork() [14:06] (child: [14:06] exec hwclock) [14:06] (parent: [14:06] wait) [14:06] // now at the next line of shell [14:07] so there's no backgrounding going on here [14:07] k, so I cannot get this information by upstart.... [14:07] the fact that hwclock has a separate pid is basically an implementation detail [14:07] how can I get then ? [14:07] for example, a particularly over-the-top shell could have hwclock as a built-in [14:07] which means it would have no pid [14:07] get it when? [14:08] nope [14:08] the pid (if there even is one) will appear and vanish pretty quickly [14:08] stop saying "nope" to me [14:08] I know what I'm talking about [14:09] let me wait for someone else to help then... [14:09] they'll either tell you the same thing [14:09] or they'll give you incorrect information [14:09] seems you are in god-mode. [14:10] no, I'm trying to help you [14:10] but you're not actually listening to anything I say [14:10] which is exasperating [14:10] by screaming at me ? [14:10] I'm not screaming at you [14:10] I'm telling you how things work [14:10] and screaming if I am saying sth ? [14:10] and you're refusing to believe me [14:10] interesting help, never tried [14:10] which is insulting [14:10] you're wrong [14:10] do you accept that there is no need for hwclock to have a separate pid at that point? [14:11] (feel free to relink /bin/sh -> /bin/busybox and run that script, if you need physical proof) [14:12] no idea whether it has got pid that point, but I need the 1) conf file name 2) PID when it ran. [14:12] you could add strace -f to the shell invocation [14:13] exactly as it happens in case the exec line in the configuration files. [14:13] that way you'd trace all the system calls, and follow forks, so you'd get the pids that way [14:13] you could use the proc connector kernel interface to follow the forks/execs of the shell and again get the pids [14:14] proc connector is racey though, because the pid may vanish before you have an opportunity to read /proc/PID/cmdline and actually figure out what it is [14:14] (strace at least blocks) [14:14] pid is not enough, as said also the config file where it was run frm. [14:14] audit, ftrace, etc. any of those kernel tracing hooks [14:14] * from [14:15] well, my original idea was the audit stuff. [14:15] you could mount a subsystem-less cgroup hierarchy and place that script into a new cgroup within it [14:15] that would collect the pids run in the tasks file [14:15] but again, that only really tells you background processes [14:15] hwclock would come and go, so unless you polled the tasks file *really hard* you may not even see the pid [14:16] (and even then, /proc/PID/cmdline would probably be gone by the time you found the pid) [14:16] yeah, but audit should not be racey. [14:16] audit is racey [14:16] or "slow". [14:16] and slow [14:16] lol [14:16] sadly [14:16] audit doesn't block the shell from doing its work [14:17] which is nifty. [14:17] so when the shell forks off hwclock, it doesn't actually stop anything [14:17] so by the time that the audit system is told about the new pid, it could have already finished running [14:17] hard to imagine. [14:17] which means there won't be any information about it - not even the fact it was hwclock at all [14:17] happens all the time [14:17] as long as the audit follows the syscalls. [14:17] you'd have to follow the fork and the exec calls [14:17] and extract the strings from the exec call (which is actually an array of strings) [14:18] that's what then makes it slow [14:18] well, not really. [14:18] it depends on the kernel configuration. [14:18] no, really ;( [14:18] you are not obligate to enable the syscall tracing capability. [14:19] no, I mean it's just slow when enabled [14:19] it can be nearly free when turned off [14:19] your best bet, really, is to get the shell to tell you the pids ;) [14:19] for example, you could write that script as [14:19] hwclock -s -u & [14:19] yeah, but that option is just one of the available audit options, you can still use audit with no syscall trace. [14:19] # you now have the PID in $! [14:19] wait $! [14:19] sure, but audit with no syscall trace means you won't get the pids ;) [14:20] mmh [14:20] so the easiest way is to modify ALL the configuration files existing. [14:20] well, it depends yeah [14:21] a better question is why do you need to know the PID of that hwclock invocation? [14:21] I cannot talk about that, sorry. Company project and NDA. [14:21] do you need to know the pid of the date and test invocations? [14:21] if it was run directly by upstart shell then yeah. [14:22] there's no such thing as "upstart shell" [14:22] that's what I was referring to earlier [14:22] wtf ? :D [14:22] everything in that script block - upstart thinks is a big string [14:22] you can put whatever you want in there [14:22] all upstart does is exec /bin/sh and pipe that string to it [14:22] I mean in the process queue, there is a shell process after the init process and I meant that shell as an upstart shell. [14:22] which is what I mean about "the application that Upstart runs" *is* /bin/sh [14:23] interesting example: [14:23] script [14:23] exec hwclock -s -u [14:23] end script [14:23] what will Upstart say the pid of that is? :-) [14:23] (hwclock) [14:23] no idea [14:23] without knowing your project [14:24] one option would be for you to patch /bin/sh itself [14:24] lol [14:24] why lol ? [14:24] funny [14:24] let's say you have this upstart job [14:24] exec /lib/foo/some-binary [14:25] if that some-binary is that shell script you paste-binned [14:25] do you *still* need the pids from it? [14:25] no [14:25] interesting, why the distinction? [14:25] :) [14:25] *shrug* [14:26] seems pretty arbitrary to me [14:26] btw, that script is *completely* bat shit insane [14:26] you don't need to run hwclock like that [14:26] no we need. [14:26] and it was just an example if you do not like, the point is you got the idea. [14:26] but I don't get the idea [14:27] do not make it with me, pleae :) [14:27] * please [14:27] I don't get why you're treating scripts pasted into an upstart conf file differently to those elsewhere [14:27] do not do it to me :) [14:27] let's give a hypothetical example: [14:27] you do not need to understand that bit. [14:27] let's say you have a system which is entirely an upstart-native boot [14:27] no, I *do* to help you [14:27] our system is that. [14:27] ok [14:28] so have upstart as the last thing run a shell [14:28] well, I can tell you why tbh. [14:28] the PIDs run by Upstart are "1 thru the PID of that shell:" [14:28] no? [14:28] but it requires a more in-depth understanding. [14:28] and not 1 min [14:28] that is why I would like to avoid it for now. [14:28] I would not like to bother you. [14:28] if, after booting, the shell's pid is 1456 then Upstart has resulted in 1,456 processes being run [14:29] yeah, a sheel after the jobs, daemons. [14:29] * shell [14:29] yeah, it is trivial [14:29] but what is your gist ? [14:29] no gist, I'm trying to understand what you're trying to do [14:30] well [14:30] for example, if you were trying to measure the overhead of using shell scripts for things rather than writing them in C [14:30] there are more shell scripts in fact, not just one. [14:30] depends on the amount of configuration file stuff [14:30] well, put it that way. [14:30] do you know Maemo, MeeGo ? [14:30] of course [14:30] define "of course". [14:30] it's hard to work out, because - for example - what if hwclock forked off an extra child? [14:31] do you want to know two pids or one? [14:31] are you a nokian ? [14:31] of course I know of Maemo/MeeGo [14:31] no, but I know lots of people who are [14:31] excellent [14:31] so we use upstart in this project. [14:31] right ? [14:31] indeed [14:31] well [14:31] well, on/off [14:31] * Keybuk checks whether there's a "u" in the day [14:32] ? [14:32] sorry, possibly an English expression [14:32] nvm... [14:32] Maemo/MeeGo uses Upstart on Sunday, Tuesday, Thursday and Saturday [14:32] but something else on Monday, Wednesday, Friday [14:32] so you know maemo, meego security framework [14:32] a joke about the fact that they change their mind every minute [14:32] no, I don't know about that I'm afraid [14:32] it is a bit different compared to the traditional posix capability stuff. [14:32] it is more fine tuned stuff, like selinux, smack [14:32] and the like [14:33] ok [14:33] btw, do not starve because of me :) [14:33] I've eaten :) [14:33] *Nifty, googles* [14:33] *giggles* [14:33] typo king, so [14:34] we use manifest files for fine-tune capability management [14:34] and upstart and daemon maintainer did something wrong :) [14:34] no manifest stuff, and all the run jobs, daemon, upstart have all the capabilities. [14:34] We wrote a debugger kernel module which gives a line like that: [14:35] [ 9.701293] captrace:allow=25|hwclock:329,sh:328,init:1 [14:35] 25 obv. comes from /usr/include/linux/capabili.* [14:35] it is a capability [14:35] that is the output during the boot-up [14:36] and I do some kind of reverse engineering with the perl script [14:36] to generate the manifest files for upstart and jobs, daemons maintainers. [14:36] can you follow ? [14:36] yup [14:36] okay, great I need the conf path to put into the manifest file. [14:36] and just the daemon binary path in case daemons [14:36] that is why it does not matter in that case. [14:37] so the jobs, daemons capabilities are separated. [14:37] because they are different debian packages. [14:37] or rpm, whateva' [14:38] e.g. /usr/bin/pulseaudio [14:38] but the stuff run by upstart. [14:38] will be dropped by the upstart package. [14:38] and we are still speaking about credentials. [14:38] is it clear ? [14:39] so [ 9.701293] captrace:allow=25|hwclock:329,sh:328,init:1 is a good line, but I need to pass it with a configuration file [14:39] from which hwclock was run actually. [14:40] hwclock pid is there, 329 [14:40] and that is what I should match with upstart debug output, if any [14:40] and get the related configuration file name [14:40] that is all about, feel free to ask if something is vague. [14:44] ah, I usee [14:44] so if you had a line saying date:330,sh:328,init:1 [14:44] you'd just ignore it? [14:47] so there's an easy way to map :,init:1 to an Upstart conf name [14:47] boot with --verbose, in the system logs upstart will list all the pids along with the job [14:47] so you could compare the syslog to that log, and extract :PID,init:1 and map all instances of those to a .conf [14:48] as for the things like hwclock and soforth run by the scripts, audit is *probably* the right place [14:49] since in those cases, hwclock's ppid remains 328, init never finds out about it - the only time init really cares about a pid is when the ppid dies and it gets reparented to init, you see [14:49] usee ? [14:50] "see" sorry [14:50] I would not ignore a line, why do you think that ? [14:50] well, date doesn't need any extra capabilities? :-) [14:50] or maybe it does [14:50] gotcha [14:50] you are right [14:50] the common linux commands do not. [14:51] but it is just a whitelisty parsy thingy. [14:51] this is not the main question here for now. [14:51] so there's an easy way to map :,init:1 to an Upstart conf name [14:51] nope ^ [14:51] :,sh:xxx,init:1 [14:52] --verbose will give you the log messages to do that [14:52] the blah process can most likely be "advanced" cmd. [14:52] advanced = extra credentials for now [14:52] most likely the one left to that in the line is gonna be a common tool as you have just mentioned. [14:52] it is a kinda process tree tho. [14:52] --verbose will give you: [14:53] Jan 14 14:52:49 deathspank init: tty6 main process (8557) [14:53] log messages [14:53] that means you know that all [14:53] boot with --verbose, in the system logs upstart will list all the pids along with the job -> nope [14:53] :,sh:8557,init:1 [14:53] as said, we do not need jobs [14:53] just commands from the scripts [14:53] no, not ALL [14:53] jobs means by me the daemons. [14:53] but you don't need all [14:53] Jan 14 14:52:49 deathspank init: tty6 main process (8557) [14:53] :,sh:8557,init:1 [14:53] see how the 8557 lines up there [14:53] so ? [14:54] every run by that script will still end ",sh:8557,init:1" [14:54] :,sh:8557,init:1 [14:54] hwclock:8558,sh:8557,init:1 [14:54] date:8559,sh:8557,init:1 [14:54] give me 2-3 mins to read your posts again. [14:54] mkdir:8560,sh:8557,init:1 [14:54] hwclock:8563,sh:8557,init:1 [14:54] for example [14:54] if that script ran at least those four commands, that's what you'd see, no? [14:54] (the ppid of each of those remains 8557) [14:55] and, in fact, the pgid and sid of even hwclock is 8557 [14:55] so that tells you that the shell script run by upstart for the "tty6.conf" ran those four commands [14:55] since in those cases, hwclock's ppid remains 328, init never finds out about it - the only time init really cares about a pid is when the ppid dies and it gets reparented to init, you see [14:55] right [14:56] -> yes, I mentioned an example in the beginning for that. [14:56] unless a process deliberately attempts to shed its session, the pgid and sid will remain the same as the shell [14:57] 16:57 < Keybuk> Jan 14 14:52:49 deathspank init: tty6 main process (8557) [14:57] well it is not enough [14:57] why not? [14:57] it is just about the shell script, not the application run by that. [14:57] why is it just about the shell script? [14:58] from what you've told me, that'd exactly what you want, isn't it? [14:58] nope [14:58] okay, give me a better example and not hwclock [14:58] no, what _do_ you want then? [14:58] * let me give you [15:01] sorry Keybuk [15:01] maybe I should have picked a better test command than "stop gdm" :p [15:01] :P [15:01] you were right [15:01] http://people.canonical.com/~scott/forkwatch.c.txt [15:01] I did not understand what I mean [15:02] * You mean [15:02] * meant..erm [15:02] if you need something that *really* can watch all the forks, that C program should help [15:02] wait [15:02] though you'll see what I mean about the cmdline race there [15:02] audit might be better [15:02] we have just discussed the verbose stuff is ok'ish [15:02] the verbose stuff should get you back to an upstart .conf file, yeah [15:03] and you can turn it off after boot with "initctl log-priority warn" [15:03] well, the pid idea was about the identification [15:03] but I need the config file only at last. [15:04] http://people.canonical.com/~scott/ -> is it your stuff ? [15:04] Keybuk: well not an issue [15:04] djszapi: yup [15:04] my stuff [15:04] as you got the point this task is because of a wrong maintainer past [15:04] so hopefully it is not gonna require continous running. [15:04] during the lifecycle of the device, os. [15:05] *nods [15:05] you can turn on --verbose btw with "initctl log-priority info" [15:05] so you don't even need to reboot to enable it [15:05] but for confirmation, let me give you a better example than hwclock was. [15:05] [ 8.318542] captrace:allow=6|aegis-loader:191,sh:188,init:1 [15:06] aegis-loader is an internal application [15:06]