[13:44] jodh: ping [13:44] djszapi: hello [13:45] jodh: I am not getting my upstart job logged [13:45] there is certainly issues with it and I have used console log [13:45] but there is no /var/log/upstart/foobar.log entry. [13:46] does the directory /var/log/upstart/ exist? [13:46] I do not see anything related: http://paste.kde.org/459902/ this is my upstart job: http://paste.kde.org/459908/ [13:46] nope [13:46] it does not exist [13:46] on this pandaboard [13:46] upstart 1.3-0ubuntu12~linaro2 [13:47] djszapi: http://upstart.ubuntu.com/cookbook/#stanzas-by-category [13:47] djszapi: console log didn't get introduced until 1.4. [13:48] by default or altogether ? [13:48] djszapi: if you run "init-checkconf foo.conf" you should get a msg about invalid stanza as "console log" won't be understood. [13:48] djszapi: the feature was added in Upstart 1.4 and you are running Upstart 1.3 [13:49] init-checkconf cannot be run as root [13:49] and I am not supposed to have another user for this. [13:49] at any rate, I trust you. [13:50] jodh: shall I use console output ? [13:50] and 2>&1 ? [13:51] djszapi: I suggest you read http://upstart.ubuntu.com/cookbook/#debugging [13:51] djszapi: it's all in there [13:52] djszapi: correct - init-checkconf runs as a normal user. [13:52] jodh: what is wrong about this: http://upstart.ubuntu.com/cookbook/#console-output [13:52] djszapi: nothing - there are many options. It's your decision :) [13:52] jodh: what do you recommend ? [13:52] there is some operational issues with my binary [13:52] it works if I run manually. [13:52] but it does not function if it is run by upstart [13:53] the binary is running and all that jazz, but no proper effect on the functionality. [13:53] it opens a serial port, and begins to listen to that [13:53] djszapi: I'd recommend reading http://upstart.ubuntu.com/cookbook/#debugging then. [13:53] and if there is a data, it parses. [13:53] and act accordingly, that is all. What I now see, it does not act as expected. [13:53] so needs to be debugged. [13:54] yes - see also http://upstart.ubuntu.com/cookbook/#debugging-a-script-which-appears-to-be-behaving-oddly [13:54] which might be the answer in your case. You'll just have to try it. [13:54] that is a complete no go [13:54] I would like to separate the logging out for my stuff [13:54] I would not like to grep in a bunch of stuff [13:56] and that is what console output is for, afterwards. [13:56] I think you have misunderstood - the 'grep' is an example in the section above. *You* don't need to grep, you need to test your script with '/bin/sh -e'. [13:57] I do not understand [13:58] Please note that I do not use script [13:58] and there is no "failing" either. [13:58] what I would actually like to get, is a 2>&1 like operation, and console output seems to be designed for that. [13:59] as in: exec /usr/bin/foobar-commandline receive > /var/log/foobar.log 2>&1 [14:00] djszapi: you can do exactly that if you want to. [14:01] what happend with Scott ? [14:01] one or two years ago I had the discussions with him. [14:01] he stepped down as the upstart maintainer ? [14:04] yes - he is now working at Google. [14:04] I know that, but I did not know he stepped down from the upstart maintenance. [14:06] interesting, after the launch: stop foobar; start foobar and the "foobar" binary works just fine... [14:06] what is the difference ? [14:07] the serialport is opened by that process after the launch, so I do not really understand. [14:08] jodh: is the serialport available for writing for such a job ? [14:23] jodh: the modem is not set up on the serial port :( [14:23] while the upstart job is running, just later. [14:23] how can that be ? [14:24] since the modem does not respond to the commands that time I send out. [14:24] so I guess the modem is not set up, but why ? [14:24] I had the impression "start on runlevel [23]" should be just fine [14:35] jodh: what should trigger the start of my job instead of the initlevel thingie ? [14:42] djszapi: you should probably add 'and X-device-added a=b c=d' for the modem device. See http://upstart.ubuntu.com/cookbook/#upstart-udev-bridge, upstart-events(7) and upstart-udev-bridge(8). [14:46] jodh: I have honestly no ideas about the subsystem :/ [14:46] djszapi: what device does your app open? /dev/ttyS0? [14:47] /dev/ttyUSB0 [14:47] PL2303 (usb-serial converter) [14:48] jodh: http://paste.kde.org/459968/ [14:48] perhaps usbdev ? [14:49] so, have your job 'start on tty-device-added DEVPATH=*ttyUSB0' [14:49] E: SUBSYSTEM=tty [14:49] udevadm info --export-db | grep -B 2 -A 2 /dev/ttyUSB0 [14:49] from this output [14:51] this one then ? start on (tty-device-added) [14:51] stop on (tty-device-removed) [14:51] hmm, the documentation does not mention the DEVPATH [14:52] just inside the log, but not for the "start/stop" lines. [14:52] I've given you the answer above. I'm guessing you either don't have your usb device plugged in or you didn't run udevadm as root? [14:53] well, the documentation does not mention any DEVPATH examples. [14:53] the documentation only speaks about the log wrt those. [14:53] ? [14:54] show me the documentation example for the DEVPATH usage. [14:55] these are the examples: start on (graphics-device-added or drm-device-added) [14:55] start on (graphics-device-added or drm-device-added) [14:55] 1) Both contain brakcets [14:56] brackets* [14:56] 2) None of the presents the usage of DEVPATH [14:57] jodh: did not help [14:58] djszapi: what did not help? Please be specific. [14:59] jodh: http://paste.kde.org/459974/ [14:59] what I told above [14:59] same symptom [14:59] the modem was not able to reply to the message sent out while running the relevant binary from the upstart job. [15:01] djszapi: I suggest you (1) comment out the 'stop on' and make the job 'start onrunlevel [23] and tty-device-added DEVPATH=*ttyUSB0'. [15:02] djszapi: also, comment out respawn. [15:02] init-checkconf /etc/init/piruag.conf [15:02] ERROR: failed to ask Upstart to check conf file [15:02] respawm is very important [15:05] djszapi: do this before running init-checkconf as presumably you're running on a console: eval `dbus-launch --auto-syntax` [15:05] jodh: I am now having only two lines: 1) start onrunlevel [23] and tty-device-added DEVPATH=*ttyUSB0 2) exec /usr/bin/foobar-commandline receive [15:06] djszapi: ...? [15:06] File /etc/init/foobar.conf: syntax ok [15:06] jodh: what is not clear ? [15:07] djszapi: so what happens now? does the job work with the 2 lines or not? [15:08] testing [15:09] jodh: I think the point is not ttyUSB0 [15:09] jodh: IMO, the point is that this specific modem is set up at that time. [15:09] if they are the same, then ok. [15:11] jodh: that way, it does not even run after the bootup [15:11] probably no space between on and runlevel [15:11] and I have just copied and pasted your line. :-) [15:17] jodh: that line actually broke the initscripts [15:17] even ssh does not work from outside anymore :( [15:18] jodh: get it back, sorry but no ... my binary does not even run anymore with that "start on runlevel..." etc line [15:19] are you sure brackets are not needed or other quirk ? [15:22] djszapi: init-checkconf will tell you if the file is valid. [15:23] jodh: at any rate, the binary is not even started this way [15:23] any ideas ? [15:25] too bad upstart is this hard to use :/ [15:34] if your app isn't setting the modem, sounds like you need to find out what is. If it is being configured by an Upstart job, you can make your job 'start on stopped ' since at that stage the modem will presumably be configured. If it's being done by a sysv script, then 'start on stopped rc'. [15:35] I have no clue what you are talking about. [15:36] my application does configure the modem, but it should obviously available for commands. As for me, upstart seems to be either broken or quite undocumented. [15:38] djszapi: it is neither. [15:38] well, it does not work [15:38] and I do not find in the documentation either how it is supposed to work either. [15:39] as for me, it is time consuming either way as a "customer". [15:40] script echo "`env`" > /dev/.initramfs/job-A.log [15:40] end script [15:40] this logged nothing, so even this part of the documentation is hilariously broken [15:41] djszapi: incorrect - that tells you the either you have a corrupt .conf file (your paste is incorrect in that 'echo' should be on the line below), or your job never started. [15:42] djszapi: have you tried "start " on a booted system maybe? do you get a log file then? [15:44] no log, no [15:44] even after start stuff [15:45] jodh: ^ [15:45] start on (runlevel [23] and tty-device-added DEVPATH=*ttyUSB0) [15:45] script echo "`env`" > /dev/.initramfs/job-A.log [15:45] end script [15:45] exec /usr/bin/... [15:47] djszapi: as stated above, that is incorrect syntax: the first 'script' should be on a line on it's own, as shown in the cookbook and as documented in init(5). [15:47] no [15:47] documentation shows this: start on (graphics-device-added or drm-device-added) [15:47] script echo "`env`" > /dev/.initramfs/job-A.log [15:47] end script [15:47] have you actually checked the documentation ? [15:48] also: init-checkconf /etc/init/piruag.conf [15:48] File /etc/init/piruag.conf: syntax ok [15:48] s/piruag/foobar/ [15:50] djszapi: what you have pasted above is therefore *not* what is in piruag.conf as it is invalid. the cookbook is correct. Maybe your web-browser is mis-rendering it. It views perfectly for me in ff and chromium. [15:50] huh ? [15:51] your irc client is wrong then [15:51] since it was a simple copy/paste [15:51] please do not argue about useless things [15:51] let us just nicely proceed [15:51] I would not like to spend yet another few hours with upstart to run a simply binary lol [15:51] once tty is set up properly. [15:52] s/simply/simple/ [15:52] I do not understand why this hard to use an initiscript. [15:53] this is a perfectly valid syntax made according to the documentation (TM): http://paste.kde.org/459992/ [15:58] djszapi: right - your problem is that you've specified both a 'script' section and an 'exec'. That's not valid, or rather the last valid entry will be selected (the exec), hence you won't get a log. [15:58] init check returns /valid/ [15:58] fix the check. [15:58] djszapi: put the call to /usr/bin/foobar-commandline at the end of the script section and comment out the exec section. [15:59] djszapi: it's valid, but not what you want. [15:59] what is the point of upstart if it is this hard to run a binary once the serial port is set up ? [16:00] it isn't hard - you just need to read the documentation. If the documentation is unclear or needs more examples, please raise bugs so we can rectify the situation. [16:00] well, you did not even provide any working solution so far [16:00] so it seems to be even hard for you. [16:01] so what can I say as a newcomer... [16:01] djszapi: I don't have a modem and I didn't write your application :) [16:01] it has nothing to do with my application [16:01] it expects a modem set up [16:01] which is obviously a sane thing if you wanna configure it .... [16:02] http://paste.kde.org/460028/ -> this is the env [16:02] quite poor from what I can say. [16:03] anyway, I have no clues... [16:03] any help is welcome... [16:03] otherwise I need to admit it is incredibly hard from upstart, and not worth spending the time with. [16:04] djszapi: so your application runs correctly from the command-line on a booted system? [16:04] like I said in the very beginnig, of course... [16:04] even from upstart ... [16:04] not during the boot, but I said it many times really :) [16:04] so clearly, the modem is not up and running [16:05] and still, the last line I got for the "start on ..." is broken [16:05] the binary is not even executed anymore [16:07] jodh: ^ [16:18] djszapi: what release of Ubuntu are you on? [16:19] upstart 1.13 [16:19] (like I said in the beginning) [16:19] why should the ubuntu version matter ? [16:19] djszapi: I wasn't here in the beginning. Please try to be a bit more polite. [16:20] djszapi: There is no such thing as 1.13 [16:20] weird, I see you were online. [16:20] of course there is. [16:20] 1.5 is the newest release of upstart [16:20] 1.3-0ubuntu12~linaro2 [16:20] yes, 1.3 [16:20] Ok, 1.3 not 1.13 :) [16:21] it drives me crazy :/ [16:21] djszapi: so if you add this to your exec line, you can get the error messages from your command logged to syslog: [16:21] 2>&1 | logger -t pickatag [16:21] that is not gonna work without console log IIRC [16:21] console output [16:21] djszapi: you may also want to increase the log priority to 'info' so that you can see upstart events being processed. 'initctl log-priority info' [16:22] djszapi: you don't have syslogging? [16:22] djszapi: ok, change that to this then: [16:22] I do have syslog [16:22] but I would rather prefer a dedicated log just for this. [16:22] >> /run/mylog.log 2> /run/myerr.log [16:23] -> /run ? [16:23] you mean /var/log/ ? [16:23] choose whatever you want [16:23] -> /run does not really fit the lfsh [16:24] ok, you get the idea. [16:25] http://paste.kde.org/460052/ [16:27] djszapi: yeah that should work [16:27] ok, it does not [16:27] like I said before [16:28] especially because console output is not there. [16:28] and even if I was putting there, I would have the same information, I already do have. [16:28] the modem does not respond. [16:28] most likely because it is not set up [16:28] so I do not understand what addition we would gain in here. [16:28] Any ideas pushing the topic forward ? [16:29] what I need is not the logging of my application, I already know where it fails. [16:29] I would need to know what to trigger the application for. [16:29] the documentation does not just work, period. [16:29] nor the aforementioned ideas. [16:30] I do not understand why one needs to spend many hours to run a binary once the serial port and the modem in there are ready. :/ [16:30] it should be this simple: I would like to run my daemon once the system is fully set up: ok, use this. [16:30] should be just one option. [16:32] djszapi: where does it fail then? [16:32] huh ? [16:32] djszapi: is there hardware missing? [16:32] have you actually read what I wrote ? :-) [16:32] djszapi: sounds like the device added event is emitted by udev too soon. [16:33] sorry, but really getting frustrated. I would not like to spend /enormous/ amount of time to run a binary once the system is fully set up. [16:33] djszapi: I'm done. Good luck with upstart. You will get no help until you can be nice. [16:33] you have not clearly read what I wrote. [16:33] I wrote that the modem does not respond back then while trying to configure it. [16:33] Ok, I typed that while you said sorry. [16:34] djszapi: so that tells me that you need an event for when the modem is ready, not when the serial port is added [16:34] I would need a common event which tells the system is up! [16:34] now you can run anything. [16:34] everything is set up. [16:34] "system is up" is hard to define. [16:34] I do not care about low-level stuff, like a certain specific device is run [16:34] mobile systems have resources that come and go. [16:35] with the certain specific vendor and so on ids. [16:35] this is quite low-level. [16:35] super low-level [16:35] "system is set up" -> done [16:35] run the binary [16:35] what is so hard about it ? [16:35] runlevel 2 should be that the system is ready to run anything statically onfigurable [16:35] configurable rather [16:36] like I said, runlevel 2 did not work. [16:36] getting no replies from the modem [16:36] Right, so there is something not-static that you are dependent on. [16:36] meanwhile if I make a stop myjob; start myjob, I do get replies [16:36] either an "OK" or an "ERROR". [16:36] djszapi: perhaps try 'stopped udevtrigger and runlevel [23]' [16:36] it is quite static, trust me :) [16:37] djszapi: udevtrigger will flush all events that the kernel knows about at system boot. [16:37] I am not sure udevtrigger stopping check is a good point for that. [16:37] Like I wrote: "start on (runlevel [23] and tty-device-added DEVPATH=*ttyUSB0)" does not even run the binary anymore [16:37] so it is already a broken check. [16:38] extending a broken check... I do not expect anything working. [16:38] start on (stopped udevtrigger and runlevel [23] and tty-device-added DEVPATH=*ttyUSB0) [16:38] I do not expect it to work either. [16:39] djszapi: it will delay the start a bit... perhaps until the modem is ready [16:40] djszapi: Otherwise, the modem needs to be polled [16:40] no way [16:40] if I stop and start, it runs fine [16:40] without doing that, it does not appear. [16:40] so no, sorry. [16:41] djszapi: let me put it another way. What actual event would you expect to signify that the modem is ready? [16:41] modem is continously ready [16:41] since it gets power supply, and not turned off. [16:41] so I would expect once /dev/ttyUSB0 is available, it should work oob [16:43] djszapi: so perhaps there's a bug in the code that emits the event about ttyUSB0 being ready when it isn't [16:43] I do not understand why I am having these troubles if it is super simple by using SystemV. [16:45] djszapi: because systemv takes 5x longer to start [16:46] djszapi: its still a race, its just that one car is WAY slower than the other :) [16:46] I do not feel it funny, sorry for that... [16:48] djszapi: you can try putting yourself way after rc runs.. 'start on stopped rc RUNLEVEL=[23]' [16:48] djszapi: but thats really just slowing down the race a bit. Not solving the issue. [16:48] huh ? [16:49] djszapi: perhaps also try 'respawn' which will retry a few times [16:49] djszapi: you need an event that is reliable. Your ttyUSB0 event is clearly not reliable. :-P [16:49] jodh said to remove the respawn... [16:49] you have nothing to prove that saying. [16:49] really. [16:49] I don't, because you won't show me the output of your program [16:50] ubuntu should drop upstart to the trash :) [16:51] it is unusable even for simple cases like this. [16:51] djszapi: this has nothign to do with upstart. [16:51] ofc it does. [16:51] since it works with other systems like aforementioned. [16:51] djszapi: upstart is operating properly... as I said, you just don't have an event you can rely on. [16:51] that makes no sense, sorry. [16:51] stop/start works fine! [16:52] djszapi: so if I tune up my engine and get my car to go 220mph, but that makes the tires melt.. its the engine's fault? [16:52] huh ? [16:52] djszapi: upstart is just the point where you're interfacing with the plumbing of the entire system [16:52] djszapi: stop/start works fine because the device has had enough time to initialize [16:52] I joined this channel more than three hours ago [16:53] and still not solved [16:53] like...wtf ? [16:53] djszapi: well you've been really difficult. I'm trying to help. [16:53] djszapi: lets back up, because I want to make sure I'm as helpful as possible. [16:53] I have not realized I would have any solution for the issue, really. [16:54] djszapi: you have this job which uses 'start on runlevel [23] and tty-device-added DEVPATH=*ttyUSB0 .. right? [16:54] to run a daemon after the system is fully up takes more than three hours :( [16:55] djszapi: and you refuse to try adding 'stopped udevtrigger' right? [16:55] not right [16:55] djszapi: and upstart tries to start the program, but the program fails with some kind of error about the serial port not being ready, right? [16:55] no, the program does not fail [16:55] ahh [16:55] <-- misunderstood [16:56] djszapi: ok, so the program never starts? [16:56] and, no, I am now trying this as recommended: start on stopped rc RUNLEVEL=[23] [16:56] it does start [16:56] djszapi: and its in 'stop/waiting' after the system fully boots? [16:57] * djszapi ponders how many hours have to wasted yet for upstart :( [16:57] be* [16:57] SpamapS: again: the binary runs with start on stopped rc RUNLEVEL=[23], but the modem does not respond with either "OK" or "ERROR" like after running stop/start [16:58] tart on runlevel [23] and tty-device-added DEVPATH=*ttyUSB0 and stopped udevtrigger -> did not work [16:58] the daemon did not even run [16:59] djszapi: is it possible that the modem doesn't initialize itself until the first time you run the program? [16:59] huh ? [16:59] like I said, the modem is fully powered [17:00] it is configured [17:00] it is working [17:00] the running ubuntu stuf makes it not responding [17:00] djszapi: what I'm saying is, could it be that the modem won't respond to the first program that talks to it. [17:00] during the bootup for probably some false bootup sequence. [17:00] I do not see how that would be possible really. [17:01] if I do not try to run the program, but I run after the bootup [17:01] everything is alright. [17:01] djszapi: ok so it still is just a race.. and we need to find the event that means "the modem is active" [17:01] again, again and again: I need to run my program once the system is set up ! [17:01] I would have expected tty-device-added to be that [17:01] darn upstart, it is not possible to do such a basic thing... [17:01] djszapi: systems are mobile [17:01] djszapi: tty devices come and go [17:02] djszapi: 'stopped udevtrigger' is "when all the hardware that is plugged in at boot time is initialized" [17:02] well [17:02] djszapi: perhaps take out the tty-device-added part [17:02] 19:58 < djszapi> tart on runlevel [23] and tty-device-added DEVPATH=*ttyUSB0 and stopped udevtrigger -> did not work [17:02] djszapi: so start on runlevel [23] and stopped udevtrigger [17:03] I am not sure it is a good thing to base a system on a such a brittle tool. [17:04] djszapi: the tool is actually quite strong. the events are just not easy to understand, because parallelism is hard. [17:04] I do not care about the events [17:04] have you still not realized ? :) [17:04] I would like to run my binary once everything is set up! [17:05] I do not care about premature optimization. [17:05] which causes -3-4 hours from more people's life. [17:05] djszapi: I actually fully understand that. I want there to be more clear, simpler events like 'boot-hardware-initialized'. I created the event 'static-network-up' for this reason. [17:06] djszapi: I also think that runlevel 2 should probably be delayed until after 'stopped udevtrigger' .. but people want the boot to proceed without waiting for all the hardware [17:06] I am not the people. [17:07] djszapi: then you must understand the events. :) [17:07] no [17:07] Again, I would like to run a binary once everything is set up! [17:08] djszapi: that is runlevel [23] and stopped udevtrigger [17:08] did not work [17:08] modem does not respond. [17:08] hmmm... one thing.. [17:08] I notice udevtrigger uses post-stop to run 'udevadm settle' [17:09] so perhaps the events will still be going after that event [17:09] *argh* [17:09] Upstart, to the trash! [17:10] djszapi: I have to go soon.. but I think this is worth a bug. It should be possible to ask for a job which waits for udevadm settle to finish. [17:10] djszapi: let me make this clear. This has nothing to do with upstart. It is just a deficiency in the system plumbing available on top of upstart in Ubuntu. [17:10] djszapi: upstart is doing everything right. Ubuntu has just failed to expose the events in a way that makes sense. [17:11] I am surprised that your tty-device-added doesn't work [17:11] is there *ANY* workaround ? [17:11] put a sleep into the script ? [17:12] but that is blocking. [17:12] unless the kernel events are async. [17:12] which is probably the case. [17:12] if the goal is to delay the startup of the job, sure, you can use a sleep [17:12] in any case, sleep would be blocking. [17:12] what do you mean by "async"? [17:12] djszapi: a sleep might work. You can also try putting 'udevadm settle' right before it running the program (or in a pre-start) [17:12] djszapi: sleep would be blocking, exactly, thats why it might work ;) [17:13] can you tell me the line exactly? [17:13] pre-start exec sleep 10? [17:13] ohh it is just sleep 10 [17:13] pre-start ? [17:13] why is that ? [17:13] slangasek: any ideas on an event for 'when all the hardware currently plugged in is available' [17:13] djszapi: or you can put it at the top of the 'script' [17:13] djszapi: so that when you respawn you don't run it again [17:14] SpamapS: I was thinking 'stopped udevtrigger' should do this, but I have the same uncertainty you do about whether post-stop scripts are required to complete before starting jobs that are 'stop on' [17:16] http://paste.kde.org/ -> slt ? [17:16] I think you missed the last part of that link [17:17]