=== Monthrect is now known as Piper-Off
neonixcoderhi team, this is general question not specific to Ubuntu but this is happening on Ubunut. So I am asking it here..03:26
neonixcoderWe have a script which monitors network connections, if there is no network connection(3G connection) it just tries to enable networking. If this is not working out, this script will reboot the system. And these reboots happens, untill we get network connectivity back.03:28
neonixcoderThe issue is after 5 bad pings, this script is not writing anything to log file. We tried to replicate this in lab, and everything working fine..03:28
neonixcoderbit strange and I am not able to conclude what is happening with the script.03:29
neonixcoderany suggestions?03:29
patdk-lapbased on that info? none at all03:29
patdk-lapit did not provide any useful info03:29
neonixcoderthe file system is rw mode, so read-only mode is eliminated..03:29
sarnoldpastebin the script?03:30
sarnoldpastebin how you're running the script?03:30
neonixcoderHow are you Arnold?03:30
neonixcoderGive me 2 mins..!03:30
patdk-lapwhat the filesystem is and what is the reboot?03:30
sarnoldnot bad, surprised how quickly the weekend went, but not bad :) you?03:30
* patdk-lap is just upset he debugged for 2hours why a patch wasn't working03:31
patdk-lapto come to find, the patch was not installed03:31
patdk-lapand to find, myself from 2years ago, was smarter than I am today :)03:32
neonixcoderI am good Arnold, thanks for asking..03:32
sarnoldpatdk-lap: oh yeah, been there. It's especially fun to find my own answers on stack overflow.03:33
patdk-lapadd some syncs in there03:33
patdk-lapnot sure what kick_watchdog.sh does03:34
patdk-lapbut it doesn't sound filesystem friendly03:34
patdk-lapsounds more like, yank power03:34
sarnoldpoor dog03:34
neonixcoderpatdk-lap: It is just a watchdog script which is poweroff the machine completely and boots after 5 seconds.03:36
patdk-lapyes, but exactly how?03:36
neonixcoderUp to 104 line it is fine, but 105 line on words I dont see any logs..03:36
patdk-lapit matters if the kernel has time to write everything to disk or not03:36
patdk-lapwatchdogs normally don't03:36
neonixcoderpatdk-lap: We have a c code in that kick_watchdog.sh script which take care of kicking power off.03:37
patdk-lapso it's not friendly03:37
patdk-lapat the top of kick_watchdog.sh add sync command03:38
patdk-lapshould help03:38
neonixcoderpatdk-lap: what you mean by not friendly?03:38
sarnoldit doesn't give the filesystem a chance to sync, umount, etc03:38
neonixcoderpatdk-lap: We can not use sync as we deliberately delayed writting to disk by 30mins in kernel.. but when we do kick_Watchdog.sh run, it have sync included.03:39
neonixcodersarnlod: Can you explain a bit?03:39
sarnoldneonixcoder: the kernel will buffer writes until some convenient time in the future03:40
sarnoldwrites to files don't happen immediately, they happen at only certain points03:40
patdk-lapeven worse if mapper stack is used03:41
sarnoldso the usual thing to do is to run sync to make sure that those changes have been flushed to disk, and if you can umount the filesystem first, that's even better03:41
neonixcodersarnold: my  script struck up there like 2 to 3 days with out writing anything to log file. just a note, we sync data to disk every 30mins.03:41
sarnoldbtw, all these 'sudo' in here seem funny03:42
sarnolddoes this run as root? or not?03:42
neonixcoderI know it sarnlod, I am going to modify it.03:42
sarnoldif it doesn't run as root, is sudo configured to allow it to run those commands without prompting and so no?03:42
neonixcoderThis is written while back, if I past you actual script, you will laugh at me ;)03:43
sarnoldnote that the sudo foo >> bar may not work as you want. everyone gets tripped up with that..03:43
neonixcodersarnold: Yup, I am going to remove sudo from it.03:45
neonixcodersarnold: will that helps?03:46
sarnoldneonixcoder: unlikely03:48
neonixcoderyes.. this script was running for a while with out any issue.03:48
neonixcoderWe are seeing this issue recently on couple of servers..03:49
neonixcoderOur servers are connected to network through 3G, if there is no 3G network, our script tries to disable/enable for some time. If its unable to bring network, it will reboot machine.03:49
neonixcoderIf there is no network for 10 mins, the system will be rebooted. At the time of reboot, system disks are synced. so there is no point why my script is unable to write to disk.03:51
neonixcoderAny suggetions?03:51
sarnoldare you sure the machines arent hung entirely?03:52
neonixcoderThey dont hung..03:53
neonixcoderY'day we sent a technician to see if can login. And he is able to do that. I ran some simple tests like checking network connectivity(Which is not there at the time of testing), weather my script is running or not(ps -ef | grep scriptname shows that script is running), I tried to create empty files to see file system is not mounted as readonly and I am able to create empty file as well.03:55
neonixcoderAnd the funny part is, I am unable to replicate in our lab.03:56
neonixcoderOur script is doing what it is meant for.03:56
sarnoldyou've got a shell ona system that's showing the issue? that's awesome; you can attach strace to it and find out what it's doing :)03:56
sarnoldstrace -f -o /tmp/strace.log -p pid03:57
patdk-lapyou sure the pci_modem status program isn't hanging?03:57
patdk-lapdue the the 3g issue03:57
neonixcodergreat, sarnold. I forgot to do this.03:58
neonixcoderI will do this when we see this issue again.03:58
neonixcoderand see if I can find, what our script is doing.03:59
sarnoldgood point, if any of the called programs hang, it might be hard to tell it apart..04:00
patdk-lapwell, we where assuming it rebooted04:01
patdk-lapbut if it isn't :)04:01
neonixcoderpatdk-lap: I too have same doubt about pci_modem status program. I will try to check next time when it happens..04:02
patdk-lapadd a timeout timer to it04:05
patdk-lapin shell script, that could be annoying though04:05
sarnolda watchdog for the watchdog script :)04:05
* sarnold runms04:05
patdk-lapwell, the watchdog script doesn't have a proper gun04:06
neonixcoderpatdk-lap: The status command worked fine till 4 bad pings.. have a look at my log file at http://pastebin.com/eztiyvub04:08
neonixcoderpatdk-lap: From the 5th bad ping, it stopeed writing to log file, I see other programs are able write to same log file successfully.04:09
=== cz2 is now known as networkmanager
=== networkmanager is now known as NetworkManager
=== Lcawte|Away is now known as Lcawte
=== Lcawte is now known as Lcawte|Away
=== NetworkManager is now known as cz2
SyncopatedFoolHaving trouble setting up postfix on Ubuntu server, I have POP3 working properly, but SMTP without authentication won't allow me to send outgoing mail, and I am having troubles setting up authentication.12:25
SyncopatedFoolmainly I have the configuration set up fine, but I have no clue how to setup the auth key to make it work12:26
SyncopatedFoolHaving trouble setting up postfix on Ubuntu server, I have POP3 working properly, but SMTP without authentication won't allow me to send outgoing mail, and I am having troubles setting up authentication.12:47
SyncopatedFoolmainly I have the configuration set up fine, but I have no clue how to setup the auth key to make it work12:47
SyncopatedFoolbeen using this guide to set up smtp auth, but failing at proving authentication http://postfix.state-of-mind.de/patrick.koetter/smtpauth/smtp_auth_mailclients.html12:49
=== Lcawte|Away is now known as Lcawte
eahmedshendyIs that right that Ubuntu 14.04 the last version with Upstart ? I'm studying Upstart to add some scripts at the boot time, but someone told me it is deprecated with systemd?14:23
TJ-eahmedshendy: it's optional on 15.04, and not used on 15.1014:30
eahmedshendyTJ-: Ok, now we are working on Ubuntu 14.04, and I am studying Upstart, Am I wrong?14:53
eahmedshendyI have this error:http://paste.ubuntu.com/13492759/14:53
eahmedshendyHi, anybody here?15:12
tewardrbasak: ping15:32
jgrimmteward, fyi rbasak is out on holiday all this week15:33
tewardthen nevermind :)15:34
* teward yawns15:34
teward(it's only the nginx merge that's blocked because testing is in failure state due to the sbuild schroots being busted by a bad bug in ubuntu)15:34
jgrimmcpaelzer, feel free to add yourself to the irc meeting rotation -> https://wiki.ubuntu.com/ServerTeam/Meeting16:23
=== Lcawte is now known as Lcawte|Away
cpaelzerjgrimm: done16:33
=== Lcawte|Away is now known as Lcawte
Sazpaimonwhen building an ubuntu server AMI, is there a known list of directories/files to clear/empty out in order to have a clean image?18:34
SazpaimonI know stuff like things in /var/log, and maybe .bash_history, but I can't think of anything else18:34
SazpaimonI also dont know if completely deleting the contents of /var/log is safe18:35
sarnoldprobably /etc/ssh stuff too, to ensure you generate a unique host key on reboot ..18:35
helowhere does openssh-server log to?18:35
heloi see it is set to use AUTH syslog, but i see nothing in syslog or auth.log18:35
sarnold/var/log/auth.log on my 14.04 lts system18:36
patdk-wkheh, /var/cache18:41
jellywhere do VMs on amazon get their entropy from? Re: > generate a unique host key on reboot ..19:49
sarnoldjelly: in theory they could use somethinglike virtio-rng http://wiki.qemu.org/Features-Done/VirtIORNG19:51
sarnoldjelly: another option is touse an entropy-as-a-service tool such as pollinate to grab some entropy from a service such as entropy.ubuntu.com19:52
sarnoldjelly: another option is to use something in cloud-init to provide a seed, but I'm not sure if that's pre-made and ready to go or not...19:53
jellythanks, I was not aware of half that stuff19:56
Logos01Howdy, folks -- I'm trying to set up an stunnel connection between an Ubuntu 14.04 machine and a CentOS 7 machine -- I'm invoking stunnel4 as the client on the Ubuntu machine. The problem is that even though I've hard-coded the config file to use TLSv1, it's still giving me SSLv3 errors21:15
Logos01As though it's not using TLSv1 at all.21:15
Logos01Anyone run into this before?21:15
Logos01I keep getting "SSL3_GET_RECORD:wrong version number" in /var/log/syslog21:17
sarnoldthat alone wouldn't worry me too much, there's enough similarity between sslv3 and tls1 that there's function re-use, keeping same names etc..21:20
sarnoldtry confirming both endpoints with openssl s_client or .. wireshark? ettercap? to try to figure out which specific protocol is  being used21:20
Logos01Well, I've tried hard-coding tlsv1 and/or sslv3 on both ends and I keep getting that error.21:23
Logos01I keep seeing disconnects as well.21:23
denbeirenhi all,.. i was thinking if there is a distro or tool that specifies on "booting a buggy computer and backing up all of the standard data to a networklocation" type of thing21:30
Logos01Hrm... openssl s_client -connect ubuntu.machine:port -tls1 -CAfile /etc/stunnel/ca-chain.pem returns "no peer certificate available"21:32
Logos01What the devil did I do ... this was working a minute ago. :(21:32
=== Lcawte is now known as Lcawte|Away
=== Lcawte|Away is now known as Lcawte
sarnolddenbeiren: knoppix is popular for that23:03
sarnolddenbeiren: ubuntu livecd, if that still works..23:03
* teward pokes sarnold23:08
sarnoldafternoon teward :)23:08
tewardsarnold: got a few moments for a PM?23:08

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!