| HazeAI | Hello folks, I've been using Ubuntu 14.04 on a bunch of beagle bone blacks at my workplace and I've been having a problem with them that I'm wondering if anyone here might be able to help me with | 17:25 |
|---|---|---|
| HazeAI | I keep the software that I wrote to run on them in a git repository, each beagle has a clone on it, and a cron pulls them every 5 minutes to keep them up to date | 17:26 |
| HazeAI | When I pull power without shutting them down first, they have about a 50-70% chance of corrupting the contents of the git repository so I have to remote in and re-clone it and patch things up | 17:26 |
| HazeAI | Everything else seems fine, but the contents of my git repo are all 0 byte files | 17:26 |
| HazeAI | I understand that powering down correctly would be best practice, but it seems odd that such a specific set of files are so likely to be corrupted when I pull power | 17:28 |
| HazeAI | Has anyone else experienced anything similar? | 17:28 |
| rbasak | Not experienced that, but I'm curious. What filesystem are you using? | 17:35 |
| rbasak | Also, have you tried a different SD card manufacturer? AIUI, much hardware lies about when they've really committed data. | 17:35 |
| infinity | HazeAI: A quick hack might be to throw a "sync && sync" at the end of your git pull cron job. | 17:35 |
| rbasak | And what SD card manufacturer are you using right now? | 17:35 |
| HazeAI | I have Ubuntu flashed to the eMMC - No SD card in use | 17:45 |
| ogra_ | well, forcing the sync is your best option then | 17:45 |
| HazeAI | Thanks for the suggestion infinity - I'll test out adding "sync && sync" to my git pull cron | 17:46 |
| rcn-ee | HazeAI, do you also have it mounted as "noatime" ? | 17:46 |
| HazeAI | Apologies rcn-ee, I'm not sure what you mean. It's mounted to /dev/mmcblk0p2, is there a command I could run to check what you are looking for? | 17:47 |
| HazeAI | And for adding sync && sync to my cron, do you mean: sync && git -C /usr/local/bin/marshal pull && sync | 17:48 |
| HazeAI | or : git -C /usr/local/bin/marshal pull && sync && sync | 17:48 |
| ogra_ | mount|grep noatime | 17:48 |
| HazeAI | looks like yes: '/dev/mmcblk0p2 on / type ext4 (rw,noatime,errors=remount-ro)' | 17:49 |
| rcn-ee | any chance can you change your procedure from yanking power to pushing the "power button"? | 17:51 |
| infinity | HazeAI: The latter. | 17:52 |
| infinity | HazeAI: Not that a forced sync will help you if you pull the plug during the sync or during the git pull, but it minimizes the window for data being in flight. | 17:53 |
| infinity | (It'll also grind the machine to a halt while it's doing it, if there's a lot to sync) | 17:54 |
| HazeAI | Great! Yeah, I am definitely training users to actually press the power button but I can't control folks 100% of the time, so I'm just trying to get the rate of occurrence down to something manageable | 17:58 |
| HazeAI | I initially told people the risk of pulling power would be negligible (ugh, my own fault), and I've been trying to find the cause of these random corruptions for a good month now | 17:59 |
| HazeAI | Didn't realize what it was until I flashed a fresh batch, then went back to test them, and most of their git repos were corrupted | 17:59 |
| HazeAI | Fantastic, that seemed to do the trick. I've pulled the plug 5 times in a row now with no corruption. Thanks for your help infinity, ogra, and rcn-ee! | 18:04 |
| ogra_ | mostly infinity ... :) | 18:04 |
| ogra_ | since he doesnt have anything to do today :) | 18:05 |
| infinity | *cough* | 18:13 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!