login
Search
Hello
I have a question for you. Which filesystem do you think of as most reliable?
My experience with ext3,reiserfs and xfs isn't great, many times important files have been lost due to powerfailure/kernelpanic(the filesystem is never cleanly umounted). The files that are destroyd is usually files-in-use, so I guess it has something to do with cached data not written to disk.
Fortunately kernelpanics and power failure hasn't been problem for me until recently. Once in a while my laptop doesn't resume from suspend-to-ram and this leads to a dirty filesystem. Which again means that some files is allmost allways broken. /var/lib/dpkg/* hurts alot or dot files in $HOME. When using ext3 the broken files usually contains some garbage, but with xfs it's even worse. _Every time_ there's a kernelpanic on suspend/resume, some files in my xfs fs at /home is filed with zero. Again the files affected are usually those in-use. Firefox bookmarks.html, beep-media-player settings,vmware files[ :( ] and so on.
It's not ok when important files like that becomes corrupted just because of a kernelpanic. I manage a network with Windows desktops(more than 600) and servers(more than 20) , and if I remember right, files on those NTFS fs have never been broken due to powerfailure.
So why is it that my xfs/ext3 fs allways fucks up mye files? Is there some way to force less caching of data? I guess that it would probably affect performance :(
It's a frustrating situation so I would love if you have something to say :-)
Regards,
anonymous coward ;)
SYNC
well, when I was in kernel-programmer business ...
I used reiserfs + "SYNC" for my /home/work-space/ (SYNC is a mount option (you can set it in /etc/fstab) ... now the kernel doesn't cache _writing_ anymore).
But please use it with care... if you've just one big partition for your system... don't do it! (you should _outsource_ atleast /tmp! (and if you've still some room I suggest to do the same with /home /var and /usr)
Force Sync in crontab
if you put the "sync" command in your crontab every minute or 5 it will make sure that the most data you can lose if under 5 (or 1) minute(s) old. It's not as foolproof as running in synch mode, but you won't sacrifice performance near as much.
thanks for replies
Seems like a good sugestion, I wonder if it prevents xfs from zero'ing the files.
You would think the suspend script in Ubuntu runs sync, but I'll check just to be sure.
Yes, XFS zeros the file becau
Yes, XFS zeros the file because the file was in mid-write when it was lost. Or the transactions may have been commited out of order. It is annoying tho. Ony my root filesystem I run in sync mode, except for software updates.
If you're going to suspend, I would think that mount / -o remount,sync would be a great idea.
With my laptop I had data lo
With my laptop I had data loss of newly written files regularly. The cause was a partition with wrong beginning and end points. Use fdisk to verify that no problems exists with your partition.
ext3 is very good, but...
I find ext3 very reliable if not the best FS around. If you have trouble with it, it's probably because you have your disk cache setup in writeback mode. Almost all todays IDE/SATA disks come with write cache turned on by default. While ext3 goes to great length to keep your data safe and consistent it actually doesn't have direct control of your disk(s), so if disk is "lying" that block is safe on platter when it actually is not (but in its volatile memory, instead) ext3 (or any other filesystem) can't do much about it. And with todays larger and larger on disk caches (4, 8, 16MB) there's a lot of data to be lost when you suddenly lose power.
OK, enough theory. Do this: mount all of your filesystems with I/O barrier turned on. Something like this (example line from my /etc/fstab):
/dev/sda5 / ext3 defaults,barrier=1 1 1
What it means is that when ext3 really needs data to be on the platter (and not in the disk cache!) the disk will comply (if it's not too old, hopefully not). That is somewhat better than mounting partitions in so called sync mode where each and every write operation would be synchronous and everything would get much slower. Try it and you will be amazed how ext3 can be good when it comes to reliability in corner cases.
And BTW, that sync from cron idea wouldn't help at all. First of all, sync is asynchronous - it doesn't guarantee that data is on disk when it returns. And second thing: ext3 already takes care not to cache things in memory too long (or when it would not be appropriate) so don't bother with sync it will just slow things down with none increased reliability.
Hope this sheds some light on your problem.
--
Zlatko
http://linux.inet.hr/
sync
If sync does not write all dirty buffers to disk before returning, then what does it do? When I have used it, there is a pause that is longer if there is a lot of stuff to be writen out.
on sync
It just schedules writes. So yes, everything is going to be written on disk but in some indefined time in the future, but you have no control over it. And sync(2) returns much before that time, probably before even first block is written on the disk.
fsync(2) is a completely different beast, it won't return before disk driver says that block is on the disk (or in the disk's buffer, if you have writeback cache enabled).
a few tips
Linux writes all the data out after 30 seconds or so, so sync in your crontab will be no help. You do want the disk to have write caching turned off (use hdparm for that). I would sync (by tradition, three times) befor suspending. Turn data journaling on for ext3, I don't know that you have a choice with reiserfs, and I think reiser4 (not in the stock kernel yet) does it by default. I don't know about xfs (or jfs, for that matter). Off the top of my head, I don't know how to tell the kernel to flush the data quicker. From what I can remember, power saving laptops tips pages will tell you how to change the flush parameters for the kernel (granted, they want to go in the other direction).
In short, sync befor suspending, probably put it in your power managment scripts as well as doing it by hand, and make sure your fs is set to data=journaled .
And frequent backups are good.