Compact Flash (and other similar media) does wear levelling, so essentially, 
even if we write to the same fixed location, in reality, it will mostly go to a 
different area on flash each time.

As Compact Flash and its wear levelling does not know about free space on the 
filesystem, the wear levelling's effectiveness can be only limited - writes 
won't spread on the whole free area of the flash.

Does anyone know how wear levelling is done in these devices? Perhaps it will 
differ from a manufacturer to manufacturer, but I guess they have a free area we 
normally use to store data, and some reserved area used just for wear levelling 

noatime implies nodiratime, so there is no need to add the latter.


-- 
Tomasz Chmielewski
http://wpkg.org

--

[ message continues ]

The wear leveling algorithm depends on the CompactFlash manufacturer.
Most manufacturers define a certain number of areas on a CompactFlash.
Each area consists of two or more sectors which are counted in the
total size of the CompactFlash, and one or more spare sectors. Wear
leveling is carried out within an area by spreading writes over the
sectors in an area. The controller in the CompactFlash stores the data
about which sectors are used for storing which data persistently. I
know of only one manufacturer who makes the CompactFlash controllers
carry out wear leveling over the whole CompactFlash. Some CompactFlash
devices even allow to query statistics about the wear leveling

Thanks for the info.

Bart.
--

[ message continues ]

How does it work, then?
How can it do wear levelling over the whole CF if some (or most) area of CF is 
already used by our precious data/metadata?
It would have to know the areas where no data is stored, but it contradicts the 
CF <-> filesystem separation.


-- 
Tomasz Chmielewski
http://wpkg.org
--

[ message continues ]

The wear leveling algorithm even works if the CompactFlash controller
has no knowledge of the sectors on which filesystem data is stored.
All that is needed is some spare space on the CompactFlash that is not
made visible through the IDE interface, a controller that is able to
remap sectors and that stores this mapping persistently.

Bart.
--

[ message continues ]

It don't necessarily need to know. It can swap two used blocks, one 
often-used and one
rarely-used.  That way the rarely-used block is rewriten over the 
previously busy
block, and the busy block is moved to the rarely used area that isn't worn.
This implies an extra write whenever a busy block is moved.  Don't know if
anybody do this, but the technique is simple enough.

CF-filesystem separation is necessary, for they can't know in advance what
filesystem or partitioning scheme will be used.  (I have ext3 on CF, for 
example...)

Helge Hafting


--

[ message continues ]

experience report :
In June 2004 we migrated to CF after bad hard disk crashes experience...

Most of the CF do NOT do wear leveling, you have to ask the manufacturer
if it does it or not, generally it is called industrial... and normally,
cost much more .
So, for low cost cf I assume you have to do wear leveling with the
kernel ...otherwise it is already "wear leveling inside"
Since the technology evolves very fast it may be possible today that the
production's cost of wear leveled CF will be the same as standard one
and wear leveling will be a standard ?


I think it should be possible to know if the Cf supports wear leveling
using the the identify device command (0xEC) and look at words 82 to 87 
(cf. CF specifications http://www.compactflash.org/specdl1.htm)
Depending on the application, some system hacks like noatime... need to
be setted up.

we use  silicon systems CF  with wear leveling inside  , we write a few
kb data back each 1/4 h  on it.(log /tmp files  are redirected to ram)

Since June 2004 ALL  hard disk  systems based replaced with CF (env. 100
units)   have not failed.

If somebody could report experience with other brand CF including wear
leveling , it will be fine
some other brand I know only the name :
www.apro-tw.com
http://www.afaya.com.tw/ (spec tells it supports wear leveling)


Best Regards
S.Ancelot

--

[ message continues ]

Silicon Systems CompactFlashes are the among the most reliable
CompactFlashes I have used in embedded devices. See also
http://www.siliconsystems.com/silicondrive/whitepapers/SSWP03-Endurance-R.pdf
for a whitepaper that explains their wear leveling and error
correction algorithms. Furthermore, Silicon Systems has a technology
called SiSMART that allows to monitor by how far the CompactFlash is
worn out, such that it can be monitored whether or not it is time to
replace the CompactFlash. (Note: I am not affiliated in any way to
Silicon Systems.)

Bart.
--

[ message continues ]

..

Mmm.. according to that document, the "dynamic wear leveling" devices
might benefit from having Linux issue CF-Erase commands to unallocated sectors,
so that the onboard CF controller can then add those sectors to the wear-leveling
rotation scheme.

There was discussion at LSF'08 about possible filesystem hooks that could
be used for this (and other) purposes when deleting files.

Cheers
--

[ message continues ]

We too have switched to Silicon Systems and are very happy with them.
And even industrial temperature versions are hardly expensive.  They may
cost more than you pay for a generic slow piece of junk at a retail
store, but you are getting a better card.

-- 
Len Sorensen
--

[ message continues ]

Interestingly, we once got a bunch of those and had a lot of problems
with them. Typically unbootable systems. We finally noticed that there
was a real corruption problem, because 3 MD5s of the kernel in a row
returned 3 different values. Our provider assured us that he never heard
about that, and insisted that we try again on a new batch. We have had
no trouble since. I guess we really got a bad batch. At least it was
noticeable early in the deployment process (the worst ones being a few
days after installation). However, we never had any problem with PQI

Willy

--

[ message continues ]

Certainly some cards will take blocks that haven't been changed very
much and exchange them for blocks that have been written more often
(this being physical blocks, not the virtual mapping shown to the
outside).  That way the wear gets spread around more even if some parts
of the flash are never changed (from the point of view of the outside).

The actual algorithm used is vendor specific.

-- 
Len Sorensen
--

[ message continues ]