For me, the motivators to wait for LogFS are mainly the facts that it

can work on traditional block devices, and not only on pure flash:
1. It works on normal block devices and it supports transparent compression
Today, a 64 GB SSD/flash-based media costs ~about the same as a 1 TB

hard disk. This makes flash very expensive to use; compression can

compensate that cost a bit (will depend on the usage, of course).
I believe there is no other Linux filesystem which can do transparent

compression on block devices.
2. It does wear-levelling also on normal block devices
Although it doesn't sound normal to do wear-levelling twice (most

flash-based block devices do wear-levelling on their own), I had a flash

corruption after just ~one month of using RAID bitmap on a IDE-flash

disk formatted with ext3. Apparently, device-level wear-levelling wasn't

spreading updates of RAID bitmap file well enough.
I too wouldn't use LogFS today in a production environment - it is still

not feature complete and not widely tested.

I wouldn't use btrfs or ext4 today for the very same reason.
--

Tomasz Chmielewski

http://wpkg.org
--

Sorry Thomasz, for me this makes zero sense. There are _much_ better file

systems for block devices. UBIFS may work on top of a block device as

well (just needs few hacks to make it possible) - it is not a problem

at all, it is just _senseless_.
JFFS2/UBIFS/LogFS is a separate _class_ of file-systems. The are designed

for _flash_, which has completely different work model then block device.

They are _native_ flash file systems.

Here are more details: http://www.linux-mtd.infradead.org/faq/general.html#L_mtd_vs_hdd
The traditional FSes _cannot_ work on top of flash. The solution for this

is using FTL, which emulates a block device on top of flash. It _hides_ the

real device, and fakes a block device for you. And you can use traditional

FSes on top of that fake block device.
The whole _point_ of this separate class of FSes is because we believe

we may do much _better_ job if we use flash _natively_, instead of using

FTL. FTL is the place where you loose performance, reliability, and so on.
And you are saying about using a native flash FS on top of a block device

like an SD card. This is just not sane: SD card first emulates a block device

for you, looses performance at this point, then you again emulate a flash

on top of this, and suffer from this again.
--

Best Regards,

Artem Bityutskiy (Артём Битюцкий)

--

I think he refers to flash disks appearing as block devices, like

usb sticks or similar.
-Andi

--

Right, I also meant that in my opinion it makes more sense to use traditional

file-systems like ext3 on USB-key/MMC and the like stuff (which I confusingly

referred as "block devices"), or may be something more "heavy-weight" like

XFS or JFS (never tried them, though).
--

Best Regards,

Artem Bityutskiy (Артём Битюцкий)

--

Well, even auto-levelling storage should benefit from a filesystem which

minimizes the total number of flash sectors churned, which means doing

as few writes as possible and to large, contiguous sections.
	-hpa

--

Exactly. At exosec, we ship one appliance which writes statistics to a

partition on a compactflash every 5 minutes. We preferred to go with JFFS2

exactly because of this reason. We never had any problem proceeding this way.

I'm not sure if it would have been the same with ext2 though.
Willy
--

Yes, as I agreed in a previous mail this may make sense in some cases.
But in general it is not a good approach. Basically, it is wastage of resources.

Indeed, first the firmware on MMC/SD/etc makes efforts to make flash look

like a block device. It gives you in-place updates, but by cost of performance

and reliability. Then you just drop this nice property, and use JFFS2, which

assumes it has only out-of-place updates. But if this solves the task you have

- fine!
--

Best Regards,

Artem Bityutskiy (Артём Битюцкий)

--

One of the problem is that unless you crash-test your flash cards, you will

never know if their wear-leveling algorithm is fine or not. And I suspect

that nowadays, due to very large consumer demand, flash cards price drop

at the cost of reliability. I think that most of those not flagged

"industrial-grade" do absolutely zero wear-leveling, because they are sold

to people using them in digital cameras, and they will never kill their
I'm certainly not the only one with this requirement. A lot of embedded

motherboards come with IDE compactflash connectors. This is very convenient,

but if you need to keep informations between reboots, you have to write to

the device anyway. If you need to do that very often, either you pray for

the device to be very reliable, or you take all the chances on your side

by adding your own wear-leveling "just in case".
Cheers,

Willy
--

Sure, I know about this problem. My point was that in this case it is wiser

to use bare flash and put JFFS2 on it, instead of using this black box
OK. Fair enough. Although stuff exists, but this does not necessarily mean

this a good design :-)
--

Best Regards,

Artem Bityutskiy (Артём Битюцкий)

--

Such as?

Flash (also on block devices) is slow and expensive (when compared to

modern hard disks) and therefore compression is *very* useful here.

Not only it can potentially save you money; reads and writes will be

smaller/faster (unless you're editing music and videos, where one won't

use flash anyway), flash will wear out slower etc.
Is there a filesystem for Linux which can provide transparent

compression and works for block devices (other than user-space NTFS or 
Do you mean using hacks like block2mtd? It's hacky, and pretty hard to

boot a system this way (need to build own initramfs, with a static

block2mtd or loads of libraries - not something an average user would 
It's a good comparison, but it doesn't show disadvantages of using

traditional filesystems on flash-based block devices.
I just mentioned the reasons why a filesystem like LogFS would be useful

on block devices: there are valid reasons to do it.
True.

Unfortunately, there is no way to access flash directly on flash-based

block devices (USB-sticks, IDE-flash disks, SSD disks etc.).
Therefore, could an answer be: use a traditional filesystem?
Unfortunately, traditional filesystems were rather designed for rotating

media / cheap disks (no transparent compression; tend to accumulate 
Performance is only one factor in the equation. Other factors are: cost

and reliability.
I speak from experience: flash-based block devices tend to have poor

wear-levelling (at least Transcend IDE-flash disks).

To reproduce:

- format a 2 GB Transcend IDE-flash disk with ext3

- write a small file (50-100 kB)

- update that file ~several hundred thousand times - as you finish,

IDE-flash disk will have 200-300 badblocks
If wear-levelling on the underlying IDE-flash device was any decent,

writes would be spread on the whole ~2GB surface, totalling in many more

successful writes.
Again: this is my experience, although it may contradict the theory

underlined in the link you gave earlier.

You have...
[ message continues ]

Well, if you are ready to trade performance to compression, then well,

go ahead :-) May be I used too strong wording, but I wanted to say then

use raw flash then. But I'd also consider implementing compression support

Well, ok, it still sounds strange for me, but you may use JFFS2 and UBIFS

Yeah, that's bad. But if you have a bad FTL, surely there is not guarantee

a flash FS will help? Isn't it better to use better hardware?
We did some experiments with MMC cards and we were unable to wear them

out with re-writing the same sectors again and again. This suggests there

_is_ better FTL hardware then that USB stick you was using.
Anyway, your original mail said Logfs can work with block devices. My answer -

UBIFS too, but this is very strange to do this IMO. But OK, it might is not

senseless, sorry for the wording. :-)
--

Best Regards,

Artem Bityutskiy (Артём Битюцкий)

--

I was thinking why my IDE-flash disk died so soon[1] and how efficient

can an internal wear-levelling be in devices which hide its "flashiness"

(USB-sticks, IDE-flash disks etc.).
Internal wear-levelling mechanism doesn't have a clue about free space

on the filesystem - it that case, how can it do any efficient

wear-levelling?
[1] Well, it didn't die, really. Once I removed the file which was

showing I/O errors and did "dd if=/dev/zero of=bigfile", there are no

badblocks anymore - probably remapped.
--

Tomasz Chmielewski

http://wpkg.org

--

Could have been soft errors.  Heaven knows how those devices report

uncorrectable errors, but claiming the block to be bad is a good guess.

Once blocks with uncorrectable errors are rewritten (the physical

blocks, not what the device reports), they are perfectly usable again.
Jörn
--

They laughed at Galileo.  They laughed at Copernicus.  They laughed at

Columbus. But remember, they also laughed at Bozo the Clown.

-- unknown

--

About a year ago I would have used roughly the same argument.  But in

the meanwhile I have accepted the fact that any piece of flash I'll use

in a notebook in the next five years will likely use SATA or USB as a

transport medium.
Jörn
--

...one more straw can't possibly matter...

-- Kirby Bakken

--

Well, Thomasz was talking not about the transport I believe. If you have a

raw flash access via SATA or USB, then this is handled on the driver level

and does not any require FS support. JFFS2 or UBIFS may also work with such a

device, because it is just raw flash. But I know that you know that most of

the FTL-enabled stuff does not give use raw flash access.
--

Best Regards,

Artem Bityutskiy (Артём Битюцкий)

--

When the day comes and I can actually use raw flash with a SATA or USB

transport medium, I will be more careful and not imply that they all

expose a block device interface.  Today they do.
[ With the exception of Alauda, which has a userbase of about 1 - me. ]
Jörn
--

It does not matter how slowly you go, so long as you do not stop.

-- Confucius

--

I'd like compressed filesystem for maps and lingvistic data... but

will the flash flesystems have 'reasonable' performance when used on

harddrive?

							Pavel

--

(english) http://www.livejournal.com/~pavelmachek

(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

--

If someone implemented readpages(), that might be possible - depending

on your definition of 'reasonable' and your workload.
Writes will hit the disk roughly in order.  When readahead requests a

bunch of pages, there is some chance of them being adjacent and having

the block layer combine most of the bios into a few large ones.
The main drawbacks are:

- No reservations.  If data is written in random order or several

  writers chew away in parallel, write order will be fairly pessimal.

- No readahead yet.  Wouldn't be hard to do.

- Garbage collection completely ignores fragmentation.  If segments are

  needed and one contains a nice long extend from a single file, that

  data will be written elsewhere, often split between two segments.

  Rinse, repeat and fragmentation will increase over time.

- File creation/deletion currently will cause disk heads to jump in

  triangles.  This hurts write performance on most flash media as well,

  so it will get changed reasonably soon.
All of those are solvable.  Some will definitely be solved because the

help performance on flash media as well.  Other may or may not.
Jörn
--

When people work hard for you for a pat on the back, you've got

to give them that pat.

-- Robert Heinlein

--