For me, the motivators to wait for LogFS are mainly the facts that it
can work on traditional block devices, and not only on pure flash:

1. It works on normal block devices and it supports transparent compression

Today, a 64 GB SSD/flash-based media costs ~about the same as a 1 TB
hard disk. This makes flash very expensive to use; compression can
compensate that cost a bit (will depend on the usage, of course).

I believe there is no other Linux filesystem which can do transparent
compression on block devices.


2. It does wear-levelling also on normal block devices

Although it doesn't sound normal to do wear-levelling twice (most
flash-based block devices do wear-levelling on their own), I had a flash
corruption after just ~one month of using RAID bitmap on a IDE-flash
disk formatted with ext3. Apparently, device-level wear-levelling wasn't
spreading updates of RAID bitmap file well enough.


I too wouldn't use LogFS today in a production environment - it is still 
not feature complete and not widely tested.
I wouldn't use btrfs or ext4 today for the very same reason.


-- 
Tomasz Chmielewski
http://wpkg.org


--

[ message continues ]

I'd like compressed filesystem for maps and lingvistic data... but
will the flash flesystems have 'reasonable' performance when used on
harddrive?
							Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
--

[ message continues ]

If someone implemented readpages(), that might be possible - depending
on your definition of 'reasonable' and your workload.

Writes will hit the disk roughly in order.  When readahead requests a
bunch of pages, there is some chance of them being adjacent and having
the block layer combine most of the bios into a few large ones.

The main drawbacks are:
- No reservations.  If data is written in random order or several
  writers chew away in parallel, write order will be fairly pessimal.
- No readahead yet.  Wouldn't be hard to do.
- Garbage collection completely ignores fragmentation.  If segments are
  needed and one contains a nice long extend from a single file, that
  data will be written elsewhere, often split between two segments.
  Rinse, repeat and fragmentation will increase over time.
- File creation/deletion currently will cause disk heads to jump in
  triangles.  This hurts write performance on most flash media as well,
  so it will get changed reasonably soon.

All of those are solvable.  Some will definitely be solved because the
help performance on flash media as well.  Other may or may not.

Jörn

-- 
When people work hard for you for a pat on the back, you've got
to give them that pat.
-- Robert Heinlein
--

[ message continues ]

Sorry Thomasz, for me this makes zero sense. There are _much_ better file
systems for block devices. UBIFS may work on top of a block device as
well (just needs few hacks to make it possible) - it is not a problem
at all, it is just _senseless_.

JFFS2/UBIFS/LogFS is a separate _class_ of file-systems. The are designed
for _flash_, which has completely different work model then block device.
They are _native_ flash file systems.
Here are more details: http://www.linux-mtd.infradead.org/faq/general.html#L_mtd_vs_hdd

The traditional FSes _cannot_ work on top of flash. The solution for this
is using FTL, which emulates a block device on top of flash. It _hides_ the
real device, and fakes a block device for you. And you can use traditional
FSes on top of that fake block device.

The whole _point_ of this separate class of FSes is because we believe
we may do much _better_ job if we use flash _natively_, instead of using
FTL. FTL is the place where you loose performance, reliability, and so on.

And you are saying about using a native flash FS on top of a block device
like an SD card. This is just not sane: SD card first emulates a block device
for you, looses performance at this point, then you again emulate a flash
on top of this, and suffer from this again.

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)
--

[ message continues ]

About a year ago I would have used roughly the same argument.  But in
the meanwhile I have accepted the fact that any piece of flash I'll use
in a notebook in the next five years will likely use SATA or USB as a
transport medium.

Jörn

-- 
...one more straw can't possibly matter...
-- Kirby Bakken
--

[ message continues ]

Well, Thomasz was talking not about the transport I believe. If you have a
raw flash access via SATA or USB, then this is handled on the driver level
and does not any require FS support. JFFS2 or UBIFS may also work with such a
device, because it is just raw flash. But I know that you know that most of
the FTL-enabled stuff does not give use raw flash access.

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)
--

[ message continues ]

When the day comes and I can actually use raw flash with a SATA or USB
transport medium, I will be more careful and not imply that they all
expose a block device interface.  Today they do.

[ With the exception of Alauda, which has a userbase of about 1 - me. ]

Jörn

-- 
It does not matter how slowly you go, so long as you do not stop.
-- Confucius
--

[ message continues ]

Such as?
Flash (also on block devices) is slow and expensive (when compared to 
modern hard disks) and therefore compression is *very* useful here.
Not only it can potentially save you money; reads and writes will be 
smaller/faster (unless you're editing music and videos, where one won't 
use flash anyway), flash will wear out slower etc.

Is there a filesystem for Linux which can provide transparent 
compression and works for block devices (other than user-space NTFS or 

Do you mean using hacks like block2mtd? It's hacky, and pretty hard to 
boot a system this way (need to build own initramfs, with a static 
block2mtd or loads of libraries - not something an average user would 

It's a good comparison, but it doesn't show disadvantages of using 
traditional filesystems on flash-based block devices.

I just mentioned the reasons why a filesystem like LogFS would be useful 
on block devices: there are valid reasons to do it.



True.
Unfortunately, there is no way to access flash directly on flash-based 
block devices (USB-sticks, IDE-flash disks, SSD disks etc.).

Therefore, could an answer be: use a traditional filesystem?

Unfortunately, traditional filesystems were rather designed for rotating 
media / cheap disks (no transparent compression; tend to accumulate 

Performance is only one factor in the equation. Other factors are: cost 
and reliability.

I speak from experience: flash-based block devices tend to have poor 
wear-levelling (at least Transcend IDE-flash disks).
To reproduce:
- format a 2 GB Transcend IDE-flash disk with ext3
- write a small file (50-100 kB)
- update that file ~several hundred thousand times - as you finish, 
IDE-flash disk will have 200-300 badblocks

If wear-levelling on the underlying IDE-flash device was any decent, 
writes would be spread on the whole ~2GB surface, totalling in many more 
successful writes.


Again: this is my experience, although it may contradict the theory 
underlined in the link you gave earlier.
You ...
[ message continues ]

Well, if you are ready to trade performance to compression, then well,
go ahead :-) May be I used too strong wording, but I wanted to say then
use raw flash then. But I'd also consider implementing compression support
Well, ok, it still sounds strange for me, but you may use JFFS2 and UBIFS
Yeah, that's bad. But if you have a bad FTL, surely there is not guarantee
a flash FS will help? Isn't it better to use better hardware?

We did some experiments with MMC cards and we were unable to wear them
out with re-writing the same sectors again and again. This suggests there
_is_ better FTL hardware then that USB stick you was using.

Anyway, your original mail said Logfs can work with block devices. My answer -
UBIFS too, but this is very strange to do this IMO. But OK, it might is not
senseless, sorry for the wording. :-)

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)
--

[ message continues ]

I was thinking why my IDE-flash disk died so soon[1] and how efficient 
can an internal wear-levelling be in devices which hide its "flashiness" 
(USB-sticks, IDE-flash disks etc.).

Internal wear-levelling mechanism doesn't have a clue about free space 
on the filesystem - it that case, how can it do any efficient 
wear-levelling?



[1] Well, it didn't die, really. Once I removed the file which was 
showing I/O errors and did "dd if=/dev/zero of=bigfile", there are no 
badblocks anymore - probably remapped.



-- 
Tomasz Chmielewski
http://wpkg.org
--

[ message continues ]

Could have been soft errors.  Heaven knows how those devices report
uncorrectable errors, but claiming the block to be bad is a good guess.
Once blocks with uncorrectable errors are rewritten (the physical
blocks, not what the device reports), they are perfectly usable again.

Jörn

-- 
They laughed at Galileo.  They laughed at Copernicus.  They laughed at
Columbus. But remember, they also laughed at Bozo the Clown.
-- unknown
--

[ message continues ]

I think he refers to flash disks appearing as block devices, like
usb sticks or similar.

-Andi
--

[ message continues ]

Right, I also meant that in my opinion it makes more sense to use traditional
file-systems like ext3 on USB-key/MMC and the like stuff (which I confusingly
referred as "block devices"), or may be something more "heavy-weight" like
XFS or JFS (never tried them, though).

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)
--

[ message continues ]

Well, even auto-levelling storage should benefit from a filesystem which 
minimizes the total number of flash sectors churned, which means doing 
as few writes as possible and to large, contiguous sections.

	-hpa
--

[ message continues ]

Exactly. At exosec, we ship one appliance which writes statistics to a
partition on a compactflash every 5 minutes. We preferred to go with JFFS2
exactly because of this reason. We never had any problem proceeding this way.
I'm not sure if it would have been the same with ext2 though.

Willy

--

[ message continues ]

Yes, as I agreed in a previous mail this may make sense in some cases.

But in general it is not a good approach. Basically, it is wastage of resources.
Indeed, first the firmware on MMC/SD/etc makes efforts to make flash look
like a block device. It gives you in-place updates, but by cost of performance
and reliability. Then you just drop this nice property, and use JFFS2, which
assumes it has only out-of-place updates. But if this solves the task you have
- fine!

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)
--

[ message continues ]

One of the problem is that unless you crash-test your flash cards, you will
never know if their wear-leveling algorithm is fine or not. And I suspect
that nowadays, due to very large consumer demand, flash cards price drop
at the cost of reliability. I think that most of those not flagged
"industrial-grade" do absolutely zero wear-leveling, because they are sold
to people using them in digital cameras, and they will never kill their

I'm certainly not the only one with this requirement. A lot of embedded
motherboards come with IDE compactflash connectors. This is very convenient,
but if you need to keep informations between reboots, you have to write to
the device anyway. If you need to do that very often, either you pray for
the device to be very reliable, or you take all the chances on your side
by adding your own wear-leveling "just in case".

Cheers,
Willy

--

[ message continues ]

Sure, I know about this problem. My point was that in this case it is wiser
to use bare flash and put JFFS2 on it, instead of using this black box

OK. Fair enough. Although stuff exists, but this does not necessarily mean
this a good design :-)

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)
--

[ message continues ]