Hello, here is the second round of UBIFS submission. Please, refer the following URL if you are not familiar what is UBIFS about: http://kerneltrap.org/mailarchive/linux-kernel/2008/3/27/1273514 Additionally, there is a excellent introductional LWN article written by Jonathan Corbet: http://lwn.net/Articles/276025/ (thanks Jonathan). We've addressed most of the requests from the first round and the following is a rough change log: * Store milliseconds for [mca]time stamps in the inodes, not only seconds (requested by Andi Kleen) * Documentation/filesystems/ubifs.txt has been added (requested by Josh Boyer) * do_div() is not anymore used with 'unsigned long long' values (requested by Andrew Morton) * The amount of debugging-related config options was lessened and module parameters were introduced instead (requested by Pekka Enberg and Christoph Hellwig) * Many assertions were removed (requested by Pekka Enberg) * Some debugging checks like custom memory leak and memory pressure checks were removed (requested by Pekka Enberg) Besides, several bugs were fixed both in UBIFS and mkfs.ubifs. The on-flash format has also been changed because of the inode time-stamp format changes (milliseconds were added). We've got few users outside of Nokia who seem to be happy with UBIFS and utilize it. There were some positive feed-backs from the community, e.g. http://kerneltrap.org/mailarchive/linux-kernel/2008/4/18/1466334 (thanks Thomas). We kindly ask for more review and feed-back. We'd also like to get into Andrew's mm tree. Thank you, Adrian Hunter Artem Bityutskiy git-diff --stat --summary -M v2.6.25 Documentation/filesystems/ubifs.txt | 163 ++ fs/Kconfig | 3 + fs/Makefile | 1 + fs/fs-writeback.c | 8 + fs/ubifs/Kconfig | 47 + fs/ubifs/Kconfig.debug | 30 + fs/ubifs/Makefile | 9 + fs/ubifs/budget.c | ...
General comment: - not supporting a flash page size different from the system page size is a horrible thing for people trying to use the same storage on multiple systems. For a block based filesystem that alone would be enough reason not to merge it. For a flash filesystem I'm not entirely sure given that flash isn't moved between systems all that often. VFS/VM interaction comments: - splitting most of the mount code out to build.c is rather odd. Most filesystems have this in super.c - calling convention for mount_ubifs is nasty because it doesn't clean up it's own errors. You might think it's easier to do all that in ubifs_umount but beeing in the process of untangling that mess for xfs I'd recomment against it. Unless there's a very good reason for it functions should always clean up the resources they allocated in the error case. - ubifs_get_sb would benefit from splitting out a ubifs_fill_super routine that allocates a new sb when it's actually needed. - why do you do the commit_on_unmount in your own ->shutdown_super instead of the normal ->put_super? If there's a good reason this at least needs a big comment explaining why. - ubifs_lookup doesn't really need to use d_splice_alias unless you want to support nfs exporting - in ubifs_new_inode you inherit the inode flags from the parent. This probably wants splitting out in a helper that documents explicitly what flags are inherited and what not. Given that you store the general indoe flags settable by chattr in there it seems like a bad idea to inherit them by default. - the read dir implementation won't ever support nfs exporting due to having to keep per open file state. Nor would it support thing like checkpoint and restart. - just opencode you mmap routine, there's nothing helpful in generic_file_mmap if you set your own vm_ops. - ubifs_trunc should never be called on anything but a regular file, so the check for it seems sup...
Err, how are we guaranteed that we do not miss wake-up events then, if we were woken up say, between run_bg_commit(c) and set_current_state(TASK_INTERRUPTIBLE) ? This was the purpose of the 'need_bgt' flag. -- Best Regards, Artem Bityutskiy (Артём Битюцкий) --
Christoph, thanks for the review. We keep full name of the last readdir()'ed direntry in file->private_data and in file->f_pos we store the direntry hash value (32 bits). The only reason we store full name in file->private_data is name hash collisions. We are able to restart readdir()s, but only if there are no hash collisions. Otherwise UBIFS will restart, but probably from the wrong directory entry (the first one in the colliding sequence). In other words, you may do readdir, telldir, close the directory, open it again, seekdir, and continue readdir-ing. This will mostly work as expected unless there was hash collision, in which case you will restart from the first direntry in the colliding sequence. I.e., telldir/seekdir mostly work, but not always. We thought seekdir/telldir was considered as horrible interface anyway so we assumed it is not that big deal if it occasionally does not work as expected. AFAIK there are other FSes which have similar issues. Do all of them solve this in one way or another? We have an idea of storing collision number in directory entries and return this number in the high 32 bits of file->f_pos (the low contain hash value) to make readdir truly restartable. But this requires some non-trivial changes and we'd want to realize if it is really needed. AFAIK, even ext3 may shrink directories and fail to support readdir restarts occasionally, but I am not 100% sure. Could you please elaborate some more why UBIFS won't ever support nfs export? Well, it is also doing write-buffer flushes, not only commits. And we were planning to add background garbage collection there as well. -- Best Regards, Artem Bityutskiy (Артём Битюцкий) --
Yes, because NFS is stateless. The seek offset has to be valid even over a reboot. If you don't handle it you should probably better forbid nfs export completely to avoid subtle problems. -Andi --
Thanks for the review. Well we would like to support nfs one day, so maybe we could just We could get rid of the per-open-file state if we could come up with a scheme that would make fpos unique. At the moment it is not unique because it is the name hash, but we could perhaps allocate a unique number to each colliding entry. We will It is named after Jœrn not journal ;-) How about jnl after Janelle? It uses the same number of characters. --
Do you have the filesystem as a single tarball or patch? This cancer of split-up patches that's spreading quickly all over the linux world makes sensible reviews quite hard unfortunately. --
Will git tree work? http://www.linux-mtd.infradead.org/doc/ubifs.html#L_source -- Best Regards, Artem Bityutskiy (Артём Битюцкий) --
A git tree of just ubifs would be perfect, but the full tree does it aswell. Not really conveniant, but much better than split patches :) --
Christoph, Here is a tarball for you: http://www.infradead.org/~dedekind/ubifs/ubifs-08-05-07.tar.bz2 -- Best Regards, Artem Bityutskiy (Артём Битюцкий) --
So as it is said at the above URL, git://git.infradead.org/~dedekind/ubifs-2.6.git is the current development tree. At the moment it is current Linus's tree + UBIFS. git-diff a15306365a16380f3bafee9e181ba01231d4acd7 would give UBIFS patch. git://git.infradead.org/~dedekind/ubifs-v2.6.25.git has v2.6.25 + UBIFS. I guess this should be ok. -- Best Regards, Artem Bityutskiy (Артём Битюцкий) --
Wait a little, I'll prepare current linux-2.6.git + ubifs tree. -- Best Regards, Artem Bityutskiy (Артём Битюцкий) --
This is one of the most important parts of UBIFS. Since all updates are out-of-place, we need to do garbage collection from time to time, which is implemented in this file. The UBIFS GC does not do much - it just move clean data to the journal and erases the cleaned-up eraseblock. The main trick is done in TNC commit which guarantees that the commit operation is always possible, even if there is no clean space, in which case it may use in-place updates provided by UBI. Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com> Signed-off-by: Adrian Hunter <ext-adrian.hunter@nokia.com> --- fs/ubifs/gc.c | 773 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 773 insertions(+), 0 deletions(-) diff --git a/fs/ubifs/gc.c b/fs/ubifs/gc.c new file mode 100644 index 0000000..7b43655 --- /dev/null +++ b/fs/ubifs/gc.c @@ -0,0 +1,773 @@ +/* + * This file is part of UBIFS. + * + * Copyright (C) 2006-2008 Nokia Corporation. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published by + * the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program; if not, write to the Free Software Foundation, Inc., 51 + * Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + * + * Authors: Adrian Hunter + * Artem Bityutskiy (Битюцкий Артём) + */ + +/* + * This file implements garbage collection. The procedure for garbage collection + * is different depending on whether a LEB as an index LEB (contains index + * nodes) or not. For non-index LEBs, garbage collection finds a LEB which + * contains a lot of dirty spac...
The UBIFS code is large, and we have a plenty of debugging stuff in there which helps to catch bugs. Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com> Signed-off-by: Adrian Hunter <ext-adrian.hunter@nokia.com> --- fs/ubifs/debug.c | 1463 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ fs/ubifs/debug.h | 392 +++++++++++++++ 2 files changed, 1855 insertions(+), 0 deletions(-) diff --git a/fs/ubifs/debug.c b/fs/ubifs/debug.c new file mode 100644 index 0000000..6af0e66 --- /dev/null +++ b/fs/ubifs/debug.c @@ -0,0 +1,1463 @@ +/* + * This file is part of UBIFS. + * + * Copyright (C) 2006-2008 Nokia Corporation + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published by + * the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program; if not, write to the Free Software Foundation, Inc., 51 + * Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + * + * Authors: Artem Bityutskiy (Битюцкий Артём) + * Adrian Hunter + */ + +/* + * This file implements most of the debugging stuff which is compiled in only + * when it is enabled. But some debugging check functions are implemented in + * corresponding subsystem, just because they are closely related and utilize + * various local functions of those subsystems. + */ + +#define UBIFS_DBG_PRESERVE_UBI + +#include "ubifs.h" +#include <linux/module.h> +#include <linux/moduleparam.h> + +#ifdef CONFIG_UBIFS_FS_DEBUG + +DEFINE_SPINLOCK(dbg_lock); + +static char dbg_key_buf0[128]; +static char dbg_key_buf1[128]; + +unsigned int ubifs_msg_flags ...
ubifs.h contains the internal stuff. ubifs-media.h contains the
on-flash format definition and might be copied to user-space
if needed. misc.h contains various inline helpers.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
Signed-off-by: Adrian Hunter <ext-adrian.hunter@nokia.com>
---
fs/ubifs/misc.h | 310 ++++++++++
fs/ubifs/ubifs-media.h | 719 ++++++++++++++++++++++
fs/ubifs/ubifs.h | 1563 ++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 2592 insertions(+), 0 deletions(-)
diff --git a/fs/ubifs/misc.h b/fs/ubifs/misc.h
new file mode 100644
index 0000000..ad0b495
--- /dev/null
+++ b/fs/ubifs/misc.h
@@ -0,0 +1,310 @@
+/*
+ * This file is part of UBIFS.
+ *
+ * Copyright (C) 2006-2008 Nokia Corporation
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; if not, write to the Free Software Foundation, Inc., 51
+ * Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ *
+ * Authors: Artem Bityutskiy (Битюцкий Артём)
+ * Adrian Hunter
+ */
+
+/*
+ * This file contains miscellaneous helper functions.
+ */
+
+#ifndef __UBIFS_MISC_H__
+#define __UBIFS_MISC_H__
+
+/**
+ * ubifs_zn_dirty - check if znode is dirty.
+ * @znode: znode to check
+ *
+ * This helper function returns %1 if @znode is dirty and %0 otherwise.
+ */
+static inline int ubifs_zn_dirty(const struct ubifs_znode *znode)
+{
+ return !!test_bit(DIRTY_ZNODE, &znode->flags);
+}
+
+/**
+ * ubifs_wake_up_bgt - wake up backgr...The LEB properties are stored and maintained on the flash media, because otherwise UBIFS would need to scan whole media on each mount. We store this per-LEB accounting information is the lprops tree (LPT) which is an on-flash B-tree. The tree is updated out-of-place, as everything in UBIFS. It has its own garbage-collector, and is kind of small independent world whose task is to maintain the array of per-eraseblock information. Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com> Signed-off-by: Adrian Hunter <ext-adrian.hunter@nokia.com> --- fs/ubifs/lpt.c | 2243 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 2243 insertions(+), 0 deletions(-) diff --git a/fs/ubifs/lpt.c b/fs/ubifs/lpt.c new file mode 100644 index 0000000..0232a66 --- /dev/null +++ b/fs/ubifs/lpt.c @@ -0,0 +1,2243 @@ +/* + * This file is part of UBIFS. + * + * Copyright (C) 2006-2008 Nokia Corporation. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published by + * the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program; if not, write to the Free Software Foundation, Inc., 51 + * Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + * + * Authors: Adrian Hunter + * Artem Bityutskiy (Битюцкий Артём) + */ + +/* + * This file implements the LEB properties tree (LPT) area. The LPT area + * contains the LEB properties tree, a table of LPT area eraseblocks (ltab), and + * (for the "big" model) a table of saved LEB numbers (lsave). The LPT area sits + * between the log and the orphan area. + * + * The LPT area is...
Because of compression and space wastage (due to paddings) it is not always possible to know whether the cached data fits the flash space or not. Sometimes this problem is called "ENOSPC" problem. UBIFS implements the budgeting sub-system to solve the issue. All the FS operations have to acquire the budget. The budgeting subsystem does pessimistic space calculations (e.g., assumes the data is not compressible) and forces write-back or garbage-collection if needed. Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com> Signed-off-by: Adrian Hunter <ext-adrian.hunter@nokia.com> --- fs/ubifs/budget.c | 857 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 857 insertions(+), 0 deletions(-) diff --git a/fs/ubifs/budget.c b/fs/ubifs/budget.c new file mode 100644 index 0000000..90a9397 --- /dev/null +++ b/fs/ubifs/budget.c @@ -0,0 +1,857 @@ +/* + * This file is part of UBIFS. + * + * Copyright (C) 2006-2008 Nokia Corporation. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published by + * the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program; if not, write to the Free Software Foundation, Inc., 51 + * Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + * + * Authors: Adrian Hunter + * Artem Bityutskiy (Битюцкий Артём) + */ + +/* + * This file implements the budgeting unit which is responsible for UBIFS space + * management. + * + * Factors such as compression, wasted space at the ends of LEBs, space in other + * journal heads, the effect of updates on the index, and so on, make it +...
This patch adds implementation of most of the VFS callbacks like ->readdir(), ->write_begin(), and so on. In most cases, it just does budgeting and calls corresponding journal function, because all new data goes first to the journal. Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com> Signed-off-by: Adrian Hunter <ext-adrian.hunter@nokia.com> --- fs/ubifs/dir.c | 978 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ fs/ubifs/file.c | 902 +++++++++++++++++++++++++++++++++++++++++++++++++ fs/ubifs/ioctl.c | 205 ++++++++++++ 3 files changed, 2085 insertions(+), 0 deletions(-) diff --git a/fs/ubifs/dir.c b/fs/ubifs/dir.c new file mode 100644 index 0000000..189296d --- /dev/null +++ b/fs/ubifs/dir.c @@ -0,0 +1,978 @@ +/* + * This file is part of UBIFS. + * + * Copyright (C) 2006-2008 Nokia Corporation. + * Copyright (C) 2006, 2007 University of Szeged, Hungary + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published by + * the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program; if not, write to the Free Software Foundation, Inc., 51 + * Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + * + * Authors: Artem Bityutskiy (Битюцкий Артём) + * Adrian Hunter + * Zoltan Sogor + */ + +/* + * This file implements directory operations. + * + * All FS operations in this file allocate budget before writing anything to the + * media. If they fail to allocate it, the error is returned. The only + * exceptions are 'ubifs_unlink()' and 'ubifs_rmdir()' which keep working even + * if they...
Extended attributes are implemented as separate inodes. This makes it very easy to implement them and to re-use nearly all the existing code. This might be not the fastest implementation, though. ACL support is not implemented. Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com> Signed-off-by: Adrian Hunter <ext-adrian.hunter@nokia.com> --- fs/ubifs/xattr.c | 582 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 582 insertions(+), 0 deletions(-) diff --git a/fs/ubifs/xattr.c b/fs/ubifs/xattr.c new file mode 100644 index 0000000..45f30bd --- /dev/null +++ b/fs/ubifs/xattr.c @@ -0,0 +1,582 @@ +/* + * This file is part of UBIFS. + * + * Copyright (C) 2006-2008 Nokia Corporation. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published by + * the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program; if not, write to the Free Software Foundation, Inc., 51 + * Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + * + * Authors: Artem Bityutskiy (Битюцкий Артём) + * Adrian Hunter + */ + +/* + * This file implements UBIFS extended attributes support. + * + * Extended attributes are implemented as regular inodes with attached data, + * which limits extended attribute size to UBIFS block size (4KiB). Names of + * extended attributes are described by extended attribute entries (xentries), + * which are almost identical to directory entries, but have different key type. + * + * In other words, the situation with extended attributes is very similar to + * directories. Indeed, any inode (bu...
Add UBIFS to Makefile and Kbuild. Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com> Signed-off-by: Adrian Hunter <ext-adrian.hunter@nokia.com> --- fs/Kconfig | 3 +++ fs/Makefile | 1 + fs/ubifs/Kconfig | 47 +++++++++++++++++++++++++++++++++++++++++++++++ fs/ubifs/Kconfig.debug | 30 ++++++++++++++++++++++++++++++ fs/ubifs/Makefile | 9 +++++++++ 5 files changed, 90 insertions(+), 0 deletions(-) diff --git a/fs/Kconfig b/fs/Kconfig index c509123..292251d 100644 --- a/fs/Kconfig +++ b/fs/Kconfig @@ -1347,6 +1347,9 @@ config JFFS2_CMODE_FAVOURLZO endchoice +# UBIFS File system configuration +source "fs/ubifs/Kconfig" + config CRAMFS tristate "Compressed ROM file system support (cramfs)" depends on BLOCK diff --git a/fs/Makefile b/fs/Makefile index 1e7a11b..fcae06a 100644 --- a/fs/Makefile +++ b/fs/Makefile @@ -100,6 +100,7 @@ obj-$(CONFIG_NTFS_FS) += ntfs/ obj-$(CONFIG_UFS_FS) += ufs/ obj-$(CONFIG_EFS_FS) += efs/ obj-$(CONFIG_JFFS2_FS) += jffs2/ +obj-$(CONFIG_UBIFS_FS) += ubifs/ obj-$(CONFIG_AFFS_FS) += affs/ obj-$(CONFIG_ROMFS_FS) += romfs/ obj-$(CONFIG_QNX4FS_FS) += qnx4/ diff --git a/fs/ubifs/Kconfig b/fs/ubifs/Kconfig new file mode 100644 index 0000000..21a6fae --- /dev/null +++ b/fs/ubifs/Kconfig @@ -0,0 +1,47 @@ +config UBIFS_FS + tristate "UBIFS file system support" + select CRC16 + select CRC32 + depends on MTD_UBI + help + UBIFS is a file system for flash devices which works on top of UBI. + +config UBIFS_FS_XATTR + bool "Extended attributes support" + depends on UBIFS_FS + default n + help + This option enables support of extended attributes. + +config UBIFS_FS_ADVANCED_COMPR + bool "Advanced compression options" + depends on UBIFS_FS + default n + help + This option allows to explicitly choose which compressions, if any, + are enabled in UBIFS. Removing compressors means inbility to read + existing file systems. + + If unsure, sa...
This sub-system keeps track of orphans - the files which were deleted but are still kept open. These files should be deleted only when the last reference goes. But if an unclean reboot happens, UBIFS has to also delete the orphans. This is why the orphans sub-system exists - it records information about all orphans to the on-flash orphan area. Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com> Signed-off-by: Adrian Hunter <ext-adrian.hunter@nokia.com> --- fs/ubifs/orphan.c | 955 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 955 insertions(+), 0 deletions(-) diff --git a/fs/ubifs/orphan.c b/fs/ubifs/orphan.c new file mode 100644 index 0000000..aa7993c --- /dev/null +++ b/fs/ubifs/orphan.c @@ -0,0 +1,955 @@ +/* + * This file is part of UBIFS. + * + * Copyright (C) 2006-2008 Nokia Corporation. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published by + * the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program; if not, write to the Free Software Foundation, Inc., 51 + * Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + * + * Author: Adrian Hunter + */ + +#include "ubifs.h" + +/* + * An orphan is an inode number whose inode node has been committed to the index + * with a link count of zero. That happens when an open file is deleted + * (unlinked) and then a commit is run. In the normal course of events the inode + * would be deleted when the file is closed. However in the case of an unclean + * unmount, orphans need to be accounted for. After an unclean unmount, the + * orphans' inodes must be...
The LEB find sub-system is responsible for maintaining lists of eraseblocks with free and dirty space. For example, when UBIFS has to do garbage collection, in needs to find the dirtiest eraseblock, because it is faster to garbage-collect it, and it asks the LEB find sub-system to do this, which usually has immediate answer. Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com> Signed-off-by: Adrian Hunter <ext-adrian.hunter@nokia.com> --- fs/ubifs/find.c | 957 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 957 insertions(+), 0 deletions(-) diff --git a/fs/ubifs/find.c b/fs/ubifs/find.c new file mode 100644 index 0000000..43ee366 --- /dev/null +++ b/fs/ubifs/find.c @@ -0,0 +1,957 @@ +/* + * This file is part of UBIFS. + * + * Copyright (C) 2006-2008 Nokia Corporation. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published by + * the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program; if not, write to the Free Software Foundation, Inc., 51 + * Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + * + * Authors: Artem Bityutskiy (Битюцкий Артём) + * Adrian Hunter + */ + +/* + * This file contains functions for finding LEBs for various purposes e.g. + * garbage collection. In general, lprops category heaps and lists are used + * for fast access, falling back on scanning the LPT as a last resort. + */ + +#include <linux/sort.h> +#include "ubifs.h" + +/** + * struct scan_data - data provided to scan callback functions + * @min_space: minimum number of bytes for which...
This is the commit-related part of the lprops sub-system.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
Signed-off-by: Adrian Hunter <ext-adrian.hunter@nokia.com>
---
fs/ubifs/lpt_commit.c | 1625 +++++++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 1625 insertions(+), 0 deletions(-)
diff --git a/fs/ubifs/lpt_commit.c b/fs/ubifs/lpt_commit.c
new file mode 100644
index 0000000..797c56d
--- /dev/null
+++ b/fs/ubifs/lpt_commit.c
@@ -0,0 +1,1625 @@
+/*
+ * This file is part of UBIFS.
+ *
+ * Copyright (C) 2006-2008 Nokia Corporation.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; if not, write to the Free Software Foundation, Inc., 51
+ * Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ *
+ * Authors: Adrian Hunter
+ * Artem Bityutskiy (Битюцкий Артём)
+ */
+
+/*
+ * This file implements commit-related functionality of the LEB properties
+ * subsystem.
+ */
+
+#include <linux/crc16.h>
+#include "ubifs.h"
+
+/**
+ * first_dirty_cnode - find first dirty cnode.
+ * @c: UBIFS file-system description object
+ * @nnode: nnode at which to start
+ *
+ * This function returns the first dirty cnode or %NULL if there is not one.
+ */
+static struct ubifs_cnode *first_dirty_cnode(struct ubifs_nnode *nnode)
+{
+ ubifs_assert(nnode);
+ while (1) {
+ int i, cont = 0;
+
+ for (i = 0; i < UBIFS_LPT_FANOUT; i++) {
+ struct ubifs_cnode *cnode;
+
+ cnode = nnode->nbranch[i].cnode;
+ if (cnode ...UBIFS keeps track of all logical eraseblock - how much data do they contain, how much of these data are dirty or clean. This space accounting information is needed all over the place - when finding an empty eraseblock to put new data to, when reporting amount of empty space, and so on. We call this subsystem "lprops" which stands for LEB properties. Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com> Signed-off-by: Adrian Hunter <ext-adrian.hunter@nokia.com> --- fs/ubifs/lprops.c | 1353 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 1353 insertions(+), 0 deletions(-) diff --git a/fs/ubifs/lprops.c b/fs/ubifs/lprops.c new file mode 100644 index 0000000..867c3c3 --- /dev/null +++ b/fs/ubifs/lprops.c @@ -0,0 +1,1353 @@ +/* + * This file is part of UBIFS. + * + * Copyright (C) 2006-2008 Nokia Corporation. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published by + * the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program; if not, write to the Free Software Foundation, Inc., 51 + * Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + * + * Authors: Adrian Hunter + * Artem Bityutskiy (Битюцкий Артём) + */ + +/* + * This file implements the functions that access LEB properties and their + * categories. LEBs are categorized based on the needs of UBIFS, and the + * categories are stored as either heaps or lists to provide a fast way of + * finding a LEB in a particular category. For example, UBIFS may need to find + * an empty LEB for the journal, or a very dirty LEB for garbage coll...
We commit the TNC from time to time, which means we update the on-flash indexing tree. The TNC commit basically implements journal commit. The UBIFS implementation allows writing while the commit is in progress. Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com> Signed-off-by: Adrian Hunter <ext-adrian.hunter@nokia.com> --- fs/ubifs/tnc_commit.c | 1097 +++++++++++++++++++++++++++++++++++++++++++++++++ fs/ubifs/tnc_misc.c | 259 ++++++++++++ 2 files changed, 1356 insertions(+), 0 deletions(-) diff --git a/fs/ubifs/tnc_commit.c b/fs/ubifs/tnc_commit.c new file mode 100644 index 0000000..32bd4bc --- /dev/null +++ b/fs/ubifs/tnc_commit.c @@ -0,0 +1,1097 @@ +/* + * This file is part of UBIFS. + * + * Copyright (C) 2006-2008 Nokia Corporation. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published by + * the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program; if not, write to the Free Software Foundation, Inc., 51 + * Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + * + * Authors: Adrian Hunter + * Artem Bityutskiy (Битюцкий Артём) + */ + +/* This file implements TNC functions for committing */ + +#include "ubifs.h" + +/** + * make_idx_node - make an index node for fill-the-gaps method of TNC commit. + * @c: UBIFS file-system description object + * @idx: buffer in which to place new index node + * @znode: znode from which to make new index node + * @lnum: LEB number where new index node will be written + * @offs: offset where new index node will be written + * @len: length of new index node...
From: Adrian Hunter <ext-adrian.hunter@nokia.com>
Similarly to MTD devices, allow UBI devices.
Signed-off-by: Adrian Hunter <ext-adrian.hunter@nokia.com>
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
---
init/do_mounts.c | 3 ++-
1 files changed, 2 insertions(+), 1 deletions(-)
diff --git a/init/do_mounts.c b/init/do_mounts.c
index 3885e70..f76bd2f 100644
--- a/init/do_mounts.c
+++ b/init/do_mounts.c
@@ -347,7 +347,8 @@ void __init prepare_namespace(void)
if (saved_root_name[0]) {
root_device_name = saved_root_name;
- if (!strncmp(root_device_name, "mtd", 3)) {
+ if (!strncmp(root_device_name, "mtd", 3) ||
+ !strncmp(root_device_name, "ubi", 3)) {
mount_block_root(root_device_name, root_mountflags);
goto out;
}
--
1.5.4.1
--Let file systems to writeback their pages and inodes when needed. This
is needed for UBIFS budgeting sub-system because it has to force
write-back from time to time.
Note, it cannot be called if one of the dirty pages is locked by
the caller, otherwise it'll deadlock.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
Signed-off-by: Adrian Hunter <ext-adrian.hunter@nokia.com>
---
fs/fs-writeback.c | 8 ++++++++
include/linux/writeback.h | 1 +
2 files changed, 9 insertions(+), 0 deletions(-)
diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c
index 0655767..4591270 100644
--- a/fs/fs-writeback.c
+++ b/fs/fs-writeback.c
@@ -573,6 +573,14 @@ void sync_inodes_sb(struct super_block *sb, int wait)
spin_unlock(&inode_lock);
}
+void writeback_inodes_sb(struct super_block *sb, struct writeback_control *wbc)
+{
+ spin_lock(&inode_lock);
+ sync_sb_inodes(sb, wbc);
+ spin_unlock(&inode_lock);
+}
+EXPORT_SYMBOL_GPL(writeback_inodes_sb);
+
/*
* Rather lame livelock avoidance.
*/
diff --git a/include/linux/writeback.h b/include/linux/writeback.h
index b7b3362..0083a0a 100644
--- a/include/linux/writeback.h
+++ b/include/linux/writeback.h
@@ -71,6 +71,7 @@ struct writeback_control {
void writeback_inodes(struct writeback_control *wbc);
int inode_wait(void *);
void sync_inodes_sb(struct super_block *, int wait);
+void writeback_inodes_sb(struct super_block *sb, struct writeback_control *wbc);
void sync_inodes(int wait);
/* writeback.h requires fs.h; it, too, is not included from here. */
--
1.5.4.1
--This looks rather similar to http://userweb.kernel.org/~akpm/mmotm/broken-out/reiser4-sb_sync_inodes.patch Which is best? --
Hi Andrew, Well, this reiser4 patch is basically doing 3 things at once: 1. Introduce ->sync_inodes() 2. Move spin_lock(&inode_lock); 3. Export generic_sync_sb_inodes() As Christoph suggested, I've attached a patch which just moves the spinlock. Could you please consider sending it to Linus or taking into your tree? Should I amend the reiser4 patch correspondingly for you? Also, what do you think about taking UBIFS into your tree? -- Best Regards, Artem Bityutskiy (
I already added gti-ubifs.patch to the -mm lineup. I haven't yet tried pulling, merging or compiling it. I take git trees pretty promiscuously, to get them a bit more exposure and merge/build testing. --
Andrew, is it OK for you if ubifs-2.6.git will have some patches which ubi-2.6.git also contains. Will your -mm system handle this? -- Best Regards, Artem Bityutskiy (Артём Битюцкий) --
It happens occasionally and I usually manage to work it out. Sometimes with git hackery, sometimes manually. It'd be better if it was a once-off thing, rather than having new conflicts arise each time I repull the trees, if possible. --
OK. The thing is that we'd like to have UBI patches in UBIFS tree to give them a test. So what I've done I've created a branch named "for_andrew". Could you please switch your scripts from "master" branch to "for_andrew" branch? This branch will have the same but UBI patches which are already in the UBI tree and would cause conflicts. Thanks. -- Best Regards, Artem Bityutskiy (Артём Битюцкий) --
Err, probably I was vague. I was asking to use this for UBIFS: git://git.infradead.org/~dedekind/ubifs-2.6.git for_andrew -- Best Regards, Artem Bityutskiy (Артём Битюцкий) --
I've split the original patch on 3 pieces: 1. VFS: move inode_lock into sync_sb_inodes 2. VFS: export sync_sb_inodes 3. VFS: introduce sync_inodes superblock operation The first 2 are exactly what UBIFS also needs. The third one is only needed for Reiser4. The patches are attached. I am not sure this is the best way to split. The second patch could just export sync_sb_inodes() without renaming it to generic_sync_sb_inodes(). But in this case we would have more complex third patch and more dependencies between reiser4 and ubifs. What is better? -- Best Regards, Artem Bityutskiy (
Well, assuming this is ok with you I've drop our VFS patch and pushed the above patches 1 and 2 instead. -- Best Regards, Artem Bityutskiy (Артём Битюцкий) --
Note, I have tested this patch and it works fine. -- Best Regards, Artem Bityutskiy (Артём Битюцкий) --
I think sync_sb_inodes should just lock/unlock the inode_lock itself as per the patch from the reiser4 queue. But before we export it I'd like to review what ubifs actually does with it. Anyone care to submit a patch to just move the locking that we could put in ASAP? --
Hi Christoph, Here is an updated UBIFS tarball for you: http://www.infradead.org/~dedekind/ubifs/ubifs-08-05-13.tar.bz2 Did you have chance to do the reviw? We would like to ask for merge at Done, the patch is sitting in our UBIFS. -- Best Regards, Artem Bityutskiy (Артём Битюцкий) --
Sorry, www.infradead.org was down during the weeked when I had some time to review it. I've grabbed a copy of the new tarball now and will look at it soon. Also I don't think it's too late for .26 as ubifs only adds new code except the writeback_inodes_sb bit which should be trivial enough for .26. --
Thanks. Though it depends on the results of your review and Linus. Let's see how it all goes. -- Best Regards, Artem Bityutskiy (Артём Битюцкий) --
This is a small sub-system which is doing eraseblock scanning. For
example, this is needed during journal replay, recovery, or garbage
collection.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
Signed-off-by: Adrian Hunter <ext-adrian.hunter@nokia.com>
---
fs/ubifs/scan.c | 362 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 362 insertions(+), 0 deletions(-)
diff --git a/fs/ubifs/scan.c b/fs/ubifs/scan.c
new file mode 100644
index 0000000..acf5c5f
--- /dev/null
+++ b/fs/ubifs/scan.c
@@ -0,0 +1,362 @@
+/*
+ * This file is part of UBIFS.
+ *
+ * Copyright (C) 2006-2008 Nokia Corporation
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; if not, write to the Free Software Foundation, Inc., 51
+ * Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ *
+ * Authors: Adrian Hunter
+ * Artem Bityutskiy (Битюцкий Артём)
+ */
+
+/*
+ * This file implements the scan which is a general-purpose function for
+ * determining what nodes are in an eraseblock. The scan is used to replay the
+ * journal, to do garbage collection. for the TNC in-the-gaps method, and by
+ * debugging functions.
+ */
+
+#include "ubifs.h"
+
+/**
+ * scan_padding_bytes - scan for padding bytes.
+ * @buf: buffer to scan
+ * @len: length of buffer
+ *
+ * This function returns the number of padding bytes on success and
+ * %SCANNED_GARBAGE on failure.
+ */
+static int scan_padding_bytes(void *buf, int len)
+{
+ int pad_len = 0, ma...Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com> Signed-off-by: Adrian Hunter <ext-adrian.hunter@nokia.com> --- Documentation/filesystems/ubifs.txt | 163 +++++++++++++++++++++++++++++++++++ 1 files changed, 163 insertions(+), 0 deletions(-) diff --git a/Documentation/filesystems/ubifs.txt b/Documentation/filesystems/ubifs.txt new file mode 100644 index 0000000..ab653b4 --- /dev/null +++ b/Documentation/filesystems/ubifs.txt @@ -0,0 +1,163 @@ +Introduction +============= + +UBIFS file-system stands for UBI File System. UBI stands for "Unsorted +Block Images". UBIFS is a flash file system, which means it is designed +to work with flash devices. It is important to understand, that UBIFS +is completely different to any traditional file-system in Linux, like +Ext2, XFS, JFS, etc. UBIFS represents a separate class of file-systems +which work with MTD devices, not block devices. The other Linux +file-system of this class is JFFS2. + +To make it more clear, here is a small comparison of MTD devices and +block devices. + +1 MTD devices represent flash devices and they consist of eraseblocks of + rather large size, typically about 128KiB. Block devices consist of + small blocks, typically 512 bytes. +2 MTD devices support 3 main operations - read from some offset within an + eraseblock, write to some offset within an eraseblock, and erase a whole + eraseblock. Block devices support 2 main operations - read a whole + block and write a whole block. +3 The whole eraseblock has to be erased before it becomes possible to + re-write its contents. Blocks may be just re-written. +4 Eraseblocks become worn out after some number of erase cycles - + typically 100K-1G for SLC NAND and NOR flashes, and 1K-10K for MLC + NAND flashes. Blocks do not have the wear-out property. +5 Eraseblocks may become bad (only on NAND flashes) and software should + deal with this. Blocks on hard drives typically do not become bad, + because hardware has mechanisms to substitu...
The TNC cache grows with time, because UBIFS caches the indexing nodes when the indexing B-tree is looked-up. But if the the file-system is large enough, the TNC may consume a lot of memory, in which UBIFS prunes it. Namely, it register memory shrinker for these purposes. Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com> Signed-off-by: Adrian Hunter <ext-adrian.hunter@nokia.com> --- fs/ubifs/shrinker.c | 322 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 322 insertions(+), 0 deletions(-) diff --git a/fs/ubifs/shrinker.c b/fs/ubifs/shrinker.c new file mode 100644 index 0000000..b100ed9 --- /dev/null +++ b/fs/ubifs/shrinker.c @@ -0,0 +1,322 @@ +/* + * This file is part of UBIFS. + * + * Copyright (C) 2006-2008 Nokia Corporation. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published by + * the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program; if not, write to the Free Software Foundation, Inc., 51 + * Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + * + * Authors: Artem Bityutskiy (Битюцкий Артём) + * Adrian Hunter + */ + +/* + * This file implements UBIFS shrinker which evicts clean znodes from the TNC + * tree when Linux VM needs more RAM. + * + * We do not implement any LRU lists to find oldest znodes to free because it + * would add additional overhead to the file system fast paths. So the shrinker + * just walks the TNC tree when searching for znodes to free. + * + * If the root of a TNC sub-tree is clean and old enough, then the children are + * also clean and...
TNC - tree node cache - the central UBIFS entity. It is basically in-RAM cache of the on-flash indexing B-tree. But TNC also indexes the journal, so that they are not always equivalent. Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com> Signed-off-by: Adrian Hunter <ext-adrian.hunter@nokia.com> --- fs/ubifs/tnc.c | 3240 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 3240 insertions(+), 0 deletions(-) diff --git a/fs/ubifs/tnc.c b/fs/ubifs/tnc.c new file mode 100644 index 0000000..881614c --- /dev/null +++ b/fs/ubifs/tnc.c @@ -0,0 +1,3240 @@ +/* + * This file is part of UBIFS. + * + * Copyright (C) 2006-2008 Nokia Corporation. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published by + * the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program; if not, write to the Free Software Foundation, Inc., 51 + * Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + * + * Authors: Adrian Hunter + * Artem Bityutskiy (Битюцкий Артём) + */ + +/* + * This file implements TNC (Tree Node Cache) which caches indexing nodes of + * the UBIFS B-tree. + * + * At the moment the locking rules of the TNC tree are quite simple and + * straightforward. We just have a mutex and lock it when we traverse the + * tree. If a znode is not in memory, we read it from flash while still having + * the mutex locked. + */ + +#include <linux/crc32.h> +#include "ubifs.h" + +/* + * Returned codes of 'matches_name()' and 'fallible_matches_name()' functions. + * @NAME_LESS: name corresponding to the fi...
All the new data first goes to the journal and sits there until it gets committed. The journal contents does not have corresponding on-flash indexing information, so the journal is like a small JFFS2 file-system. Once the journal is committed, the indexing information is written to the flash media. Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com> Signed-off-by: Adrian Hunter <ext-adrian.hunter@nokia.com> --- fs/ubifs/journal.c | 1264 ++++++++++++++++++++++++++++++++++++++++++++++++++++ fs/ubifs/log.c | 799 +++++++++++++++++++++++++++++++++ 2 files changed, 2063 insertions(+), 0 deletions(-) diff --git a/fs/ubifs/journal.c b/fs/ubifs/journal.c new file mode 100644 index 0000000..9caf93f --- /dev/null +++ b/fs/ubifs/journal.c @@ -0,0 +1,1264 @@ +/* + * This file is part of UBIFS. + * + * Copyright (C) 2006-2008 Nokia Corporation. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published by + * the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program; if not, write to the Free Software Foundation, Inc., 51 + * Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + * + * Authors: Artem Bityutskiy (Битюцкий Артём) + * Adrian Hunter + */ + +/* + * This file implements UBIFS journal. + * + * The journal consists of 2 parts - the log and bud LEBs. The log has fixed + * length and position, while a bud logical eraseblock is any LEB in the main + * area. Buds contain file system data - data nodes, inode nodes, etc. The log + * contains only references to buds and some other stuff like commit + * start ...
This is the UBIFS journal commit implementation. The journal commit does not mean the data is physically moved anywhere - we just update the indexing information and find new eraseblocks for the journal. Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com> Signed-off-by: Adrian Hunter <ext-adrian.hunter@nokia.com> --- fs/ubifs/commit.c | 709 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 709 insertions(+), 0 deletions(-) diff --git a/fs/ubifs/commit.c b/fs/ubifs/commit.c new file mode 100644 index 0000000..636e508 --- /dev/null +++ b/fs/ubifs/commit.c @@ -0,0 +1,709 @@ +/* + * This file is part of UBIFS. + * + * Copyright (C) 2006-2008 Nokia Corporation. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published by + * the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program; if not, write to the Free Software Foundation, Inc., 51 + * Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + * + * Authors: Adrian Hunter + * Artem Bityutskiy (Битюцкий Артём) + */ + +/* + * This file implements functions that manage the running of the commit process. + * Each affected module has its own functions to accomplish their part in the + * commit and those functions are called here. + * + * The commit is the process whereby all updates to the index and LEB properties + * are written out together and the journal becomes empty. This keeps the + * file system consistent - at all times the state can be recreated by reading + * the index and LEB properties and then replaying the journal. + * + * ...
This file implement various helper functions to work with UBIFS keys. The keys are part of the UBIFS index which is a B-tree. For example, directory entry key consists of the parent inode number and directory entry hash. Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com> Signed-off-by: Adrian Hunter <ext-adrian.hunter@nokia.com> --- fs/ubifs/key.h | 507 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 507 insertions(+), 0 deletions(-) diff --git a/fs/ubifs/key.h b/fs/ubifs/key.h new file mode 100644 index 0000000..7c5bed7 --- /dev/null +++ b/fs/ubifs/key.h @@ -0,0 +1,507 @@ +/* + * This file is part of UBIFS. + * + * Copyright (C) 2006-2008 Nokia Corporation. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published by + * the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program; if not, write to the Free Software Foundation, Inc., 51 + * Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + * + * Authors: Artem Bityutskiy (Битюцкий Артём) + * Adrian Hunter + */ + +/* + * This header contains various key-related definitions and helper function. + * UBIFS allows several key schemes, so we access key fields only via these + * helpers. At the moment only one key scheme is supported. + * + * Simple key scheme + * ~~~~~~~~~~~~~~~~~ + * + * Keys are 64-bits long. First 32-bits are inode number (parent inode number + * in case of direntry key). Next 3 bits are node type. The last 29 bits are + * 4KiB offset in case of inode node, and direntry hash in case of a direntry + * node....
The recovery sub-system is responsible for recovering from unclean reboots. It makes sure every-thing is consistent, rolls-back the last broken and un-finished FS operation, and so on. Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com> Signed-off-by: Adrian Hunter <ext-adrian.hunter@nokia.com> --- fs/ubifs/recovery.c | 1439 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 1439 insertions(+), 0 deletions(-) diff --git a/fs/ubifs/recovery.c b/fs/ubifs/recovery.c new file mode 100644 index 0000000..67bcea0 --- /dev/null +++ b/fs/ubifs/recovery.c @@ -0,0 +1,1439 @@ +/* + * This file is part of UBIFS. + * + * Copyright (C) 2006-2008 Nokia Corporation + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published by + * the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program; if not, write to the Free Software Foundation, Inc., 51 + * Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + * + * Authors: Adrian Hunter + * Artem Bityutskiy (Битюцкий Артём) + */ + +/* + * This file implements functions needed to recover from unclean un-mounts. + * When UBIFS is mounted, it checks a flag on the master node to determine if + * an un-mount was completed sucessfully. If not, the process of mounting + * incorparates additional checking and fixing of on-flash data structures. + * UBIFS always cleans away all remnants of an unclean un-mount, so that + * errors do not accumulate. However UBIFS defers recovery if it is mounted + * read-only, and the flash is not modified in that case. + */ + +#include ...
This patch contains the superblock and master node implementations. The UBIFS superblock is read-only and contains only static data like the default compression type. The superblock sits at the fixed position and may be changed only with user-space tools. The master node contains dynamic information like the position of the root indexing node of the UBIFS indexing B-tree, and so on. The master node is updated out-of-place. Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com> Signed-off-by: Adrian Hunter <ext-adrian.hunter@nokia.com> --- fs/ubifs/master.c | 387 +++++++++++++++++++++++++++++++++ fs/ubifs/sb.c | 612 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 999 insertions(+), 0 deletions(-) diff --git a/fs/ubifs/master.c b/fs/ubifs/master.c new file mode 100644 index 0000000..d9dfc95 --- /dev/null +++ b/fs/ubifs/master.c @@ -0,0 +1,387 @@ +/* + * This file is part of UBIFS. + * + * Copyright (C) 2006-2008 Nokia Corporation. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published by + * the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program; if not, write to the Free Software Foundation, Inc., 51 + * Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + * + * Authors: Artem Bityutskiy (Битюцкий Артём) + * Adrian Hunter + */ + +/* This file implements reading and writing the master node */ + +#include "ubifs.h" + +/** + * scan_for_master - search the valid master node. + * @c: UBIFS file-system description object + * + * This function scans the master node LEBs an...
Hi Artem,
Here's one proposal. Please add the no compressed mode support.
In my simple test. it's working with no compress mode.
Yes I know, it makes the performance poor. but we can measure the
metadata operation overhead compare to the previous flash filesystems.
How do you think?
Thank you,
#ifdef CONFIG_UBIFS_FS_NO_COMPR
#define DEFAULT_COMPRESSOR UBIFS_COMPR_NONE
#else
#define DEFAULT_COMPRESSOR UBIFS_COMPR_LZO
#ifdef CONFIG_UBIFS_FS_NO_COMPR
/* Don't use the compression */
ino->flags = 0;
#else
/* Set compression enabled by default */
ino->flags = cpu_to_le32(UBIFS_COMPR_FL);
#endif
Of course it also adds the UBIFS_FS_NO_COMPR to fs/ubifs/Kconfig properly.
--