Re: [PATCH 05/10] AXFS: axfs_profiling.c

Previous thread: [PATCH 04/10] AXFS: axfs_inode.c by Jared Hulbert on Wednesday, August 20, 2008 - 10:45 pm. (16 messages)

Next thread: [PATCH 06/10] AXFS: axfs_super.c by Jared Hulbert on Wednesday, August 20, 2008 - 10:45 pm. (15 messages)
From: Jared Hulbert
Date: Wednesday, August 20, 2008 - 10:45 pm

Profiling is a fault instrumentation and /proc formating system.
This is used to get an accurate picture of what the pages are actually used.
Using this info the image can be optimized for XIP

Signed-off-by: Jared Hulbert <jaredeh@gmail.com>
---
diff --git a/fs/axfs/axfs_profiling.c b/fs/axfs/axfs_profiling.c
new file mode 100644
index 0000000..30d881c
--- /dev/null
+++ b/fs/axfs/axfs_profiling.c
@@ -0,0 +1,594 @@
+/*
+ * Advanced XIP File System for Linux - AXFS
+ *   Readonly, compressed, and XIP filesystem for Linux systems big and small
+ *
+ * Copyright(c) 2008 Numonyx
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * Authors:
+ *  Eric Anderson
+ *  Jared Hulbert <jaredeh@gmail.com>
+ *  Sujaya Srinivasan
+ *  Justin Treon
+ *
+ * More info and current contacts at http://axfs.sourceforge.net
+ *
+ * axfs_profiling.c -
+ *   Tracks pages of files that enter the page cache.  Outputs through a proc
+ *   file which generates a comma separated data file with path, page offset,
+ *   count of times entered page cache.
+ */
+
+#include <linux/axfs.h>
+#ifdef CONFIG_AXFS_PROFILING
+#include <linux/module.h>
+#include <linux/vmalloc.h>
+#include <linux/proc_fs.h>
+
+#define AXFS_PROC_DIR_NAME "axfs"
+
+struct axfs_profiling_manager {
+	struct axfs_profiling_data *profiling_data;
+	struct axfs_super *sbi;
+	u32 *dir_structure;
+	u32 size;
+};
+
+#define MAX_STRING_LEN 1024
+
+/* Handles for our Directory and File */
+static struct proc_dir_entry *axfs_proc_dir;
+static u32 proc_name_inc;
+
+/******************************************************************************
+ *
+ * axfs_init_profile_dir_structure
+ *
+ * Description:
+ *   Creates the structures for tracking the page usage data and creates the
+ *   proc file that will be used to get the data.
+ *
+ * Parameters:
+ *   ...
From: Carsten Otte
Date: Thursday, August 21, 2008 - 1:44 am

Exporting profiling data for a file system in another file system 
(/proc) seems not very straigtforward to me. I think it is worth 
considering to export this information via the same mount point.
--

From: David Woodhouse
Date: Thursday, August 21, 2008 - 1:49 am

I would have said sysfs, rather than 'the same mount point'.

-- 
David Woodhouse                            Open Source Technology Centre
David.Woodhouse@intel.com                              Intel Corporation



--

From: Carsten Otte
Date: Thursday, August 21, 2008 - 3:20 am

Well, filesystems are usually not represented in the device model. 
It'd be possible to add a system device for it, but that does'nt feel 
like the right solution to me.
--

From: Arnd Bergmann
Date: Thursday, August 21, 2008 - 4:39 am

Let me throw in debugfs as my preferred option. sysfs is for stable
interfaces, while profiling generally fits into the debugging category.

	Arnd <><
--

From: Jared Hulbert
Date: Thursday, August 21, 2008 - 7:55 am

Three responses, three suggestions....

1) same mount point -
I don't see how this works without an ioctl.  I can't just make up
files in my mounted filesystem.   You expect the mounted version to
match input to the mkfs.  I'd not be happy with an ioctl.  You can
just read it.

2) sysfs -
I agree with Carsten, I don't see how this fits in the sysfs hierarchy.

3) debugfs -
I don't know diddly about this.

So why not /proc?
--

From: Arnd Bergmann
Date: Thursday, August 21, 2008 - 8:06 am

I think what Carsten was suggesting is that you create extra files
in the file system that behave like your current procfs files.
This limits the choice for names inside of the file system, and

You can create attributes below the device you have mounted.
Technically possible, the main issue I see with this is having

/proc has the same ABI restrictions as sysfs. We more or less stopped
allowing new files in /proc some 5 years ago for this reason. I didn't
even read beyond the word /proc to know that what you do here is wrong.
debugfs is normally easier to use than procfs as well, you just
define some file_operations with read/write callbacks and call
debugfs_create_file with the path name below /sys/kernel/debug.

If I may give yet another suggestion:

4) no profiling at all
The profiling code has certainly been useful to you during development,
and you should keep that code around for your own work on it,
but maybe you should not push that upstream, because regular users
are not going to need it.

	Arnd <><
--

From: Jared Hulbert
Date: Thursday, August 21, 2008 - 8:17 am

no /proc.
thanks for the explanation.


Nope.  Profiling is absolutely fundamental to how AXFS works.  Read
the [PATCH 00/10] thread again.
--

From: Arnd Bergmann
Date: Thursday, August 21, 2008 - 8:50 am

Ok, understood it now. So it actually is a stable interface into
the file system, which means that debugfs might not be the best
solution.

I need to think about this some more. So far none of the options
are perfect. What I can think of so far includes:

1. An ioctl on the mount point of the fs
2. An ioctl that you need to call on each file
3. sysfs files is /sys/fs/axfs/
4. A new virtual file system to be mounted to /sys/fs/axfs/
5. A file below the device in sysfs, e.g. /sys/block/mtdblk0/axfs-profile

I've also taken a look at the format of the profiling data file.
I'm not sure that it is ideal if you want to be able to share
identical data blocks between files. Do you currently do that
in your mkfs? The fs format certainly allows it. I would expect
that if you checksum each page, you can find duplicates on a page
basis and save some space this way. However, it can make profiling
harder if you count based on blocks but report the data based on
the file name. Not sure what a better solution would look like.

	Arnd <><
--

From: Carsten Otte
Date: Friday, August 22, 2008 - 12:26 am

I agree, the profiling part is the sweet spot. Profiling should be in, 
probably even selectable at runtime as opposed to a config-switch. 
This way we could have it in enterprise distros that would'nt build 
another kernel image for this.
--

From: Geert Uytterhoeven
Date: Thursday, August 21, 2008 - 8:18 am

The profiling is needed to feedback into mkfs.axfs, to decide which pages to
make XIP (in NOR) or not (in NAND). So yes, normal users need it when creating
file systems for a mixed NOR/NAND FLASH system.

With kind regards,

Geert Uytterhoeven
Software Architect

Sony Techsoft Centre Europe
The Corporate Village 
From: Arnd Bergmann
Date: Friday, August 22, 2008 - 1:37 pm

Ok, so now yet another suggestion, which may sound a little strange:

oprofilefs

I believe you can use the oprofile infrastructure to record data
about file accesses, even independent of the file system you
are looking at.

It's probably a lot of work to get it right, but I would be worth it.

	Arnd <><
--

Previous thread: [PATCH 04/10] AXFS: axfs_inode.c by Jared Hulbert on Wednesday, August 20, 2008 - 10:45 pm. (16 messages)

Next thread: [PATCH 06/10] AXFS: axfs_super.c by Jared Hulbert on Wednesday, August 20, 2008 - 10:45 pm. (15 messages)