Hi,

I committed the module framework that I was working on. It will be a while
before I can spend more time on it, and others want to work on it, so here
it is. Ordinarily I would have sent this for review. In this case I thought
it was better to start with a working implementation in order to minimize
the level of bikeshedding.

The change is here:

http://mail-index.netbsd.org/source-changes/2008/01/16/0013.html	

The basic idea is that a kernel facility like ext2fs should have code
difference to be loadable as a module - the initialization is the same
whether or not it is built into the kernel. All that should be needed is
some build glue like a Makefile under sys/modules. Features: it has an in
kernel loader, does module dependencies and has hooks for the boot loader
to load modules and provide them to the kernel.

Here's how to get a module loaded for testing/fun. Note that only the linker
has been tested on i386 and amd64 only:

- Build and install new libc. 
- Edit sbin/modload/Makefile, sbin/modunload/Makefile.
- Build and install sbin/modstat, modload, modunload.
- Make sure your PATH is finding /sbin/modstat and not /usr/bin/modstat.
- Compile kernel with options MODULAR.
- mkdir /modules
- cd sys/modules/example
- make
- mv example.o /modules/example
- modload example

What needs to happen for it to replace LKMs:

- Conversion of the modules under sys/lkm.
- Linker relocation code for mips, hppa.
- Some changes to improve robustness in a couple of places.
- Locking for kern_ksyms.c

Please review.

Thanks,
Andrew
[ message continues ]

"should have no code difference".

Andrew
[ message continues ]

On Wed, 16 Jan 2008 13:03:42 +0000
Very nice!


		--Steve Bellovin, http://www.cs.columbia.edu/~smb
[ message continues ]

I thought of a few more things this morning:

- We need some new calls for modules to make so that they can link what
  they provide into the system. Currently, kern_lkm.c takes care of most of
  this. One goal was to remove this indirection because it's complicated and
  difficult to understand what is going on. So the new style modules call
  directly into whatever subsystem they interact with to say "I'm here". 
  For file systems those calls already exist: vfs_attach() and vfs_detach(). 
  We need calls like syscall_attach() and so on. Picking sysvshm as an
  example, it's entries would never appear in the syscall table at boot,
  even if built into the system. When the module is initialized it would
  enter them.

- The syscall case raises a question about compat code, because in some
  places the compat code calls optional syscalls directly. We could either
  go through the syscall table to get at optional features (call *sy_call),
  or make compat code always depend on the optional features like sysvshm.
  The latter is problematic because then we grow a conflict between
  config(8) and the module dependencies so I think indirection is the way
  to go here.

- Device drivers are difficult because autoconf isn't really a good fit
  for loadables. As of right now the "built into the kernel" and "loadable"
  cases would probably have to be different for everything but pseudo
  devices. So what do we do here?

- Relocations need to be done in two stages (right now they are all done in
  one pass). First the local relocs so that the module's modinfo_t record is
  correct and can be read safely.  Then any requisite modules can be loaded.
  Once they are loaded, relocs depending on symbols external to the module
  can be done, and the module initialized as done now. Otherwise, we would
  fail to find symbols from a required module and fail the module load.

Thanks,
Andrew
[ message continues ]

One problem here is that the compat code probably doesn't want to call
the sys_foo() function itself, but rather then next function down that
is executed once all the parameters are in kernel space.

I think that means that if sys_foo() is in an LKM, then compat_bar_sys_foo()
must also be in an LKM.  Then the LKM dependecies can be used to ensure
all the required functionality is present.

	David

-- 
David Laight: david@l8s.co.uk
[ message continues ]

Hi, Andrew



Some time ago I have changed ddb to accept new command tables from  
LKM. I have added two external visible functions [1], db_register_tbl,  
db_unregister_tbl. With this interface LKM can add/remove functions to  
ddb interface. I can rename them to something more standard ddb_attach/ 
ddb_detach + I can add test ddb command table to example module, too.  
If it is needed. There is no locking for now because I thought that it  
is not needed is this still true ?

[1]http://opengrok.netbsd.org/source/xref/sys/ddb/db_command.c#450

[snip]


Regards

Adam.

[ message continues ]

Hiya Andrew, hi folks,


AFAIR Linux/FreeBSD's modules are kernel compilation dependent; i.e. if you 
recompile the kernel you recompile the modules. If the config file could 
identify things like

	auixp* at pci? dev ? function ?

hardwires the module into the kernel like default

	auixp* at pci? dev ? function ? module

explicitly creates a module containing the files needed for that config 
line but leaves it to a bootloader/userland program to issue the modload.

	auixp* at pci? dev ? function ? module bootprobe

then the config attach function could create a module containing the files 
needed for that config line and create an autoconf veneer that loads the 
module, tries its attach and if fails unloads the module; if it matches the 
module stays and gets refcounted.

	umodem* at uhub? port ? configuration ? module insertprobe

could be used for say USB insert stuff; code is generated on USB device 
detection that could probe the module like above....

Well these are just things i just thought of; 
suggestions/comments/feasability?

With regards,
Reinoud
[ message continues ]