Matt Dillon [story] provides an interesting and detailed explanation of future development plans with regards to DragonFly's I/O subsystem. Originally inspired by the PIPE code improvements of FreeBSD's Alan Cox, and demonstrated in DragonFly's unique XIO and MSFBUF APIs, the goal of this work is to avoid KVA mappings for I/O requests and the resulting overhead of interprocessor interrupts in SMP systems. In theory, this equates to high performance through the benefit of efficient I/O in combination with the ability of any subsystem layer to transfer data to busdma with zero memory-to-memory copies. Matt expands:

"What we are going to do is extend the msf_buf abstraction to cover these needs and provide a set of API calls that allows upper layers to supply data in any form and lower level layers to request data in any form, including with address restrictions. msf_buf's already have a page-list (XIO) and KVA mapping abstraction. We are going to add a bounce-buffer abstraction and then work on a bunch of new API calls for msf_bufs to cover the needs of various subsystems."

There appears to be a lot of interesting work going on in DragonFly, read more for the entirety of Matt's post.

From: Matthew Dillon [email blocked]
To: dragonfly-kernel
Subject: I/O consolidation and direct-to-DMA plans
Date: 2004-12-20 20:10:54


    Hiten and I have come up with a roadmap for the I/O path cleanup and
    direct-to-dma plans (avoiding having to map KVA buffers).

    We've looked at all existing structures... msf_buf's, sfbufs, struct buf,
    XIO's, UIO's, and BUSDMA (bounce_page and bus_dmamap).  Each of these
    structures represents a piece of the larger 'integrated I/O' puzzle.

    The primary issue is that we need a structure which covers both ends of
    the equation... we need the capability in higher layers to specify either
    a KVA mapped buffer or a page list, and we need the capability in lower
    layers to require either a KVA mapped buffer or a page list, depending on
    the requirements of the layer.   For example, in some instances CAM 
    may supply a buffer pointer to simple SCSI request structures while in
    others CAM might want to pass data from a struct BUF or UIO that it has
    a page list for.  In some cases a driver will have access to a DMA
    engine and require a page list, while in others a driver might need a
    mapped buffer, or might need to create bounce pages.

    What we are going to do is extend the msf_buf abstraction to cover these
    needs and provide a set of API calls that allows upper layers to supply
    data in any form and lower level layers to request data in any form,
    including with address restrictions.  msf_buf's already have a page-list
    (XIO) and KVA mapping abstraction.  We are going to add a bounce-buffer
    abstraction and then work on a bunch of new API calls for msf_bufs to
    cover the needs of various subsystems.

    As a starter we'll have these functions:


    msf_init(struct msf_buf *msf)

	Initialize an msf_buf for use (i.e. zero its fields).

    struct msf_buf *
    msf_create_from_buf(struct msf_buf *opt_msf, void *buf, size_t bytes)

	Populate an existing msf_buf or allocate a new one and install the
	supplied KVA buffer pointer and size.

    struct msf_buf *
    msf_create_from_xio(struct msf_buf *opt_msf, struct xio *xio)

	Populate an existing msf_buf or allocate a new one and install the
	supplied XIO (page list).

    int
    msf_require_buf(struct msf_buf *msf)

	Require that an msf_buf have a mapped KVA buffer.  If the msf_buf
	already has a mapped KVA buffer this is a NOP.  If the msf_buf
	contains a page list a KVA buffer will be allocated and mapped
	based on the page list.

    int
    msf_require_xio(struct msf_buf *msf)

	Require that an mfs_buf have a page list.  If the msf_buf already
	has a page list this is a NOP.  Otherwise a page list is constructed
	from the msf_buf's KVA buffer.

    void
    msf_release(struct msf_buf *msf)

	Release an msf_buf, freeing any resources that were created as side
	effects to the above API calls and zeroing out any resources
	that were originally supplied.


    The plan is to start embedding msf_buf's in various system layers
    (struct buf, busdma, etc...) as independant entities to begin with.  As
    more of this work is accomplished the various layers using msf_buf's 
    will start to become adjacent to each other and we will be able to then
    have one layer pass its msf_buf directly to another without having to
    re-create this.

    Along the way all the various disparate I/O related structures will
    be consolidated.

    Eventually this will allow e.g. the buffer cache to pass an msf_buf
    all the way down to BUSDMA without having to KVA map the buffer, thus
    achieving our goal.

					-Matt
					Matthew Dillon 
					

• Archive of the Above Thread