Hello everyone,

The following patchset implements a Contiguous Memory Allocator.  For
those who have not yet stumbled across CMA an excerpt from
documentation:

   The Contiguous Memory Allocator (CMA) is a framework, which allows
   setting up a machine-specific configuration for physically-contiguous
   memory management. Memory for devices is then allocated according
   to that configuration.

   The main role of the framework is not to allocate memory, but to
   parse and manage memory configurations, as well as to act as an
   in-between between device drivers and pluggable allocators. It is
   thus not tied to any memory allocation method or strategy.

For more information please refer to the second patch from the
patchset which contains the documentation.


Links to the previous versions of the patchsets:
v2: <http://article.gmane.org/gmane.linux.kernel.mm/50986/>
v1: <http://article.gmane.org/gmane.linux.kernel.mm/50669/>


This is the third version of the patchset.  All of the changes are
concentrated in the second, the third and the fourth patch -- the
other patches are almost identical.


Major observable changes between the second (the previous) and the
third (this) versions are:

1. The command line parameters have been removed (and moved to
   a separate patch, the fourth one).  As a consequence, the
   cma_set_defaults() function has been changed -- it no longer
   accepts a string with list of regions but an array of regions.

2. The "asterisk" attribute has been removed.  Now, each region has an
   "asterisk" flag which lets one specify whether this region should
   by considered "asterisk" region.

3. SysFS support has been moved to a separate patch (the third one in
   the series) and now also includes list of regions.


Major observable changes between the first and the second versions
are:

1. The "cma_map" command line have been removed.  In exchange, a SysFS
   entry has been created under kernel/mm/contiguous.
   
   The intended way ...
[ message continues ]

Are there any comments or ack?

We hope this method included at mainline kernel if possible.
It's really needed feature for our multimedia frameworks.

Thank you,
Kyungmin Park

On Fri, Aug 6, 2010 at 10:22 PM, Michal Nazarewicz
--

[ message continues ]

Is there a git tree and/or link to the latest version that is based on
top 2.6.36-rc1?? I somehow seem to have lost the v3 of these patches.
--

[ message continues ]

I'm currently working on a v4 of the patchset after some comments from
Hans Verkuil on the #v4l.  I should manage to post it today (Korean time).

-- 
Best regards,                                        _     _
| Humble Liege of Serenely Enlightened Majesty of  o' \,=./ `o
| Computer Science,  Michał "mina86" Nazarewicz       (o o)
+----[mina86*mina86.com]---[mina86*jabber.org]----ooO--(_)--Ooo--

--

[ message continues ]

On Wed, 18 Aug 2010 12:01:35 +0900

You got any comments from mm people?

Virtually, this adds a new memory allocator implementation that steals
some memory from memory allocator during boot process. Its API looks
completely different from the API for memory allocator. That doesn't
sound appealing to me much. This stuff couldn't be integrated well
into memory allocator?
--

[ message continues ]

What kind of integration do you mean?  I see three levels:

1. Integration on API level meaning that some kind of existing API is used
    instead of new cma_*() calls.  CMA adds notion of devices and memory
    types which is new to all the other APIs (coherent has notion of devices
    but that's not enough).  This basically means that no existing API can be
    used for CMA.  On the other hand, removing notion of devices and memory
    types would defeat the whole purpose of CMA thus destroying the solution
    that CMA provides.

2. Reuse of memory pools meaning that memory reserved by CMA can then be
    used by other allocation mechanisms.  This is of course possible.  For
    instance coherent could easily be implemented as a wrapper to CMA.
    This is doable and can be done in the future after CMA gets more
    recognition.

3. Reuse of algorithms meaning that allocation algorithms used by other
    allocators will be used with CMA regions.  This is doable as well and
    can be done in the future.

-- 
Best regards,                                        _     _
| Humble Liege of Serenely Enlightened Majesty of  o' \,=./ `o
| Computer Science,  Michał "mina86" Nazarewicz       (o o)
+----[mina86*mina86.com]---[mina86*jabber.org]----ooO--(_)--Ooo--

--

[ message continues ]

You can create something similar to the existing API for memory
allocator.

For example, blk_kmalloc/blk_alloc_pages was proposed as memory
allocator API with notion of an address range for allocated memory. It
wasn't merged for other reasons though.

I don't mean that this is necessary for the inclusion (I'm not the
person to ack or nack this). I just expect the similarity of memory

Well, why can't we do the above before the inclusion?

Anyway, I think that comments from mm people would be helpful to merge
this.
--

[ message continues ]

That may be tricky.  cma_alloc() takes four parameters each of which is
required for CMA.  No other existing set of API uses all those arguments.
This means, CMA needs it's own, somehow unique API.  I don't quite see
how the APIs may be unified or "made similar".  Of course, I'm gladly

Because it's quite a bit of work and instead of diverting my attention I'd
prefer to make CMA as good as possible and then integrate it with other
subsystems.  Also, adding the integration would change the patch from being
4k lines to being like 40k lines.

What I'm trying to say is that I don't consider that a work for now but

Yes, I agree.

-- 
Best regards,                                        _     _
| Humble Liege of Serenely Enlightened Majesty of  o' \,=./ `o
| Computer Science,  Michał "mina86" Nazarewicz       (o o)
+----[mina86*mina86.com]---[mina86*jabber.org]----ooO--(_)--Ooo--

--

[ message continues ]

On Fri, 20 Aug 2010 08:38:10 +0200

Have you even tried to search 'blk_kmalloc' on google? I wrote

4k to 40k? I'm not sure. But If I see something like the following, I
suspect that there is a better way to integrate this into the existing
infrastructure.

mm/cma-best-fit.c                   |  407 +++++++++++++++
--

[ message continues ]

I have and I haven't seen any way how

   void *()(struct request_queue *q, unsigned size, gfp_t gfp);

prototype could be applied to CMA.  I admit that I haven't read the whole
discussion of the patch and maybe I'm missing something about Andi's patches
but I don't see how CMA could but from what I've understood blk_kmalloc() is

Yes, but I don't really know what you have in mind.  CMA is similar to various
APIs in various ways: it's similar to any allocator since it takes size in bytes,
it's similar to coherent since it takes device, it's similar to bootmem/memblock/etc
since it takes alignment.  I would appreciate if you could give some examples of what

Ah, sorry.  I misunderstood you.  I thought you were replying to both 2. and 3.
above.

If we only take allocating algorithm then you're right.  Reusing existing one
should not increase the patch size plus it would be probably a better solution.

No matter, I would rather first work and core CMA without worrying about reusing
kmalloc()/coherent/etc. code especially since providing a plugable allocator API
integration with existing allocating algorithms can be made later on.  To put it
short I want first to make it work and then improve it.

-- 
Best regards,                                        _     _
| Humble Liege of Serenely Enlightened Majesty of  o' \,=./ `o
| Computer Science,  Michał "mina86" Nazarewicz       (o o)
+----[mina86*mina86.com]---[mina86*jabber.org]----ooO--(_)--Ooo--

--

[ message continues ]

On Fri, 20 Aug 2010 10:10:45 +0200

why don't take gfp_t flags?

Something like dev_alloc_page is more appropriate name?

Or something similar to dmapool API (mm/dmapool.c) might work
better. The purpose of dmapool API is creating a pool for consistent
memory per device. It's similar to yours, creating a pool for

I don't think that bootmem/memblock matters here since it's not the

I'm not sure that's how a new feature is merged.
--

[ message continues ]

Because they are insufficient.  Either that or I don't understand gfp_t.

With CMA, platform can define many memory types.  For instance, if there are
two memory bans there can be two memory types for the two banks.  For at least one
of the device I'm in contact with, another type for it's firmware is also needed.

Two things: I'd prefer a "cma" prefix rather then "dev" and I think it should
be "pages", right? Then, size should be given in pages rather then bytes.


I'll try to look at it later on and think about it.  I'm still somehow reluctant
to change the names but still, thank you for suggestions.

-- 
Best regards,                                        _     _
| Humble Liege of Serenely Enlightened Majesty of  o' \,=./ `o
| Computer Science,  Michał "mina86" Nazarewicz       (o o)
+----[mina86*mina86.com]---[mina86*jabber.org]----ooO--(_)--Ooo--

--

[ message continues ]