Re: Examining the VM splay tree effectiveness

Previous thread: letting glabel recognise a media change by Alexander Best on Wednesday, September 29, 2010 - 2:12 pm. (24 messages)

Next thread: Re: Examining the VM splay tree effectiveness by Ivan Voras on Thursday, September 30, 2010 - 10:51 am. (1 message)
From: Roman Divacky
Date: Thursday, September 30, 2010 - 10:24 am

are you aware of Summer of Code 2008 project by Mayur Shardul?

quoting: http://www.freebsd.org/projects/summerofcode-2008.html

Project: VM Algorithm Improvement
Student: Mayur Shardul
Mentor: Jeff Roberson
Summary:
A new data structure, viz. radix tree, was implemented and used for management of the resident pages. The
objective is efficient use of memory and faster performance. The biggest challenge was to service insert requests
on the data structure without blocking. Because of this constraint the memory allocation failures were not
acceptable, to solve the problem the required memory was allocated at the boot time. Both the data structures were
used in parallel to check the correctness and we also benchmarked the data structures and found that radix trees
gave much better performance over splay trees.

Ready to enter CVS/SVN: We will investigate some more approaches to handle allocation failures before the new data
structure goes in CVS.


_______________________________________________
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
From: Andre Oppermann
Date: Thursday, September 30, 2010 - 10:46 am

I remember that there was this project but I never saw any numbers
or other outcome of it.  Haven't checked p4 to look at the code
though.

-- 
_______________________________________________
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
From: Roman Divacky
Date: Thursday, September 30, 2010 - 10:49 am

there's a little something: 

	http://wiki.freebsd.org/MayurShardul/VM_Algorithm_Improvement
_______________________________________________
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
From: Roman Divacky
Date: Thursday, September 30, 2010 - 11:04 am

and the actual patch + userspace implementation + benchmarking suite
is here:

http://code.google.com/p/google-summer-of-code-2008-freebsd/downloads/detail?name=Mayu...

it looks like a solid work with solid results to me
_______________________________________________
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
From: Andre Oppermann
Date: Thursday, September 30, 2010 - 2:44 pm

I just took a look (not in-depth though) at the patch and can't follow
your conclusion. It is not ready to go into svn in its current state.

Even though it is called a radix trie it doesn't look like one.  On first
impression it looks much more like an AA tree.  A radix trie, which we
already have in our network routing table code, is a variable length
(mask) tree that does path compression.  See net/radix.[ch] and
  http://en.wikipedia.org/wiki/Radix_tree

Extrapolating in a complete guesstimating way from the lookup function
I'd say it may perform only slightly better in an ideal case than a RB
tree but with the added overall expense of requiring external memory to
store the index and branch nodes.  This is probably a nasty disadvantage.

-- 
Andre
_______________________________________________
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
From: Andre Oppermann
Date: Thursday, September 30, 2010 - 3:06 pm

Thinking more about this I can't see any advantage of the radix trie
becoming useful for the vmmap.  The index of the vmmap is the address
range and can't be expressed in a path compressable power of 2 quantity
and is non-overlapping.  The key for the trie, a pointer, has a fixed
small size and can't be optimized away.  And it is not a balanced tree.
Many keys in the same region lead to long node traversals.

In short: VMmap and a radix trie have an impedance mismatch and are unfit
for each other imho.

-- 
Andre
_______________________________________________
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
From: Freddie Cash
Date: Thursday, September 30, 2010 - 3:50 pm

For the curious, DragonflyBSD went through this back in 2005.  All the
gory details are in the thread with Subject: "splay tree and red-black
tree for vm_map entry lookups" [1] While things are most likely
different now between the FreeBSD VM and the DragonflyBSD VM, it may
be worthwhile checking out what they did, and why.  They considered
the FreeBSD splay-tree and compared it to red-black tree, and went
with red-black.

There's mention in that thread that NetBSD uses red-black trees.  No
idea if this is still correct.

[1] http://leaf.dragonflybsd.org/mailarchive/kernel/2005-01/msg00121.html

-- 
Freddie Cash
fjwcash@gmail.com
_______________________________________________
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
From: Matthew Dillon
Date: Thursday, September 30, 2010 - 9:49 pm

I don't remember the reference but I read a comprehensive comparison
    between various indexing methods about a year ago and the splay tree
    did considerably better than a RB-tree.  The RB-tree actually did
    fairly poorly.

    Any binary tree-like structure makes fairly poor use of cpu caches.
    Splay trees work somewhat better as long as there is some locality
    of reference in the lookups, since the node being looked up is moved
    to the root.  It isn't a bad trade-off.

    On the negative side all binary-tree-like structures tend to be
    difficult to MP-lock in a fine-grained manner (not that very many
    structures are locked at that fine a grain anyway, but it's a data
    point).  Splay-trees are impossible to lock at a fine-grain due to
    the massive modifications made on any search and the movement
    of the root, and there are MP access issues too.

    --

    What turned out to be the best indexing mechanism was a chained
    hash table whos hoppers were small linear arrays instead of single
    elements.  So instead of pointer-chaining each element you have a small
    for() loop for 4-8 elements before you chain.  The structure being
    indexed would NOT be integrated into the index directly, the index
    would point at the final structure from the hopper.

    For our purposes such linear arrays would contain a pointer and
    an indexing value in as small an element as possible (8-16 bytes),
    the idea being that you make the absolute best use of your cache line
    and L1 cache / memory burst.  One random access (initial hash index),
    then linear accesses using a small indexing element, then one final
    random access to get to the result structure and validate that
    it's the one desired (at that point we'd be 99.9% sure that we have
    the right structure because we have already compared the index value
    stored in the hopper).  As a plus the initial hash index also makes
    MP locking the base of the chains ...
From: Adrian Chadd
Date: Thursday, September 30, 2010 - 10:53 pm

Sounds like B+tree style stuff. Minimise the "seek" operations, as
random lookup times are orders of magnitude slower than sequential
access times.

(Memory is hierarchial, who would've thunk. :-)


Adrian
_______________________________________________
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
From: Andre Oppermann
Date: Friday, October 1, 2010 - 2:21 am

It heavily depends on the access pattern of data structure.  Is
it lookup, modify or insert/delete dominated?  Or a mix of any
of them.  How heavily is the data structure shared across CPU's?
Without this information it is impossible to make a qualified choice.

Making general comparative statements on indexing methods without
taking the access pattern and SMP/CPU cache behavior into account
is going to lead to wrong approach 90% of the time. (Made that number

Again, it hugely depends on how good the locality is and how expensive
the CPU cache line dirtying of the splay rotation is.  You can quickly
fall off an amortization *cliff* here.

I agree with binary tree structures being a bit less optimal for CPU
caches because the tree node is embedded with the data element.  On
the plus side not many other data structures are either.  And as long
as memory is only read it can be cached on multiple CPU's.  Touching
it throws it out everywhere else and causes a high latency memory access

I doubt that fine grained locking of such data structures is beneficial
in many cases.  Fine grained locking implies more locks, more bus lock
cycles, more memory barriers and more CPU cache dirtying.  As long as
a data structure's global lock is not significantly contended on and
based on that a finer locking doesn't lead to parallel operation it just

This makes a lot of sense if the index is sufficiently small, lets say
one or two int's.  When you go beyond that the advantage quickly fades


Agreed with the emphasis on including lock/atomic cycles and CPU

Without first studying the accesses pattern and applying it to
the various data structures this is a very speculative statement
to make.

-- 
Andre
_______________________________________________
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
From: Attilio Rao
Date: Thursday, September 30, 2010 - 2:05 pm

The final code (Jeff took Mayur initial patches and further refined)
resulted in a very variadic performance for simple tests, so I think
that the last posted codes had the same performance, overall, than the
splay tree.
I recall, however, that there were still rooms for improvement. Ask
Jeff about the last patches.

Thanks,
Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
_______________________________________________
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
Previous thread: letting glabel recognise a media change by Alexander Best on Wednesday, September 29, 2010 - 2:12 pm. (24 messages)

Next thread: Re: Examining the VM splay tree effectiveness by Ivan Voras on Thursday, September 30, 2010 - 10:51 am. (1 message)