[...]
Quite true. It is a trade-off: additional complexity in the client
permits reduced latency and increased throughput. But is the additional
complexity -- including administrative and access control headaches --
worth it? As you say, the "complex" clients must join the data network.
Hardware manufacturers are putting so much effort into zero-copy and
RDMA. The client-to-many approach mimics that trend by minimizing
latency and data copying (and permitting use of more exotic or unusual
hardware).
But the client-to-many approach is not as complex as you make out. A
key attribute is simply for a client to be able to store new objects and
metadata on multiple servers in parallel. Once the data is stored
redundantly, the metadata controller may take quick action to
commit/abort the transaction. You can even shortcut the process further
by having the replicas send confirmations to the metadata controller.
That said, the biggest distributed systems seem to inevitably grow their
own "front end server" layer. Clients connect to N caching/application
servers, each of which behaves as you describe: the caching/app server
connects to the control and data networks, and performs the necessary
load/store operations.
Personally, I think the most simple thing for _users_ is where
semi-smart clients open multiple connections to an amorphous cloud of
servers, where the cloud is self-optimizing, self-balancing, and
self-repairing internally.
Jeff
--
