On Feb 20, 2008 8:34 AM, Erez Zilber <erezz@voltaire.com> wrote:
quoted text
> Bart Van Assche wrote:
> > Or: data sent during the first burst is not transferred via one-sided
> > remote memory reads or writes but via two-sided send/receive
> > operations. At least on my setup, these operations are as fast as
> > one-sided remote memory reads or writes. As an example, I obtained the
> > following numbers on my setup (SDR 4x network);
> > ib_write_bw: 933 MB/s.
> > ib_read_bw: 905 MB/s.
> > ib_send_bw: 931 MB/s.
>
> According to these numbers one can think that you don't need RDMA at
> all, just send iSCSI PDUs over IB.

Sorry, but you are misinterpreting what I wrote.

quoted text
> The benchmarks that you use are
> synthetic IB benchmarks that are not equivalent to iSCSI over iSER. They
> just send IB packets. I'm not surprised that you got more or less the
> same performance because, AFAIK, ib_send_bw doesn't copy data (unlike
> iSCSI that has to copy data that is sent/received without RDMA).

I agree that ib_write_bw / ib_read_bw / ib_send_bw performance results
are not equivalent to iSCSI over iSER. The reason that I included
these performance results was to illustrate that two-sided data
transfers over IB are about as fast as one-sided data transfers.

quoted text
> When you use RDMA with iSCSI (i.e. iSER), you don't need to create iSCSI
> PDUs and process them. The CPU is not busy as it is with iSCSI over TCP
> because no data copies are required. Another advantage is that you don't
> need header/data digest because the IB HW does that.

As far as I know, when using iSER, the FirstBurstLength bytes of data
are sent via two-sided data transfers, and there is no CPU
intervention required to transfer the data itself over the IB network.

Bart Van Assche.
--
unsubscribe notice
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/