--=-cjWXoD/BdHvppsJ7ytbJ
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

I've been trying to write some code using librdmacm and I've run across
a few shortcomings in the library.

1)  When you listen for connections, the event includes a new cm_id
struct attached to the listen event channel.  Attempts to change this
channel make the cm_id unusable (rdma_create_qp fails).  This is
suboptimal in situations where you want the listen channel to produce
listen events only.  A function such as rdma_modify_channel(cm_id,
new_channel); would work to solve this.

2)  When you create a new cm_id with the intent of connecting to another
machine, it is again desirable to get your events related to the
establishment of the connection in a separate channel from those events
related to already established connections (amongst other things, if you
are sharing a channel with a different thread that is responsible for
tearing down connections on error, then which thread gets the
ADDR_RESOLVED or ROUTE_RESOLVED events is up in the air...to make sure
it gets delivered properly, I currently have the connecting thread
pthread_mutex_lock the connection construct, set connection->cm_waiting
=3D 1, then issue the rdma_resolve_route, then pthread_mutex_lock again so
it deadlocks, and then other thread gets the event, checks
connection->cm_waiting =3D=3D 1, and if true it places the event pointer in
connection->event, clears connection->cm_waiting, then
pthread_mutex_unlock's the connection...how gross is that).  So, using a
separate event channel up until the connection is established, then
calling rdma_modify_channel() would also solve this problem.

3)  The man pages on rdma_connect() and rdma_accept() aren't really
clear on the role of the connection parameters struct that gets passed
in.  Specifically, it doesn't say whether or not the initiator_depth and
responder_resources in the parm struct present in the listen event are
what the other side set, or if they are already swapped to indicate the
minimum/maximum that we can set on our side of the connection.  Also,
the initial message pointer is not detailed.  When we call
rdma_accept/rdma_reject, does our parm struct need to have that same
pointer?  Do we need to free that mem?  Can we supply a new initial
message and not leak the memory associated with the incoming initial
message?

Otherwise I haven't had any problems so far.

--=20
Doug Ledford <dledford@redhat.com>
              GPG KeyID: CFBFF194
              http://people.redhat.com/dledford

Infiniband specific RPMs available at
              http://people.redhat.com/dledford/Infiniband

--=-cjWXoD/BdHvppsJ7ytbJ
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)

iD8DBQBHCO6ATzP9PM7pjw8RAiwHAJ4rIMAIThBwp7yvwrnbFHQ7+p/bDwCcC9PT
cX9hX7JLIREn0LTa3BRUlxI=
=v/x1
-----END PGP SIGNATURE-----

--=-cjWXoD/BdHvppsJ7ytbJ--