Re: [PATCH] Broadcom 8603 SAS/SATA driver, rough draft

Previous thread: [PATCH] use pgd_list_add/pgd_list_del by Akinobu Mita on Saturday, September 22, 2007 - 9:56 pm. (1 message)

Next thread: [PATCH] leds: add missing header by Pierre Ossman on Sunday, September 23, 2007 - 2:14 am. (3 messages)
To: <linux-scsi@...>
Cc: LKML <linux-kernel@...>
Date: Sunday, September 23, 2007 - 12:04 am

Rather than sitting on this for far too long, I wanted to go ahead and
get this out there. I heard some chips might be trickling out into
public hands.

This is a bare bones Broadcom 8603 SAS+SATA driver, attempting to use
the vaunted libsas. Notes:

* A quick glance at the FIXMEs will tell you obviously doesn't work.

* The hardware is quite simple and straightforward and easy to program
in an efficient way: each SAS port has a command queue (DMA ring) and
a response queue (DMA ring). Or if in SATA mode, just a command
queue.

* The SAS/SATA negotiation is largely out of our hands. The silicon
does its thing, and then tells us what type of device connected. We
are then expected to switch the port to either SAS mode or SATA mode,
accordingly.

* There is no firmware or anything. Just DMA and register bitbanging.
We have plenty of low-level control.

* The state of SAS/SATA integration is perpetually pathetic. Updates
in this area are likely. There's a rumor Brian King @ IBM may look
into this area too.

* This driver pretty much completely lacks exception handling.

As an aside, I am also writing a driver for Marvell chips that behave
quite similarly to this chip. It seems the future of storage might look
like these Broadcom and Marvell SAS+SATA DMA ring interfaces, in the
volume marketspace at least.

The 'broadsas' branch of
master.kernel.org:/pub/scm/linux/kernel/git/jgarzik/misc-2.6.git broadsas

contains the following updates:

drivers/scsi/Kconfig | 10 +
drivers/scsi/Makefile | 1 +
drivers/scsi/broadsas.c | 997 +++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 1008 insertions(+), 0 deletions(-)
create mode 100644 drivers/scsi/broadsas.c

Jeff Garzik (1):
Add rough draft Broadcom 8603 SAS/SATA driver.

diff --git a/drivers/scsi/Kconfig b/drivers/scsi/Kconfig
index 6f2c71e..44fa3a9 100644
--- a/drivers/scsi/Kconfig
+++ b/drivers/scsi/Kconfig
@@ -486,6 +486,16 @@ config SCSI_AIC7XXX_OLD
...

To: Jeff Garzik <jeff@...>
Cc: <linux-scsi@...>, LKML <linux-kernel@...>
Date: Sunday, September 23, 2007 - 7:05 pm

The first thing to note is about the specs and the pre-production
hardware: the Linux Foundation has mechanism to get both into the hands
of interested developers; if you can point me to contacts, I can at

The first thing I really noted is that SMP and STP protocol support is
stubbed out ... you really can't do anything other than direct device

That's not such a bad way of doing it ... it pretty much mimics the wire

I also note there's a slight nomenclature issue which will trip up SAS
people. All through the driver, you seem to use the word "port" to
refer to a physical phy. the struct bs_port seems to actually be a phy
descriptor ... unless there's some missing phy<->port setup logic that
will be in the final driver? The trouble is that phys and ports are

If you have a contact here too, I can get the LF NDA and hardware
programmes rolling.

James

-

To: James Bottomley <James.Bottomley@...>
Cc: <linux-scsi@...>, LKML <linux-kernel@...>
Date: Sunday, September 23, 2007 - 7:43 pm

Well, are there interested, motivated, skilled developers with time? :)

That was sarcasm :)

libsas is a big 'ole pain, that I'm finding has many aic94xx-isms buried

That's sorta the way I read the hardware docs, too.

I have some engineering questions pending with Broadcom, but from my

Yep. The hardware (on my end of the spectrum) seems to be moving
towards forcing software to generate all "packets," except (a) data
frames [generated via DMA engine] or (b) special frames that need to

Nomemclature came straight from the hardware docs, I'm afraid.

Comparing with the Marvell hardware, I can see how (with Marvell) wide
ports can be set up, and the port/phy distinction is easily programmable
depending on the situation.

Not so with BCM8603.

The only places where the docs mention SMP and STP at all is in the SAS
outgoing DMA descriptor docs, when you fill in connection type. The
_only_ other mention of SMP or STP at all is a note saying neither is
supported. So, even the docs contradict themselves, but overall I have
the feeling that SMP/STP are out of my hands.

I wonder if Broadcom's interface is born out of the closed RAID-on-chip
product that this is descended from.

Hopefully more knowledge will be gained soon, as I debug simple SAS and

Same response as at the top :) Marvell is actually better at responding
than Broadcom, but I'm quite reluctant to make /another/ introduction
(already did so for one hacker) that leads nowhere.

Jeff

-

To: Jeff Garzik <jeff@...>
Cc: <linux-scsi@...>, LKML <linux-kernel@...>
Date: Sunday, September 23, 2007 - 7:59 pm

Well, that's hardly surprising ... it was written by Adaptec and there

That would effectively render the device pretty much useless. The maing
benefit of SAS is that you can support expanders, which are the
predominant connection infrastructure. Direct connect SAS is OK, but

SMP and STP, by the way, are simple frame in, frame out. If it
identifies the initiator and target protocols and allows us to send

Same answer ... GregKH has a legion ... lets use it.

James

-

To: James Bottomley <James.Bottomley@...>
Cc: <linux-scsi@...>, LKML <linux-kernel@...>
Date: Sunday, September 23, 2007 - 8:14 pm

This is a good project for somebody who knows how SAS behaves on the
wire, which is a rather limited group.

I'm willing to help anyone who proactively moves in this direction, but
they need to be proactive about it... I am the person who will have to
make introductions at Broadcom, and convince Broadcom that new Person is
a highly capable SAS software engineer.

Telling bcm "I don't know this guy, its doubtful he knows SAS, but we

That's the way it's looking. There are a few avenues for exposing
IDENTIFY and OPEN frames and related details, but no obvious "any frame,
no problem" method like with the Marvell chip.

IMO it's also indicative that Marvell's chip uses a single set of
command and response queues, whereas Broadcom has command/response
queues for each "port" (bcm's term).

Jeff

-

To: Jeff Garzik <jeff@...>
Cc: <linux-scsi@...>, LKML <linux-kernel@...>
Date: Monday, September 24, 2007 - 10:06 am

I wasn't actually planning that ... I was planning to say we have an NDA
programme to allow them to get their docs and early access silicon in to

Heh, OK ... I'm happy to bet that the market won't be too appreciative
of a chip like this, unless its sold as pure SATA. The only real reason
for HBAs to speak SAS as well as SATA is that most dual SAS/SATA
enclosures have internal expanders, which this chip won't be able to
talk to.

On the other hand, I think I can find a nice lever to move Marvell with,
so I'll take this on without needing potentially to compromise your
contacts.

James

-

To: James Bottomley <James.Bottomley@...>
Cc: <linux-scsi@...>, LKML <linux-kernel@...>
Date: Monday, September 24, 2007 - 11:30 am

FWIW Marvell is moving quite nicely... they are actively providing docs
under NDA, and sometimes sample code (or even a GPL driver) to active
developers. Still trying to push them on opening docs, but one step at
a time :)

Jeff

-

To: Jeff Garzik <jeff@...>
Cc: <linux-scsi@...>, LKML <linux-kernel@...>
Date: Sunday, September 23, 2007 - 5:59 pm

Jeff,
Is the lack of SMP support a driver limitation or is it
the silicon?

How about support for wide ports (i.e. when 2 or more HBA
phys are attached to remote phys which have the same SAS
addresses)?

Last question: can the chip run in SCSI target mode?

Doug Gilbert
-

To: <dougg@...>
Cc: <linux-scsi@...>, LKML <linux-kernel@...>, James Bottomley <James.Bottomley@...>
Date: Sunday, September 23, 2007 - 7:46 pm

Just for everybody's information, the Marvell SAS/SATA chip for which
I'm also writing a driver definitely supports all of that: SMP, STP,
wire ports, SCSI target mode, even SATA target mode.

Jeff

-

To: <dougg@...>
Cc: <linux-scsi@...>, LKML <linux-kernel@...>, James Bottomley <James.Bottomley@...>
Date: Sunday, September 23, 2007 - 7:47 pm

s/wire/wide/ of course

-

To: <dougg@...>
Cc: <linux-scsi@...>, LKML <linux-kernel@...>
Date: Sunday, September 23, 2007 - 7:33 pm

Open question (pending w/ BCM). It looks like the answer is "silicon

AFAICT, no.

Jeff

-

Previous thread: [PATCH] use pgd_list_add/pgd_list_del by Akinobu Mita on Saturday, September 22, 2007 - 9:56 pm. (1 message)

Next thread: [PATCH] leds: add missing header by Pierre Ossman on Sunday, September 23, 2007 - 2:14 am. (3 messages)