KernelTrap - SCSI

Block Data Integrity

Jeremy — Wed, 11 Jun 2008 16:46:38 +0000

"These patches allow data integrity information (checksum and more) to be attached to I/Os at the block/filesystem layers and transferred through the entire I/O stack all the way to the physical storage device," began Martin Petersen. He went on to explain, "the integrity metadata can be generated in close proximity to the original data. Capable host adapters, RAID arrays and physical disks can verify the data integrity and abort I/Os in case of a mismatch." He noted that support currently only exists for SCSI disks, but that work is underway to also add support for SATA drives and SCSI tapes, "with a few minor nits due to protocol limitations the proposed SATA format is identical to the SCSI". Explaining how this works, Martin continued:

"SCSI drives can usually be reformatted to 520-byte sectors, yielding 8 extra bytes per sector. These 8 bytes have traditionally been used by RAID controllers to store internal protection information. DIF (Data Integrity Field) is an extension to the SCSI Block Commands that standardizes the format of the 8 extra bytes and defines ways to interact with the contents at the protocol level. [...] When writing, the HBA (Host Bus Adapter) will DMA 512-byte sectors from host memory, generate the matching integrity metadata and send out 520-byte sectors on the wire. The disk will verify the integrity of the data before committing it to stable storage. When reading, the drive will send 520-byte sectors to the HBA. The HBA will verify the data integrity and DMA 512-byte sectors to host memory."

SCSI Targets

Jeremy — Thu, 31 Jan 2008 18:59:36 +0000

"As you probably know there is a trend in enterprise computing towards networked storage. This is illustrated by the emergence during the past few years of standards like SRP (SCSI RDMA Protocol), iSCSI (Internet SCSI) and iSER (iSCSI Extensions for RDMA)," began Bart Van Assche, proposing that SCST be merged into the mainline kernel. He suggested that while similar to the STGT project which has been part of the mainline kernel since 2.6.20, "SCST is superior to STGT with respect to features, performance, maturity, stability, and number of existing target drivers. Unfortunately the SCST kernel code lives outside the kernel tree, which makes SCST harder to use than STGT."

SCSI subsystem maintainer, James Bottomley, was not convinced, explaining:

"The two target architectures perform essentially identical functions, so there's only really room for one in the kernel. Right at the moment, it's STGT. Problems in STGT come from the user<->kernel boundary which can be mitigated in a variety of ways. The fact that the figures are pretty much comparable on non IB networks shows this. I really need a whole lot more evidence than at worst a 20% performance difference on IB to pull one implementation out and replace it with another. Particularly as there's no real evidence that STGT can't be tweaked to recover the 20% even on IB."

Tracking Merge Candidates

Jeremy — Mon, 21 Jan 2008 16:58:05 +0000

"Yes, I know ... another tree, just what everyone wants," quipped James Bottomley, announcing his new merge candidate (-mc) tree:

"This one has a specific purpose: It's my tree tracking everyone else's git and quilt trees so I get early warning if there are going to be any merge issues. However, it struck me it might be useful to anyone wishing to track what's going upstream more closely."

James noted that his new tree is available in git, and being automatically built each night. "As you can see from the reverts and the skips, we have trouble even now (and that's after I fixed up most of the failures in SCSI). ACPI and the x86 trees clash hideously, so I kicked out x86. Jens' block tree has two patches which clash with Bart's ide quilt. Greg actually has one patch in his tree that clashes with one of mine." He also noted, "this tree is currently very storage centric (i.e. I haven't included net trees or quilts because I didn't think they'd be likely to clash with my SCSI trees). However, if it could be more generally useful, I could add other trees and quilts to it."

Asynchronous Event Notification Infrastructure

Jeremy — Mon, 29 Oct 2007 21:25:42 +0000

Jeff Garzik posted a two patch series introducing an asynchronous event notification infrastructure, "enabling SATA Asynchronous Notification ('AN') for CD/DVD devices that support it." He summarized:

"For devices that support SATA AN (only very recent ones do), this means that HAL and other userspace utilities no longer need to repeatedly poll the CD/DVD device to determine if the user has changed the media."

The first patch is for the SCSI driver and is based on work originally done by Kristen Carlson Accardi, along with "copious input from James Bottomley". The second patch updates libata to utilize the new SCSI event infrastructure.

SCSI Utility sdparm 1.02

Jeremy — Wed, 10 Oct 2007 21:02:17 +0000

Douglas Gilbert announced the 1.02 release of the sdparm utility. Originally written for Linux, it has also been ported to FreeBSD, Solaris, Tru64 and Windows. Douglas described the program:

"sdparm is a command line utility designed to get and set SCSI device parameters (cf hdparm for ATA disks). The parameters are held in mode pages. Apart from SCSI devices (e.g. disks, tapes and enclosures) sdparm can be used on any device that uses a SCSI command set. Almost all CD/DVD drives use the SCSI MMC set irrespective of the transport. sdparm also can decode VPD pages including the device identification page. Commands to start and stop the media; load and unload removable media and some other housekeeping functions are supported."

Whitelists and Blacklists

Jeremy — Fri, 14 Sep 2007 03:52:44 +0000

"It turns out that USB devices suck when it comes to powermanagement issues :(" lamented Greg KH in posting some patches to handle USB autosuspend problems. He noted that the patches were intended for inclusion in the upcoming 2.6.23 kernel, "a number of patches have been submitted near the end of this kernel release cycle that add new device ids to the quirk table in the kernel to disable autosuspend for specific devices. However, a number of developers are very worried that even with the testing that has been done, once 2.6.23 is released, we are going to get a whole raft of angry users when their devices break in nasty ways." He proved an example, "it seems that almost 2/3 of all USB printers just can not handle autosuspend. And there's a _lot_ of USB printers out there..."

Later in the discussion, Linux creator Linus Torvalds commented, "in general, I think the USB blacklist/whitelists are generally a sign of some deeper bug." He continued on to point out a number of quirks in the USB layer that need to be addressed and added:

"We used to have a lot of those things due to simply incorrect SCSI probing, causing devices to lock up because Linux probed them with bad or unexpected modepages etc. I suspect we still have old blacklist entries from those days that just never got cleaned up, because nobody ever dared remove the blacklist entry.

"We should strive to make the default behaviour be so safe that we never need a black-list (or a whitelist), and basically consider blacklists to be not a way to 'fix up a device', but a way to avoid some really serious AND *RARE* error."