Protocol weds Ethernet and ATA drives

Applications need massive amounts of storage, but servers have limited disk space. Moving storage out of a server and putting it on a network makes expanding storage easy. The ATA-over-Ethernet storage protocol (AoE) combines Ethernet and low-cost disks to create a simple way to connect storage to a network.

Much like the Fibre Channel storage protocol, which uses SCSI disk commands over fibre optics, AoE is designed to transport ATA disk commands over standard Ethernet without using TCP/IP. Eliminating the complexities of TCP/IP and Fibre Channel makes AoE storage inexpensive and easy to use. AoE enables unlimited scalability, and disks can be shared by any servers on a network.

AoE is a command/response protocol that puts Ethernet connectors on disk drives. AoE clients use a block device driver (initiator), which lets a very large number of AoE devices (targets) appear as local disks. The AoE protocol enables a driver to discover target devices using configuration information stored in those devices.

Two types of messages are transferred via AoE. One carries ATA disk commands, and the other is used for discovering AoE targets. The beginning of each message identifies a target's physical location, carries a correlation tag and defines the type of message. The physical location is recorded in a 16-bit major and an 8-bit minor address.

The major address is usually an assigned chassis, or shelf, number. The minor address is a disk slot in the chassis or, in the case of a RAID target, a logical unit number. A correlation tag is used by the initiator to uniquely identify each message. This allows multiple outstanding requests at any given time.

ATA commands
The most common AoE message is an ATA command and data to be written to a target. The ATA command can be thought of as an operation code and a group of parameters. The target receives the command, copies the parameters into the disk's registers, issues the command and then monitors the status of the disk. On completion, the disk returns status and error register values, along with any data that has been read.

The second type of message is used in a configuration/discovery process. Each AoE target device has a short string of bytes that can be set, read and queried. Using Ethernet broadcast packets and the prefix match command, AoE clients can discover AoE targets available for use on a network.

AoE storage devices can be single disks or groups of disks assembled into RAID volumes. They can be partitioned like any disk. AoE devices can be co-operatively claimed by a single AoE client, or simultaneously shared by multiple AoE clients in clustered environments. Because AoE devices are block storage, they can be used as raw storage disks or mounted with any disk file system. AoE devices can be managed with volume-management software tools and become part of large storage systems. AOE storage devices are suited for applications that require low-cost and scalability, including server clusters, disk-to-disk backup, e-mail servers, databases, medical images and records, video surveillance and many others.

Most popular Linux distributions include AoE drivers, and hundreds of users are benefiting from expanded storage using AoE.

Brantley Coile is the founder and chief technical officer of Coraid and a co-author of the AoE protocol.