Computers generate, process and delete data. However, they can only store data for very short periods. Therefore, computers move data to storage devices such as tape libraries and the disk subsystems discussed in the previous chapter for long-term storage and fetch it back from these storage media for further processing. So-called I/O techniques realize the data exchange between computers and storage devices. This chapter describes I/O techniques that are currently in use or that the authors believe will very probably be used in the coming years.This chapter first considers the I/O path from the CPU to the storage system (Sect-ion 3.1). An important technique for the realization of the I/O path is SCSI (Section 3.2).To be precise, SCSI defines a medium (SCSI cable) and a communication protocol (SCSI protocol). The idea of Fibre Channel SAN is to replace the SCSI cable by a network that is realized using Fibre Channel technology: servers and storage devices exchange data as before using SCSI commands, but the data is transmitted via the Fibre Channel network instead of via the SCSI cable (Sections 3.3, 3.4). An alternative to Fibre Channel SAN is IP storage. Like Fibre Channel, IP storage connects several servers and storage devices via a network on which data exchange takes place using the SCSI protocol. In contrast to Fibre Channel, however, the devices are connected by TCP/IP and Ethernet (Section 3.5). With InfiniBand, networks move even closer to the CPU. The objective of InfiniBand is to replace the PCI bus in the computer by a serial network (Section 3.6). These new transfer technologies (Fibre Channel, Gigabit Ethernet and in particular InfiniBand) form the basis for lightweight communication connections such as Virtual Interfaces (VI) and Remote Direct Memory Access (RDMA), which permit the fast and efficient exchange of data (Section 3.7). Finally we present an emerging protocol family which combinesthese lightweight communication connections with TCP/IP. Namely we present RDMAover TCP, the Socket Direct Protocol (SDP) and the iSCSI Extension for RDMA (iSER)(Section 3.8).
THE PHYSICAL I/O PATH FROM THE CPUTO THE STORAGE SYSTEM
In the computer, one or more CPUs process data that is stored in the CPU cache or in themain memory (Random Access Memory, RAM). CPU cache and main memory are very fast; however, they cannot store data after the power has been switched off. Furthermore, main memory is expensive in comparison to disk and tape storage. Therefore, the data is moved from the main memory to the storage devices such as disk subsystems and tape libraries via system bus, host bus and I/O bus (Figure 3.1). Although storage devices are slower than CPU cache and main memory, they compensate for this by being cheaper and
Figure 3.1
The physical I/O path from the CPU to the storage system consists of system bus, host I/O bus and I/O bus.More recent technologies such as InfiniBand, Fibre Channel and iSCSI replace individual buses with a serial network. For historic reasons the corresponding connections are still called host I/O bus or I/O bus
THE PHYSICAL I/O PATH FROM THE CPU TO THE STORAGE SYSTEM 51
by their ability to store data even when the power is switched off. Incidentally, the sameI/O path also exists within a disk subsystem between the connection ports and the disksubsystem controller and between the controller and the internal hard disk (Figure 3.2).At the heart of the computer, the system bus ensures the rapid transfer of data between CPUs and main memory. The system bus must be timed at a very high frequency so that
it can supply the CPU with data sufficiently quickly. It is realized in the form of printed conductors on the main circuit board. Due to physical properties, high system speeds require short printed conductors. Therefore, the system bus is kept as short as possible and thus connects only CPUs and main memory.
In modern computers as many tasks as possible are moved to special processors such as graphics processors in order to free up the CPU for the processing of the application. These cannot be connected to the system bus due to the physical limitations mentioned above. Therefore, most computer architectures realize a second bus, the so-called host I/O bus. So-called bridge communication chips provide the connection between system bus and
host I/O bus. Peripheral Component Interconnect (PCI) is currently the most widespread technology for the realization of host I/O buses. InfiniBand is an emerging technology that will very probably replace the parallel PCI bus by a serial network (Section 3.6). Device drivers are responsible for the control of and communication with peripheral devices of all types. The device drivers for storage devices are partially realized in the form of software that is processed by the CPU. However, part of the device driver for the communication with storage devices is almost always realized by firmware that is processed by special processors (Application Specific Integrated Circuits, ASICs). These ASICs are currently partially integrated into the main circuit board, such as on-board SCSI controllers, or connected to the main board via add-on cards (PCI cards). These add-on cards are usually called network cards (Network Interface Controller, NIC) or simply controllers. Storage devices are connected to the server via the host bus adapter (HBA) or via the on-board controller. The communication connection between controller and peripheral device is called the I/O bus. The most important technologies for I/O buses are currently SCSI (Small Computer
System Interface) and Fibre Channel. SCSI defines a parallel bus that can connect up to 16 servers and storage devices with one another. Fibre Channel, on the other hand, defines different topologies for storage networks that can connect several millions of servers and storage devices. As an alternative to Fibre Channel, the industry is currently experimenting with different options for the realization of storage networks by means of
TCP/IP and Ethernet (IP storage). It is worth noting that all new technologies continue to use the SCSI protocol for device communication. The Virtual Interface Architecture (VIA) is a further I/O protocol. The VIA permits rapid and CPU-saving data exchange between two processes that run on two different servers or storage devices. In contrast to the I/O techniques discussed previously the Virtual Interface Architecture defines only a protocol. As a medium it requires the exis-tence of a powerful and low-error communication path, which is realized, for example, by means of Fibre Channel, Gigabit Ethernet or InfiniBand. VIA could become an important technology for storage networks and server clusters. There are numerous other I/O bus technologies on the market that will not be discussedfurther in this book, for example, Serial Storage Architecture (SSA), IEEE 1394 (Apple's Fire wire, Sony's i.Link), High-Performance Parallel Interface (HIPPI), Advanced Tech-nology Attachment (ATA)/Integrated Drive Electronics (IDE), Serial ATA (SATA) and Universal Serial Bus (USB). All have in common that they are either used by veryfew manufacturers or are not powerful enough for the connection of servers and storage devices. Some of these technologies can form small storage networks. However, none is anywhere near as flexible and scalable as the Fibre Channel and IP Storage technologiesdescribed in this book.
No comments:
Post a Comment