BASICES THE FIBRE CHANNEL PROTOCOL STACK,FIBRE CHANNEL SWITCHES AND LINKS PORTS AND TOPOLOGIES

BASICES THE FIBRE CHANNEL PROTOCOL STACK

Fibre Channel is currently the technique for the realization of storage networks. Interestingly, Fibre Channel was originally developed as a backbone technology for the connection of LANs. The original development objective for Fibre Channel was to supersede Fast-Ethernet (100 Mbit/s) and Fibre Distributed Data Interface (FDDI). Now it looks as if Gigabit Ethernet and 10 Gigabit Ethernet have become prevalent or will become prevalent in this market segment. By coincidence, the design goals of Fibre Channel are covered by the requirements of a transmission technology for storage networks such as:• serial transmission for high speed and long distances;

• low rate of transmission errors;

• low delay (latency) of the transmitted data;

• implementation of the Fibre Channel protocol in hardware on host bus adapter cards to free up the server CPUs.

In the early 1990s, Seagate was looking for a technology that it could position against IBM's Serial Storage Architecture (SSA). With the support of the Fibre Channel industry, Fibre Channel was expanded by the arbitrated loop topology, which is cheaper than the originally developed fabric topology. This led to the breakthrough of Fibre Channel for the realization of storage networks. Fibre Channel is only one of the transmission technologies with which storage area net-

works (SANs) can be realized. Nevertheless, the terms 'Storage Area Network' and 'SAN' are often used synonymously with Fibre Channel technology. In discussions, newspaper articles and books the terms 'storage area network' and SAN are often used to mean a storage area network that is built up using Fibre Channel. The advantages of storage area networks and server-centric IT architectures can, however, also be achieved using other

technologies for storage area networks, for example, iSCSI. In this book we have taken great pains to express ourselves precisely. We do not use the

terms 'storage area network' and 'SAN' on their own. For unambiguous differentiation we always also state the technology, for example, 'Fibre Channel SAN' or 'iSCSI SAN'.In statements about storage area networks in general that are independent of a specific technology we use the term 'storage network'. We use the term 'Fibre Channel' without the suffix 'SAN' when we are referring to the transmission technology that underlies a Fibre Channel SAN.For the sake of completeness we should also mention that the three letters 'SAN' are also used as an abbreviation for 'System Area Network'. A System Area Network is a

network with a high bandwidth and a low latency that serves as a connection between computers in a distributed computer system. In this book we have never used the abbreviation SAN in this manner. However, it should be noted that the VIA standard, for example, does use this second meaning of the abbreviation 'SAN'.The Fibre Channel protocol stack is subdivided into five layers (Figure 3.8). The lowerfour layers, FC-0 to FC-3 define the fundamental communication techniques, i.e. the physical levels, the transmission and the addressing. The upper layer, FC-4, defines how application protocols (upper layer protocols, ULPs) are mapped on the underlying Fibre Channel network. The use of the various ULPs decides, for example, whether a realFibre Channel network is used as an IP network, a Fibre Channel SAN (i.e. as a storage network) or both at the same time. The link services and fabric services are located quasi-

adjacent to the Fibre Channel protocol stack. These services will be required in order to administer and operate a Fibre Channel network. Basic knowledge of the Fibre Channel standard helps to improve understanding of the possibilities for the use of Fibre Channel for a Fibre Channel SAN. This section

(Section 3.3) explains technical details of the Fibre Channel protocol. We will restrict the level of detail to the parts of the Fibre Channel standard that are helpful in the administration or the design of a Fibre Channel SAN. Building upon this, the next section(Section 3.4) explains the use of Fibre Channel for storage networks.

Links, ports and topologies

The Fibre Channel standard defines three different topologies: fabric, arbitrated loop and point-to-point (Figure 3.9). Point-to-point defines a bi-directional connection between two devices. Arbitrated loop defines a unidirectional ring in which only two devices can ever exchange data with one another at any one time. Finally, fabric defines a network in which several devices can exchange data simultaneously at full bandwidth. A fabric basically requires one or more Fibre Channel switches connected together to form a control centre between the end devices. Furthermore, the standard permits the connection of one or

more arbitrated loops to a fabric. The fabric topology is the most frequently used of all topologies, and this is why more emphasis is placed upon the fabric topology than on the two other topologies in the following. Common to all topologies is that devices (servers, storage devices and switches) must

be equipped with one or more Fibre Channel ports. In servers, the port is generally realized by means of so-called host bus adapters (HBAs, for example, PCI cards) that are also fitted in the server. A port always consists of two channels, one input and one output channel. The connection between two ports is called a link. In the point-to-point topology and in the fabric topology the links are always bi-directional: in this case the input channel and the output channel of the two ports involved in the link are connected together by

 

a cross, so that every output channel is connected to an input channel. On the other hand, the links of the arbitrated loop topology are unidirectional: each output channel is connected to the input channel of the next port until the circle is closed. The cabling of an arbitrated loop can be simplified with the aid of a hub. In this configuration the end devices are bi-directionally connected to the hub; the wiring within the hub ensures that the unidirectional data flow within the arbitrated loop is maintained. The fabric and arbitrated loop topologies are realized by different, incompatible protocols. We can differentiate between the following port types with different capabilities:

• N-Port (Node Port): originally the communication of Fibre Channel was developed around N-Ports and F-Ports, with 'N' standing for 'node' and 'F' for 'fabric'. An N-Port describes the capability of a port as an end device (server, storage device), also called node, to participate in the fabric topology or to participate in the point-to-point topology as a partner.

• F-Port (Fabric Port): F-Ports are the counterpart to N-Ports in the Fibre Channel switch.The F-port knows how it can pass a frame that an N-Port sends to it through the Fibre Channel network on to the desired end device.

• L-Port (Loop Port): the arbitrated loop uses different protocols for data exchange than the fabric. An L-Port describes the capability of a port to participate in the arbitrated loop topology as an end device (server, storage device). More modern devices are now fitted with NL-Ports instead of L-Ports. Nevertheless, old devices that are fitted with an L-Port are still encountered in practice.

• NL Port (Node Loop Port): an NL-Port has the capabilities of both an N-Port and an L-port. An NL-Port can thus be connected both in a fabric and in an rbitrated loop. Most modern host bus adapter cards are equipped with NL-Ports.

• FL-Port (Fabric Loop Port): an FL-Port allows a fabric to connect to a loop. However, this is far from meaning that end devices in the arbitrated loop can communicate with end devices in the fabric. More on the subject of connecting fabric and arbitrated loop can be found in Section 3.4.3.

• E-Port (Expansion Port): two Fibre Channel switches are connected together by E-Ports. E-Ports transmit the data from end devices that are connected to two different

Fibre Channel switches. In addition, Fibre Channel switches smooth out information over the entire Fibre Channel network via E-ports. • G-Port (Generic Port): modern Fibre Channel switches configure their ports automatically. Such ports are called G-Ports. If, for example, a Fibre Channel switch is connected to a further Fibre Channel switch via a G-Port, the G-Port configures itself as an E-Port. • B-Port (Bridge Port): B-Ports serve to connect two Fibre Channel switches together via ATM or SONET/SDH. Thus Fibre Channel SANs that are a long distance apart can be connected together using classical WAN techniques. In future versions of the Fibre Channel standard we can expect B-Ports to also support Ethernet and IP. Some Fibre Channel switches have further, manufacturer-specific port types over and above those in the Fibre Channel standard: these port types provide additional functions. When using such port types, it should be noted that you can sometimes bind yourself to the Fibre Channel switches of a certain manufacturer, which cannot subsequently be replaced by Fibre Channel witches of a different manufacturer.