Learn more on Basics Interoperability of Fibre Channel SAN

Learn more on Basics Interoperability of Fibre Channel SAN

Fibre Channel SANs are currently being successfully used in production environments. Nevertheless, interoperability is an issue with Fibre Channel SAN, as in all new cross manufacturer technologies. When discussing the interoperability of Fibre Channel SAN we must differentiate between the interoperability of the underlying Fibre Channel network layer, the interoperability of the Fibre Channel application protocols, such as FCP (SCSI over Fibre Channel) and the interoperability of the applications running on the Fibre Channel SAN. The interoperability of Fibre Channel SAN stands and falls by the interoperability of

FCP. FCP is the protocol mapping of the FC-4 layer, which maps the SCSI protocol on a Fibre Channel network (Section 3.3.8). The FCP is a complex piece of software that can only be implemented in the form of a device driver. The implementation of hardware-like device drivers alone is a task that attracts errors as if by magic. The developers of FCP device drivers must therefore test extensively and thoroughly.

Two general conditions make it more difficult to test the FCP device driver. The server initiates the data transfer by means of the SCSI protocol; the storage device only responds to the requests of the server. However, the idea of storage networks is to consolidate storage devices, i.e. for many servers to share a few large storage devices. Therefore, with storage networks a single storage device must be able to serve several parallel requests from different servers simultaneously. For example, it is typical for a server to be exchanging data with a storage device just when another server is scanning the Fibre Channel SAN for available storage devices. This situation requires end devices to be able to multitask. When testing multitasking systems the race conditions of the tasks to be performed come to bear: just a few milliseconds delay can lead to a completely different test result. The second difficulty encountered during testing is due to the large number of components that come together in a Fibre Channel SAN. Even when a single server is connected to a single storage device via a single witch, there are numerous possibilities that cannot all be tested. If, for example, a Windows server is selected, there is still the choicebetween NT, 2000 and 2003, each with different service packs. Several manufacturers offer several different models of the Fibre Channel host bus adapter card in the server. If we take into account the various firmware versions for the Fibre Channel host bus adapter cards we find that we already have more than 50 combinations before we even

select a switch. Companies want to use their storage network to connect servers and storage devicesfrom various manufacturers, some of which are already present. The manufacturers of Fibre Channel components (servers, switches and storage devices) must therefore perform interoperability tests in order to guarantee that these components work with devices from third-party manufacturers. Right at the top of the priority list are those combinations that are required by most customers, because this is where the expected profit is the highest. The result of the interoperability test is a so-called support matrix. It specifies, for example, which storage device supports which server model with which operating system versions and Fibre Channel cards. Manufacturers of servers and storage devices often limit the Fibre Channel switches that can be used. Therefore, before building a Fibre Channel SAN you should carefully check whether the manufacturers in question state that they support the planned configuration. If the desiredconfiguration is not listed, you can negotiate with the manufacturer regarding the payment of a surcharge to secure manufacturer support. Although non-supported configurations canwork very well, if problems occur, you are left without support in critical situations. If in any doubt you should therefore look for alternatives right at the planning stage. All this seems bsolutely terrifying at first glance. However, manufacturers now support a number of different configurations. If the manufacturers' support matrices are taken into consideration, robust Fibre Channel SANs can now be operated. The operation of up-to- date operating systems such as Windows NT/2000, AIX, Solaris, HP-UX and Linux is particularly unproblematic. Fibre Channel SANs are based upon Fibre Channel networks. The incompatibility of the fabric and arbitrated loop topologies and the networking of fabrics and arbitrated loops has already been discussed in Section 3.4.3. Within the fabric, the incompatibility of the Fibre Channel switches from different manufacturers should also be mentioned. At the end of 2003 we still recommend that when installing a Fibre Channel SAN only the switches

and directors of a single manufacturer are used. Even though routing between switches and directors of different manufacturers may work as expected, and basic functions of the fabric topology such as aliasing, name server and zoning work well across different vendors in so-called 'compatibility modes'. But bear in mind that there is still only a very small installed base of mixed switch vendor configurations. A standard has been passed that addresses the interoperability of these basic functions, meaning that it is now just a matter of time before these basic functions work across every manufacturers' products.However, for new functions such as SAN security, inter-switch-link trunking or B-Ports,teething troubles with interoperability must once again be expected. In general, applications can be subdivided into higher applications that model and support the business processes and system-based applications such as file systems, databases and back-up systems. The system-based applications are of particular interest from the point of view of storage networks and storage management. The compatibility of network file systems such as NFS and CIFS is now taken for granted and hardly ever queried. As storage networks penetrate into the field of file systems, cross-manufacturer standards are becoming ever more important in this area too. A first offering is Network Data Management Protocol (NDMP, Section 7.9.4) for the back-up of NAS servers. Further down the road we expect also a customer demand for cross-vendor standards in the emerging field of storage virtualization (Chapter 5). The subject of interoperability will preoccupy manufacturers and customers in the field of storage networks for a long time to come. Virtual Interface Architecture (VIA), Infini- Band and Remote Direct Memory Access (RDMA) represent emerging new technologies that must also work in a cross-manufacturer manner. The same applies for Internet SCSI (iSCSI) and its variants like iSCSI Extensions over RDMA (iSER). iSCSI transmits the

SCSI protocol via TCP/IP and, for example, Ethernet. Just like FCP, iSCSI has to serialize he SCSI protocol bit-by-bit and map it onto a complex network topology. Interoperability will therefore also play an important role in iSCSI.

FREE TUTORS ON THE FIBRE CHANNEL PROTOCOL STACK COMMON SERVICES Link services: login and addressing and (Fabric services: name server and co)

THE FIBRE CHANNEL PROTOCOL STACK COMMON SERVICES (Fabric services: name server and co)

FC-3 has been in its conceptual phase since 1988; in currently available products FC-3 is empty. The following functions are being discussed for FC-3:

• Striping manages several paths between multiport end devices. Striping could distribute the frames of an exchange over several ports and thus increase the throughput between the two devices.

• Multi patching combines several paths between two multiport end devices to form a logical path group. Failure or overloading of a path can be hidden from the higher protocol layers.

• Compressing the data to be transmitted, preferably realized in the hardware on the host bus adapter.

• Encryption of the data to be transmitted, preferably realized in the hardware on the host bus adapter.

• Finally, mirroring and other RAID levels are the last example that are mentioned in the Fibre Channel standard as possible functions of FC-3. However, the fact that these functions are not realized within the Fibre Channel protocol does not mean that they are not available at all. For example, multipathing functions are currently provided both by suitable additional software in the operating system (Section 6.3.1) and also by some more modern Fibre Channel switches

(ISL Trucking).3.3.6 Link services: login and addressing Link services and the fabric services discussed in the next section stand next to the Fibre

Channel protocol stack. They are required to operate data traffic over a Fibre Channel network. Activities of these services do not result from the data traffic of the application protocols. Instead, these services are required to manage the infrastructure of a Fibre Channel network and thus the data traffic on the level of the application protocols. For example, at any given time the switches of a fabric know the topology of the whole network. Login Two ports have to get to know each other before application processes can exchange data over them. To this end the Fibre Channel standard provides a three-stage login mechanism (Figure 3.20): 1. Fabric login (FLOGI)The fabric login establishes a session between an N-Port and a corresponding F-Port. The fabric login takes place after the initialization of the link and is an absolute prerequisite for the exchange of further frames. The F-Port assigns the N-Port a dynamic

address. In addition, service parameters such as the buffer-to-buffer credit are negotiated. The fabric login is crucial for the point-to-point topology and for the fabric topology. An N-Port can tell from the response of the corresponding port whether it is a fabric topology or a point-to-point topology. In arbitrated loop topology the fabric login is optional. 2. N-Port login (PLOGI) N-Port login establishes a session between two N-ports. The N-Port login takes place after the fabric login and is a compulsory prerequisite for the data exchange at FC-4 level. N-Port login negotiates service parameters such as end-to-end credit. N-Port login

is optional for Class 3 communication and compulsory for all other service classes.3. Process login (PRLI) Process login establishes a session between two FC-4 processes that are based upon two different N-Ports. These could be system processes in Unix systems and system partitions in mainframes. Process login takes place after the N-Port login. Process login is optional from the point of view of FC-2. However, some FC-4 protocol mappings call for a process login for the exchange of FC-4-specific service parameters.

Addressing

Fibre Channel differentiates between addresses and names. Fibre Channel devices (servers, switches, ports) are differentiated by a 64-bit identifier. The Fibre Channel standard defines different name formats for this. Some name formats guarantee that such a 64-bit identifier will only be issued once world-wide. Such identifiers are thus also known as World Wide Names (WWPN). On the other hand, 64-bit identifiers that can be issued several times inseparate networks are simply called Fibre Channel Names (FCN).In practice this fine distinction between WWN and FCN is hardly ever noticed, with all 64-bit identifiers being called WWNs. In the following we comply with the general usage and use only the term WWN. World Wide Names are differentiated into World Wide Port Names (WWPNs) andWorld Wide Node Names (WWNNs). As the name suggests, every port is assigned its own World Wide Name in the form of a World Wide Port Name and in addition the entire device is assigned its own World Wide Name in the form of a World Wide Node Name. The differentiation between World Wide Node Name and World Wide Port Name allows us to determine which ports belong to a common multiport device in the FibreChannel network. Examples of multiport devices are intelligent disk subsystems with several Fibre Channel ports or servers with several Fibre Channel host bus adapter cards.WWNNs could also be used to realize services such as striping over several redundant physical paths within the Fibre Channel protocol. As discussed above (Section 3.3.5,

'FC-3: common services'), the Fibre Channel standard unfortunately does not support these options, so that such functions are implemented in the operating system or by manufacturer-specific expansions of the Fibre Channel standard. In the fabric, each 64-bit World Wide Port Name is automatically assigned a 24-bit port address (N-Port identifier, N-Port ID) during fabric login. The 24-bit port addresses are used within a Fibre Channel frame for the identification of transmitter and receiver of the frame. The port address of the transmitter is called the Source Identifier (S ID) and that of the receiver the Destination Identifier (D ID). The 24-bit addresses are hierarchically structured and mirror the topology of the Fibre Channel network. As a result, it is a simple matter for a Fibre Channel switch to recognize which port it must send an incoming frame to from the destination ID (Figure 3.21). Some of the 24-bit addresses are reserved

for special purposes, so that 'only' 15.5 million addresses remain for the addressing of devices. In the arbitrated loop every 64-bit World Wide Port Name is even assigned only an eight-bit address, the so-called Arbitrated Loop Physical Address (AL PA). Of the 256possible eight-bit addresses, only those for which the 8b/10b encoded transmission word contains an equal number of zeros and ones may be used. Some ordered sets for the configuration of the arbitrated loop are parametrized using AL PAs. Only by limiting thevalues for AL PAs is it possible to guarantee a uniform distribution of zeros and onesin the whole data stream. After the deduction of a few of these values for the control ..., Fibre Channel differentiates end devices using World Wide Node Names (WWPN). Each connection port is assigned its own World Wide Port Name (WWPN). For addressing in the fabric WWNNs or WWPNs are converted into shorter Port IDs that reflect the network topology of the arbitrated loop, 127 addresses of the 256 possible addresses remain. One of these addresses is reserved for a Fibre Channel switch so only 126 servers or storage devices can be connected in the arbitrated loop.

Fabric services: name server and co

In a fabric topology the switches manage a range of information that is required for the operation of the fabric. This information is managed by the so-called fabric services. All services have in common that they are addressed via FC-2 frames and can be reached by defined addresses (Table 3.3). In the following we introduce the fabric login server, the fabric controller and the name server. The fabric login server processes incoming fabric login requests under the address

'0×FF FF FE'. All switches must support the fabric login under this address. The fabric controller manages changes to the fabric under the address '0×FF FF FD'.

N-Ports can register for state changes in the fabric controller (State Change Registration, SCR). The fabric controller then informs registered N-Ports of changes to the fabric (Registered State Change Notification, RSCN). Servers can use this service to monitor their storage devices. The name server (Simple Name Server to be precise) administers a database on N-Ports under the address '0×FF FF FC'. It stores information such as port WWN, node WWN, port address, supported service classes, supported FC-4 protocols, etc. N-Ports can register

their own properties with the name server and request information on other N-Ports. Like all services, the name server appears as an N-Port to the other ports. N-Ports must log on with the name server by means of port login before they can use its services.

FREE TUTORS ON THE FIBRE CHANNEL PROTOCOL STACK FLOW CONTROL (Service classes)

Free Tutors on THE FIBRE CHANNEL PROTOCOL STACK

FC-1: 8b/10b encoding, ordered sets and link control protocolFC-1 defines how data is encoded before it is transmitted via a Fibre Channel cable(8b/10b encoding). FC-1 also describes certain transmission words (ordered sets) that are required for the administration of a Fibre Channel connection (link control protocol). 8b/10b encoding In all digital transmission techniques, transmitter and receiver must synchronize their clock-pulse rates. In parallel buses the bus rate is transmitted via an additional data line. By contrast, in the serial transmission used in Fibre Channel only one data line is available through which the data is transmitted. This means that the receiver must regenerate the transmission rate from the data stream.

The receiver can only synchronize the rate at the points where there is a signal change in the medium. In simple binary encoding (Figure 3.11) this is only the case if the signal changes from '0' to '1' or from '1' to '0'. In Manchester encoding there is a signal change for every bit transmitted. Manchester encoding therefore creates two physical signals for each bit transmitted. It therefore requires a transfer rate that is twice as high as that for binary encoding. Therefore, Fibre Channel – like many other transmission techniques – uses binary encoding, because at a given rate of signal changes more bits can be transmitted than is the case for Manchester encoding. The problem with this approach is that the signal steps that arrive at the receiver are not always the same length (jitter). This means that the signal at the receiver is sometimes a little longer and sometimes a little shorter (Figure 3.12). In the escalator analogy this means that the escalator bucks. Jitter can lead to the receiver losing synchronization with the received signal. If, for example, the transmitter sends a sequence of ten zeros, the receiver cannot decide whether it is a sequence of nine, ten or eleven zeros. If we nevertheless wish to use binary encoding, then we have to ensure that the data stream generates a signal change frequently enough that jitter cannot strike. The so-called 8b/10b encoding represents a good compromise. 8b/10b encoding converts an eight-bit

byte to be transmitted into a ten-bit character, which is sent via the medium instead of the eight-bit byte. For Fibre Channel this means, for example, that a useful transfer rate of 100 MByte/s requires a raw transmission rate of 1 Gbit/s instead of 800 Mbit/s. Incidentally, 8b/10b encoding is also used for the Enterprise System Connection Architecture (ESCON), Serial Storage Architecture (SSA), Gigabit Ethernet and InfiniBand. Finally, it should be noted that 1 Gigabyte Fibre Channel uses the 64b/66b encoding variant for a certain cable type (single lane with serial transmission). Expanding the eight-bit data bytes to ten-bit transmission character gives rise to the following advantages: • In 8b/10b encoding, of all available ten-bit characters, only those that generate a bit

sequence that contains a maximum of five zeros one after the other or five ones one after the other for any desired combination of the ten-bit character are selected. There- fore, a signal change takes place at the latest after five signal steps, so that the clock synchronization of the receiver is guaranteed.

• A bit sequence generated using 8b/10b encoding has a uniform distribution of zeros and ones. This has the advantage that only small direct currents flow in the hardware that processes the 8b/10b encoded bit sequence. This makes the realization of Fibre Channel hardware components simpler and cheaper.

• Further ten-bit characters are available that do not represent eight-bit data bytes. These additional characters can be used for the administration of a Fibre Channel link.

Ordered sets

Fibre Channel aggregates four ten-bit transmission characters to form a 40-bit transmission word. The Fibre Channel standard differentiates between two types of transmission word: data words and ordered sets. Data words represent a sequence of four eight-bit data bytes. Data words may only stand between a Start-of-Frame delimiter (SOF delimiter) and an End-of-Frame delimiter (EOF delimiter).Ordered sets may only stand between an EOF delimiter and a SOF delimiter, with SOF sand EOFs themselves being ordered sets. All ordered sets have in common that they begin with a certain transmission character, the so-called K28.5 character. The K28.5 character includes a special bit sequence that does not occur elsewhere in the data stream. The input channel of a Fibre Channel port can therefore use the K28.5 character to divide the continuous incoming bit stream into 40 bit transmission words when initializing a Fibre Channel link or after the loss of synchronization on a link. Link control protocol With the aid of ordered sets, FC-1 defines various link level protocols for the initialization and Administration of a link. The initialization of a link is the prerequisite for data exchange by means of frames. Examples of link level protocols are the initialization and arbitration of an arbitrated loop.

3.3.4 FC-2: data transfer

FC-2 is the most comprehensive layer in the Fibre Channel protocol stack. It determines how larger data units (for example, a file) are transmitted via the Fibre Channel network. It regulates the flow control that ensures that the transmitter only sends the data at a speed that the receiver can process it. And it defines various service classes that are tailored to the requirements of various applications. Exchange, sequence and frame FC-2 introduces a three-layer hierarchy for the transmission of data (Figure 3.13). At the top layer a so-called exchange defines a logical communication connection between two end devices. For example, each process that reads and writes data could be assigned its own exchange. End devices (servers and storage devices) can simultaneously maintain several exchange relationships, even between the same ports. Different exchanges help the FC-2 layer to deliver the incoming data quickly and efficiently to the orrect receiver in the higher protocol layer (FC-3). A sequence is a larger data unit that is transferred from a transmitter to a receiver. Only one sequence can be transferred after another within an exchange. FC-2 guarantees that sequences are delivered to the receiver in the same order they were sent from the transmitter; hence the name 'sequence'. Furthermore, sequences are only delivered to the next protocol layer up when all frames of the sequence have arrived at the receiver (Figure 3.13). A sequence could represent the writing of a file or an individual database transaction. A Fibre Channel network transmits control frames and data frames. Control frames contain no useful data, they signal events such as the successful delivery of a data frame. Data frames transmit up to 2112 bytes of useful data. Larger sequences therefore have to be broken down into several frames. Although it is theoretically possible to agree upon

different maximum frame sizes, this is hardly ever done in practice. A Fibre Channel frame consists of a header, useful data (payload) and a CRC checksum

(Figure 3.14). In addition, the frame is bracketed by a Start-of-Frame delimiter (SOF) and an End-of-Frame delimiter (EOF). Finally, six filling words must be transmitted by means of a link between two frames. In contrast to Ethernet and TCP/IP, Fibre Channel is an integrated whole: the layers of the Fibre Channel protocol stack are so well harmonizedwith one another that the ratio of payload to protocol overhead is very efficient at up to 98%. The CRC checking procedure is designed to recognize all transmission errors if the underlying medium does not exceed the specified error rate of 10−12 Error correction takes place at sequence level: if a frame of a sequence is wrongly transmitted, the entire sequence is retransmitted. At gigabit speed it is more efficient to resend a complete sequence than to extend the Fibre Channel hardware so that individual lost frames can be resent and inserted in the correct position. The underlying protocol

layer must maintain the specified maximum error rate of 10−12so that this procedures efficient.

Flow control

Flow control ensures that the transmitter only sends data at a speed that the receiver can receive it. Fibre Channel uses the so-called credit model for this. Each credit represents the capacity of the receiver to receive a Fibre Channel frame. If the receiver awards the transmitter a credit of '4', the transmitter may only send the receiver four frames. The transmitter may not send further frames until the receiver has acknowledged the receipt of at least some of the transmitted frames. FC-2 defines two different mechanisms for flow control: end-to-end flow control and link flow control (Figure 3.15). In end-to-end flow control two end devices negotiate the end-to-end credit before the data exchange. The end-to-end flow control is realized on the host bus adapter cards of the end devices. By contrast, link flow control takes place at each physical connection. This is achieved by two communicating ports negotiating the buffer-to-buffer credit. This means that the link flow control also takes place at the Fibre Channel switches.

Service classes

The Fibre Channel standard defines six different service classes for data exchange between end devices. Three of these defined classes (Class 1, Class 2 and Class 3) are realized in products available on the market, with hardly any products providing the connection- oriented Class 1. Almost all new Fibre Channel products (host bus adapters, switches, storage devices) support the service classes Class 2 and Class 3, which realize a packet- oriented service (datagram service). In addition, Class F serves for the data exchange between the switches within a fabric. Class 1 defines a connection-oriented communication connection between two node ports: a Class 1 connection is opened before the transmission of frames. This specifies a route through the Fibre Channel network. Thereafter, all frames take the same route through the Fibre Channel network so that frames are delivered in the sequence in which they were transmitted. A Class 1 connection guarantees the availability of the full bandwidth. A port thus cannot send any other frames while a Class 1 connection is open.

Class 2 and Class 3, on the other hand, are packet-oriented services (datagram services): no dedicated connection is built up, instead the frames are individually routed through the Fibre Channel network. A port can thus maintain several connections at the same time. Several Class 2 and Class 3 connections can thus share the bandwidth. Class 2 uses end-to-end flow control and link flow control. In Class 2 the receiver acknowledges each received frame (acknowledgement, Figure 3.16). This acknowledge-ment is used both for end-to-end flow control and for the recognition of lost frames. A missing acknowledgement leads to the immediate recognition of transmission errors byFC-2, which are then immediately signalled to the higher protocol layers. The higherprotocol layers can thus initiate error correction measures straight away (Figure 3.18).Users of a Class 2 connection can demand the delivery of the frames in the correct order.Class 3 achieves less than Class 2: frames are not acknowledged (Figure 3.17). Thismeans that only link flow control takes place, not end-to-end flow control. In addition, the higher protocol layers must notice for themselves whether a frame has been lost. The loss of a frame is indicated to higher protocol layers by the fact that an expected sequence is not delivered because it has not yet been completely received by the FC-2 layer. A switch may dispose of Class 2 and Class 3 frames if its buffer is full. Due to greater time-outvalues in the higher protocol layers it can take much longer to recognize the loss of aframe than is the case in Class 2 (Figure 3.19).We have already stated that in practice only Class 2 and Class 3 are important. In practice the service classes are hardly ever explicitly configured, meaning that in current Fibre Channel SAN implementations the end devices themselves negotiate whether theycommunicate by Class 2 or Class 3. From a theoretical point of view the two service classes differ in that Class 3 sacrifices some of the communication reliability of Class 2in favour of a less complex protocol. Class 3 is currently the most frequently used service class. This may be because the current Fibre Channel SANs are still very small, so that

frames are very seldom lost or overtake each other. The linking of current Fibre Channel SAN islands to a large SAN could lead to Class 2 playing a greater role in future due toits faster error recognition.

 

 

 

 

 

BASICES THE FIBRE CHANNEL PROTOCOL STACK,FIBRE CHANNEL SWITCHES AND LINKS PORTS AND TOPOLOGIES

BASICES THE FIBRE CHANNEL PROTOCOL STACK

Fibre Channel is currently the technique for the realization of storage networks. Interestingly, Fibre Channel was originally developed as a backbone technology for the connection of LANs. The original development objective for Fibre Channel was to supersede Fast-Ethernet (100 Mbit/s) and Fibre Distributed Data Interface (FDDI). Now it looks as if Gigabit Ethernet and 10 Gigabit Ethernet have become prevalent or will become prevalent in this market segment. By coincidence, the design goals of Fibre Channel are covered by the requirements of a transmission technology for storage networks such as:• serial transmission for high speed and long distances;

• low rate of transmission errors;

• low delay (latency) of the transmitted data;

• implementation of the Fibre Channel protocol in hardware on host bus adapter cards to free up the server CPUs.

In the early 1990s, Seagate was looking for a technology that it could position against IBM's Serial Storage Architecture (SSA). With the support of the Fibre Channel industry, Fibre Channel was expanded by the arbitrated loop topology, which is cheaper than the originally developed fabric topology. This led to the breakthrough of Fibre Channel for the realization of storage networks. Fibre Channel is only one of the transmission technologies with which storage area net-

works (SANs) can be realized. Nevertheless, the terms 'Storage Area Network' and 'SAN' are often used synonymously with Fibre Channel technology. In discussions, newspaper articles and books the terms 'storage area network' and SAN are often used to mean a storage area network that is built up using Fibre Channel. The advantages of storage area networks and server-centric IT architectures can, however, also be achieved using other

technologies for storage area networks, for example, iSCSI. In this book we have taken great pains to express ourselves precisely. We do not use the

terms 'storage area network' and 'SAN' on their own. For unambiguous differentiation we always also state the technology, for example, 'Fibre Channel SAN' or 'iSCSI SAN'.In statements about storage area networks in general that are independent of a specific technology we use the term 'storage network'. We use the term 'Fibre Channel' without the suffix 'SAN' when we are referring to the transmission technology that underlies a Fibre Channel SAN.For the sake of completeness we should also mention that the three letters 'SAN' are also used as an abbreviation for 'System Area Network'. A System Area Network is a

network with a high bandwidth and a low latency that serves as a connection between computers in a distributed computer system. In this book we have never used the abbreviation SAN in this manner. However, it should be noted that the VIA standard, for example, does use this second meaning of the abbreviation 'SAN'.The Fibre Channel protocol stack is subdivided into five layers (Figure 3.8). The lowerfour layers, FC-0 to FC-3 define the fundamental communication techniques, i.e. the physical levels, the transmission and the addressing. The upper layer, FC-4, defines how application protocols (upper layer protocols, ULPs) are mapped on the underlying Fibre Channel network. The use of the various ULPs decides, for example, whether a realFibre Channel network is used as an IP network, a Fibre Channel SAN (i.e. as a storage network) or both at the same time. The link services and fabric services are located quasi-

adjacent to the Fibre Channel protocol stack. These services will be required in order to administer and operate a Fibre Channel network. Basic knowledge of the Fibre Channel standard helps to improve understanding of the possibilities for the use of Fibre Channel for a Fibre Channel SAN. This section

(Section 3.3) explains technical details of the Fibre Channel protocol. We will restrict the level of detail to the parts of the Fibre Channel standard that are helpful in the administration or the design of a Fibre Channel SAN. Building upon this, the next section(Section 3.4) explains the use of Fibre Channel for storage networks.

Links, ports and topologies

The Fibre Channel standard defines three different topologies: fabric, arbitrated loop and point-to-point (Figure 3.9). Point-to-point defines a bi-directional connection between two devices. Arbitrated loop defines a unidirectional ring in which only two devices can ever exchange data with one another at any one time. Finally, fabric defines a network in which several devices can exchange data simultaneously at full bandwidth. A fabric basically requires one or more Fibre Channel switches connected together to form a control centre between the end devices. Furthermore, the standard permits the connection of one or

more arbitrated loops to a fabric. The fabric topology is the most frequently used of all topologies, and this is why more emphasis is placed upon the fabric topology than on the two other topologies in the following. Common to all topologies is that devices (servers, storage devices and switches) must

be equipped with one or more Fibre Channel ports. In servers, the port is generally realized by means of so-called host bus adapters (HBAs, for example, PCI cards) that are also fitted in the server. A port always consists of two channels, one input and one output channel. The connection between two ports is called a link. In the point-to-point topology and in the fabric topology the links are always bi-directional: in this case the input channel and the output channel of the two ports involved in the link are connected together by

 

a cross, so that every output channel is connected to an input channel. On the other hand, the links of the arbitrated loop topology are unidirectional: each output channel is connected to the input channel of the next port until the circle is closed. The cabling of an arbitrated loop can be simplified with the aid of a hub. In this configuration the end devices are bi-directionally connected to the hub; the wiring within the hub ensures that the unidirectional data flow within the arbitrated loop is maintained. The fabric and arbitrated loop topologies are realized by different, incompatible protocols. We can differentiate between the following port types with different capabilities:

• N-Port (Node Port): originally the communication of Fibre Channel was developed around N-Ports and F-Ports, with 'N' standing for 'node' and 'F' for 'fabric'. An N-Port describes the capability of a port as an end device (server, storage device), also called node, to participate in the fabric topology or to participate in the point-to-point topology as a partner.

• F-Port (Fabric Port): F-Ports are the counterpart to N-Ports in the Fibre Channel switch.The F-port knows how it can pass a frame that an N-Port sends to it through the Fibre Channel network on to the desired end device.

• L-Port (Loop Port): the arbitrated loop uses different protocols for data exchange than the fabric. An L-Port describes the capability of a port to participate in the arbitrated loop topology as an end device (server, storage device). More modern devices are now fitted with NL-Ports instead of L-Ports. Nevertheless, old devices that are fitted with an L-Port are still encountered in practice.

• NL Port (Node Loop Port): an NL-Port has the capabilities of both an N-Port and an L-port. An NL-Port can thus be connected both in a fabric and in an rbitrated loop. Most modern host bus adapter cards are equipped with NL-Ports.

• FL-Port (Fabric Loop Port): an FL-Port allows a fabric to connect to a loop. However, this is far from meaning that end devices in the arbitrated loop can communicate with end devices in the fabric. More on the subject of connecting fabric and arbitrated loop can be found in Section 3.4.3.

• E-Port (Expansion Port): two Fibre Channel switches are connected together by E-Ports. E-Ports transmit the data from end devices that are connected to two different

Fibre Channel switches. In addition, Fibre Channel switches smooth out information over the entire Fibre Channel network via E-ports. • G-Port (Generic Port): modern Fibre Channel switches configure their ports automatically. Such ports are called G-Ports. If, for example, a Fibre Channel switch is connected to a further Fibre Channel switch via a G-Port, the G-Port configures itself as an E-Port. • B-Port (Bridge Port): B-Ports serve to connect two Fibre Channel switches together via ATM or SONET/SDH. Thus Fibre Channel SANs that are a long distance apart can be connected together using classical WAN techniques. In future versions of the Fibre Channel standard we can expect B-Ports to also support Ethernet and IP. Some Fibre Channel switches have further, manufacturer-specific port types over and above those in the Fibre Channel standard: these port types provide additional functions. When using such port types, it should be noted that you can sometimes bind yourself to the Fibre Channel switches of a certain manufacturer, which cannot subsequently be replaced by Fibre Channel witches of a different manufacturer.