Learn more on Basics Interoperability of Fibre Channel SAN

Learn more on Basics Interoperability of Fibre Channel SAN

Fibre Channel SANs are currently being successfully used in production environments. Nevertheless, interoperability is an issue with Fibre Channel SAN, as in all new cross manufacturer technologies. When discussing the interoperability of Fibre Channel SAN we must differentiate between the interoperability of the underlying Fibre Channel network layer, the interoperability of the Fibre Channel application protocols, such as FCP (SCSI over Fibre Channel) and the interoperability of the applications running on the Fibre Channel SAN. The interoperability of Fibre Channel SAN stands and falls by the interoperability of

FCP. FCP is the protocol mapping of the FC-4 layer, which maps the SCSI protocol on a Fibre Channel network (Section 3.3.8). The FCP is a complex piece of software that can only be implemented in the form of a device driver. The implementation of hardware-like device drivers alone is a task that attracts errors as if by magic. The developers of FCP device drivers must therefore test extensively and thoroughly.

Two general conditions make it more difficult to test the FCP device driver. The server initiates the data transfer by means of the SCSI protocol; the storage device only responds to the requests of the server. However, the idea of storage networks is to consolidate storage devices, i.e. for many servers to share a few large storage devices. Therefore, with storage networks a single storage device must be able to serve several parallel requests from different servers simultaneously. For example, it is typical for a server to be exchanging data with a storage device just when another server is scanning the Fibre Channel SAN for available storage devices. This situation requires end devices to be able to multitask. When testing multitasking systems the race conditions of the tasks to be performed come to bear: just a few milliseconds delay can lead to a completely different test result. The second difficulty encountered during testing is due to the large number of components that come together in a Fibre Channel SAN. Even when a single server is connected to a single storage device via a single witch, there are numerous possibilities that cannot all be tested. If, for example, a Windows server is selected, there is still the choicebetween NT, 2000 and 2003, each with different service packs. Several manufacturers offer several different models of the Fibre Channel host bus adapter card in the server. If we take into account the various firmware versions for the Fibre Channel host bus adapter cards we find that we already have more than 50 combinations before we even

select a switch. Companies want to use their storage network to connect servers and storage devicesfrom various manufacturers, some of which are already present. The manufacturers of Fibre Channel components (servers, switches and storage devices) must therefore perform interoperability tests in order to guarantee that these components work with devices from third-party manufacturers. Right at the top of the priority list are those combinations that are required by most customers, because this is where the expected profit is the highest. The result of the interoperability test is a so-called support matrix. It specifies, for example, which storage device supports which server model with which operating system versions and Fibre Channel cards. Manufacturers of servers and storage devices often limit the Fibre Channel switches that can be used. Therefore, before building a Fibre Channel SAN you should carefully check whether the manufacturers in question state that they support the planned configuration. If the desiredconfiguration is not listed, you can negotiate with the manufacturer regarding the payment of a surcharge to secure manufacturer support. Although non-supported configurations canwork very well, if problems occur, you are left without support in critical situations. If in any doubt you should therefore look for alternatives right at the planning stage. All this seems bsolutely terrifying at first glance. However, manufacturers now support a number of different configurations. If the manufacturers' support matrices are taken into consideration, robust Fibre Channel SANs can now be operated. The operation of up-to- date operating systems such as Windows NT/2000, AIX, Solaris, HP-UX and Linux is particularly unproblematic. Fibre Channel SANs are based upon Fibre Channel networks. The incompatibility of the fabric and arbitrated loop topologies and the networking of fabrics and arbitrated loops has already been discussed in Section 3.4.3. Within the fabric, the incompatibility of the Fibre Channel switches from different manufacturers should also be mentioned. At the end of 2003 we still recommend that when installing a Fibre Channel SAN only the switches

and directors of a single manufacturer are used. Even though routing between switches and directors of different manufacturers may work as expected, and basic functions of the fabric topology such as aliasing, name server and zoning work well across different vendors in so-called 'compatibility modes'. But bear in mind that there is still only a very small installed base of mixed switch vendor configurations. A standard has been passed that addresses the interoperability of these basic functions, meaning that it is now just a matter of time before these basic functions work across every manufacturers' products.However, for new functions such as SAN security, inter-switch-link trunking or B-Ports,teething troubles with interoperability must once again be expected. In general, applications can be subdivided into higher applications that model and support the business processes and system-based applications such as file systems, databases and back-up systems. The system-based applications are of particular interest from the point of view of storage networks and storage management. The compatibility of network file systems such as NFS and CIFS is now taken for granted and hardly ever queried. As storage networks penetrate into the field of file systems, cross-manufacturer standards are becoming ever more important in this area too. A first offering is Network Data Management Protocol (NDMP, Section 7.9.4) for the back-up of NAS servers. Further down the road we expect also a customer demand for cross-vendor standards in the emerging field of storage virtualization (Chapter 5). The subject of interoperability will preoccupy manufacturers and customers in the field of storage networks for a long time to come. Virtual Interface Architecture (VIA), Infini- Band and Remote Direct Memory Access (RDMA) represent emerging new technologies that must also work in a cross-manufacturer manner. The same applies for Internet SCSI (iSCSI) and its variants like iSCSI Extensions over RDMA (iSER). iSCSI transmits the

SCSI protocol via TCP/IP and, for example, Ethernet. Just like FCP, iSCSI has to serialize he SCSI protocol bit-by-bit and map it onto a complex network topology. Interoperability will therefore also play an important role in iSCSI.

Basics Hardware components for Fibre Channel SAN

Basics Hardware components for Fibre Channel SAN

Within the scope of this book we can only introduce the most important product groups. It is not worth trying to give an overview of specific products or a detailed description of individual products due to the short product cycles. This section mentions once again some product groups that have been discussed previously and introduces some product groups that have not yet been discussed. It is self-evident that servers and storage devices are connected to a Fibre Channel network. In the server this can be achieved by fiting the host bus adapter cards (HBAs) of different manufacturers, with each manufacturer offering different HBAs with differing performance features. In storage devices the same HBAs are normally used. However, the manufacturers of storage devices restrict the selection of HBAs. Of course, cables and connectors are required for cabling. In Section 3.3.2 we discussed different copper and fiber-optic cables and their properties. Various connector types are currently on offer for all cable types. It may sound banal, but in practice the installation of a Fibre Channel SAN is sometimes delayed because the connectors on the cable do not fit the connectors on the end devices, hubs and switches and a suitable adapter is not

to hand. A further, initially improbably, but important device is the so-called Fibre Channel-to- SCSI bridge. As the name suggests, a Fibre Channel-to-SCSI bridge creates a connection between Fibre Channel and SCSI (Figure 3.30). These bridges have two important fields of application. First, old storage devices often cannot be converted from SCSI to Fibre Channel. If the old devices are still functional they can continue to be used in the Fibre Channel SAN by the deployment of a Fibre Channel-to-SCSI bridge. Second, new tape libraries in particular often initially only support SCSI; the conversion to Fibre Channel is often not planned until later.With a Fibre Channel-to-SCSI bridge the newest tape libraries can be operated directly in a Fibre Channel SAN and Fibre Channel connections retrofitted as soon as they become available. Unfortunately, the manufacturers have not agreed upon consistent name for this type of device. In addition to Fibre Channel-to-SCSI bridge, terms such as SAN router or storage gateway are also common. The switch is the control centre of the fabric topology. It provides routing and aliasing, name server and zoning functions. Fibre Channel switches support both cut-through routing and the buffering of frames. In new switches a number of ports between eight and about 250 and a data transfer rate of 200 MByte/s should currently (2003) be viewed as standard. In Fibre Channel SANs that have already been installed, however, a large base of switches exists that still work at 100 MByte/s.

 

 

Resilient, enterprise-class switches are commonly referred to as 'directors', named after the switching technology used in mainframe ESCON cabling. Like Fibre Channel switches they provide routing, alias names, name server and zoning functions. Fibre Channel direc- tors are designed to avoid any single point of failure, having for instance two backplanes and two controllers. Current directors (2003) have between 64 and 256 ports. Designing a SAN often raises the question whether several complementary switches or a single director should be preferred. As described, directors are more fault-tolerant than switches, but they are more expensive per port. Therefore, designers of small entry-level SANs commonly choose two complementary Fibre Channel switches, with mutual traffic fail-over in case of a switch or a I/O path failure (Figure 3.31). Designers of larger Fibre Channel SANs often favour directors due to the number of ports currently available per device and the resulting layout simplicity. However, this argument in favour of directors becomes more and more obsolete since today switches with a greater number of ports are available as well. SANs running especially critical applications, e.g. stock market banking or flight control,

would use complementary directors with mutual traffic failover, even though these directors already avoid internal single points of failure. This is similar to wearing trousers with a belt and braces in addition: protecting against double or triple failures. In less critical cases, a single director or a dual complementary switch solution will be considered sufficient. If we disregard the number of ports and the cost, the decision for a switch or a director in an Open Systems Fibre Channel network primarily comes down to fault-tolerance of

 

an individual component. For the sake of simplicity we will use the term 'Fibre Channel switch' throughout this book in place of 'Fibre Channel switch or Fibre Channel director'. A hub simplifies the cabling of an arbitrated loop. Hubs are transparent from the point of view of the connected devices. This means that hubs send on the signals of the connected devices; in contrast to a Fibre Channel switch, however, the connected devices do not communicate with the hub. Hubs change the physical cabling from a ring to a star-shape. Hubs bridge across defective and switched-off devices, so that the physical

ring is maintained for the other devices. The arbitrated loop protocol is located above this cabling. Hubs are divided into unmanaged hubs, managed hubs and switched hubs. Unman- aged hubs are the cheap version of hubs: they can only bridge across switched-off devices. However, they can neither intervene in the event of protocol infringements by an end device nor indicate the state of the hub or the arbitrated loop to the out- side world. This means that an unmanaged hub cannot itself notify the administrator if one of its components is defective. A very cost-conscious administrator can build up a small SAN from PC systems, JBODs and unmanaged hubs. However, the upgrade path to a large Fibre Channel SAN is difficult: in larger Fibre Channel SANs it is questionable whether the economical purchase costs compensate for the higher administration costs. In contrast to unmanaged hubs, managed hubs have administration and diagnosis functions like those that are a matter of course in switches and directors.Managed hubs monitor the power supply, serviceability of fans, temperature, and the status of the individual ports. In addition, some managed hubs can, whilst remaining invisible to the connected devices, intervene in higher Fibre Channel protocol layers, for example, to deactivate the port of adevice that frequently sends invalid Fibre Channel frames. Managed hubs, like switches and directors, can inform the system administrator about events via serial interfaces, Telnet, HTTP and SNMP (see also Chapter 8). Finally, the switched hub is mid-way between a hub and a switch. In addition to the properties of a managed hub, with a switched hub several end devices can exchange data at full bandwidth. Fibre Channel switched hubs are cheaper than Fibre Channel switches, so in some cases they represent a cheap alternative to switches. However, it should benoted that only 126 devices can be connected together via hubs and that services such as aliasing and zoning are not available. Furthermore, the protocol cost for the connection or the removal of a device in a loop is somewhat higher than in a fabric (keyword 'Loop Initialisation Primitive Sequence', 'LIP'). Finally, so- alled link extenders should also be mentioned. Fibre Channel supports a maximum cable length of several ten kilometres (Section 3.3.2). A link extender can

increase the maximum cable length of Fibre Channel by transmitting Fibre Channel frames using MAN/WAN techniques such as ATM, SONET or TCP/IP (Figure 3.32).When using link extenders it should be borne in mind that long distances between end devices significantly increase the latency of a connection. Time-critical applications such as database transactions should therefore not run over a link extender. On the other hand,Fibre Channel SANs with link extenders offer new possibilities for applications such as back-up, data sharing and asynchronous data mirroring.

Fibre Channel SAN is a comparatively new technology. In many data centres in which Fibre Channel SANs are used, it is currently (2003) more likely that there will be several islands of small Fibre Channel SANs than one large Fibre Channel SAN (Figure 3.33).Over 80% of the installed Fibre Channel SANs consist only of up to four Fibre Channel switches. A server can only indirectly access data stored on a different SAN via the LAN and a second server. The reasons for the islands of small Fibre Channel SANs are that they are simpler to manage than one large Fibre Channel SAN and that it was often unnecessary to install a large one.

Originally, Fibre Channel SAN was used only as an alternative to SCSI cabling. Until now the possibility of flexibly dividing the capacity of a storage device etween several servers (storage pooling) and the improved availability of dual SANs have been the main reasons for the use of Fibre Channel SANs. Both can be realized very well with several small Fibre Channel SAN islands. However, more and more applications are now exploiting the possibilities offered by a Fibre Channel SAN. Applications such as back-up

 

(Chapter 7), remote data mirroring and data sharing over Fibre Channel SAN and storage virtualization (Chapter 5) require that all servers and storage devices are connected via a single SAN. Incidentally, the connection of Fibre Channel SANs to form a large SAN could be one field of application in which a Fibre Channel director is preferable to a Fibre Channel switch (Figure 3.34). As yet these connections are generally not critical. In the future, however, this could change (extreme situation: virtualization over several data centres). In our opinion these connection points between two storage networks tend to represent a single point of failure, so they should be designed to be particularly fault-tolerant.

Free Tutors on Fibre Chanel cables, plugs and signal encoding

Tutors on Fibre Chanel cables, plugs and signal encoding

Fibre Chanel defines the physical transmission medium (cable, plug) and specifies which physical signals are used to transmit the bits '0' and '1'. In contrast to the SCSI bus, in which each bit has its own data line plus additional control lines. Fibre Channel transmits the bits sequentially via a single line. In general, buses come up against the problem that the signals have a different transit time on the different data lines (skew), which means tha the speed can only be increased to a limited degree in buses. The different signal transit times can be visualized as the hand rail in an escalator that runs faster or slower than the escalator stairs themselves. Fibre Channel therefore transmits the bits serially. This means that, in contrast to the parallel bus, a high transfer rate is possible even over long distances. The high transfer rate of serial transmission more than compensates for the parallel lines of a bus. Transfer rates of 200 MByte/s are currently (2003) standard; we expect that in 2004 the firs products will support 400 MByte/s and 1 GByte/s. When considering the transfer rate i should be noted that in the fabric and point-to-point topologies the transfer is bi-directional and full-duplex, which means that today the transfer rate of 200 MByte/s is available in

each direction. Fibre Channel defines various cable types (Table 3.2) for copper and fibre-optic cable where the higher speeds only support fiber-optic. Various plug types are defined both for copper cable and for fiber-optic cable. Figure 3.10 shows various plug types for fiber-optic cable. Apart from their different dimensions, no technical advantages are associated with the various types. Copper cables are subdivided into 'intracabinet' cables and 'intercabinet' cables. Intra-cabinet cables are designed for cabling within an enclosure, they are less well shielded against electromagnetic interference – and thus cheaper – than inter cabinet cable, which can be used to connect up devices outside the limits of enclosures. Fiber-optic cables are more expensive than copper cables. They do, however, have some advantages:

• greater distances possible than with copper cable;

• insensitivity to electromagnetic interference;

• no electromagnetic radiation;

• no electrical connection between the devices;

• no danger of 'cross-talking'.

Different cable and plug types are also defined for fiber-optic cable. Cables for long distances are more expensive than those for short distances. The dentitions of various cables makes it possible to choose the most economical technology for each distance to be bridged. With 1 GByte Fibre Channel there will be some innovations in fiber-optic cables. First, a new cable type has been introduced – the 50 micron high bandwidth cable – with which greater distances can be spanned than with a conventional 50 micron cable. Second, it will be possible to multiplex the data stream over four connections. This may occur first

by distributing the data over four fiber-optic pairs (4 lines). In another variant, Coarse Wavelength Division Multiplexing (CWDM), these four physical lines are replaced by four signals in different frequency ranges, so that one physical pair of lines is sufficient. In practice, we will have to wait and see which of these different cable variants the industry will actually support with real products for 1 GByte Fibre Channel. For all media, the Fibre Channel standard demands that a single bit error may occur at most once in every 1012 transmitted bits. On average, this means that for a 100 Mbit/s connection under full load a bit error may occur only every 16.6 minutes. The error recognition and handling mechanisms of the higher protocol layers are optimized for the maintenance of this error rate. Therefore, when installing a Fibre Channel network it is recommended that the cable is properly laid so that the bit error rate of 1012 is, where

possible, also achieved for connections from end device to end device, i.e. including all components connected in between such as repeaters and switches.

The distance information in Table 3.2 specifies the minimum distances at which the error rate can reliably be kept within the stipulated figure, given the current state of technology and proper laying of the cable during the timeframe the standard was ratified. Technical improvements and proper laying of the cable make it possible for even greater distances to be bridged in actual installations. Today (2003), distances up to several 10 kilometres are supported for 200 MByte/s. The reduction in the supported cable lengths could represent a problem when upgrading the equipment of an existing Fibre Channel SAN to a higher speed, thus it should be checked in advance, whether a given distance can be bridged at the higher speed as well.