Free Tutors on Interoperability of Fibre Channel SAN and IP STORAGE

Free Tutors on Interoperability of Fibre Channel SAN and IP STORAGE

Fibre Channel SANs are currently being successfully used in production environments. Nevertheless, interoperability is an issue with Fibre Channel SAN, as in all new cross manufacturer technologies. When discussing the interoperability of Fibre Channel SAN we must differentiate between the interoperability of the underlying Fibre Channel network layer, the interoperability of the Fibre Channel application protocols, such as FCP (SCSI over Fibre Channel) and the interoperability of the applications running on the Fibre Channel SAN. The interoperability of Fibre Channel SAN stands and falls by the interoperability of

FCP. FCP is the protocol mapping of the FC-4 layer, which maps the SCSI protocol on a Fibre Channel network (Section 3.3.8).The FCP is a complex piece of software that can only be implemented in the form ofa device driver. The implementation of hardware-like device drivers alone is a task that attracts errors as if by magic. The developers of FCP device drivers must therefore test extensively and thoroughly. Two general conditions make it more difficult to test the FCP device driver. The server initiates the data transfer by means of the SCSI protocol; the storage device only responds to the requests of the server. However, the idea of storage networks is to consolidate storage devices, i.e. for many servers to share a few large storage devices. Therefore, with storage networks a single storage device must be able to serve several parallel requests from different servers simultaneously. For example, it is typical for a server to be exchanging data with a storage device just when another server is scanning the Fibre Channel SAN for available storage devices. This situation requires end devices to be able to multitask. When testing multitasking systems the race conditions of the tasks to be performed come to bear: just a few milliseconds delay can lead to a completely different test result.The second difficulty encountered during testing is due to the large number of components that come together in a Fibre Channel SAN. Even when a single server is connected to a single storage device via a single switch, there are numerous possibilities that can-

not all be tested. If, for example, a Windows server is selected, there is still the choice between NT, 2000 and 2003, each with different service packs. Several manufacturers offer several different models of the Fibre Channel host bus adapter card in the server. If we take into account the various firmware versions for the Fibre Channel host bus adapter cards we find that we already have more than 50 combinations before we even select a switch. Companies want to use their storage network to connect servers and storage devices from various manufacturers, some of which are already present. The manufacturers of Fibre Channel components (servers, switches and storage devices) must therefore perform interoperability tests in order to guarantee that these components work with devices from third-party manufacturers. Right at the top of the priority list are those combinations that are required by most customers, because this is where the expected profit is the highest. The result of the interoperability test is a so-called support matrix. It specifies, for example, which storage device supports which server model with which operating system versions and Fibre Channel cards. Manufacturers of servers and storage devices often limit the Fibre Channel switches that can be used. Therefore, before building a Fibre Channel SAN you should carefully check whether the manufacturers in question state that they support the planned configuration. If the desired configuration is not listed, you can negotiate with the manufacturer regarding the payment of a surcharge to secure manufacturer support. Although non-supported configurations can work very well, if problems occur, you are left without support in critical situations. If

in any doubt you should therefore look for alternatives right at the planning stage. All this seems absolutely terrifying at first glance. However, manufacturers now support a number of different configurations. If the manufacturers' support matrices are taken into consideration, robust Fibre Channel SANs can now be operated. The operation of up-to-date operating systems such as Windows NT/2000, AIX, Solaris, HP-UX and Linux is particularly unproblematic. Fibre Channel SANs are based upon Fibre Channel networks. The incompatibility of the fabric and arbitrated loop topologies and the networking of fabrics and arbitrated loops

has already been discussed in Section 3.4.3. Within the fabric, the incompatibility of the Fibre Channel switches from different manufacturers should also be mentioned. At the end of 2003 we still recommend that when installing a Fibre Channel SAN only the switches and directors of a single manufacturer are used. Even though routing between switches and directors of different manufacturers may work as expected, and basic functions of the fabric topology such as aliasing, name server and zoning work well across different vendors in so-called 'compatibility modes'. But bear in mind that there is still only a very small installed base of mixed switch vendor configurations. A standard has been passed that addresses the interoperability of these basic functions, meaning that it is now just a matter of time before these basic functions work across every manufacturers' products. However, for new functions such as SAN security, inter-switch-link trunking or B-Ports, teething troubles with interoperability must once again be expected.In general, applications can be subdivided into higher applications that model and support the business processes and system-based applications such as file systems, databases and back-up systems. The system-based applications are of particular interest from the point of view of storage networks and storage management. The compatibility of network file systems such as NFS and CIFS is now taken for granted and hardly ever queried. As storage networks penetrate into the field of file systems, cross-manufacturer standards are becoming ever more important in this area too. A first offering is Network Data Management Protocol (NDMP, Section 7.9.4) for the back-up of NAS servers. Further down the road we expect also a customer demand for cross-vendor standards in the emerging field of storage virtualization (Chapter 5).

The subject of interoperability will preoccupy manufacturers and customers in the field of storage networks for a long time to come. Virtual Interface Architecture (VIA), Infini Band and Remote Direct Memory Access (RDMA) represent emerging new technologies that must also work in a cross-manufacturer manner. The same applies for Internet SCSI (iSCSI) and its variants like iSCSI Extensions over RDMA (iSER). iSCSI transmits the SCSI protocol via TCP/IP and, for example, Ethernet. Just like FCP, iSCSI has to serialize the SCSI protocol bit-by-bit and map it onto a complex network topology. Interoperability will therefore also play an important role in iSCSI.

IP STORAGE

Fibre Channel SANs are currently (2003) being successfully implemented in production environments. Nevertheless, the industry is at pains to establish storage networks based upon IP (IP storage) and Ethernet as an alternative to Fibre Channel. This section first introduces various protocols for the transmission of storage data traffic via TCP/IP (Section 3.5.1). Then we explain to what extent TCP/IP and Ethernet are suitable transmission techniques for storage networks at all (Section 3.5.2). Finally, we discuss a migration path from SCSI and Fibre Channel to IP storage (Section 3.5.3).

 

 

FREE TUTORS ON FIBRE CHANNEL SAN Point-to-point topology

FREE TUTORS ON FIBRE CHANNEL SAN Point-to-point topology

The previous section introduced the fundamentals of the Fibre Channel protocol stack This section expands our view of Fibre Channel with the aim of realizing storage network with Fibre Channel. To this end, we will first consider the three Fibre Channel topologies point-to-point, fabric and arbitrated loop more closely (Sections 3.4.1. to 3.4.3). We will then introduce some hardware components that are required for the realization of a Fibr Channel SAN (Section 3.4.4). Building upon this, the networking of small storage network islands to form a large SAN will be discussed (Section 3.4.5). Finally, the question interoperability in Fibre Channel SANs will

Point-to-point topology

The point-to-point topology connects just two devices and is not expandable to three or more devices. For storage networks this means that the point-to-point topology connects a server to a storage device. The point-to-point topology may not be very exciting, but it offers two important advantages compared to SCSI cabling. First, significantly greater cable lengths are possible with Fibre Channel than with SCSI because Fibre Channel supports distances up to ten kilometres without repeaters, whilst SCSI supports only up to 25 metres. Second, Fibre Channel defines various fiber-optic cables in addition tocopper cables. Optical transmission via fiber-optic is robust in relation to electromagnetic interference and does not emit electromagnetic signals. This is particularly beneficial in technical environments. Fibre Channel cables are simpler to lay than SCSI cables. For example, the SCSISAN shown in Figure 3.7 can very simply be realized using the point-to-point topology.Application servers for the control of production can be set up close to the production machines and the data of the application server can be stored on the shared storage systems, which are located in a room that is protected against unauthorized access and physical influences such as fire, water and extremes of temperature. 3.4.2 Fabric topology The fabric topology is the most flexible and scalable of the three Fibre Channel topologies.

A fabric consists of one or more Fibre Channel switches connected together. Servers and storage devices are connected to the fabric by the Fibre Channel switches. In theory a fabric can connect together up to 15.5 million end devices. However, Fibre Channel SANs connected to several hundreds of end devices are currently (2003) still the exception. Most installations use two to four switches. There are, however, a few 'power users',who operate significantly larger Fibre Channel SANs. End Devices (servers and storage devices) connected to the various Fibre Channel switches can exchange data by means of switch-to-switch connections (inter-switch links,ISLs). Several inter-switch links can be installed between two switches in order to increase the bandwidth. A transmitting end device only needs to know the Node ID of the target device; the necessary routing of the Fibre Channel frame is taken care of by the Fibre Channel switches. Fibre Channel switches generally support so-called cut-through routing: cut-through routing means that a Fibre Channel switch forwards an incoming frame before it has been fully received. The latency describes the period of time that a component requires to transmit a signal or the period of time that a component requires to forward a frame. Figure 3.23 compares the latency of different Fibre Channel SAN components. Light requires approximately 25microseconds to cover a distance of ten kilometres. A ten kilometre-long Fibre Channel cable thus significantly increases the latency of an end-to-end connection. For hardware

components the rule of thumb is that a Fibre Channel switch can forward a frame in two to four microseconds; a Fibre Channel host bus adapter requires two to four milliseconds to process it. Additional Fibre Channel switches between two end devices therefore only increase the latency of the network to an insignificant degree.One special feature of the fabric is that several devices can send and receive data simultaneously at the full data rate. All devices thus have the full bandwidth available to them at the same time. Figure 3.24 shows a Fibre Channel SAN with three servers and three storage devices, in which each server works to its own storage device. Each of the three logical connections over the Fibre Channel SAN has the full bandwidth of 200MByte/s available to them.A prerequisite for the availability of the full bandwidth is good design of the Fibre Channel network. Figure 3.25 shows a similar structure to that in Figure 3.24, the only difference is that the single switch has been replaced by two switches, which are connected via one inter-switch link (ISL). It is precisely this inter-switch link that represents the limiting factor because all three logical connections now pass through the same inter-switch link. This means that all three connections have, on average, only a third of the maximum bandwidth available to them. Therefore, despite cut-through routing, switches have a certain number of buffers (frame buffers) available to them, with which they can temporarily bridge such bottlenecks. However, the switch must still reject valid frames if the flow control does not engage quickly enough. In addition to routing, switches realize the basic services of aliasing, name server and

zoning. As described in Section 3.3.6, end devices are differentiated using 64-bit WorldWide Node Names (WWNNs) or by 64-bit World Wide Port Names (WWPNs) and addressed via 24-bit port addresses (N-Port ID). To make his job easier the administratorcan issue alias names to WWNs and ports.

throughput of the three connections is limited by the ISL The name server supplies information about all end devices connected to the Fibre Channel SAN (Section 3.3.7). If an end device is connected to a switch, it reports to this and registers itself with the name server. At the same time it can ask the name server which other devices are still connected to the SAN. The name server administers end devices that are currently active; switched off end devices are not listed in the name server. Finally, zoning makes it possible to define subnetworks within the Fibre Channel net-work. This has two main advantages. First, zoning limits the visibility of end devices. With zoning, servers can only see and access storage devices that lie in the same zone. Zoning therefore helps to protect sensitive data. Furthermore, incompatible Fibre Channel host bus adapters can be separated from each other by different zones. Second, individual ports of a multiport disk subsystem – and thus a certain bandwidth – can be reserved for important applications. The bandwidth of inter-switch links (ISLs) cannot be reserved in

this manner since switches currently (2003) do not support this, or at least not officially. Although the Fibre Channel standard defines service classes that reserve a certain band- width (Classes 1, 4 and 6), these service classes are not implemented in most current Fibre Channel devices. There are many variants of zoning, for which unfortunately no consistent terminology exists. Different manufacturers use the same term for different types of zoning and differ-

ent terms for the same type of zoning. Therefore, when selecting Fibre Channel switches do not let yourself get fobbed off with statements such as 'the device supports hard zon- ing'. Rather, it is necessary to ask very precisely what is meant by 'hard zoning'. In the following we introduce various types of zoning.

In zoning, the administrator brings together devices that should see each other in Fibre Channel SAN into a zone, whereby zones can overlap. Zones are described by World Wide Node Names, World Wide Port Names, port addresses or by their alias names. The description on the basis of WWNNs and WWPNs has the advantage that zoning is robust

in relation to changes in cabling: it does not need to be changed for a device to be plugged into a different switch port. By contrast, zoning on the basis of port addresses must be altered since every port in the switch has a different port address.Soft zoning restricts itself to the information of the name server. If an end device asks the name server about other end devices in the Fibre Channel network, it is only informed of the end devices with which it shares at least one common zone. If, however, an end device knows the address (Port ID) of another device, it can still communicate with it. Soft zoning thus does not protect access to sensitive data. Soft zoning is problematic in relation to operating systems that store the WWNs of Fibre Channel devices that have been found in an internal database or in which WWNs are announced in configuration files because this means that WWNs remain known to the operating system even after

a system reboot. Thus in soft zoning operating systems continue to have access to all known devices despite changes to the zoning, regardless of whether they lie in a common zone or not. Hard zoning offers better protection. In hard zoning only devices that share at least one common zone can actually communicate with one another. Both hard zoning and soft zoning can be based upon port addresses or WWNs. Nevertheless, port-based zoning is sometimes known as hard zoning. Some more modern Fibre Channel switches support LUN masking – described in Section 2.7.3 in relation to disk subsystems – within the switch. To achieve this they read the first bytes of the payload of each Fibre Channel frame. Although reading part of the Fibre Channel payload increases the latency of a Fibre Channel switch, this increase in latency is so minimal that it is insignificant in comparison to the latency of the host bus adapter in the end devices.

So-called virtual storage area networks (virtual SAN, VSAN) represent a further inno- vation. In this technique, several ports or WWNs and thus several end devices of a Fibre Channel fabric, are grouped together to form a virtual fabric. This means that several virtual Fibre Channel fabrics that are logically separate from one another can be operated over one physical Fibre Channel network. In addition, separate fabric services such asname server and zoning are realized for each virtual storage network. In addition to pure zoning, virtual storage networks thus not only limit the mutual visibility of end devices but also the mutual visibility of the fabric configuration. This is particularly advantageous in installations which aim to offer storage services for various customers over a consol-

idated infrastructure. Here, in particular, it is not desirable for a customer to be able to read which end devices belonging to other customers are still connected in the storage network, or even change their configuration, over the name server.