High Salary Software Jobs

High Salary Software jobs Interview Process Maps Interview preparation Tips Resume preparation Tips Salary Negotiation Videos Tutorials

How to Make Money Online Learn Secrets for FREE from Experts

Citrix Interview Questions

Windows Sysadmin Interview Question

Know more about extension of the snia shared storage model to tape functions logical and physical structure of tapes and Extension of the model

Know more on Asymmetric ﬁle services: NAS/ﬁle server metadata manager Object-based storage device (OSD)

Free tutors on Storage network attached block storage Block storage aggregation in a storage device: Network attached block storage with metadata and File server controller: NAS heads

Free tutors on Clustering Storage, data and information The service subsystem examples of disk-based storage architectures

Learn more on Combination of the block and ﬁle/record layers Access paths Caching Access paths

KNOW MORE ON THE SNIA SHARED STORAGE MODEL ,THE LAYERS AND THE MODEL 317

Free Tutors the snia shared storage model Graphical representations An elementary overview and The components

Free tutors on the SNIA Shared Storage Model and The functional approach

Know more about THE IEEE 1244 STANDARD FOR REMOVABLE MEDIA MANAGEMENT Operational characteristics of the media manager

FREE TUTORS ABOUT THE IEEE 1244 STANDARD FOR REMOVABLE MEDIA MANAGEMENT SYSTEM ARCHITECTURE

Learn more on Life cycle management

Tutors on Monitoring management of removable media

Free tutors on Media tracking Grouping, pooling Drive pools

Tutors on Authorization for access to individual objects , Access synchronization and Access prioritization and mount request queuing

How to use Efficient use of the available resources and Access Control

Know more on problems and requirements related to removable media management

Free Tutors on Control of the media changer SCSI and Fibre Channel interface

Free Tutors on Management features of removable media Tape libraries and drives

Learn more about Removable Media Management THE SIGNIFICANCE OF REMOVABLE MEDIA LIKE TAPE CD, DVD and magneto-optical media

TUTORS ON OPERATIONAL ASPECTS OF THE MANAGEMENTOF STORAGE NETWORKS AND Basic of CMIP and DMI

Learn more About the WBEM architecture Storage Management Initiative Speciﬁcation (SMI-S)

Tutor on Use of SNMP SNMP architecture for Free and CIM and WBEM

Free tutors on Out Band Management

Know more About The zoning problem

Free Tutors On In-band management in Fibre Channel SAN Discovery, Massaging and Monitoring

Free Tutors On Band Management

Learn more on Standardized mechanisms and Proprietary mechanisms

Know more about Management Interfaces Storage networks( STANDARDIZED AND PROPRIETARY MECHANISMS)

FREE TUTOIRS SUPPORT BY MANAGEMENT SYSTEMS

LEARN MORE ON BASEIC REQUIREMENTS OF MANAGEMENT SYSTEMS

Tutors on How Do Management of Storage Networks

KNOW MORE ABOUT ORGANIZATIONAL ASPECTS OF BACK-UP

Free Gide On Next generation back-up of databases

Free Tutors On How Classical back-up of databases works

Tutor on how take Backup of databases And Operating method of database systems

Know more on The Network Data Management Protocol (NDMP)

Know more The Network Data Management Protocol (NDMP)

Free tutors How to take Back-up of NAS servers

Know basics of Back-up of ﬁle systems

KNOW MORE ABOUT BACK-UP OF FILE SYSTEMS AND BACKUP OF FILE SERVER

Learn more on Data protection using remote mirroring

Free Tutors on Tape library sharing and Back-up using instant copies

Know more on LAN-free back-up

Know more on LAN-free back-up ,LAN-free back-up with shared disk ﬁle systems

Free tutors about Next Generation Backups, Server-free back-up

Free tutors on Back-up server and application server on the same physical computer

Know more how to get the opportunities for increasing performance on Several back-up servers

Learn more on How to take performance bottlenecks of networks backup

Learn more on How to BACK-UP clients , (PERFORMANCE GAINS AS A RESULT OF NETWORK BACK-UP Application-speciﬁc performance bottlenecks)

Learn more on Network backup ,Server components(Job scheduler, Error handling, Metadata database, Media manager,)

Free tutors on Network Back-up administration ,GENERAL CONDITIONS FOR BACK-UP, NETWORK BACK-UP SERVICES

Free Tutors on APPLICATION OF STORAGE NETWORKS in Web applications based upon the ‘travel portal’ case study

Know more about ADAPTABILITY AND SCALABILITY OF IT SYSTEMS and Clustering for load distribution, Web architecture

Free Storage tutors Failure of virtualization in the storage network and case study on the Failure of a data centre

Citrix Interview Questions

2008-12-07T20:25:00.000-08:00

2008-09-02T22:11:00.000-07:00

Many of you search for a way to make money online. Here is a Simple,EASY & FREE way to learn How to make Money Online.

You can make money online if you just have a service or a product which can be sold or you can have money because of some simple things like writing articles, creating content etc. With all those things you might make just few hundred dollars a month. But if you go through this link
"Search Engine Optimization" you can make a lot more money. Since using Search Engine Optimization you can get hundreds of visitors who are very much looking for the service or product you are selling.

This is FREE hence I am writing about it go here "FREE Secrets to Make Money Online"

This is not some cheap ebook they are going to send you a Video DVD along with lot more for almost FREE & this DVD has several Videos which explain how to make money online.

Go here Order for FREE watch this Video you will know this thing which they are giving away for FREE is worth a thousand dollar.

This product is from the industry leading team called Stompernet . Lots of people pay them to get the same secrets.

------
Subject: "Stomping the Search Engines 2" and "The Net Effect"
for HOW MUCH?

Hey

Andy Jenkins has finally given me the all-clear to spill the
beans on this insane offer that StomperNet has cooked up.

Tomorrow, Sept. 3rd at 3pm Eastern, you can get StomperNet's
big daddy expert SEO Video Course, "Stomping the Search Engines
2"... for FREE.

That's right. FREE.

All you need to do is just TRY their new monthly printed Action
Journal called "The Net Effect" - and guess what?...

You get the PREMIER ISSUE of "The Net Effect" for FREE TOO!

You don't pay one penny more than Shipping and Handling unless
you LOVE it and want to get issue 2 a month from now.

That's NUTS. They are betting the FARM that you will LOVE this
stuff and stick around for more. That takes GUTS, and and HUGE
confidence in the quality of their stuff.

But then again, it's StomperNet. I've SEEN the stuff, and can
vouch. It would be worth FULL PRICE.

But for FREE? You'd be FOOLISH not to check this out.

Don't believe it? Watch this video they've released to the
public. No fooling - this is a FOR-REAL DEAL.

https://member.stompernet.net/?r=1324&i=68

This MIGHT just change your online business fortunes...
forever.

P.S. There's no hint of scarcity here - they've got tons of
BOTH products ready to ship. But still - be there EARLY. If I
hadn't already gotten my "insider" review copy, I'd be the
FIRST one on this page tomorrow.

2008-04-17T09:54:00.001-07:00

1. What is the requirement for Citrix server installation.

2. What is Data store

3. What is Data collector

4. What is LHC

5. What is Client Lock Down

6. What is Printer terminology in Citrix

7. How to use datastore for database

8. What is the difference between all citrix versions

9. What are different load evaluators are available in Citrix

10. How to implement Policies in Citrix

11. What you will check when any user is not able to launch citrix application.

12. What is IMA

2008-04-17T09:43:00.000-07:00

1. What is Active Directory schema?

2. What are the domain functional level in Windows Server 2003?

3. What are the forest functional level in Windows Server 2003?

4. What is global catalogue server?

5. How we can raise domain functional & forest functional level in Windows Server 2003?

6. Which is the default protocol used in directory services?

7. What is IPv6?

8. What is the default domain functional level in Windows Server 2003?

9. What are the physical & logical components of ADS

10. In which domain functional level, we can rename domain name?

11. What is multimaster replication?

The Active Directory schema contains formal definitions of every object class that can be created in an Active Directory forest it also contains formal definitions of every attribute that can exist in an Active Directory object. Active Directory stores and retrieves information from a wide variety of applications and services. So that it can store and replicate data from a potentially infinite variety of sources, Active Directory standardizes how data is stored in the directory. By standardizing how data is stored, the directory service can retrieve, update, and replicate data while ensuring that the integrity of the data is maintained.

Schema master is a set of rules which is used to define the structure of active directory. It contains definitions of all the objects which are stored in AD. It maintains information and detail information of objects.

.What is global catalog server?

A global catalogue server is a domain controller it is a master searchable database that contains information about every object in every
domain in a forest. The global catalogue contains a complete replica of all
objects in Active Directory for its host domain, and contains a partial replica
of all objects in Active Directory for every other domain in the forest.
It have two important functions:
i)Provides group membership information during logon and authentication
ii)Helps users locate resources in Active Directory

1. The two technologies in DFS are as follows:

DFS Replication. New state-based, multimaster replication engine that is optimized for WAN environments. DFS Replication supports replication scheduling, bandwidth throttling, and a new byte-level compression algorithm known as remote differential compression (RDC).

DFS Namespaces. Technology that helps administrators group shared folders located on different servers and present them to users as a virtual tree of folders known as a namespace. DFS Namespaces was formerly known as Distributed File System in Windows 2000 Server and Windows Server 2003.

1. 1. DNS(Domain Name Service):
—————————-
It’s mainly used to resolve from host name(FQDN-Fully Qualified Domain Name) to IP address and IP address to host name.DNS mainly used in Internet. DNS divide in form of hierarchical.

2. DHCP(Dynamic Host Configuration Protocol):
———————————————
DHCP use for provide IP address dynamically to client machine. If that client not able to find DHCP server then client machine will go for APIPA(We have range for APIPA which is 169.254.0.1-169.254.255.254).

3. HUB and SWITCH:
——————
Switch is expensive than hub. If more then one user try to send packet at a time collision will occure but in switch we can send. Switch is full duplex. Maximum bandwidth is 100 Mhz and that bandwidth is shared by all of the PC’s connected to the hub. Data can be sent in both directions simultaneously, the maximum available bandwidth is 200 Mbps, 100 Mbps each way, and there are no other PC’s with which the bandwidth must be shared.

2008-04-04T01:18:00.000-07:00

Know more about extension of the snia shared storage model to tape functions logical and physical structure of tapes and Extension of the model

The SNIA Shared Storage Model described previously concentrates upon the modelling of disk-based storage architectures. In a supplement to the original model, the SNIA Technical Council deﬁnes the necessary extensions for the description of tape functions and back-up architectures.

The SNIA restricts itself to the description of tape functions in the Open Systems environment, since the use of tapes in the mainframe environment is very difﬁcult to model and differs fundamentally from the Open Systems environment. In the Open Systems ﬁeld, tapes are used almost exclusively for back-up purposes, whereas in the ﬁeld of mainframes tapes are used much more diversely. Therefore, the extension of the SNIA model concerns

itself solely with the use of tape in back-up architectures. Only the general use of tapes in shared storage environments is described in the model.

The SNIA does not go into more depth regarding the back-up applications themselves. We have already discussed network back-up in Chapter 7. More detailed information on tapes can be found in Section 9.2.1. First of all, we want to look at the logical and physical structure of tapes from the

point of view of the SNIA Shared Storage Model (10.3.1). Then we will consider the differences between disk and tape storage (10.3.2) and how the model is extended for the description of the tape functions (10.3.3).

Logical and physical structure of tapes

Information is stored on tapes in so-called tape images, which are made up of the following

logical components (Figure 10.20):

• Tape extent A tape extent is a sequence of blocks upon the tape. A tape extent is comparable with a volume in disk storage. The IEEE Standard 1244 (Section 9.5) also uses the term volume but it only allows volumes to reside exactly on one tape and not span multiple tapes.

• Tape extent separator The tape extent separator is a mark for the division of individual tape extents.

• Tape header The tape header is an optional component that marks the start of a tape.

• Tape trailer The tape trailer is similar to the tape header and marks the end of a tape. This, too, is an optional component. In the same way as logical volumes of a volume manager extend over several physical disks, tape images can also be distributed over several physical tapes. Thus, there may be precisely one logical tape image on a physical tape, several logical tape images on a physical tape, or a logical tape image can be distributed over several physical tapes. So- called tape image separators are used for the subdivision of the tape images (Figure 10.21). 10.3.2 Differences between disk and tape At ﬁrst glance, disks and tapes are both made up of blocks, which are put together to form long sequences. In the case of disks these are called volumes, whilst in tapes they are called extents. The difference lies in the way in which they are accessed, with disks being designed for random access, whereas tapes can only be accessed sequentially. Consequently, disks and tapes are also used for different purposes. In the Open Sys- tems environment, tapes are used primarily for back-up or archiving purposes. This is completely in contrast to their use in the mainframe environment, where ﬁle structures – so-called tape ﬁles – are found that are comparable to a ﬁle on a disk. There is no deﬁnition of a tape ﬁle in the Open systems environment, since several ﬁles are generally bundled to form a package, and processed in this form, during back-up and archiving.

This concept is, therefore, not required here.

Extension of the model

The SNIA Shared Storage Model must take into account the differences in structure and application between disk and tape and also the different purposes for which they are used. To this end, the ﬁle/record layer is expanded horizontally. The block layer, which produces the random access to the storage devices in the disk model, is exchanged for a sequential access block layer for the sequential access to tapes. The model is further supplemented by the following components (Figure 10.22):

• Tape media and tape devices Tape media are the storage media upon which tape images are stored. A tape devices is a special physical storage resource, which can process removable tape media. This differentiation between media and devices is particularly important in the context

of removable media management (Chapter 9). The applicable standard, IEEE 1244, denotes tape media as cartridge and tape device as drive.

• Tape applications The SNIA model concentrates upon the use of tapes for back-up and archiving. Special tape applications, for example, back-up software, are used for back-up. This software can deal with the special properties of tapes.

• Tape format system In the tape format system, ﬁles or records are compressed into tape extents and tape images. Speciﬁcally in the Open Systems environment, the host generally takes over this task. However, access to physical tape devices does not always have to go through the tape format system. It can also run directly via the extent aggregation layer described below or directly on the device.

• Extent aggregation layer The extent aggregation layer works in the same way as the block aggregation layer (Section 10.1.7), but with extents instead of blocks. However, in contrast to the random access of the block aggregation layer, access to the physical devices takes place

sequentially. Like the access paths, the data ﬂows between the individual components are shown as arrows.

2008-04-04T00:10:00.001-07:00

Know more on Asymmetric ﬁle services: NAS/ﬁle server metadata manager Object-based storage device (OSD)

A ﬁle server metadata manager (Figure 10.18) works in the same way as asymmetric storage virtualization on ﬁle level (Section 5.7.2):

• Hosts and storage devices are connected via a storage network.

• A metadata manager positioned outside the data path stores all ﬁle position data, i.e. metadata, and makes this available to the hosts upon request.

• Hosts and metadata manager communicate over an expanded ﬁle-oriented protocol.

• The actual user data then ﬂows directly between hosts and storage devices by means of a block-oriented protocol. This approach offers the advantages of fast, direct communication between host and storage devices, whilst at the same time offering the advantages of data sharing on

ﬁle level. In addition, in this solution the classic ﬁle sharing services can be offere

in a LAN over the metadata manager.

Object-based storage device (OSD)

The SNIA Shared Storage Model deﬁnes the so-called object-based storage device (OSD) The idea behind this architecture is to move the position data of the ﬁles and the access rights to a separate OSD. OSD offers the same advantages as a ﬁle sharing solution,combined with increased performance due to direct access to the storage by the hosts, and central metadata management of the ﬁles. The OSD approach functions as follows

(Figure 10.19):

• An OSD device exports a large number of byte vectors instead of the LUNs used in block-oriented storage devices. Generally, a byte vector corresponds to a single ﬁle.

• A separate OSD metadata manager authenticates the hosts and manages and checks the access rights to the byte vectors. It also provides appropriate interfaces for the hosts.

• After authentication and clearance for access by the OSD metadata manager, the hosts access the OSD device directly via a ﬁle-oriented protocol. This generally takes place via a LAN, i.e. a network that is not specialized for storage trafﬁc.

2008-04-04T00:01:00.001-07:00

Free tutors on Storage network attached block storage Block storage aggregation in a storage device: Network attached block storage with metadata and File server controller: NAS heads

Multi-site block storage file servers File server controller: NAS heads

The connection from storage to host via a storage network can be represented in the Shared Storage Model as shown in Figure 10.12. In this case:

• Several hosts share several storage devices.

• Block-oriented protocols are generally used.

• Block aggregation can be used in the host, in the network and in the storage device.

Block storage aggregation in a storage device:

SAN appliance Block aggregation can also be implemented in a specialized device or server of the storage network in the data path between hosts and storage devices, as in the symmetric storage virtualization (Figure 10.13, Section 5.7.1). In this approach:

• Several hosts and storage devices are connected via a storage network.

• A device or a dedicated server – a so-called SAN appliance – is placed in the data path between hosts and storage devices to perform block aggregation, and data and metadata trafﬁc ﬂows through this.

Network attached block storage with metadata

server: asymmetric block services The asymmetric block services architecture is identical to the asymmetric storage virtualization approach (Figure 10.14, Section 5.7.2):

• Several hosts and storage devices are connected over a storage network.

• Host and storage devices communicate with each other over a protocol on block level.

• The data ﬂows directly between hosts and storage devices.

• A metadata server outside the data path holds the information regarding the position of the data on the storage devices and maps between logical and physical blocks.

Multi-site block storage

Figure 10.15 shows how data replication between two locations can be implemented means of WAN techniques. The data can be replicated on different layers of the mo using different protocols:

• between volume managers on the host;

• between specialized devices in the storage network; or

• between storage systems, for example disk subsystems.

If the two locations use different network types or protocols, additional converters can be installed for translation.

File server

A ﬁle server (Section 4.2) can be represented as shown in Figure 10.16. The following points are characteristic of a ﬁle server:

• the combination of server and normally local, dedicated storage;

• ﬁle sharing protocols for the host access;

• normally the use of a network, for example, a LAN, that is not specialized to the storage trafﬁc;

• optionally, a private storage network can also be used for the control of the dedicated storage.

File server controller: NAS heads

In contrast to ﬁle servers, NAS heads (Figure 10.17, Section 4.2.2) have the following properties:

• They separate storage devices from the controller on the ﬁle/record layer, via which the hosts access.

• Hosts and NAS heads communicate over a ﬁle-oriented protocol.

• The hosts use a network for this that is generally not designed for pure storage trafﬁc, for example a LAN.

• When communicating downwards to the storage devices, the NAS head uses a block-oriented protocol.

NAS heads have the advantage over ﬁle servers that they can share the storage systems with other hosts that access them directly. This makes it possible for both ﬁle and block services to be offered by the same physical resources at the same time. In this manner, IT architectures can be designed more ﬂexibly, which in turn has a positive effect upon scalability.

2008-04-03T23:35:00.001-07:00

Free tutors on Clustering Storage, data and information The service subsystem examples of disk-based storage architectures

A cluster is deﬁned in the SNIA Shared Storage Model as a combination of resources with the objective of increasing scalability, availability and management within the shared storage environment (Section 6.4.1). The individual nodes of the cluster can share their resources via distributed volume managers (multi-node LVM) and cluster ﬁle systems (Figure 10.8, Section 4.3).

Storage, data and information

The SNIA Shared Storage Model differentiates strictly between storage, data and information. Storage is space – so-called containers – provided by storage units, on which the data is stored. The bytes stored in containers on the storage units are called data. Information is the meaning – the semantics – of the data. The SNIA Shared Storage Model names the following examples in which data–container relationships arise (Table 10.1).

10.1.14 Resource and data sharing In a shared storage environment, in which the storage devices are connected to the host via a storage network, every host can access every storage device and the data stored upon it (Section 1.2). This sharing is called resource sharing or data sharing in the SNIA model, depending upon the level at which the sharing takes place (Figure 10.9). If exclusively the storage systems – and not their data content – are shared, then we talk of resource sharing. This is found in the physical resources, such as disk subsystems and tape libraries, but also within the network. Data sharing denotes the sharing of data between different hosts. Data sharing is significantly more difﬁcult to implement, since the shared data must always be kept consistent, particularly when distributed caching is used. Heterogeneous environments also require additional conversion steps in order to convert the data into a format that the host can understand. Protocols such as NFS or CIFS are used in the more frequently used data sharing within the ﬁle/record layers

. For data sharing in the block layer, server clusters with shared disk ﬁle systems or parallel databases are used

The service subsystem

Up to now we have concerned ourselves with the concepts within the layers of the SNIA Shared Storage Model. Let us now consider the service subsystem (Figure 10.10). Within the service subsystem we ﬁnd the management tasks which occur in a shared storage environment and which we have, for the most part, already discussed in Chapter 8. In this connection, the SNIA Technical Council mention:

• discovery and monitoring

• resource management

• conﬁguration

• security

• billing (charge-back)

• redundancy management, for example, by network back-up

• high availability

• capacity planning.

The individual subjects are not yet dealt with in more detail in the SNIA Shared Storage Model, since the required deﬁnitions, speciﬁcations and interfaces are still being developed (Section 8.7.3). At this point we expressly refer once again to the check list in the

Appendix B, which reﬂects a cross-section of the questions that crop up here.

EXAMPLES OF DISK-BASED STORAGE ARCHITECTURES

In this section we will present a few examples of typical storage architectures and their properties, advantages and disadvantages, as they are represented by the SNIA in the Shared Storage Model. First of all, we will discuss block-based architectures, such as the direct connection of storage to the host (Section 10.2.1), connection via a storage network (Section 10.2.2), symmetric and asymmetric storage virtualization in the network

(Section 10.2.3 and Section 10.2.4) and a multi-site architecture such as is used for data replication between several locations (Section 10.2.5). We then move on to the ﬁle/record layer and consider the graphical representation of a ﬁle server (Section 10.2.6), a NAS head (Section 10.2.7), the use of metadata controllers for asymmetric ﬁle level virtualization (Section 10.2.8) and an object-based storage device (OSD), in which the position

data of the ﬁles and their access rights is moved to a separate device, a solution that combines ﬁle sharing with increased performance due to direct ﬁle access and central metadata management of the ﬁles (Section 10.2.9). 10.2.1 Direct attached block storage Figure 10.11 shows the direct connection from storage to the host in a server-centric architecture. The following properties are characteristic of this structure:

• No connection devices, such as switches or hubs, are needed.

• The host generally communicates with the storage device via a protocol on block level.

• Block aggregation functions are possible both in the disk subsystem and on the host.

2008-04-03T23:25:00.001-07:00

Learn more on Combination of the block and ﬁle/record layers Access paths Caching Access paths

Figure 10.5 shows how block and ﬁle/record layer can be combined and represented in the SNIA shared storage model:

• Direct attachment The left-hand column in the ﬁgure shows storage connected directly to the server, as is normally the case in a server-centric IT architecture (Section 1.1).

• Storage network attachment

In the second column we see how a disk array is normally connected via a storage network in a storage-centric IT architecture, so that it can be accessed by several host computers (Section 1.2).

• NAS head (NAS gateway) The third column illustrates how a NAS head is integrated into a storage network between SAN storage and a host computer connected via LAN.

• NAS server The right-hand column shows the function of a NAS server with its own dedicated storage in the SNIA Shared Storage Model.

Access paths

Read and write operations of a component on a storage device are called access paths in the SNIA Shared Storage Model. An access path is descriptively deﬁned as the list of components that are run through by read and write operations to the storage devices and responses to them. If we exclude cyclical access paths, then a total of eight possible access paths from applications to the storage devices can be identiﬁed in the SNIA Shared

Storage Model (Figure 10.6):

1. Direct access to a storage device.

2. Direct access to a storage device via a block aggregation function.

3. Indirect access via a database system.

4. Indirect access via a database system based upon a block aggregation function.

5. Indirect access via a database system based upon a ﬁle system.

6. Indirect access via a database system based upon a ﬁle system, which is itself based

upon a block aggregation function.

7. Indirect access via a ﬁle system.

8. Indirect access via a ﬁle system based upon a block aggregation function.

Caching

Caching is the method of shortening the access path of an application – i.e. the number of the components to be passed through – to frequently used data on a storage device. To this end, the data accesses to the slower storage devices are buffered in a faster cache storage. Most components of a shared storage environment can have a cache. The cache can be implemented within the ﬁle/record layer, within the block layer or in both.

In practice, several caches working simultaneously on different levels and components are generally used. For example, a read cache in the ﬁle system may be combined with a write cache on a disk array and a read cache with pre-fetching on a hard disk (Figure 10.7). In addition, a so-called cache-server (Section 5.7.2), which temporarily stores data for other components on a dedicated basis in order to reduce the need for network capacity or to accelerate access to slower storage, can also be integrated into the storage network. However, the interaction between several cache storages on several components means that consideration must be given to the consistency of data. The more components that use cache storage, the more dependencies arise between the functions of individual components. A classic example is the use of a snapshot function on a component in the block layer, whilst another component stores the data in question to cache in the ﬁle/record layer. In this case, the content of the cache within the ﬁle/record layer, which we will assume to be consistent, and the content of a volume on a disk array that is a component of the block layer can be different. The content of the volume on the array is thus inconsistent. Now, if a snapshot is taken of the volume within the disk array, a virtual

copy is obtained of an inconsistent state of the data. The copy is thus unusable. Therefore, before the snapshot is made within the block layer, the cache in the ﬁle/record layer on the physical volume must be destaged, so that it can receive a consistent copy later.

Access control

Access control is the name for the technique that arranges the access to data of the shared

storage environment. The term access control should thus be clearly differentiated from

the term access path, since the mere existence of an access path does not include the right

to access. Access control has the following main objectives:

• Authentication

Authentication establishes the identity of the source of an access.

• Authorization

Authorization grants or refuses actions to resources.

• Data protection

Data protection guarantees that data may only be viewed by authorized persons. All access control mechanisms ultimately use a form of secure channel between the data on the storage device and the source of an access. In its simplest form, this can be a check to establish whether a certain host is permitted to have access to a speciﬁc storage device. Access control can, however, also be achieved by complicated cryptographic proce-

dures, which are secure against the most common external attacks. When establishing a control mechanism it is always necessary to trade off the necessary protection and efﬁciency against complexity and performance sacriﬁces. In server-centric IT architectures, storage devices are protected by the guidelines on the host computers and by simple physical measures. In a storage network, the storage devices, the network and the network components themselves must be protected against unauthorized access, since in theory they can be accessed from all host computers. Access control becomes increasingly important in a shared storage environment as the number of components used, the diversity of heterogeneous hosts and the distance between the individual devices rise. Access controls can be established at the following points of a shared storage environment:

• On the host In shared storage environments, access controls comparable with those in server-centric environments can be established at host level. The disadvantage of this approach is, however, that the access rights have to be set on all host computers. Mechanisms that reduce the amount of work by the use of central instances for the allocation and distribution of rights must be suitably protected against unauthorized access. Database

systems and ﬁle systems can be protected in this manner. Suitable mechanisms for the block layer are currently being planned. The use of encryption technology for the host's network protocol stack is in conﬂict with performance requirements. Suitable ofﬂoad engines, which process the protocol stack on the host bus adapter themselves, are available for some protocols.

• In the storage network Security within the storage network is achieved in Fibre Channel SANs by zoning and virtual storage networks (Virtual SAN (VSAN), Section 3.4.2) and in Ethernet-based storage networks by so-called virtual LANs (VLAN). This is always understood to be the subdivision of a network into virtual subnetworks, which permit communication between a number of host ports and certain storage device ports. These guidelines can, however, also be deﬁned on ﬁner structures than ports.

• On the storage device The normal access control procedure on SAN storage devices is the so-called LUN masking, in which the LUNs that are visible to a host are restricted. Thus, the computer sees only those LUNs that have been assigned to it by the storage device (Section 2.7.3).

2008-04-03T23:15:00.001-07:00

KNOW MORE ON THE SNIA SHARED STORAGE MODEL ,THE LAYERS AND THE MODEL 317

The SNIA Shared Storage Model deﬁnes four layers (Figure 10.2):

I. Storage devices

II. Block aggregation layer

III. File/record layer

IIIb. Database

IIIa. File system

IV Applications

Applications are viewed as users of the model and are thus not described in the model. They are, however, implemented as a layer in order to illustrate the point in the model to which they are linked. In the following we'll consider the ﬁle/record layer (Section 10.1.6), the block layer (Section 10.1.7) and the combination of both (Section 10.1.8) in detail.

10.1.6 The ﬁle/record layer

The ﬁle/record layer maps database records and ﬁles on the block-oriented volume of the storage devices. Files are made up of several bytes and are therefore viewed as byte vectors in the SNIA model. Typically, ﬁle systems or database management systems take over these functions. They operate directories of the ﬁles or records, check the access, allocate storage space and cache the data (Chapter 4). The ﬁle/record layer thus works on volumes that are provided to it from the block layer below. Volumes themselves consist of several arranged blocks, so-called block vectors. Database systems map one or more records, so-called tuple of records, onto volumes via tables and table spaces:

Tuple of records −→ tables −→ table spaces −→ volumes

In the same way, ﬁle systems map bytes onto volumes by means of ﬁles:

Bytes −→ ﬁles −→ volumes

Some database systems can also work with ﬁles, i.e. byte vectors. In this case, bloc vectors are grouped into byte vectors by means of a ﬁle system – an additional abstraction level. Since an additional abstraction level costs performance, only smaller databases word in a ﬁle-oriented manner. In large databases the additional mapping layer of byte to bloc vectors is dispensed with for performance reasons. The functions of the ﬁle/record layers can be implemented at various points (Figure 10.3 Section 5.6):

• Exclusively on the host In this case, the ﬁle/record layer is implemented entirely on the host. Databases an the host-based ﬁle systems work in this way.

• Both in the client and also on a server component The ﬁle/record layer can also be implemented in a distributed manner. In this case th

functions are distributed over a client and a server component. The client component is realized on a host computer, whereas the server component can be realized on th following devices:

• NAS/ﬁle server A NAS/ﬁle server is a specialized host computer usually with a locally connected dedicated storage device (Section 4.2.2).

• NAS head A host computer that offers the ﬁle serving services, but which has access to external storage connected via a storage network. NAS heads correspond with the devices called NAS gateways in our book (Section 4.2.2). In this case, client and server components work over network ﬁle systems such as NFS or CIFS (Section 4.2).

The block layer

The block layer differentiates between block aggregation and the block-based storage devices. The block aggregation in the SNIA model corresponds to our deﬁnition of the virtualization on block level (Section 5.5). SNIA thus uses the term 'block aggregation' to mean the aggregation of physical blocks or block vectors into logical blocks or block vectors. To this end, the block layer maps the physical blocks of the disk storage devices onto

logical blocks and makes these available to the higher layers in the form of volumes (block vectors). This either occurs via a direct (1 : 1) mapping, or the physical blocks are ﬁrst aggregated into logical blocks, which are then passed on to the upper layers in the form of volumes (Figure 10.4). In the case of SCSI, the storage devices of the storage device layer exist in the form of one or more so-called logical units (LU). Further tasks of the block layer are the labelling of the logical units using so-called logical unit numbers (LUNs), caching and – increasingly in the future – access control.

Block aggregation can be used for various purposes, for example:

• Volume/space management The typical task of a volume manager is to aggregate several small block vectors to form one large block vector. On SCSI level this means aggregating several logical units

THE MODEL 317

to form a large volume, which is passed on to the upper layers such as the ﬁle/record layer (Section 4.1.4).

• Striping In striping, physical blocks of different storage devices are aggregated to one volume. This increases the I/O throughput of the read and write operations, since the load is distributed over several physical storage devices (Section 2.5.1).

• Redundancy In order to protect against failures of physical data carriers, RAID (Section 2.5) and remote mirroring (Section 2.7.2) are used. Snapshots (instant copies) can also be used for the redundant storage of data (Section 2.7.1). The block aggregation functions of the block layer can be realized at different points of the shared storage environment (Section 5.6):

• On the host Block aggregation on the host is encountered in the form of a logical volume manager software, in device drivers and in host bus adapters.

• On a component of the storage network The functions of the block layer can also be realized in connection devices of the storage network or in specialized servers in the network.

• In the storage device Most commonly, the block layer functions are implemented in the storage devices themselves, for example, in the form of RAID or volume manager functionality. In general, various block aggregation functions can be combined at different points of the shared storage environment. In practical use, RAID may, for example, be used in the disk subsystem with additional mirroring from one disk subsystem to another via the volume manager on the host computer (Section 4.1.4). In this setup, RAID protects against the failure of physical disks of the disk subsystem, whilst the mirroring by means of the volume manager on the host protects against the complete failure of a disk subsystem. Furthermore, the performance of read operations is increased in this set-up, since the volume manager can read from both sides of the mirror (Section 2.5.2).

2008-04-03T23:05:00.001-07:00

Free Tutors the snia shared storage model Graphical representations An elementary overview and The components

The SNIA Shared Storage Model further deﬁnes how storage architectures can be graphically illustrated. Physical components are always represented as three-dimensional objects, whilst functional units should be drawn in two-dimensional form. The model itself also deﬁnes various colours for the representation of individual component classes. In the black and white format of the book, we have imitated these using shades of grey. A coloured version of the illustrations to this chapter can be found on our home page Thick lines in the model represent the data transfer,

whereas thin lines represent the metadata ﬂow between the components.

An elementary overview

The SNIA Shared Storage Model ﬁrst of all deﬁnes four elementary parts of a shared storage environment (Figure 10.1):

1. File/record layer The ﬁle/record layer is made up of database and ﬁle system.

2. Block layer The block layer encompasses the storage devices and the block aggregation. The SNIA Shared Storage Model uses the term 'aggregation' instead of the often ambiguously used term 'storage virtualization'. In Chapter 5, however, we used the term 'storage virtualization' to mean the same thing as 'aggregation' in the SNIA model, in order to avoid ambiguity.

3. Services subsystem The functions for the management of the other components are deﬁned in the services subsystem.

4. Applications Applications are not discussed further by the model. They will be viewed as users o the model in the widest sense.

The components

The SNIA Shared Storage Model deﬁnes the following components:

• Interconnection network The interconnection network represents the storage network, i.e. the infrastructure, that connects the individual elements of a shared storage environment with one another. The interconnection network can be used exclusively for storage access, but it can

also be used for other communication services. Our deﬁnition of a storage network (Section 1.2) is thus narrower than the dentitions of the interconnection network in the SNIA model. The network must always provide a high-performance and easily scalable connection for the shared storage environment. In this context, the structure of the interconnection network – for example redundant data paths between two components to increase fault-tolerance – remains just as open as the network techniques used. It is therefore a prerequisite of the model that the components of the shared storage environment are connected over a network without any deﬁnite communication protocols or transmission techniques being speciﬁed.

In actual architectures or installations, Fibre Channel, Fast Ethernet, Gigabit Ethernet, InﬁniBand and many other transmission techniques are used (Chapter 3). Communication protocols such as SCSI, Fibre Channel FCP, TCP/IP, RDMA, CIFS or NFS are based upon these.

• Host computer Host computer is the term used for computer systems that draw at least some of their storage from the shared storage environment. According to SNIA, these systems were often omitted from classical descriptive approaches and not viewed as part of the environment. The SNIA shared storage model, however, views these systems as part of the entire shared storage environment because storage-related functions can be implemented on them. Host computers are connected to the storage network via host bus adapters or network

cards, which are operated by means of their own drivers and software. Drivers and software are thus taken into account in the SNIA Shared Storage Model. Host computers can be operated fully independently of one another or they can work on the resources of the storage network in a compound, for example, a cluster

• Physical storage resource All further elements that are connected to the storage network and are not host computers are known by the term 'physical storage resource'. This includes simple hard disk drives, disk arrays, disk subsystems and controllers plus tape drives and tape libraries. Physical storage resources are protected against failures by means of redundant data paths (Section 6.3.1), replication functions such as snapshots and mirroring (Section 2.7) and RAID (Section 2.5).

\• Storage device A storage device is a special physical storage resource that stores data.

• Logical storage resource The term 'logical storage resource' is used to mean services or abstract compositions of physical storage resources, storage management functions or a combination of these. Typical examples are volumes, ﬁles and data movers.

• Storage management functions The term 'storage management function' is used to mean the class of services that monitor and check (Chapter 8) the shared storage environment or implement logical storage resources. These functions are typically implemented by software on physical

storage resources or host computers.

2008-04-02T23:07:00.001-07:00

Free tutors on the SNIA Shared Storage Model and The functional approach

The fact that there is a lack of any uniﬁed terminology for the description of storage architectures has already become apparent at several points in previous chapters. There are thus numerous components in a storage network which, although they do the same thing, are called by different names. Conversely, there are many systems with the same name, but fundamentally different functions. A notable example is the term' data mover' relating to server-free back-up (Section 7.8.1) in storage networks. When this term is used it is always necessary to check whether the component in question is one that functions in the sense of the 3rd-party SCSI Copy Command for, for example, a software component of back-up software on a special server, which implements the server-free back-up without 3rd-party SCSI. This example shows that the type of product being offered by a manufacturer and the functions that the customer can ultimately expect from this product are often unclear. This makes it difﬁcult for customers to compare the products of individual manufacturers and ﬁnd out the differences between the alternatives on offer. There is no uniﬁed model for this with clearly deﬁned descriptive terminology. For this reason, in 2001 the Technical Council of the Storage Networking Industry Association (SNIA) introduced the so-called Shared Storage Model in order to unify the terminology and descriptive models used by the storage network industry. Ultimately, the SNIA wants to use the SNIA Shared Storage Model to establish a reference model, which will have the same importance for storage architectures as the seven-tier OSI model has for computer networks. In this chapter, we would ﬁrst like to introduce the disk-based Shared Storage Model (Section 10.1) and then show, based upon examples (Section 10.2), how the model can be used for the description of typical disk storage architectures. In Section 10.3 we introduce the extension of the SNIA model to the description of tape functions. We then discuss examples of tape-based back-up architectures (Section 10.4). Whilst describing the SNIA Shared Storage Model we often refer to text positions in this book where the subject in question is discussed in detail, which means that this chapter also serves as a summary of the entire book.

THE MODEL

In this book we have spoken in detail about the advantages of the storage-centric architecture in relation to the server-centric architecture. The SNIA sees its main task as being to communicate this paradigm shift and to provide a forum for manufacturers and developers so that they can work together to meet the challenges and solve the problems in this ﬁeld. In the long run, an additional reason for the development of the Shared Storage Model by SNIA was the creation of a common basis for communication between the manufacturers who use the SNIA as a platform for the exchange of ideas with other manufacturers. Storage-centric IT architectures are called shared storage environments by the SNIA. We will use both terms in the following. First of all, we will describe the functional approach of the SNIA model (Section 10.1.1) and the SNIA conventions for graphical representation (Section 10.1.2). We will then consider the model (Section 10.1.3), its components (Section 10.1.4) and the layers' ﬁle/record layer' and 'block layer' in detail (Section 10.1.5 to Section 10.1.8). Then we will introduce the deﬁnitions and representation of concepts from the SNIA

model, such as access paths (Section 10.1.9), caching (Section 10.1.10), access control (Section 10.1.11), clustering (Section 10.1.12), data (Section 10.1.13) and resource and data sharing (Section 10.1.14). Finally, we will take a look at the service subsystem (Section 10.1.15).

The functional approach

The SNIA Shared Storage Model ﬁrst of all describes functions that have to be provided in a storage-centric IT architecture. This includes, for example, the block layer or the ﬁle/record layer. The SNIA model describes both the tasks of the individual functions and also their interaction. Furthermore, it introduces components such as server ('host computer') and storage networks ('interconnection network'). Due to the separation of functions and components, the SNIA Shared Storage Model is suitable for the description of various architectures, speciﬁc products and concrete

installations. The fundamental structures, such as the functions and services of a shared storage environment, are highlighted. In this manner, functional responsibilities can be assigned to individual components and the relationships between control and data ﬂows in the storage network worked out. At the same time, the preconditions for interoperability between individual components and the type of interoperability can be identiﬁed. In addition to providing a clear terminology for the elementary concepts, the model should be simple to use and, at the same time, extensive enough to cover a large number of possible storage network conﬁgurations. The model itself describes, on the basis of examples, possible practicable storage architectures and their advantages and disadvantages. We will discuss these in Section 10.2 without evaluating them or showing any preference for speciﬁc architectures. Within the model deﬁnition, however, only a few selected examples will be discussed in order to highlight how the model can be applied for the description of storage-centred environments and further used.

2008-04-02T22:47:00.001-07:00

Know more about THE IEEE 1244 STANDARD FOR REMOVABLE MEDIA MANAGEMENT Operational characteristics of the media manager

From the point of view of the client, the media manager works as a server that waits for MMP commands, which the client sends via a TCP/IP connection. The media manage executes these commands, generates appropriate responses and sends these back to the clients. All commands are given unambiguous task identiﬁers. The responses contain the task identiﬁer of the command in question. The response takes place in two stages. First, the successful receipt of the command is acknowledged. In a second response, the application is informed whether the command has been successfully executed and which responses the system has supplied. Example 1 An application wants to mount the volume with the name back-up-1999-12 31. To this end it, sends the following command to the media manager: mount task[' '1' '] volname [' 'back-up-1999-12-31' ']

report [MOUNTLOGICAL.' 'MountLogicalHandle' ']; The media manager has recognized the command and accepted it for processing and

therefore sends the following response: response task[' '1' '] accepted; Now the media manager will transport the cartridge containing the volume into a drive to which the application has access. Once the cartridge has been successfully inserted, a response is generated that could look like this:

response task[' '1' '] success text [' '/dev/rmt0' ']; The media manager stores all commands in a task queue until all resources required for execution are available. Once all the resources are available, the media manager removes the command from the task queue and executes it. If several commands are present that require the same resources, the media manager selects the next command to be carried out on the basis of priorities or on a ﬁrst come, ﬁrst served basis. All other commands remain in the task queue until the resources in question become free again. In this manner

libraries, drives and also cartridges can be shared. Commands that are in the task queue can be removed again using the Cancel command

Operational characteristics of the library and drive managers The library manager receives the media manager's commands via the library manage

ment protocol (LMP) and converts these into the speciﬁc commands for the hardware in question. From the point of view of the media manager, a uniﬁed abstract interface tha conceals the properties of the hardware in question thus exists for all libraries. New hard ware can thus be integrated into the management system using a suitable library manager without having to make changes to the whole system. Accordingly, drive manager implementations of the abstract drive management protocol (DMP) are interfaces for a certain drive hardware. However, drive management

must also take into account the speciﬁc properties of the various client platforms upon which the applications that want to use the media management system run. If such an application is running on a UNIX-compatible platform, the drive manager must provide the corresponding names of a device special ﬁle for access to the drive. Under Windows, such a drive manager must supply a windows-speciﬁc ﬁle name, such as \\.\TAPE0.

Privileged and non-privileged clients The media manager carries out requests from clients that want to take advantage of the media management services. From the point of view of the media manager there are privileged and non-privileged clients:

• Non-privileged clients, such as back-up systems, can only handle objects for which they have been granted an appropriate authorization.

• Privileged clients, usually administrative applications, may perform all actions and manipulate all objects. They serve primarily to include non-privileged applications in the system and to establish suitable access controls. The IEEE 1244 data model (Table 9.1) In addition to the architecture and the logs for communication, the standard also describes a complete data model, which includes all objects, and their attributes, that are necessary for the representation of the media management system. Objects can be provided with additional application-speciﬁc attributes. The object model can thus be dynamically and ﬂexibly adapted to the task at hand, without changes being necessary to the underlying management system. 9.5.3 Media Management Protocol (MMP) The media management protocol (MMP) is used by the applications to make use of

the media management services of an IEEE 1244-compatible system. MMP is a text based protocol, which exchanges messages over TCP/IP. The syntax and semantics of the individual protocol messages are speciﬁed in the MMP speciﬁcation IEEE 1244.3. MMP permits applications to allocate and mount volumes, read and write metadata and to manage and share libraries and drives platform-independently. Due to the additional bstraction levels, the application is decoupled from the direct control of the hardware. Thus, applications can be developed independently of the capability of the connected hard- ware and can be made available to a large number of different types of removable media.

Table 9.1 The most important objects of the IEEE 1244 data model

Object Description

APPLICATION Authorized client application. Access control is performed on the basis of applications. User management is not part of this standard, since it is assumed that it is not individual users, but applications that already manage their users, that will

use the services of the media management system. AI Authorized instances of a client application. All instances of an application have unrestricted access to resources that are assigned to the application. LIBRARY Automatic or manually operated libraries.

LM Library managers know the details of a library. The library manager protocol serves as a hardware-independent interface between media

manager and library manager. BAY Part of a LIBRARY (contains DRIVES and SLOTS). SLOT Individual storage space for CARTRIDGEs within a

BAY. SLOTGROUP Group of SLOTS to represent a magazine, for example, within a LIBRARY. SLOTTYPE Valid types for SLOTs, for example 'LTO',

'DL-Tor3480', 'QIC' or 'CDROM'. DRIVE Drives, which can accept CARTRIDGEs for writing or reading. DRIVEGROUP Groups of drives.

DRIVEGROUPAPPLICATION This object makes it possible for applications to access drives in a DRIVEGROUP. This connection can be assigned a priority so that several DRIVEGROUPs with different priorities are available. The media manager selects a suitable drive according to priority.

DM Drive manager. Drive managers know the details of a drive and make this available to the media manager. The drive manager protocol serves as a hardware-independent interface between media manager and drive manager. CARTRIDGE Removable data carrier; media. CARTRIDGEGROUP Group of CARTRIDGEs. CARTRIDGEGROUPAPPLICATION This object makes it possible for applications to access CARTRIDGEs in a CARTRIDGEGROUP.

This connection can be assigned a priority so that several CARTRIDGEGROUPs with different priorities are available. The media manager selects a

suitable CARTRIDGE according to priority in order to allocate a VOLUME, if no further entries are made. (continued overleaf )

2008-04-02T21:51:00.001-07:00

FREE TUTORS ABOUT THE IEEE 1244 STANDARD FOR REMOVABLE MEDIA MANAGEMENT SYSTEM ARCHITECTURE

As early as 1990, the IEEE Computer Society set up the 1244 project for the development of standards for storage systems. The Storage System Standards Working Group was also established with the objective of developing a reference model for mass storage systems (Mass Storage System Reference Model/MSSRM). This reference model has signiﬁcantly inﬂuenced the design of some storage systems that are in use today. The model was then revised a few times and in 1994 released as the IEEE Reference Model for Open Storage Systems Interconnection (OSSI). Finally, in the year 2000, after further revisions, the 1244 Standard for Media Management Systems was released. This standard consists of a series of documents that describe a platform-independent,distributed management system for removable media. It also deﬁnes both the architecture for a removable media management system and its interfaces towards the outside world. The architecture makes it possible for software manufacturers to implement very

scalable, distributed software systems, which serve as generic middleware between application software and library and drive hardware. The services of the system can thus be consolidated in a central component and from there made available to all applications. The speciﬁcation paid particular attention to platform-independence and the heterogeneous environment of current storage networks was thus taken into account mazingly early on. Systems that build upon this standard can manage different types of media. In addition to the typical media for the computer ﬁeld such as magnetic tape, CD, DVD or optical media, audio and video tapes, ﬁles and video disks can also be managed. In actual fact, there are no assumptions about the properties of a medium in IEEE 1244- compliant systems. Their characteristic features (number of sides, number of partitions, etc.) must be deﬁned for each media type that the system is to support. There is a series of predeﬁned types, each with their own properties. This open design makes it possible to specify new media types and their properties at any time and to add them to the current system. In addition to neutrality with regard to media types, the standard permits the management of both automatic and manually-operated libraries. An operator interface, which is

also documented, and with which messages are sent to the appropriate administrators of a library, serves this purpose. In the following sections we wish to examine more closely the architecture and functionality of a system based upon the IEEE standard.

Media management system architecture

The IEEE 1244 standard describes a client/server architecture (Figure 9.9). Applications such as network back-up systems take on the role of the client that makes use of the services of the removable media management system. The following components are individually deﬁned:

• a media management component, which serves as a central repository for the metadata and provides mechanisms for controlling and co-ordinating the use of media, libraries and drives;

• a library manager component, which controls the library hardware on behalf of the media manager and transmits the properties and the content of the library to the media manager;

• a drive manager component, which manages the drive hardware on behalf of the media manager and transmits the properties of the drives to the media manager. In addition, the standard deﬁnes the interfaces for the communication with these components:

• the Media Management Protocol (MMP) for the communication between application (client) and media manager (server);

• the Library Management Protocol (LMP) for the communication between library manager and media manager;

• the Drive Management Protocol (DMP) for the communication between drive manager and media manager.

These protocols use TCP/IP as the transport layer. As in the popular Internet applications HTTP, FTP or SMTP, commands are sent via TCP/IP in the form of text messages. These protocols can be implemented and used just as simply on different platforms. The advantage of this approach is that the media manager can implement components as a generic application, i.e. independently of the speciﬁc library and drive hardware used. The differences, in particular with the control, are encapsulated in the library manager or drive manager for the hardware in question. For a new tape library, therefore, only a new library manager component needs to be implemented that converts the speciﬁc interface of the library into the library management protocol, so that this can be linked into an existing media manager installation. The next sections describe how communication takes place between clients and servers and how the media manager processes the commands.

2008-04-02T06:59:00.001-07:00

Learn more on Life cycle management

The life cycle of a cartridge describes a series of states that a cartridge can take on over the course of time. Essentially, the following events, which lead to state transitions, can be observed in the life cycle:

• Initialization Cartridges are announced to the system.

• Allocation of access rights Applications are permitted to access certain cartridges.

• Use Cartridges are used for reading and writing.

• deallocation The data on a cartridge is no longer needed; the storage space can once again be made available to other applications.

• Retirement The cartridge has reached the end of its life cycle and is removed from the system. These events directly yield a series of states for cartridges. States of media In addition to the location of the media, it is also important that storage administrators are aware of the state of the media. For example, a cartridge may not be removed from the system until no more logical volumes have been allocated to it. Otherwise, there is a

danger of data loss, because back-up software can no longer access this volume. During its life cycle (Figure 9.8) a cartridge can take on the following states:

• Undeﬁned (unknown) No further information is known about a cartridge. This is the initial state of a cartridge before it is taken into a management system.

• Deﬁned A cartridge is announced to the system. Information about the type, cartridge label, etc. are given, and the cartridge is thus described.

• Available In addition to the information that a cartridge exists in the system, information about whether, and where, data can be still written to the cartridge is also important. If this information is known, the cartridge is available for applications

• Allocated In order that an application can use a cartridge, suitable storage space on a cartridge

must be allocated by the system. This allocation of storage space generally leads to the placing of a volume on a partition of a cartridge. The application should be able to freely choose the identiﬁcation of the volume. In general, the state of the ﬁrst volume placed determines the state of the entire cartridge. To this end, as soon as the ﬁrst volume has been placed on a cartridge, the state of the cartridge is set to 'allocated'.

• deallocated If the storage space (the volume) is no longer required by an application, the volume can be de-allocated. The system should then delete all information about the volume.

The application should continue to be able to reallocate the same storage space. If the

storage space is to be made available to other applications, a cartridge must be recycled.

• Recycled Depending upon system conﬁguration, once all volumes have been removed from a cartridge the entire storage space can be made available to other applications.

• Purged A cartridge, and all information about this cartridge, is completely removed from the system. In general, this state is reached at the end of a cartridge's life cycle. Typically, this state exists only for a short time, since the cartridge is immediately placed in the undeﬁned state.

Policy-based life cycle management Certain tasks should be automated so that as little manual intervention as possible is required during the management of the data carriers. In life cycle management, some tasks positively demand to be performed automatically. These include:

• the monitoring of retention periods;

• transportation to the next storage location (movement or rotation);

• the copying of media when it reaches a certain age;

• the deletion of media at the end of the storage period;

• the recycling or automatic removal of the cartridge from the system.

The individual parameters for the automated tasks are speciﬁed by suitable policies.

Individual cartridges, or groups of cartridges, are assigned suitable policies.

2008-04-02T06:30:00.000-07:00

Tutors on Monitoring management of removable media

The large number of devices and media that have to be monitored in a data centre makes it almost impossible for monitoring to be performed exclusively by administrators. Automatic control of the system, or at least of parts of it, is therefore absolutely necessary for installations above a certain size. For removable media, in particular, it is important that monitoring is well constructed because in daily operation there is too little time to verify every back-up. If errors creep in whilst the system is writing to tape this may not be recognized until the data needs to be restored – when it is too late. If there is no second copy, the worst conceivable incident for the datacenter has occurred: data loss! Modern tape drives permit a very good monitoring of their state. This means that the number of read-write errors that cannot be rectiﬁed by the built-in ﬁrmware, and also the

number of load operations, are stored in the drive. Ideally, this data will be read by the management system and stored so that it is available for further evaluations. A further step would be to have this data automatically analyzed by the system. If certain error states are reached, actions can be triggered automatically so that at least no further error states are permitted. Under certain circumstances, errors can even be rectiﬁed automatically, for example by switching a drive off and back on again. In the worst case, it is only possible to mark the drive as defective so that it is not used further. In these tasks, too, a mechanism controlled by means of rules can help and signiﬁcantly take the pressure off the administrator.

The data stored on the drives not only provides information on the drives themselves, but also on the loaded tapes. This data can be used to realize a tape quality management, which, for example, monitors the error rates when reading and writing and, if necessary, copies the data to a new tape if a certain threshold is exceeded.

Reporting

In addition to media management and drive and library sharing a powerful system requires the recording of all actions. For certain services it is even a legal requirement that so-called security audits are performed. Therefore, all actions must be precisely logged. In addition, he log data must be protected against manipulation in an appropriate manner. With the aid of a powerful interface, it should be possible to request data including

the following:

• When was a cartridge incorporated into the system?

• Who allocated which volume to which cartridge when?

• Who accessed which volume when?

• Was this volume just read or also written?

• Which drive was used?

• Was this an authorized access or was access refused? The following requirements should be fulﬁlled by the reporting module of a removable

media management system:

• Audit trails As already mentioned, it should be possible to obtain a complete list of all accesses to

a medium. Individual entries in this list should give information about who accessed a

medium, for how long, and with what access rights.

• Usage statistics Data about when the drives were used, and for how long they were used, is important

in order to make qualitative statements about the actual utilization of all drives. At

any point in time, were sufﬁcient drives available to carry out all mount requests? Are

more drives available than the maximum amount needed at the same time over the last

twelve months? The answers to such questions can be found in the report data. Like

the utilization of the drives, the available storage space is, of course, also of interest.

Was enough free capacity available? Were there bottlenecks?

• Error statistics Just like the data on the use of resources, data regarding the errors that occurred during use is also of great importance for the successful use of removable media. Have the storage media of manufacturer X caused less read-write errors in the drives of manufacturer Y than the media of manufacturer Z? Appropriate evaluations help considerably in the optimization of the overall performance of a system.

• Future planning Predictions for the future can be made from the above-mentioned statistics. How will the need for storage grow? How many drives will be used in twelve months? And MANAGEMENT OF REMOVABLE MEDIA how many slots and cartridges? A management system should be able to help in the search for answers to these questions. In order that future changes can also be simply carried out, the addition of further drives or cartridges must also be possible without any problems and must not require any changes to the existing applications that use the management services.

2008-04-02T05:21:00.000-07:00

Free tutors on Media tracking Grouping, pooling Drive pools

As an integral part of a disaster recovery solution, a management system must ensure that all removable media plus the appropriate metadata remain in the system until they are deleted by an appropriately authorized user. Thus, under no circumstances may tapes be 'lost' or removed from the system without authorization. Furthermore, it must be possible to determine the storage location of each medium at all times. If access is available to the media online, for example, in automatic tape libraries in which tapes can be automatically identiﬁed by the reading of a barcode label by a scanner, a suitable audit can be performed at any time. In such an audit, the content of the inventory is compared with the real existing tapes. Such libraries also automatically report the opening of a door. After the door has been closed an audit should once again be automatically performed in order to ensure that no media has been removed from the system without authorization. If a part of the media is withdrawn from direct access, either to a well-protected safe or to another manually-operated library, this storage place must be managed with appropriate care by the responsible administrators. Ideally, the management software provides an interface for this vaulting. In order to increase the reliability of a disaster recovery concept and to fulﬁl statutory provisions, a two-stage or multi-stage strategy made up of online and ofﬂine storage is often performed. Storage media are ﬁrst written in automatic libraries, then stored ofﬂine for a certain period of time and subsequently either taken out of circulation or reused (Figure 9.5). Media that are in transit from an online library to an ofﬂine storage place must be identiﬁed. A management system for removable media should serve both as a central repository for all resources and as a universal interface for applications. It should not be possible for any application to withdraw itself from the control of the system and access or move media in an uncontrolled manner. Only thus is it actually guaranteed that all media in the system can be located at any time. Such a central interface is always in danger of becoming a single point of failure. It is therefore very wise to use appropriate measures to guarantee a high level of availability for the entire solution. Consequently, care must be taken to ensure that all components of the entire system, from the hardware through the operating systems used with the media management to the back-up software, are designed to have a high level of availability. The hardware often offers suitable options. Drives and media changers are available with a redundant power supply and redundant access paths. Modern operating systems such as AIX or Solaris can automatically use such redundantly designed access paths in the event of a fault.

Grouping, pooling

Systems for the management of removable media must be able to deal with a great many media and facilitate access to many applications. In order to plan and execute access control in a sensible manner and to guarantee its effective use, it is necessary to combine cartridges and drives into groups. Grouping also allows budgets for storage capacities or drives to be created, which are made available to the applications. Scratch pools

A scratch pool contains unused cartridges that are available to authorized applications so that they can place volumes upon them. As soon as an application has placed a volume upon a cartridge from such a scratch pool, this cartridge is no longer available to all other applications and is thus removed from the scratch pool. If several scratch pools are available, they cannot only be grouped for different media, it is also possible to deﬁne groups for certain application purposes. For example, an administrator should be able to make a separate pool of cartridges available for important

back-up jobs, whilst cartridges from a different scratch pool are used for 'normal' back-up jobs. The back-up application can choose which pool it would like to have a cartridge from. If it is possible to assign priorities to scratch pools on the basis of which the management system can decide from which pool a new cartridge will be provided, then such a request can be automated to a certain degree without the back-up application

having to know all available pools. To this end, the request for a new tape must be given an appropriate priority, whereupon the management system searches for a cartridge from a scratch pool with a suitably high priority. In addition to the requirement that all cartridges can be located at all times, a management system for removable media should be capable of offering free storage space to an application at any time. Only thus can back-up windows actually be adhered to. As is the case for media tracking, in order to fulﬁl this requirement a high-availability solution

covering all levels, from the hardware to the application software, should be pursued. In addition, scratch pools can help to contain the cartridges from two or more libraries (Figure 9.6). They also offer the guarantee that, even in the event of the failure of indi- vidual libraries, cartridges in the other libraries will remain usable. It is precisely in this case that the advantages of a storage network, together with an intelligent management

system for removable media, fully come to bear in the optimal utilization of resources that are distributed throughout the entire system.

In order to be able to react ﬂexibly to changes, scratch pools should be dynamically expandable. To this end, an administrator must make additional storage space available to the system dynamically, whether by the connection of a new library or by the addition of previously unused cartridges. Ideally, this can be achieved without making changes to the applications that have previously accessed the scratch pool. An adjustable minimum size (low water mark) makes the management of a scratch pool easier. If this threshold is reached, measures must be taken to increase the size of the

pool, as otherwise there is the danger that the system will cease to be able to provide free storage space in the foreseeable future. The management system can help here by ﬂexibly offering more options. Many actions are possible here, from the automatic enlargement of the scratch pools – as long as free media are available in the libraries – to the 'call home' function, in which an administrator is notiﬁed. If a cartridge has several partitions, then it would occasionally be desirable to collect just the free partitions – rather than the complete cartridges – into a scratch pool. Then

it would be possible to manage free storage capacity with a ﬁner granularity and thus achieve an optimal utilization of the total amount of available storage capacity. Since, owever, the individual partitions of a medium cannot be accessed at the same time, a cartridge is currently generally managed and allocated to an application as the smallest unit of a scratch pool. As capacity increases, however, the additional use of partitions for

this may also be required.

Drive pools

The mere fact that cartridges are available does not actually mean that the storage space can be used. In addition, drives must be available that can mount the cartridges for reading or writing. Similarly to cartridges, it is also possible to combine drives into pools. A pool of high- priority drives can, for example, always be kept to fulﬁl mount requests if all other drives are fully utilized. In order to save the applications from having to know and request all drive pools, it is a good idea to have a priority attribute that is used by the management system to automatically locate a drive with an appropriate priority. If several libraries are available, drive pools should include drives from several libraries (Figure 9.7). This ensures that drives are still available even if one library has failed. At the very least, this helps when writing new data if free cartridges are still available.

2008-04-02T05:13:00.001-07:00

Tutors on Authorization for access to individual objects , Access synchronization and Access prioritization and mount request queuing

It is currently still common to use authorization procedures for entire cartridges only and not for their components. However, in order to be prepared for future developments, such as the 1-terabyte tape cartridge, a management system should, even now, have appropriately detailed access protection for subdivisions of cartridges such as sides, partitions and volumes. All components of a cartridge are suitable for access control. The application purpose determines whether the user receives access to a side, a partition or a volume. An authorization is always applicable to all elements of the authorized object. If, for example, the right to access a cartridge is granted, this right also applies to all sides, partitions and

volumes of this cartridge. It is not only cartridges that should be provided with access control. For example, it is a good idea to restrict applications’ access to cartridges that are still available. To this end, the available cartridges are combined into one or more scratch pools. The applications

are then granted the right to access only certain scratch pools. Usually, access control is less important here than an optimal utilization of the free storage capacity. Drives are also suitable for access control. However, here, too, it is usually the optimal utilization of the drives that is sought. An allocation of drives to certain users or applications should, however, be possible. As a result, users can be granted exclusive access to drives. Naturally, the grouping of drives is again an option here for simplifying management and increasing drive utilization.

Access synchronization

As already mentioned several times, a library or a drive cannot receive and process several commands from various applications in parallel. Therefore, synchronization is required that serializes all commands received at the same time and forwards them to the drives one after the other. As a result of this functionality, devices can be used ‘quasi’ simultaneously, in the same way as operating systems allow a single processor to be

made available to several processes one after the other for a limited duration. This type of synchronization corresponds with dynamic tape library sharing and has already been described in Section 6.2.1.

Access prioritization and mount request queuing

Despite intelligent access control, there may be more mount requests than available drives at a certain point in time. Ideally, a system should collect requests into a request queue in this case (Figure 9.4). This queue can be available for each drive and also for each group of drives and collect all mount requests that cannot be carried out immediately. Once a drive becomes available again, the system can perform the next request and remove

this from the queue in question. In the search for the next request to be carried out, a scheduler can sometimes also evaluate the priority of requests and change their sequence accordingly. Request queues that are not bound to drives have the advantage that several free drives may be available for each new request that is taken out of the queue. The assignment of mount request and drive must be re-evaluated accordingly, taking into account utilization and priority in order to increase the utilization of the system as a whole. Depending upon realization, this can also lead to an application being again withdrawn from a drive, so that it has to interrupt the access to the tape. It should be possible to remove requests from a queue and change the priorities of requests via an administrative interface.

2008-04-02T05:09:00.001-07:00

How to use Efficient use of the available resources and Access Control

A great advantage of the use of well-designed storage networks is the fact that the available hardware resources are better utilized. In contrast to directly connected devices, the available storage space is available to many applications and can therefore be used signiﬁcantly more effectively. In the ﬁeld of removable media this is achieved by the better utilization of the free storage capacity and the sharing of drives. Efﬁcient use of the storage capacity The disk storage pooling described in Section 6.2.1 is transferable to removable media one-to-one. In this case, free storage space should be taken to mean both unused removable media and also free slots for removable media, which must be kept in reserve due to the continuous growth in data. Ideally, this takes place in a cross-library manner. To this end, all free cartridges from all libraries are managed in a so-called scratch pool (Section 9.4.6), which is available to all applications, so that the remaining free storage capacity can be ﬂexibly assigned.

Efﬁcient use of the drives What applies for the effective use of the free storage capacity also applies in the same way for the use of the drives. If drives are directly connected to servers they cannot be used by other servers, even if they currently have no cartridge loaded into them.

By contrast, drives in storage networks can be assigned to the applications that need them at the time. Thus, it can be ensured that all drives that are installed are also actually used. The utilization of drives can be further increased by mount request queuing. However, more time is then required to perform the mount requests in the queue. This is a typical time-versus-space optimization problem. With more drives, more mount requests can be

carried out in the same time. If, however, a lot of mount requests are not urgent, fewer drives can execute the requests one after the other. Ideally, the mount requests in the queues are prioritizable, so that urgent tasks are actually performed sooner.

Access control

Reliable control to prevent unauthorized access to media is indispensable. Users and applications must be authenticated. Successfully authenticated users can then be given suitable authorization to access certain resources. Authentication Users, and also applications, that want to make use of removable media management services must be registered with the system. A sufﬁciently strict authentication mechanism should ensure that only users and applications that have been unambiguously identiﬁed can use the system. Authorization Authorization is necessary to prevent unauthorized users from being able to view, or even change, data belonging to other users. Authorization can both apply for certain operations and also arrange access to certain objects (cartridges, partitions, volumes, drives, etc.). A successful authentication is a necessary prerequisite for authorization.

By means of an appropriate authorization, users or applications can be assigned the following rights regarding certain objects in the management system:

• generation and deleting of objects (e.g. the allocation and deallocation of volumes);

• read access to objects (e.g. read access to own volumes);

• write access to objects (e.g. addition of cartridges to a scratch pool);

• mount and unmount of cartridges, sides, partitions or volumes;

• moving of cartridges within libraries;

• import and export of cartridges;

• activation and deactivation of libraries or drives.

The use of various authorization levels allows access control to be modiﬁed according to the user's role. The following roles and activities are currently used in systems for the list serves as an example only. These roles and activities can also be assigned differently

depending upon the speciﬁc requirements.

The system administrator is responsible for:

• installation of the system

• installation/deinstallation of libraries

• user and application management

• management of disk and cartridge groups.

The storage administrator is responsible for:

• management of disk and cartridge groups

• cartridge life cycle management

• planning of the future requirements for resources.

The library administrator is responsible for:

• management of disk and cartridge groups for individual libraries

• planning of the future requirements for resources

• starting and continuance of the operation of individual libraries

• starting and continuance of the operation of individual drives.

The library operator is responsible for:

• starting and continuance of the operation of individual libraries

• monitoring the operation of libraries

• starting and continuance of the operation of individual drives

• monitoring of the operation of drives in the libraries

• manual import and export of cartridges into and out of libraries

• performance of mount/unmount operations in manually operated libraries

• moving cartridges within a library.The users/applications may:

• allocate and de-allocate volumes to cartridges

• mount and unmount volumes

• read and write volumes

• list and display volumes that they have allocated

• list and display cartridges upon which volumes they have allocated have been put

• list and display scratch cartridges, which are included in cartridge groups to which there is an access right

• list and display drives that are included in drive groups to which there is an access right.

2008-04-02T05:00:00.001-07:00

Know more on problems and requirements related to removable media management

The problems and requirements relating to the integration of removable media in the storage network can be divided into two areas:

1. the management of removable media; and

2. the sharing of the associated resources.

In large environments, removable media management must be able to catalogue hundreds of thousands of media, storing not only the media and their attributes, but also accessing these media with corresponding data about errors, duration of use, etc. In contrast to hard disk storage, it is possible to store the media separately from drives, which means that the system must additionally know the location of a medium at all

times. Since this location can be a manually-managed store or an automatic library in which the cassettes are automatically located, special solutions are required that take these requirements into account. The second important ﬁeld of application for a removable media management system

in the storage network is the sharing of libraries, drives and media. This sharing between several applications connected to the storage network requires corresponding mechanisms for access control, access synchronization and access prioritization. These mechanisms control who may access which hardware when, so that potentially all applications can access all resources available in the storage network. In order to be able to fulﬁl these requirements, there is an increasing need for management layers for removable media in storage networks. These layers link existing

applications to the hardware connected via the storage network (Figure 9.3). They control and synchronize all accesses and should remain as transparent as possible to the applications. As a central interface, this middleware should therefore be capable of managing all resources and also the sharing, i.e. the sharing of libraries, drives and cartridges by various applications. We already discussed in Section 6.2.2 the various options for realizing library and drive sharing for removable media in storage networks. As also mentioned at that point, we believe that an architecture that shields the applications from the complex internal processes during management and sharing represents the best and most promising solution.

After all, the management is familiar with all components and their interaction. Therefore, an optimal control over the use of resources can also be implemented there. Individually, the following problems and requirements can be deﬁned:

• Resource Utilization: Efﬁcient use of all available resources by intelligent sharing.

• Access control: Applications and users must be authenticated. Applications and users may only be given access to the media for which suitable authorization exists.

• Access synchronization: Accesses to libraries and drives must be synchronized. • Access prioritization:

Prioritization can be used if several accesses to a resource, for example a drive, are to

be performed.

• Media tracking: It must be guaranteed at all times that every medium can be found and accessed.

• Grouping, pooling: It should be possible to dynamically aggregate both media and drives into groups or pools in order to simplify management and sharing.

• Monitoring: Automatic monitoring of the system.

• Reporting: Accesses to media must be logged. Audit trails should be possible.

• Life cycle management: Media run through a life cycle. They are written, read, written again and after a certain time taken out of circulation.

• Vaulting: Management of ofﬂine storage locations.

In what follows we investigate the above-mentioned problems and requirements further.

2008-04-02T04:56:00.001-07:00

Free Tutors on Control of the media changer SCSI and Fibre Channel interface

Applications must be able to control the media changers in automatic libraries. To this end, these libraries are equipped with suitable interfaces, which the applications use to send commands and receive return messages. In the Open Systems environment the direct connection via SCSI or Fibre Channel (in-band interface) is the most widespread. On the other hand, proprietary out-band interfaces tend to be used more in the mainframe

environment.

SCSI and Fibre Channel interface

Two procedures have established themselves for the control of the media changer via the SCSI or Fibre Channel FCP interface. In one case the media changer is equipped with its own controller and can be addressed as a separate device over its own SCSI target ID. In the other case the media changer shares the controller with the tape drives (Figure 9.1). Then it is either visible as an independent device with separate LUN (inde- pendent media changer) or is controlled via the drive LUN using special commands (attached media changer). As is often the case in the IT world, there are two contrasting philosophies here, that reveal their speciﬁc advantages and disadvantages depending upon the application case.If the media changer shares the same controller with the tape drive, then the bandwidth available to the drive is reduced. However, as only a relatively small number of commands are transferred and carried out for media changers, the reduction of the bandwidth available for the drive is low. This is particularly true in the Open Systems environment, where tapes are predominantly used in streaming mode. If, on the other hand, access is mainly ﬁle-based and if the ﬁles are located on several tapes, the ratio of media changer to drive commands increases correspondingly. In this case it can be worthwhile conducting the communication with the media changer over an addi- tional controller. However, both this additional controller and the additional SAN com- ponents make such a solution more expensive and involve additional management costs. The addressing of the media changer over a second LUN of the drive controller has a further major advantage in addition to the low costs. Normally, several drives are ﬁtted in a large library. Additional access paths make it possible to also control the media changer over the second LUN of a different drive controller. If a drive should fail, the media changer remains accessible. Furthermore, drives are often provided with a redundant power supply or the controllers possess an additional port, which can automatically be used if the ﬁrst path fails.

can be reached as an additional device with a different LUN over the SCSI port of the drive. Attached media changers form a unit with the drive. In order to move the media changer, special SCSI commands such as

READ ELEMENT STATUS ATTACHED and MOVE MEDIUM ATTACHED

must be used Proprietary interfaces In addition to SCSI interfaces, further interfaces have established themselves, particularly in the mainframe environment. These interfaces offer a higher level of abstraction than SCSI and often also a rudimentary management of the media. Typically, such interfaces are out-of-band, i.e. not accessible over the data path (SCSI or Fibre Channel connection), but instead over TCP/IP or RS-232 (Figure 9.2).

The commands that are exchanged over such interfaces are generally executed by a control unit that is ﬁtted in the library. This control unit can usually accept and execute commands from several applications at the same time, without this leading to conﬂicts. Likewise, additional services such as the management of scratch pools can be made available to all applications.

2008-04-02T04:50:00.001-07:00

Free Tutors on Management features of removable media Tape libraries and drives

In what follows we give an overview of the most important features and terms, plus a brief explanation:

Cartridge

A cartridge is a physical medium upon which storage space is available. A cartridge can be moved and has one or more sides. External cartridge label A label that is applied to the outside of a cartridge and serves to identify the cartridge, for example, a mechanically readable barcode. Internal cartridge label A dataset in a certain format at a certain position on the data carrier that serves to identify the cartridge. Side A physical part of a cartridge that provides storage space. A side contains one or more partitions. Tapes normally have only one side. DVDs and magneto-optical media

are also available in double-sided variants. Partition Part of a side that provides storage space as a physical unit of the cartridge.

Volume A volume is a logical data container. It serves to reserve storage space for applications on data carriers. A partition can hold as many volumes as desired. Please note that the term volume may have different meanings depending on the context it is used: in terms of the SNIA Shared Storage Model (Chapter 10) a tape volume is called a tape extent and may span multiple physical tape cartridges. In terms of back-up

software and mainframes a volume is often used synonymously with cartridge. Scratch tape A new tape without any content or a tape, the content of which is no longer of interest, and the entire storage capacity of which can be used for new purposes. Access handle An identiﬁer that an application can use to access the data of a volume. Under UNIX operating systems an access handle is equivalent to the name of a device special ﬁle (for example: /dev/rmt0). Mount request The command to place a certain cartridge in a drive. Audit trail Audit trails consist of a series of data sets, which describe the processes that have been performed by a computer system. Audit trails are used primarily in security-critical ﬁelds in order to record and check access to data. As already mentioned, a system for the management of removable media should also be able to represent the logical and physical properties and features of a cartridge. Ideally a cartridge can consist of as many sides as desired (tapes generally have only one side, optical media often have two, holographic media could, at least theoretically, provide even more sides). Each side can hold one or more partitions, and any desired number of volumes can be allocated to each partition.

LIBRARIES AND DRIVES

Operating systems and applications that use removable data carriers must be able to deal with a large amount of different library hardware. In general, libraries possess a media changer, slots to accept cartridges, and drives with which the cartridges can be read and written. The media changer takes cartridges from the slots and transports them to the drives. In automatic libraries, the media changer can be controlled via an interface. This interface can be realized in the form of an in-band interface (e.g. SCSI) or an out- band interface depending upon the device (cf. Section 9.3.3). The bandwidth of these automatic libraries ranges from individual small autoloaders with 1–2 drives and a few slots through to large automatic tape libraries, in which one or more media changers can transport thousands of cartridges in dozens or possibly even hundreds of drives.

In addition to automatic libraries, a removable media management system should also consider manually operated libraries. In these, an operator takes on the function of the media changer and inserts the cartridges in the drives accordingly. It is thus possible to include even individual (standalone) drives in the system as a whole. Depending upon the level of abstraction, a shelf or a safe ﬁlled with cartridges and without any drives can also be viewed as a library. These libraries are also called vaults or vaulting locations. Particularly if both automatic libraries and vaults are used, it is wise to choose the level of abstraction for the management of the media so that vaults can also be handled like manual libraries without drives. This means that, for all libraries of whatever type, the same procedures can be applied for auditing (the requesting of all components, particularly the cartridges of a library), export (the removal of a cartridge from a less library) and import (the insertion of a cartridge into a library). In what follows we will consider the individual components of libraries.

Drives

Like hard drives, drives for removable media are currently equipped with a SCSI or Fibre Channel interface in the Open Systems environment and are connected to the storage network via these. In the mainframe ﬁeld, ESCON and FICON are dominant. As already mentioned, tape drives in particular can only work at full speed if they read and write many blocks one after the other (streaming). Although it is possible, and in the mainframe environment totally normal, to write individual ﬁles consisting of just one logical block to tape, or to read them from tape, different drives are necessary for this than those used in the Open Systems back-up operation. These enterprise drives have larger motors and can position the read-write heads signiﬁcantly more quickly and precisely over a certain logical block.

Media changers

Media changers have the job of transporting cartridges within a library. The start and end of a transport operation can either be a slot or a drive. To this end, a library has an inventory in which all elements of the library and their attributes are noted. The media changer has access to this inventory. Like drives, media changers have an interface for the control and checking of their functions. It is normal to use this interface for requesting data from the inventory, as well as for controlling the transport operations. The following information can therefore be requested via the media changer interface:

• the number of drives and their properties (addresses, type, etc.);

• the number of slots and their properties;

• the number of cartridges and their properties (slot, label, etc.);

• the number of further media changers and their properties.

2008-04-02T03:43:00.001-07:00

Learn more about Removable Media Management THE SIGNIFICANCE OF REMOVABLE MEDIA LIKE TAPE CD, DVD and magneto-optical media

Removable media is a central component of the storage architectures of large data centers The use of storage networks means that several servers – and thus various different applications – can now use media and libraries jointly. The management of removable media in storage networks is therefore becoming increasingly important. Hence this chapter describes the network based virtualization of tape libraries and other removable media resources. In the following section we ﬁrst of all explain why, in spite of the ever-increasing capacity of hard disks and intelligent disk subsystems, removable media is indispensable (Section 9.1). Then we consider various types of removable media (Section 9.2) and libraries (Section 9.3), giving special consideration to media management. We then discuss the problems and requirements related to the management of removable media (Section 9.4). Finally, we introduce the IEEE 1244 Standard for Removable Media Management – an approach that describes both the architecture of a system for the management of removable media and also its communication with applications.

The significance of removable media

Articles with such titles as 'Tapes Have No Future' or 'Is Tape Dead?' keep appearing in the press. Some storage manufacturers proclaimed the end of tapes as early as twenty years ago. Then it was to have been all over in the last few years ... After all, they said, hard disks (e.g. serial ATA disks) have now become so cheap that it is unnecessary to move data to other data carriers. In our opinion (in 2003), removable media is, and will remain, an important building block in the storage architectures of data center In addition to their high capacity and low price, for many companies the fact that

removable media can be stored separately from read and write devices and thus withdrawn from direct access is particularly relevant. Viruses, worms and other 'animals' are thus denied the possibility of propagating themselves uncontrollably, as they could on storage that is continuously available online. Furthermore, with removable media a very large quantity of data can be stored in a very small and possibly well-protected area at low storage costs. WORM (Write Once Read Multiple) properties, which are now available not only for optical media but also for magnetic tapes, additionally increase security. Furthermore, the requirement for storage capacity is increasing continuously. Progress in storage density and the capacity of cartridges can scarcely keep up with the ever-growing requirement, which means that the number of cartridges is also growing continuously. For the ﬁlm The Lord of the Rings alone, 160 computer animators generated and edited a data volume of one terabyte every day, which was stored to tape. At the end of the three-year production period, the digital material for the ﬁnal version of the ﬁlm – 150 terabytes in size – was stored on tape. In the scientiﬁc ﬁeld, and also in the ﬁeld of medicine and bioinformation, data volumes in the petabyte range have been handled for a long time. This immense requirement for storage space cannot be provided exclusively in the form of storage that is available online, such as hard disks. Power consumption, heat and space requirements would drive the costs of this so high that this type of storage could not currently be justiﬁed by the shorter access time. For example, the power consumption of an average 120 GB S-ATA drive is currently (mid-2003) approximately 13 Watts. An installation with 400 terabytes of storage thus has a power consumption of more than 42 kW! This corresponds with approximately the average power consumption for 120 german single family homes. A further important advantage of removable media in comparison to hard disks is their robustness. They are less sensitive to impact and a service life of up to 30 years is possible for media in the ﬁeld of high-end tapes stored in the correct manner.

REMOVABLE MEDIA

Various types of removable media are currently in use. These are primarily magnetic tapes (Section 9.2.1), optical media such as CDs and DVDs and magneto-optical media (Section 9.2.2). In these sections we are primarily interested in how the special properties of the various media types should be taken into consideration in the management of removable media.

Tapes

Tapes have ﬁrmly established themselves as a back-up and archiving medium for large data quantities due to their very low costs per megabyte storage space in comparison to other media. However, tapes can only be accessed sequentially. The position of the head of a tape drive cannot, therefore, be chosen at will, but must be determined by the appropriate fast-forwarding and rewinding of the tape. This movement of the tape costs

signiﬁcantly more time than the movement of the head of a hard disk drive and an optimal speed can, therefore, only be achieved if as many associated data blocks as possible are read and written one after the other, i.e. sequentially. Access to back-up and archive data at will is often unnecessary. The speed at which the large quantities of data can be backed up and restored is likely to be a signiﬁcantly more important factor than the random access to individual ﬁles. Back-up and archiving applications available today utilize this special property of tapes by aggregating the data to be backed up into a stream of blocks and then writing these blocks onto tapes sequentially (cf. Section 9.3.1). Such programs use an internal management system to ensure that they are capable of identifying at any time both the tape on which a ﬁle or database is saved and also the position of the start of the ﬁle or database on the tape (cf. Section 7.3.4) Today (end of 2003), the term 'removable media' is primarily used to refer to tapes and tape libraries. In almost all large data centres, tape libraries with several drives and a great many tape cartridges are used. They are currently the most commonly used medium for back-up and archiving purposes. Therefore, systems for the management of removable media are currently used primarily where tapes have to be managed and tape libraries have to be controlled. Nevertheless, in view of future developments of new storage technologies, current management systems should also support media that possess several sides and several partitions.

CD, DVD and magneto-optical media

When writing to CDs, DVDs and magneto-optical media, a ﬁle system (e.g. ISO-9660) is generally applied. When writing to these media, the same limitations apply as for tapes, since only one application can write to the data carrier at any one time. Normally, this application also writes a large portion – if not the whole – of the available storage space. However, once these data carriers have been written, applications can access them like

hard disk drives. As a result, the applications have available to them the full support of the operating system for the read access to optical media, which is why in this case they behave like write-protected hard disks and can be shared accordingly. Magneto-optical media are generally readable and writeable on both sides. Depending upon the drive, the cartridge may have to be turned over in order to access the second side. This property makes it necessary for management systems to be able to manage a second side of a cartridge and control the changing mechanism so that

the cartridge can be turned over. Furthermore, the WORM properties must be suitably represented for these data carriers.

2008-04-02T02:45:00.001-07:00

TUTORS ON OPERATIONAL ASPECTS OF THE MANAGEMENTOF STORAGE NETWORKS AND Basic of CMIP and DMI

For the sake of completeness we will now list two further protocol standards for out-band management: the Common Management Information Protocol (CMIP) and the Desktop Management Interface (DMI). However, neither protocol has as yet made any inroads into storage networks and they have up until now been used exclusively for the monitoring of servers.

CMIP

At the end of the 1980s the Common Management Information Protocol (CMIP) was originally developed as a successor of SNMP and, together with the Common Management Information Services (CMIS), it forms part of the Open Systems Interconnect Speciﬁcation (OSI). Due to its complexity it is, however, very difﬁcult to program and for this reason is not widespread today. CMIP uses the same basic architecture as SNMP. The management information is also held in variables similar to the MIBs. However, in contrast to the MIBs in SNMP, variables in CMIP are comparatively complex data structures. Like SNMP, CMIP provides corresponding operations for the reading and changing of variables and also incorporates messaging by means of traps. In addition, actions can be deﬁned in CMIP that are triggered by the value change of a variable. CMIP has the advantage over SNMP that it has a proper authentication mechanism. The disadvantage of CMIP is that it is very resource-hungry during operation, both on the NMS side and also at the managed device.

DMI

The Desktop Management Interface was also speciﬁed by the DMTF. It describes a mechanism by means of which management information can be sent to a management system over a network. The architecture of the DMI consists of a service layer, a database in the management information format (MIF), a management interface (MI) and a component interface (Figure 8.12). The service layer serves to exchange information between

the managed servers and a management system. All properties of a managed server are stored in the MIF database. A DMI-capable management system can access a server and its components via the management interface. The component information is provided to the management interface by component interfaces. DMI thus provides an open standard for the management of servers, but is nowhere near as widespread as SNMP.

OPERATIONAL ASPECTS OF THE MANAGEMENT OF STORAGE NETWORKS

In large heterogeneous environments the introduction of a management system appears indispensable for those wishing to take control of management costs and make full use of the storage network. For small environments, the implementation of a management system is recommended if the environment is expected to grow strongly in the medium term. Entry in a small environment offers the additional advantage that the management system grows with the environment and you have plenty of time to get used to the product in question. If the storage network reaches its critical size at a later date you will already be better prepared for the more difﬁcult management. Because, by this time, the installed

tools will already be well known, the optimal beneﬁt can be drawn from them. If you have the choice between standardized or proprietary mechanisms, then you should go for standardized mechanisms. Many device manufacturers have already built support for the standards in question into their products. Other manufacturers will follow this example. When purchasing new devices, the devices' support for standards is a critical

selection criterion and should thus be checked in advance. When choosing a management system you should ensure corresponding support for the various standards. It should, however, also have interfaces for the support of proprietary mechanisms. The calling up of element managers from the management console is the minimum requirement here. Only thus can many older devices be integrated into the management system. Which strategies should a management system use: in-band or out-band? This question cannot be answered in a straightforward manner since the success of a management system depends to a large degree upon the available interfaces and mechanisms of the devices used. In general, however, the following advantages and disadvantages of in-band and out-band management can be worked out. The main advantage of the use of the in-band interface is that it is available as standard in the storage network. By the use of various protocol levels (transport and ULP) a great deal of detailed information about the storage network can be read. In environments where an additional out-band interface is not available or cannot be implemented, in-band monitoring may represent the only option for monitoring the status of devices. The great disadvantage of in-band management is that a management agent connected to the storage network is required because the in-band management functions can only be

used through such an agent. This can give rise to additional costs and sometimes increase the complexity of the storage network. On the developer side of management systems this naturally also increases the development and testing cost for suitable agent software. Agent software can be used for additional services. These can be operating system-speciﬁc functions or more extensive functions such as a storage virtualization integrated

into the management system (Chapter 5). When using management agents in Fibre Channel SAN, it should be noted that they are subject to the zoning problem. This can be remedied by the measures described or – where already available – by more recent options such as fabric device management interfaces. Out-band management has the advantage that it is not bound to the storage net- work infrastructure and technology in question. This dispenses with the necessity of support for the management system by in-band protocols such as Fibre Channel or iSCSI. Furthermore, abstract data models can be implemented with SNMP-MIBs and CIM that are independent of the infrastructure. These must, however, be supplemented by infrastructure-speciﬁc data models. The fabric element MIB and the Fibre Channel management MIB are two examples in the ﬁeld of SNMP. A further advantage of the use of the out-band interface is that no dedicated management agent is required in order to gain access to management functions. A management system can communicate directly with the SNMP agents and CIM providers in question without

having to make the detour via a management agent. In the event of problems with the management interface, this source of errors can be ruled out in advance. The corresponding costs and administrative effort associated with the management agent are therefore also not incurred.

A great disadvantage of out-band management is that up until now there has been no access to the operational services that are available in-band. Although this would be technically possible it has not yet been implemented. Finally, even the additional interface that is required can prevent the use of out-band management in cases where the implementation of a further interface is not possible or not desirable. Therefore, a management system should, where possible, support both interfaces in order to get the best of both worlds. This means that it is also capable of integrating devices that only have one of the two interfaces. For devices that have access to both in-band and out-band interfaces an additional connection can be very helpful, particularly for error isolation. If, for example, the in-band connection to a device has failed, then an in-band management system would report both the failure of the line and also the failure of the device. A management system that operates both interfaces would still be able to reach the device out-band and thus trace the error to the failed connection.

SUMMARY

In this chapter we have dealt with the requirements and the possibilities that exist for the management of storage networks. The realization of these comprehensive requirements in a management system implies a complexity that should not be underestimated. In general, storage networks impose higher requirements on management than do storage-centric IT architectures. The objective of effective storage management is the use of a central management system that is capable of integrating both the proprietary and the standardized management mechanisms of the individual devices. Both in-band and out-band management mechanisms are offered. We discussed the Fibre Channel Generic Services for the in-band management of a Fibre Channel SAN. Out-band, standardized management mechanisms are available in the form of the important protocols SNMP, CIM/WBEM

and SMI-S. Today (end of 2003) SNMP is an important protocol for the management of storage networks. As development progresses and the power of CIM/WBEM and SMI-S increases, the SNMP MIBs may gradually be forced from the market. Suitable extensively implemented CIM providers could then also supersede the in-band interface. From an operational point of view it is to be hoped that, on the one hand, the management techniques continue to be further developed in an open and standardized – and thus interoperable – manner and, on the other, that even more manufacturers feel obliged to establish corresponding standards for their devices. In the next chapter we continue the discussion of storage management: Removable media and large tape libraries are central components of large data centres and we have not covered them so far. Thus the next chapter deals with the management of removable media.

2008-04-02T02:26:00.000-07:00

Learn more About the WBEM architecture Storage Management Initiative Speciﬁcation (SMI-S)

WBEM describes the architecture of a WBEM management system using the three pillars CIM, xmlCIM and CIM operations over HTTP. To this end, WBEM deﬁnes the following elements (Figure 8.10):

• CIM managed object The object to be managed is called a CIM managed object. This can, for example, be

a storage device or an application.

• CIM provider The CIM provider supplies the management data of a managed object. In the terminology of CIM this means that it provides the instances of the object that are deﬁned in the CIM model for the managed device. The interface between CIM provider and CIM managed object is not described by WBEM, with this interface being the starting point for the integration of other protocols such as SNMP.

• CIMOM – CIM object manager The CIM object manager implements the CIM repository and provides interfaces for CIM provider and CIM clients (e.g. central management applications). The speciﬁcation of the interfaces between CIM provider and CIMOM is also not part of WBEM, so manufacturer-speciﬁc mechanisms are used here too.

• CIM repository The CIM repository contains templates for CIM models and object instances.

• CIM client The CIM client corresponds to a management system. The CIM client contacts the CIMOM in order to recognize managed objects and to receive management data from the CIM provider. The communication between CIM client and CIMOM is based upon the techniques xmlCIM and CIM operations over HTTP described above. These should facilitate interoperability between CIM clients and CIMOMs of different manufacturers.

The CIM speciﬁcation also provides a mechanism for sounding an alert in case of a state change of an object. The recognition of a state change such as creation, deletion, update or access of a class instance is called a Trigger. A Trigger will result in the creation of a short-living object called Indication which is used to communicate the state change information to a CIM client through the WBEM architecture. For that the CIM client needs to subscribe for indications with the CIMOM previously. The use of CIM and WBEM for the management of storage networks In the past, WBEM/CIM has proved itself useful in the management of homogeneous environments. For example, the management of Windows servers is based upon these

techniques. Storage networks, on the other hand, contain components from very different manufacturers. Experience from the past has shown that WBEM and CIM alone are not sufﬁcient to guarantee the interoperability between CIM clients, i.e. management systems, and CIMOMs in the ﬁeld of storage networks. In the next section we will introduce Blueﬁn, a technique that aims to ﬁll this gap.

Storage Management Initiative Speciﬁcation (SMI-S)

At the start of this chapter we sketched out the necessity of a central management system from which all components of a storage network can be managed from different points of view. The so-called Blueﬁn initiative aimed to close the gaps of WBEM/CIM with the objective of deﬁning an open and manufacturer-neutral interface (API) for the discovery, monitoring and conﬁguration of storage networks. Blueﬁn thus aimed to deﬁne

a standardized management interface for heterogeneous storage networks, so that they can be managed in a consistent manner. The Blueﬁn initiative was set up in 2001 by a group of manufacturers under the name of Partner Development Process (PDP). Since August 2002 the further development of Blueﬁn has been in the hands of the SNIA. The standardization itself is now being driven forward by the SNIA Storage Management Initiative (SMI) under the name Storage Management Initiative Speciﬁcation (SMI-S). SMI-S is based upon the WBEM architecture and expands this in two directions. First, it reﬁnes the classes of the CIM Common Schemas to include classes for the management of storage networks. For example, it introduces classes for host, fabric, LUN, zoning, etc. Second, it extends the WBEM architecture by two new services: the Directory Manager and the Lock Manager (Figure 8.11). The Directory Manager aims to simplify discovery in a storage network. SMI-S also deﬁnes how resources (physical and virtual) report to the Directory Manager by means of the Service Location Protocol (SLP, IETF standard since 1997), so that management systems can interrogate the resources in a storage network through a central service, i.e. the Directory Manager. The Lock Manager aims to assist the concurrent access to resources from different management applications in a storage network. Access to CIM objects can be protected by locks, so that a transaction model can be implemented. To this end, SMI-S-compliant management systems must demand the corresponding rights from the Lock Manager in the role of the lock management client, in order to be allowed to call up the protected methods. The SMI is a working group that co-ordinates the various subgroups of the SNIA that are working on the standardization of various aspects of storage networks. In the opinion of SNIA around 70% of the required functions for the management of a storage network have already been passed. The current standard (end of 2003) primarily describes the components of a Fibre Channel SAN. SMI-S still has to be expanded for techniques such as NAS and iSCSI. Reference implementations already exist for the Directory Manager, but there are still none for the Lock Manager. As the next step, the CIM classes

that are still lacking must be deﬁned and the interoperability of the implementations from different manufacturers tested. Therefore SNIA has established the Interoperability Conformance Testing Program (ICTP), which provides unique test standards for all participating products to demonstrate proven interoperability and standard compliance.

2008-04-02T02:20:00.001-07:00

Tutor on Use of SNMP SNMP architecture for Free and CIM and WBEM

The Simple Network Management Protocol (SNMP) was ratiﬁed by the Internet Engineering Task Force (IETF) and was originally a standard for the management of IP networks. Although there are, even now, protocols for this ﬁeld that can be better adapted to the devices to be managed, SNMP is still the most frequently used protocol due to its simple architecture. Perhaps this is also the reason why SNMP has gained such great importance

in the ﬁeld of storage networks. Management information bases In SNMP, management information is organized into so-called management information bases (MIBs). An MIB is a collection of so-called managed objects. Managed objects are represented by variables. Since an MIB can also exist as precisely one managed object, managed objects are also called MIB objects or even just MIB. In this manner a managed object is identiﬁed with its MIB. We differentiate between two types of managed objects:

• Scalar objects Scalar objects deﬁne precisely one object instance.

• Tabular objects Tabular objects bring together several related object instances to form a so-called MIB table. All the MIBs on the market can be divided into two groups:

• Standard MIBs General management functions of certain device classes are covered by standard MIBs.

• Private or enterprise MIBs Private or so-called enterprise MIBs permit individual companies to develop their own MIBs. Management functions can thus be offered that are specially tailored to individual devices and extend beyond the functions of the standard MIBs. In order to differentiate between the individual managed objects there is an MIB hierarchy with a tree structure (Figure 8.4). The various standardization organizations form

the top level of the tree. From there, the tree branches to the individual standards of this organization and then to the actual objects, which form the leaves of the hierarchy tree. In this manner an individual MIB object can be clearly deﬁned by means of its position within the MIB hierarchy. In addition, each managed object is given a unique identiﬁcation number, the so-called object identiﬁer. The object identiﬁer is a sequence of digits

that are separated by points. Each individual digit stands for a branch in the MIB tree and each point for a junction. The full object identiﬁer describes the route from the root to the MIB object in question. For example, all MIB objects deﬁned by the IBM Corporation hang under the branch 1.3.6.1.4.1.2 or in words iso.org.dod.internet.private.enterprises.ibm (Figure 8.4). Thus all object identiﬁers of the MIB objects that have been deﬁned by IBM Corporation begin with this sequence of numbers.

SNMP architecture

SNMP deﬁnes three components (Figure 8.5): • Managed device

A managed device is a connection device or an end device that carries an SNMP agent. Managed devices collect and save their management data in an MIB.

• SNMP agent The SNMP agent is a software module that runs on a managed device. Its task is to translate the MIB information collected by the managed device into an SNMP- compatible form upon request.

• Network management system (NMS) An application for the monitoring and conﬁguration of various managed devices runs on the NMS. If the NMS knows the MIB of the device to be managed, then it can interrogate or change individual MIB objects by appropriate requests to the SNMP agent. The information regarding the MIB in question is loaded into the NMS in advance by means of a so-called MIB ﬁle. In our context the NMS corresponds with the management system. However, even the Syslog-Daemon of a Unix system can be

used as an NMS. SNMP operations SNMP deﬁnes four operations for the monitoring and conﬁguration of managed devices:

• Get Get is used by NMS in order to request the values of one or more MIB object instances from an agent.

• GetNext GetNext allows the NMS to request the next value of an object instance within an MIB table from an agent after a prior Get request.

• Set Set allows the NMS to set the value of an object instance.

• Trap The Trap operation allows the SNMP agent to inform the NMS independently about value changes of object instances. Security with SNMP

SNMP has no secure authentication options. Only so-called community names are issued. Each NMS and each SNMP agent is allocated such a community name. The allocation of community names creates individual administrative domains. Two communication partners (an NMS and an SNMP agent) may only talk to each other if they have the same community name. The most frequently used community name is 'public'.

If, for example, an NMS makes a Set request of an SNMP agent, then it sends its community name with it. If the community name of the NMS corresponds with that of the SNMP agent, then this performs the Set operation. Otherwise it is rejected. Thus anyone who knows the community name can make changes to the values of an object instance. This is one reason why many providers of SNMP-capable devices avoid the

implementation of Set operations on their SNMP agent, because community names only represent a weak form of authentication. In addition, they are transmitted over the network unencrypted. Application in storage networks An SNMP agent can be installed upon the various devices such as servers, storage devices or connection devices. SNMP thus covers the entire range of devices of a storage network.

• Discovery A management system can address the SNMP agent on a connection device in order to interrogate the properties of a device and to obtain information from it about the connected devices. It thus also gets to know the immediate neighbours and with that information can continue scanning the end devices insofar as all devices lying on this route support SNMP. In this manner a management system ﬁnally obtains the topology

of a storage network.

• Monitoring SNMP also supports the management system in the monitoring of the storage network. The SNMP agents of the end nodes can be addressed in order to ask for device-speciﬁc status information. Corresponding error and performance statistics can thus be requested

from the SNMP agent of a connection node, for example, from a Fibre Channel switch.

• Messages Due to the Trap operation SNMP is also familiar with the concept of messages. In SNMP jargon these are called traps. In this manner an SNMP agent on a device in the storage network can send the management system information via IP if, for example, the status has changed. To achieve this only the IP address of the so-called trap recipient has to be registered on the SNMP agent. In our case, the trap recipient would be the

management system. In contrast to the RSCN, in the in-band management of a Fibre Channel SAN, in which all registered nodes are informed about changes to a device by means of a message, an SNMP trap only reaches the trap recipients registered in the SNMP agent. In addition, the connection-free User Datagram Protocol (UDP) is used for sending a trap, which does not guarantee the message delivery to the desired recipient.

• Conﬁguration SNMP also offers the option of changing the conﬁguration of devices. If the device MIB is known, this can be performed by changing the value of the MIB variables on a managed device by means of the Set operation. Standard MIBs for Fibre Channel SAN There are two important standard MIBs for the management of a Fibre Channel SAN:

• Fabric element MIB This standard MIB developed by the Storage Networking Industry Association (SNIA) is specialized for Fibre Channel switches and supplies detailed information on port states and port statistics. Likewise, connection information can be read over this.

• Fibre Channel management MIB This MIB was developed by the Fibre Alliance. It can be used to request connection information, information on the device conﬁguration or the status of a device. Access to the fabric name server and thus the collection of topology information is

also possible.

CIM and WBEM

Today, numerous techniques and protocols are used for system management that, when taken together, are very difﬁcult to integrate into a single, central management system. Therefore, numerous tools are currently used for system management that all address only a subsection of the system management. Web Based Enterprise Management (WBEM) is an initiative by the Distributed Management Task Force (DMTF), the aim of which is to make possible the management of the entire IT infrastructure of a company (Figure 8.6). WBEM uses web techniques such as XML and HTTP to access and represent management information. Furthermore, it deﬁnes interfaces for integrating conventional techniques such

as SNMP. WBEM deﬁnes three columns that standardize the interfaces between resources and management tools (Figure 8.7):

• Common Information Model (CIM) The Common Information Model (CIM) deﬁnes an object-oriented model that can describe all aspects of system management. It is left up to the components participating in a WBEM environment how they realize this model, for example in C++ or in

Java. The only important thing is that they provide the semantics of the model, i.e. provide the deﬁned classes and objects plus the corresponding methods outwards to other components.

• xmlCIM Encoding Speciﬁcation The xmlCIM Encoding Speciﬁcation describes the transfer syntax in a WBEM environment. It thus deﬁnes precisely the XML formats in which method calls of the CIMobjects and the corresponding returned results are encoded and transmitted. As a result, it is pos-

sible for two components to communicate with each other in the WBEM architecture, regardless of how they locally implement the CIM classes and CIM objects.

• CIM operations over HTTP Finally, CIM operations over HTTP provide the transport mechanism in a WBEM environment that makes it possible for two components to send messages encoded in xmlCIM back and forth. This makes it possible to call up methods of CIM objects that are located on a different component. Common Information Model (CIM) CIM itself is a method of describing management data for systems, applications net-

works, devices, etc. CIM is based upon the concept of object-oriented modelling (OOM). Understanding CIM requires knowledge of OOM. OOM is based upon the concept of object-oriented programming. However, OOM is not a programming language, but a formal modelling language for the description of circumstances of the real world on abstract level. In OOM, real existing objects are represented by means of instances. An instance has certain properties, which are called attributes, and allows for execution of speciﬁc actions, which are called methods. A class in OOM is the abstract description of an instance, i.e. it is the instance type. To illustrate: a Porsche Cabriolet in a car park is an object of the real world and is represented in OOM as an object instance. A Porsche Cabriolet is of the type car, thus it belongs to the class car. Thus, from an abstract point of view a Porsche Cabriolet – just like a BMW, Mercedes, etc. – is nothing more than a car. We hope that Porsche fans will forgive us for this comparison!

Classes can have subclasses. Subclasses inherit the attributes and methods of the parent class (Figure 8.8). Examples of subclasses of the class car are sports cars or convertibles. An inherited property in this case is that they have a chassis and four wheels. As we see, OOM can also be used to great effect for the description of non-computer- related circumstances. In order to describe complex states of affairs between several classes, a further construct is required in OOM: the association (Figure 8.8). An association is a class that contains two or more references to other classes. In that way it represents a relationship between two or more objects. Such relationships exist between individual classes and can themselves possess properties. Let us consider the class 'person'. In the language of OOM 'man' and 'woman' are subclasses of the parent class 'person'. A relationship between the class 'man' and the class 'woman' could be 'marriage'. A property of this relationship would be the date of the wedding, for example. Complex management environments can be described in abstract terms with the aid of the class and relationship constructs. An abstract description of a management environment using OOM is called a schema in CIM. CIM has three different types of schema: the Core Schema (also known as Core Model), the Common Schema (also known as Common Model) and further Extension Schemas The basis of the CIM model is the Core Schema. The Common Schema supplies abstract classes and relationships on the basis of the Core Schema for all those components that the different management environments have in common. By means of Extension Schemas the abstract basic Core Schema and Common Schema can be

concretized and expanded The Core Schema deﬁnes the abstract classes and their relationships necessary for the description and analysis of complex management environments. The Core Schema speciﬁes the basic vocabulary for management environments. For example, the Core Schema speciﬁes that in a management environment elements to be managed exist, which themselves have logical and physical components. A strict differentiation is always made between logical and physical units or properties. Systems, applications or net- works represent such elements to be managed and can be realized in CIM as extensions of this Core Schema. The Core Schema thus yields the conceptual template for

all extensions. The Common Schema builds upon the Core Schema to supply the abstract classes and relationships for all those components that the different management environments have in common, regardless of the underlying techniques or implementations. The abstract classes of the Common Schema all arise by inheritance from the classes of the Core Schema. The Common Schema deﬁnes the following submodels:

• System model The system model brings together all objects that belong in a management environment.

• Device model The device model represents the logical units of such a system that provide the system functions on a purely logical level in order to remain independent of the physical implementation. This makes sense because physical aspects of devices change but their importance on a logical level generally remains the same for the management of a system as a whole. For example, a Fibre Channel switch differs from an iSCSI switch

only in terms of its physical properties. The logical property of acting as a connection node to connect other nodes in the network together is ommon to both components.

• Application model The application model deﬁnes the aspects that are required for the management of applications. The model is designed so that it can be used to describe both single location applications and also complex distributed software.

• Network model The network model describes the components of the network environment such as topology, connections and the various services and protocols that are required for the operation of and access to a network.

• Physical model The physical model makes it possible to also describe the physical properties of a management environment. However, these are of low importance for the management, since the important aspects for the management are on a logical and not a physical level. Extension Schemas permit the abstract basic Core Schema and Common Schema to be further concretized and expanded. The classes of the Extension Schema must be formed by inheritance from the classes of the Core and Common Schemas and the rules of the CIM model have to be applied. In this manner it is possible to use CIM for the description of the extremely different management data of the components of a storage network, which can be used for management by means of the following WBEM architecture.

2008-04-01T23:27:00.001-07:00

Free tutors on Out Band Management

Out-band management goes through a different interface than the interface used by data trafﬁc. In Fibre Channel SANs, for example, most devices have a separate IP interface for connection to the LAN, over which they offer management functions (Figure 8.3). For out-band management an IP connection must exist between the computer of the central management system and the device to be managed. For security reasons it can be a good idea to set up a separate LAN for the management of the storage network in addition to the conventional LAN for the data transfer. The protocol that is currently (end of 2003) most frequently used for out-band man- agement is the Simple Network Management Protocol (SNMP, Section 8.7.1). In addition there are more recent developments such as the Common Information Model (CIM) and the Web Based Enterprise Management (WBEM), which can be used instead of SNMP (Section 8.7.2). Finally, SMI-S represents a further development of WBEM and CIM that is specially tailored to the management of storage networks (Section 8.7.3). Furthermore, there are other protocols such as CMIP and DMI that specialize in server monitoring

2008-04-01T21:18:00.001-07:00

Know more About The zoning problem

The identiﬁcation and monitoring of a node in the Fibre Channel SAN usually fail if this is located in a different zone than the management agent since in this situation direct access is no longer permitted. This problem can be rectiﬁed by the setting up of special management zones, the placing of a management agent in several zones or the placing of further management agents. The Fibre Channel protocol, however, has an even more elegant method of solving this problem. It deﬁnes services that permit the collection or entering of information. A management agent is capable of requesting the necessary information about these services, since these are not affected by the zoning problem. Two examples are:

• Platform registration This service is offered by the fabric conﬁguration server and makes it possible for an end node, such as a server or a storage device, to enter information about itself in the conﬁguration server.

• End point node information This is a new proposal for the Fibre Channel protocol, which was entered under the name of fabric device management interface and which allows the fabric itself to collect information about the properties and state statistics via ports.

2008-04-01T21:16:00.003-07:00

Free Tutors On In-band management in Fibre Channel SAN Discovery, Massages and Monitoring

The Fibre Channel Methodologies for Interconnects (FC-MI) and Fibre Channel Generic Services 4 (FC-GS-4) standards deﬁned by the American National Standards Institute (ANSI) form the basis for the in-band management in the Fibre Channel SAN. The FC-MI standard describes general methods to guarantee interoperability between various devices. In particular, this deﬁnes the prerequisites that a device must fulﬁl for in-band management. The FC-GS-4 standard deﬁnes management services that are made

available over the so-called Common Transport Interface of the Fibre Channel protocol. Services for management There are two Fibre Channel services that are important to Fibre Channel SAN management: the directory service and the management service. Each service deﬁnes one or more so-called servers. In general, these servers – split into individual components – are implemented in distributed form via the individual connection nodes of a Fibre Channel SAN but are available as one single logical unit. If an individual component cannot answer a management query, then the query is forwarded to a different server component on a

different node. This implementation is comparable to the Domain Name Services (DNS) that we know from IP networks.

The Fibre Channel standard deﬁnes, amongst other things, the following servers that are of interest for the management of storage networks:

• Name server The name server is deﬁned by the directory service. It is an example of an operational service. Its beneﬁt for a management system is that it reads out connection information and the Fibre Channel speciﬁc properties of a port (node name, port type).

• Conﬁguration server The conﬁguration server belongs to the class of management-speciﬁc services. It is provided by the management service. It allows a management system to recognize the topology of a Fibre Channel SAN.

• Zone server The zone server performs both an operational and an administrative task. It permits the zones of a Fibre Channel SAN fabric to be conﬁgured (operational) and recognized (management-speciﬁc). These services make it possible for a management system to recognize and conﬁgure the devices, the topology and the zones of the SAN.

Discovery

The conﬁguration server is used to identify devices in the Fibre Channel SAN and to recognize the topology. The so-called RNID function (Request Node Identiﬁcation Data) is also available to the management agent via its host bus adapter API, which it can use to request identiﬁcation information from a device in the Fibre Channel SAN. The RTIN function (Request Topology INformation) allows information to be called up about connected devices. Suitable chaining of these two functions ﬁnally permits a management system to rec- ognize the entire topology of the Fibre Channel SAN and to identify all devices and properties. If, for example, a device is also reachable out-band via a LAN connection, then its IP address can be requested in-band in the form of a so-called management address. This can then be used by the software for subsequent out-band management.

Monitoring

Since in-band access always facilitates communication with each node in a Fibre Channel SAN, it is simple to also request link and port state information. Performance data can also be determined in this manner. For example, a management agent can send a request to a node in the Fibre Channel SAN so that this transmits its counters for error, retry and trafﬁc. With the aid of this information, the performance and usage proﬁle of the Fibre Channel SAN can be derived. This type of monitoring requires no additional management entity on the nodes in question and also requires no out-band access to them. The FC-GS-4 standard also deﬁned extended functions that make it possible to call up state information and error statistics of other nodes. Two commands that realize the collection of port statistics are: RPS (Read Port Status Block) and RLS (Read Link Status Block).

Messages

In addition to the passive management functions described above, the Fibre Channel protocol also possesses active mechanisms such as the sending of messages, so-called events. Events are sent via the storage network in order to notify the other nodes of status changes of an individual node or a link. Thus, for example, in the occurrence of the failure of a link at a switch, a so-called

Registered State Change Notiﬁcation (RSCN) is sent as an event to all nodes that have registered for this service. This event can be received by a registered management agent and then transmitted to the management system.

2008-04-01T21:16:00.001-07:00

Free Tutors On Band Management

In-band management runs over the same interface as the one that connects devices to the storage network and over which normal data transfer takes place. This interface is thus available to every end device node and every connection node within the storage network. The management functions are implemented as services that are provided by the protocol in question via the nodes.

In-band services can be divided into the following two groups:

• Operational services Operational services serve to fulﬁl the actual tasks of the storage network such as making the connection and data transfer.

• Management-speciﬁc services Management-speciﬁc services supply the functions for discovery, monitoring and the conﬁguration of devices. However, not only the management-speciﬁc services are of interest from a management point of view. The operational services can also be used for system management. In order to be able to use in-band services, a so-called management agent is normally needed that is installed in the form of software upon a server connected to the storage network. This agent communicates with the local host bus adapter over an API in order to call up appropriate in-band management functions from an in-band management service For Fibre Channel, the Storage Networking Industry Association (SNIA) has already released the Fibre Channel Common HBA API, which gives a management agent easy and cross-platform access to in-band management services. The computer on which this management agent runs is also called the management agent. For a central management system, this either means that it acts as such a management agent itself and must be connected to the storage network or that within the

storage network there must be computers upon which – in addition to the actual applications – a management agent software is installed, since the in-band management services otherwise could not be used. If the management system uses such a decentral agent, then a communication between the central management console and the management agent must additionally be created so that the management information can travel from the agents to the central console (Figure 8.2). Normally this takes place over a LAN connection. Typically, the management agent is also used for more services than merely to provide access to the management services of the in-band protocol. Some possibilities are the collection of information about the operating system, about the ﬁle systems or about the applications of the server. This information can then also be called up via the central console of the management system. In the following we will explain which management functions can be realized with in-band management based upon the example of the in-band management of a Fibre Channel SAN.

2008-04-01T21:04:00.001-07:00

Learn more on Standardized mechanisms and Proprietary mechanisms

Standardized mechanisms

Many standardization organizations invest a great deal of development work in the standardization of management interfaces. For in-band management the developments occur on different protocol levels:

• In-band transport levels The management interfaces for Fibre Channel, TCP/IP and InﬁniBand are deﬁned on the in-band transport levels. In Section 8.6.1 we will discuss in detail the management interface of the transport levels of the Fibre Channel protocol.

• In-band upper layer protocols (ULP) Primarily SCSI variants such as Fibre Channel FCP and iSCSI are used as an upper layer protocol. SCSI has its own mechanisms for requesting device and status information: the so-called SCSI Enclosure Services (SES). In addition to the management functions

on transport levels a management system can also operate these ULP operations in order to identify an end device and request status information.

Special protocols such as SNMP (Section 8.7.1) and WBEM with CIM (Section 8.7.2) as well as SMI-S (Section 8.7.3) are used for the out-band management.

Proprietary mechanisms

The proprietary (this generally means manufacturer-speciﬁc and usually even device speciﬁc) mechanisms fall into three categories:

• APIs In this approach, individual devices have programming interfaces – so-called Application Programming Interfaces (APIs) – which they can use to call up special management functions. They are generally implemented out-band. It requires a considerable development and testing cost to be able to use these APIs in a central management system. The manufacturers of management systems must develop appropriate software

modules for various device APIs in order to access the management functions of each. This increases the complexity and the costs of such a management system. Some manufacturers do not shy away from this cost. The advantage of such an approach can be a device-near support of the management system. Thus such a management system can supply better results than one that only operates standardized mechanisms.

• Telnet Many devices can also be conﬁgured out-band via Telnet. Although Telnet itself is not a proprietary mechanism, it is subject to the same problems as an API regarding connection to a central management system. For this reason we will count it amongst the proprietary mechanisms.

• Element manager An element manager is a device-speciﬁc management interface. It is frequently found in the form of a GUI (graphical user interface) on a further device or in the form of a WUI (web user interface) implemented over a web server integrated in the device itself. Since the communication between element manager and device generally takes place via a separate channel next to the data channel, element managers are classiﬁed amongst the out-band management interfaces. Element managers have largely the same disadvantages in a large heterogeneous storage network as the proprietary APIs. How- ever, element managers can be more easily integrated into a central management system than can an API. To achieve this, the element manager only needs to know and call up the appropriate start routines. WUIs are started by means of the Internet browser.

To call up a GUI this must be installed upon the computer on which the management system runs. In that way element managers to a certain degree form the device-speciﬁc level in the software architecture of the management system. In the following we will look in more detail at the standardized mechanisms and how these are used for in-band (Section 8.6) and out-band management (Section 8.7). Let us begin with in-band management.

2008-04-01T20:54:00.000-07:00

Know more about Management Interfaces Storage networks STANDARDIZED AND PROPRIETARY MECHANISMS

Storage networks consist of two main types of devices • Connection devices Connection devices include the switches, hubs and bridges that are used to create the connections in the storage network.

• Endpoint devices Endpoint devices are the servers and storage devices that are connected to the connection devices. Devices – i.e. connection devices and endpoint devices – in a storage network are also called nodes because they seem to form the nodes of the network. Endpoint devices are

correspondingly called end nodes. The interfaces for the management of endpoint devices and connection devices are differentiated into in-band and out-band interfaces

• In-band All devices of a storage network have an in-band interface as standard. Devices are connected to the storage network via the in-band interface and data transfer takes place through this interface. In addition, management functions for discovery, monitoring and conﬁguration of connection devices and endpoint devices are made available on this interface. These are generally realized in the form of components of the current

protocol of the in-band interface. Thus, for example, in a Fibre Channel SAN the Fibre Channel protocol makes the appropriate in-band management functions available. The use of these services for the management of storage networks is then called in-band management.

• Out-band Most connection devices and complex endpoint devices possess one or more further interfaces in addition to the in-band interface. These are not directly connected to the storage network, but are available on a second, separate channel. In general, these are LAN connections and serial cable. This channel is not intended for data transport, but is provided exclusively for management purposes. This interface is therefore called out-of-band or out-band for short. Management functions are made available over this additional interface using a suitable protocol. Thus Fibre Channel SAN devices generally have an additional LAN interface and frequently possess a serial port in addition to their Fibre Channel ports to the storage network. The use of the management services that are provided by means of the out-band interface is called out-band management. In-band and out-band have their advantages and disadvantages. Depending upon the implementation environment, either an in-band or an out-band anagement approach will fulﬁl its purpose better. This depends primarily upon the interfaces that the devices in the storage network possess. In some storage network environments, however, even the use of an additional out-band interface is not possible for security reasons or because there is no LAN connection, for example. A management system can operate one or both interfaces at the same time. This requires that it uses the appropriate protocol for each interface. In the following we now wish to ﬁrst consider standardized and proprietary mechanisms (Section 8.5) and then consider in detail which mechanisms are available for in-band and out-band (Section 8.7) interfaces.

STANDARDIZED AND PROPRIETARY MECHANISMS

Standardized and proprietary mechanisms are used for the realization of management functions on in-band and out-band interfaces. Standardized mechanisms offer the advantage in relation to proprietary interfaces that management systems can address various devices via a uniﬁed interface. This means that the developer of a management system does not need to implement a proprietary mechanism in the software for each device, which is complicated and thus expensive. Proprietary mechanisms, on the other hand, have the advantage over standardized mechanisms that they can provide more management functions for a certain device and thus permit deeper and more device-speciﬁc management

interventions.

2008-04-01T20:29:00.001-07:00

FREE TUTOIRS SUPPORT BY MANAGEMENT SYSTEMS

As explained in the previous section, a storage network raises numerous questions for and imposes many requirements upon a management system. Many software manufacturers tackle individual aspects of this problem and offer various management systems that address the various problem areas. Some management systems concern themselves more with the commercial aspects, whereas others tend to concentrate upon administrative interests. Still other management systems specialize in certain components of the storage network such as applications, resources, data or the network itself. This results in numerous different system management products being used in a complex and heterogeneous storage network. The lack of a comprehensive management system increases the complexity of and the costs involved in system management. Analysts therefore assume that in addition to the simple costs for hardware and software of a storage network, up to ten times these costs will have to be spent upon its management. In this context it makes sense to develop a management system which makes it possible for as many aspects of the storage network as possible to be managed from a central point by means of a management console. In order to guarantee the management of all components, a management system must operate the existing interfaces of applications and resources and – where this is not possible – try to integrate existing management mechanisms into the central software or to re-establish them. In order to permit the full management of a storage network in daily operation, a management system should have the following ﬁve core components: • Discovery The task of a discovery component is to recognize the applications and resources used in the storage network. It collects information about the properties and the current conﬁguration of resources. Finally, it correlates and evaluates all gathered information and supplies the data for the representation of the network topology. • Monitoring The monitoring components are used to monitor the status of the applications and resources of the network. In the event of an application crash or the failure of a resource, it must take appropriate measures to raise the alert based upon the severity of the error that has occurred. The monitoring components operate error isolation by trying to ﬁnd the actual cause of the fault in the event of the failure of part of the storage network.

• Conﬁguration The conﬁguration of applications and resources can be changed by the conﬁguration components. They further make it possible to simulate in advance the effects of changing the conﬁguration.

• Analysis This allows trend analyses, in particular of the commercial aspects, to be called up. It also evaluates the availability and scalability requirements of the storage network. However, it can also be used to track down single points of failure within the storage network.

• Data control These components are concerned with all aspects of data such as performance, backups, archiving or migration and thus control the efﬁcient use and availability of data and resources. By using policies it allows the administrator to control the ﬂow and placement of data. In the following we want to deal in more detail with the mechanisms that are needed for the ﬁrst three core components: discovery, monitoring and conﬁguration. Due to the complexity of the methods used in the analysis component it is not possible to deal with these in detail. A model – storage virtualization – has already been introduced to deal with the requirements that crop up for the data control component

2008-04-01T07:29:00.000-07:00

LEARN MORE ON BASEIC REQUIREMENTS OF MANAGEMENT SYSTEMS

The management of storage networks is of different signiﬁcance to various technical ﬁelds For example, the classical network administrator is interested in the question of how the data should be transported and how it is possible to ensure that the transport functions correctly. Further aspects for him are the transmission capacity of the transport medium, redundancy of the data paths or the support for and operation of numerous protocols (Fibre Channel FCP, iSCSI, NFS, CIFS, etc.). In short: to a network administrator it is important how the data travels from A to B and not what happens to it when it ﬁnally arrives at its destination. This is where the ﬁeld of interest of a storage administrator begins. He is more interested

in the organization and storage of the data when it has arrived at its destination. He is concerned with the allocation of LUNs to the servers (LUN mapping) of intelligent storage systems or the RAID levels used. A storage administrator therefore assumes that the data has already arrived intact at point B and concerns himself with aspects of storage. The data transport in itself has no importance to him. An industrial economist, on the other hand, assumes that A, B and the route between them function correctly and concerns himself with the question of how long it takes for the individual devices to depreciate or when an investment in new hardware and software must be made. Abalancedmanagement systemmust ultimately live up to all these different requirements equally. It should cover the complete bandwidth from the start of the conceptual phase through the implementation of the storage network to its daily operation. Therefore, right from the conception of the storage network, appropriate measures should be put in place to subsequently make management easier in daily operation. A good way of taking into account all aspects of such a management system for

a storage network is to orientate ourselves with the requirements that the individual components of the storage network will impose upon a management system. These components include:

• Applications These include all software that processes data in a storage network.

• Data Data is the term used for all information that is processed by the applications, transported over the network and stored on storage resources.

• Resources The resources include all the hardware that is required for the storage and the transportf the data and the operation of applications.

• Network The term network is used to mean the connections between the individual resources. Diverse requirements can now be formulated for these individual components with regard to monitoring, availability, performance or scalability. Some of these are requirements such as monitoring that occur during the daily operation of a storage network, others are requirements such as availability that must be taken into account as early as

the implementation phase of a storage network. For reasons of readability we do not want to investigate the individual requirements in more detail at this point. In Appendix

B you will ﬁnd a detailed elaboration of these requirements in the form of a checklist. We now wish to turn our attention to the possibilities that a management system can offer in daily operation.

2008-04-01T07:14:00.001-07:00

Tutors on How Do Management of Storage Networks

In the course of this book we have dealt with the different techniques that are used in storage networks and the beneﬁts that can be derived from them. As we did so it became clear that storage networks are complex architectures, the management of which imposes stringent demands on administrators. In this chapter we therefore want to look at the management of storage networks Our primary objective here is to show how the management of a storage network can be supported by appropriate software. To this end we will ﬁrst consider new requirements of the system management that arise as a result of the developmental transition from server-centric to storage-centric IT architecture (Section 8.1). It is evident that various technical ﬁelds impose different requirements that should be taken into account in a management system (Section 8.2). Therefore, a management system must provide a range of functions (Section 8.3) and operate existing interfaces such as in-band and out-band interfaces (Section 8.4). These interfaces are provided via proprietary and standardized mechanisms (Section 8.5). Then we want to deal in more detail with the possibilities that the standardized methods offer by means of in-band (Section 8.6) and out-band (Section 8.7) interfaces. Finally, we discuss operational aspects of the management of storage networks (8.8).

SYSTEM MANAGEMENT

In the conventional server-centric IT architecture it is assumed that a server has directly connected storage. From the perspective of system management, therefore, there are two units to manage: the server on the one hand and the storage on the other. The connection

between server and storage does not represent a unit to be managed. It is primarily a question of how and where the data is stored and not how it is moved. The transition to storage-centric IT architecture – i.e. the introduction of storage net- works – has greatly changed the requirements of system management. In a storage net- work the storage is no longer local to the servers but can instead be located in different buildings or even different parts of the city. In the network between the servers and storage devices, numerous devices (host bus adapters, hubs, switches, gateways) are used, which can each change the data ﬂow. In a storage network there are thus many more units to manage than in a server-centric IT architecture. Now administrators have to think not only about their data on the storage devices, but also about how the data travels from the

servers to the storage devices. The management of a storage network is therefore com- parable to the management of a LAN. However, LAN management methods are better known than the methods for the management of new technologies such as Fibre Channel SAN, iSCSI or InﬁniBand.

The manufacturers of these new storage technologies are still involved in implementing suitable management functions and integrating these into the management tools for LAN and servers. The objective is to develop a central management system in which all resources of a data center from hardware to the applications can be managed. Therefore, in the management of storage networks we meet long familiar representatives from the LAN and the server world. The most prominent example is the SNMP protocol (Section 8.7.1), which has already been used for a long time in the ﬁeld of LAN

2008-04-01T07:07:00.001-07:00

KNOW MORE ABOUT ORGANIZATIONAL ASPECTS OF BACK-UP

In addition to the necessary technical resources, the personnel cost of backing data up is also often underestimated. We have already discussed (1) how the back-up of data has to be continuously adapted to the ever-changing IT landscape; and (2) that it is necessary to continuously monitor whether the back-up of data is actually performed according to plan. Both together quite simply take time, with the time cost for these activities often being underestimated. As is the case for any activity, human errors cannot be avoided in back-up, particularly if time is always short due to staff shortages. However, in the ﬁeld of data protection these human errors always represent a potential data loss. The costs of data loss can be

enormous: for example, Marc Farley (Building Storage Networks, 2000) cites a ﬁgure of US$ 1000 per employee as the cost for lost e-mail databases. Therefore, the personnel requirement for the back-up of data should be evaluated at least once a year. As part of this process, personnel costs must always be compared to the cost of lost data. The restoration of data sometimes fails due to the fact that data has not been fully backed up, tapes have accidentally been overwritten with current data or tapes that were already worn and too old have been used for back-ups. The media manager can prevent most of these problems. However, this is ineffective if the back-up software is not correctly conﬁgured. One of the three authors can well remember a situation more than ten years ago in which he was not able to restore the data after a planned repartitioning of a disk drive. The script for the back-up of the data contained a single typing error. This error resulted in an empty partition being backed up instead of the partition containing the data. The restoration of data should be practised regularly so that errors in the back-up are detected before an emergency occurs, in order to practise the performance of such tasks and in order to measure the time taken. The time taken to restore data is an important cost variable: for example, a multi-hour failure of a central application such as SAP R/3 can involve signiﬁcant costs. Therefore, staff should be trained in the following scenarios, for example:

• restoring an important server including all applications and data to equivalent hardware;

• restoring an important server including all applications and data to new hardware;

• restoring a subdirectory into a different area of the ﬁle system;

• restoring an important ﬁle system or an important database;

• restoring several computers using the tapes from the off-site store;

• restoring old archives (are tape drives still available for the old media?).

The cost in terms of time for such exercises should be taken into account when calculating the personnel requirement for the back-up of data.

Storage networks and intelligent storage subsystems open up new possibilities for solving the performance problems of network back-up. However, these new techniques are significantly more expensive than classical network back-up over the LAN. Therefore, it is ﬁrst necessary to consider at what speed data really needs to be backed up or restored. Only then is it possible to consider which alternative is the most economical: the new tech-

niques will be used primarily for heavyweight clients and for 24 × 7 applications. Simple clients will continue to be backed up using classical methods of network back-up and for medium-sized clients there remains the option of installing a separate LAN for the back-up of data. All three techniques are therefore often found in real IT systems nowadays. Data protection is a difﬁcult and resource-intensive business. Network back-up systems

allow the back-up of data to be largely automated even in heterogeneous environments. This automation takes the pressure off the system administrator and helps to prevent errors such as the accidental overwriting of tapes. The use of network back-up systems is indispensable in large environments. However, it is also worthwhile in smaller environments. Nevertheless, the personnel cost of back-up must not be underestimated.

This chapter started out by describing the general conditions for back-up: strong growth in the quantity of data to be backed up, continuous adaptation of back-up to ever-changing IT systems and the reduction of the back-up window due to globalization. The transition to network back-up was made by the description of the back-up, archiving and hierarchical storage management (HSM). We then discussed the server components necessary for the implementation of these services (job scheduler, error handler, media manager and meta- data database) plus the back-up client. At the centre was the incremental-forever strategy and the storage hierarchy within the back-up server. Network back-up was also considered from the point of view of performance: we ﬁrst showed how network back-up systems can contribute to using the existing infrastructure more efﬁciently. CPU load, the clogging of the internal buses and the inefﬁciency of the TCP/IP/Ethernet medium were highlighted as performance bottlenecks. Then, proposed solutions for increasing performance that are possible within a server-centric IT architecture were discussed, including their limitations. This was followed by proposed solutions to overcome the performance bottlenecks in a storage-centric IT architecture. Finally, the back-up of large ﬁle systems and databases was described and organizational questions regarding network back-up were outlined. This chapter ends our consideration of the use of storage networks. In the remaining three chapters we concern ourselves with management of storage networks, removable media management, and the SNIA Shared Storage Model.

2008-04-01T06:05:00.001-07:00

Free Gide On Next generation back-up of databases

The methods introduced in the previous section for the back-up of databases (cold backup, hot back-up and fuzzy back-up) are excellently suited for use in combination with storage networks and intelligent storage subsystems. In the following we show how the back-up of databases can be performed more efﬁciently with the aid of storage networks and intelligent storage subsystems. The linking of hot back-up with instant copies is an almost perfect tool for the back-up of databases. Individually, the following steps should be performed:

1. Switch the database over into hot back-up mode so that there is a consistent data set in the storage system.

2. Create the instant copy.

3. Switch the database back to normal mode.

4. Back up the database from the instant copy.

This procedure has two advantages: ﬁrst, access to the database is possible throughout the process. Second, steps 1–3 only take a few seconds, so that the database system only has to catch up comparatively few transactions after switching back to normal mode. Application server-free back-up expands the back-up by instant copies in order to additionally free up the database server from the load of the back-up

The concept shown in Figure 7.11 is also very suitable for databases. Due to the large quantity of data involved in the back-up of databases, LAN-free back-up is often used – unlike in the ﬁgure – in order to back up the data generated using instant copy. In the previous section (Section 7.10.2) we explained that the time of the last back-up is decisive for the time that will be needed to restore a database to the last data state. If the last back-up was a long time ago, a lot of archive log ﬁles have to be reapplied. In order to reduce the restore time for a database it is therefore necessary to increase the frequency of database back-ups. The problem with this approach is that large volumes of data are moved during a

complete back-up of databases. This is very tie-consuming and uses a lot of resources, which means that the frequency of back-ups can only be increased to a limited degree. Likewise, the delayed copying of the log ﬁles to a second system (Section 6.3.5) and the holding of several copies of the data set on the disk subsystem by means of instant copy can only seldom be economically justiﬁed due to the high hardware requirement and the

associated costs. In order to nevertheless increase the back-up frequency of a database, the data volume to be transferred must therefore be reduced. This is possible by means of an incremental back-up of the database on block level. The most important database systems offer backup tools for this by means of which such database increments can be generated. Many network back-up systems provide special adapters (back-up agents) that are tailored to the back-up tools of the database system in question. However, the format of the increments is unknown to the back-up software, so that the incremental-forever strategy cannot be realized in this manner. This would require manufacturers of database systems to publish

the format of the increments. The back-up of databases using the incremental-forever strategy therefore requires that the back-up software knows the format of the incremental back-ups, so that it can calculate the full back-ups from them. To this end, the storage space of the database must be

provided via a ﬁle system that can be incrementally backed up on block level using the appropriate back-up client. The back-up software knows the format of the increments so the incremental-forever strategy can be realized for databases via the circuitous route of file systems.

2008-04-01T05:09:00.000-07:00

Free Tutors On How Classical back-up of databases works

As in all applications, the consistency of backed up data also has to be ensured in databases. In databases, consistency means that the property of atomicity of the transactions is maintained. After the restoration of a database it must therefore be ensured that only the results of completed transactions are present in the data set. In this section we discuss various back-up methods that guarantee precisely this. In the next section we explain how storage networks and intelligent storage systems help to accelerate the back-up of databases (Section 7.10.3).

The simplest method for the back-up of databases is the so-called cold back-up. For cold back-up, the database is shut down so that all transactions are concluded, and then the ﬁles or volumes in question are backed up. In this method, databases are backed up in exactly the same way as ﬁle systems. In this case it is a simple matter to guarantee the consistency of the backed up data because no transactions are taking place during the back-up. Cold back-up is a simple to realize method for the back-up of databases. However, it has two disadvantages. First, in a 24 × 7 environment you cannot afford to shut down databases for back-up, particularly as the back-up of large databases using conventional methods can take several hours. Second, without further measures all changes since the last back-up would be lost in the event of the failure of a disk subsystem. For example, if a database is backed up overnight and the disk subsystem fails on the following evening

all changes from the last working day are lost. With the aid of the archive log ﬁle the second problem, at least, can be solved. The latest

state of the database can be recreated from the last back-up of the database, all archive log ﬁles backed up since and the active log ﬁles. To achieve this, the last back-up of the database must ﬁrst of all be restored from the back-up medium – in the example above the back-up from the previous night. Then all archive log ﬁles that have been created since the last back-up are applied to the data set, as are all active log ﬁles. This procedure, which is also called forward recovery of databases, makes it possible to restore the latest state even a long time after the last back-up of the database. However, depending upon the size of the archive log ﬁles this can take some time. The availability of the archive log ﬁles is thus an important prerequisite for the successful forward recovery of a database. The ﬁle system for the archive log ﬁles should,

therefore, be stored on a different hard disk to the database itself (Figure 7.19) and additionally protected by a redundant RAID procedure. Furthermore, the archive log ﬁles should be backed up regularly. Log ﬁles and archive log ﬁles form the basis of two further back-up methods for databases: hot back-up and fuzzy back-up. In hot back-up, the database system writes pending changes to the database to the log ﬁles only. The actual database remains unchanged at this time, so that the consistency of the back-up is guaranteed. After the end of the back-up, the database system is switched back into the normal state. The database system can then incorporate the changes listed in the log ﬁles into the database. Hot back-up is suitable for situations in which access to the data is required around the clock. However, hot back-up should only be used in phases in which a relatively low number of write accesses are taking place. If, for example, it takes two hours to back up the database and the database is operating at full load, the log ﬁles must be dimensioned so that they are large enough to be able to save all changes made during the back-up. Furthermore, the system must be able to complete the postponed transactions after the back-up in addition to the currently pending transactions. Both together can lead to performance bottlenecks. Finally, fuzzy back-up allows changes to be made to the database during its back-up so that an inconsistent state of the database is backed up. The database system is nevertheless

capable of cleaning the inconsistent state with the aid of archive log ﬁles that have been written during the back-up. With cold back-up, hot back-up and fuzzy back-up, three different methods are avail- able for the back-up of databases. Network back-up systems provide back-up clients for databases, which means that all three back-up methods can be automated with a network back-up system. According to the principle of keeping systems as simple as possible, cold back-up or hot back-up should be used whenever possible.

2008-04-01T05:07:00.001-07:00

Tutor on how take Backup of databases And Operating method of database systems

Databases are the second most important organizational form of data after the ﬁle systems discussed in the previous section. Despite the measures introduced in Section 6.3.5, it is sometimes necessary to restore a database from a back-up medium. The same questions are raised regarding the back-up of the metadata of a database server as for the back-up of ﬁle servers (Section 7.9.1). On the other hand, there are clear differences between the back-up of ﬁle systems and databases. The back-up of databases requires a fundamental understanding of the operating method of databases (Section 7.10.1). Knowledge of the operating method of databases helps us to perform both the conventional

back-up of databases without storage networks (Section 7.10.2) and also the back-up of databases with storage networks and intelligent storage subsystems (Section 7.10.3) more efficiently.

Operating method of database systems

One requirement of database systems is the atomicity of transactions, with transactions bringing together several write and read accesses to the database to form logically coherent units. Atomicity of transactions means that a transaction involving write access should be performed fully or not at all. Transactions can change the content of one or more blocks that can be distributed over several hard disks or several disk subsystems. Transactions that change several blocks are problematic for the atomicity. If the database system has already written a few of the blocks to be changed to hard disk and has not yet written others and then the database server goes down due to a power failure or a hardware fault, the transaction has only partially been performed. Without additional measures the transaction can neither be completed nor undone after a reboot of the database server because the information necessary for this is no longer available. The database would therefore be inconsistent. The database system must therefore store additional information regarding transactions that have not yet been concluded on the hard disk in addition to the actual database. The database system manages this information in so-called log ﬁles. It ﬁrst of all notes every pending change to the database in a log ﬁle before going on to perform the changes to the blocks in the database itself. If the database server fails during a transaction, the database system can either complete or undo incomplete transactions with the aid of the log ﬁleafter the reboot of the server.

• Database: storing the logical data structure to block-oriented storage First, the database system organizes the data into a structure suitable for the applications and stores this on the block-oriented hard disk storage. In modern database systems the relational data model, which stores information in interlinked tables, is the main model used for this. To be precise, the database system stores the logical data directly onto the hard disk, circumventing a ﬁle system, or it stores it to large ﬁles. The advantages and disadvantages of these two alternatives have already been discussed in Section 4.1.1.

• Transaction machine: changing the database Second, the database system realizes methods for changing the stored information. To this end, it provides a database language and a transaction engine. In a rela- tional database the users and applications initiate transactions via the database language SQL and thus call up or change the stored information. Transactions on the logical, application-near data structure thus bring about changes to the physical blocks on the hard disk. The transaction system ensures, amongst other things, that the changes to

the data set caused by a transaction are either completed or not performed at all. As described above, this condition can be guaranteed with the aid of log ﬁles even in the event of computer or database system crashes. The database system changes blocks in the data area, in no speciﬁc order, depending on how the transactions occur. The log ﬁles, on the other hand, are always written sequentially, with each log ﬁle being able to store a certain number of changes. Database systems are generally conﬁgured with several log ﬁles written one after the other. When all log ﬁles have been fully written, the database system ﬁrst overwrites the log ﬁle that was written ﬁrst, then the next, and so on.

A further important function for the back-up of databases is the back-up of the log ﬁles. To this end, the database system copies full log ﬁles into a ﬁle system as ﬁles and numbers these sequentially: logﬁle 1, logﬁle 2, logﬁle 3, etc. These copies of the log ﬁles are also called archive log ﬁles. The database system must be conﬁgured with enough log ﬁles that there is sufﬁcient time to copy the content of a log ﬁle that has just been fully ritten into an archive log ﬁle before it is once again overwritten.

2008-04-01T05:01:00.001-07:00

Know more on The Network Data Management Protocol (NDMP)

The Network Data Management Protocol (NDMP) deﬁnes an interface between NAS servers and network back-up systems that makes it possible to back up NAS servers with- out providing a speciﬁc back-up client for them. More and more manufacturers – both of NAS servers and network back-up systems – are supporting NDMP. The current version of NDMP is Version 4; Version 5 is in preparation. NDMP uses the term 'data management operations' to describe the back-up and restoration of data. A so-called data management application (DMA) – generally a back-up system – initiates and controls the data management operations, with the execution of a data management operation generally being called an NDMP session. The DMA cannot directly access the data; it requires the support of so-called NDMP services (Figure 7.13). NDMP services manage the current data storage, such as ﬁle systems, back-up media and tape libraries. The DMA creates an NDMP control connection for the control of every participating NDMP service; for the actual data ﬂow between source medium and back-up medium a so-called NDMP data connection is established between the NDMP services in question. Ultimately, the NDMP describes a client-server architecture, with the DMA taking on the role of the NDMP client. An NDMP server is made up of one or more NDMP services. Finally, the NDMP host is the name for a computer that accommodates one or more NDMP servers. NDMP deﬁnes different forms of NDMP services. All have in common that they only manage their local state. The state of other NDMP services remains hidden to an NDMP service. Individually, NDMP Version 4 deﬁnes the following NDMP services: • NDMP Data Service The NDMP data service forms the interface to primary data such as a ﬁle system on a NAS server. It is the source of back-up operations and the destination of restore operations. To back-up a ﬁle system, the NDMP Data Service converts the content of the ﬁle system into a data stream and writes this in an NDMP data connection, which is generally created by means of a TCP/IP connection. To restore a ﬁle system it reads the data stream from an NDMP data connection and from this reconstructs the content of a ﬁle system. The Data Service only permits the back-up of complete ﬁle systems;

it is not possible to back up individual ﬁles. By contrast, individual ﬁles or directories can be restored in addition to complete ﬁle systems.

The restoration of individual ﬁles or directories is also called 'direct access recovery'. To achieve this, the Data Service provides a so-called ﬁle history interface, which it uses to forward the necessary metadata to the DMA during the back-up. The ﬁle history stores the positions of the individual ﬁles within the entire data stream. The DMA cannot read this so-called ﬁle locator data, but it can forward it to the NDMP tape

service in the event of a restore operation. The NDMP tape service then uses this information to wind the tape to the appropriate position and read the ﬁles in question. • NDMP Tape Service The NDMP Tape Service forms the interface to the secondary storage. Secondary storage, in the sense of NDMP, means computers with connected tape drive, connected tape library or a CD burner. The Tape Service manages the destination of a back-up or the source of a data restoration operation. For a back-up, the Tape Service writes an incoming data stream to tape via the NDMP data connection; for a restoration it reads the content of a tape and writes this as a data stream in a NDMP data connection. The Tape Service has only the information that it requires to read and write, such as tape size or block size. It has no knowledge of the format of the data stream. It requires the assistance of the DMA to change tapes in a tape library. • NDMP SCSI Pass Through Service

The SCSI Pass Through Service makes it possible for a DMA to send SCSI commands to a SCSI device that is connected to a NDMP server. The DMA requires this service, for example, for the changing of tapes in a tape library. The DMA holds the threads of an NDMP session together: it manages all state information of the participating NDMP services, takes on the management of the back-up media and initiates appropriate recovery measures in the event of an error. To this end the DMA maintains an NDMP control connection to each of the participating NDMP

services, which – like the NDMP data connections – are generally based upon TCP/IP. Both sides – DMA and NDMP services – can be active within an NDMP session. For example, the DMA sends commands for the control of the NDMP services, whilst the NDMP services for their part send messages if a control intervention by the DMA is required. If, for example, an NDMP Tape Service has ﬁlled a tape, it informs the DMA. This can then initiate a tape change by means of an NDMP SCSI Pass Through Service. The fact that both NDMP control connections and NDMP data connections are based upon TCP/IP means that ﬂexible conﬁguration options are available for the back-up of data using NDMP. The NDMP architecture supports back-up to a locally connected tape drive (Figure 7.14) and likewise to a tape drive connected to another computer, for example a second NAS server or a back-up server (Figure 7.15). This so-called remote back-up has the advantage that smaller NAS servers do not need to be equipped with a tape library. Further ﬁelds of application of remote back-up are the replication of ﬁle systems (disk-to-disk remote back-up) and of back-up tapes (tape-to-tape remote back-up). In remote back-up the administrator comes up against the same performance bottlenecks as in conventional network back-up over the LAN (Section 7.6). Fortunately, NDMP local back-up and LAN-free back-up of network back-up systems complement each other excellently: a NAS server can back up to a tape drive available in the storage network, with the network back-up system co-ordinating access to the tape drive outside of NDMP by means of tape library sharing In Version 5, NDMP will have further functions such as multiplexing, compressing and encryption. To achieve this, NDMP Version 5 expands the architecture to include the so-called translator service (Figure 7.17). Translator services process the data stream (data stream processor): they can read and change one or more data streams. The implementation of translator services is in accordance with that of previous NDMP services. This means that the control of the translator service lies with the DMA; other participating NDMP

services cannot tell whether an incoming data stream was generated by a translator service or a different NDMP service. NDMP Version 5 deﬁnes the following translator services:

• Data stream multiplexing The aim of data stream multiplexing is to bundle several data streams into one data stream (N:1-multiplexing) or to generate several data streams from one (1:M-multi- plexing). Examples of this are the back-up of several small, slower ﬁle systems onto a

faster tape drive (N:1-multiplexing) or the parallel back-up of a large ﬁle system onto several tape drives (1:M-multiplexing).

• Data stream compression In data stream compression the translator service reads a data stream, compresses it and sends it back out. Thus the data can be compressed straight from the hard disk, thus freeing up the network between it and the back-up medium.

• Data stream encryption Data stream encryption works on the same principle as data stream compression, except that it encrypts data instead of compressing it. Encryption is a good idea, for example, for the back-up of small NAS servers at branch ofﬁces to a back-up server in a data centre via a public network. NDMP offers many opportunities to connect NAS servers to a network back-up system. The prerequisite for this is NDMP support on both sides. NDMP data services cover approximately the functions that back-up clients of network back-up systems provide. One weakness of NDMP is the back-up of the NAS server metadata, which makes the restoration of a NAS server after the full replacement of hardware signiﬁcantly more difﬁcult (Section 7.9.1). Furthermore, there is a lack of support for the back-up of ﬁle

systems with the aid of snapshots or instant copies. Despite these missing functions NDMP has established itself as a standard and so we believe that it is merely a matter of time before NDMP is expanded to include these functions.

2008-04-01T04:59:00.000-07:00

Know more The Network Data Management Protocol (NDMP)

it is not possible to back up individual ﬁles. By contrast, individual ﬁles or directories can be restored in addition to complete ﬁle systems.

services cannot tell whether an incoming data stream was generated by a translator service or a different NDMP service. NDMP Version 5 deﬁnes the following translator services:

faster tape drive (N:1-multiplexing) or the parallel back-up of a large ﬁle system onto several tape drives (1:M-multiplexing).

2008-04-01T04:53:00.001-07:00

Free tutors How to take Back-up of NAS servers

NAS servers are preconﬁgured ﬁle servers; they consist of one or more internal servers, preconﬁgured disk capacity and usually a stripped-down or speciﬁc operating system (Section 4.2.2). NAS servers generally come with their own back-up tools. However, just like the back-up tools that come with operating systems, these tools represent an isolated solution (Section 7.1). Therefore, in the following we speciﬁcally consider the linking of the back-up of NAS servers into an existing network back-up system. The optimal situation would be if there were a back-up client for a NAS server that was adapted to suit both the peculiarities of the NAS server and also the peculiarities of the network back-up system used. Unfortunately, it is difficult to develop such a back-up client in practice:

• If the NAS server is based upon a speciﬁc operating system the manufacturers of the network back-up system sometimes lack the necessary interfaces and compilers to develop such a client. Even if the preconditions for the development of a speciﬁc back- up client were in place, it is doubtful whether the manufacturer of the network back-up system would develop a speciﬁc back-up client for all NAS servers: the necessary development cost for a new back-up client is still negligible in comparison to the testing cost that would have to be incurred for every new version of the network back-up system.

• Likewise, it is difficult for the manufacturers of NAS servers to develop such a client. The manufacturers of network back-up systems publish neither the source code nor the interfaces between back-up client and back-up server, which means that a client cannot be developed. Even if such a back-up client already exists because the NAS server is based upon on a standard operating system such as Linux, Windows or Solaris, this does not mean that customers may use this client: in order to improve the Plug&Play- capability of NAS servers, customers may only use the software that has been tested and certiﬁed by the NAS manufacturer. If the customer installs non-certiﬁed software, then he can lose support for the NAS server. Due to the testing cost, manufacturers of NAS servers may be able to support some, but certainly not all network back-up systems. Without further measures being put in place, the only possibility that remains is to back the NAS server up from a client of the NAS server (Figure 7.12). However, this approach, too, is doubtful for two reasons:

• First, this approach is only practicable for smaller quantities of data: for back-up the ﬁles of the NAS server are transferred over the LAN to the network ﬁle system client on which the back-up client runs. Only the back-up client can write the ﬁles to the back-up medium using advanced methods such as LAN-free back-up.

• Second, the back-up of metadata is difﬁcult. If a NAS server supports the export of the local ﬁle system both via CIFS and also via NFS then the back-up client only accesses one of the two protocols on the ﬁles – the metadata of the other protocol is lost. NAS servers would thus have to store their metadata in special ﬁles so that the network back-up system can back these up. There then remains the question of the cost for the restoring of a NAS server or a ﬁle system. The metadata of NAS servers and ﬁles has to be re-extracted from these ﬁles. It is dubious whether network back-up systems can automatically initiate this process. As a last resort for the integration of NAS servers and network back-up systems, there remains only the standardization of the interfaces between the NAS server and the network back-up system. This would mean that manufacturers of NAS servers would only have to develop and test one back-up client that supports precisely this interface. The back-up systems of various manufacturers could then back up the NAS server via this interface. In such an approach the extensively of this interface determines how well the back-up of NAS servers can be linked into a network back-up system. The next section introduces a standard for such an interface – the Network Data Management Protocol (NDMP).

2008-04-01T04:49:00.001-07:00

Know basics of Back-up of ﬁle systems

For the classical network back-up of ﬁle systems, back-up on different levels (block level, ﬁle level, ﬁle system image) has been discussed in addition to the incremental-forever strategy. The introduction of storage networks makes new methods available for the backup of ﬁle systems such as server-free back-up, application server-free back-up, LAN-free back-up, shared disk ﬁle systems and instant copies. The importance of the back-up of ﬁle systems is demonstrated by the fact that manufacturers of ﬁle systems are providing new functions speciﬁcally targeted at the acceleration of back-ups. In the following we introduce two of these new functions – the so-called archive bit and block level incremental back-up. The archive bit supports incremental back-ups at ﬁle level such as, for example, the incremental-forever strategy. One difﬁculty associated with incremental back-ups is ﬁnding out quickly which ﬁles have changed since the previous back-up. To accelerate this decision, the ﬁle system adds an archive bit to the metadata of each ﬁle: the network back-up system sets this archive bit immediately after it has backed a ﬁle up on the back-up server. Thus the archive bits of all ﬁles are set after a full back-up. If a ﬁle is altered, the ﬁle system automatically clears its archive bit. Newly generated ﬁles are thus not given an archive bit. In the next incremental back-up the network back-up system knows that it only has to back up those ﬁles for which the archive bits have been

cleared. The principle of the archive bit can also be applied to the individual blocks of a ﬁle system in order to reduce the cost of back-up on block level. In Section 7.4 a comparatively expensive procedure for back-up on block level was introduced: the cost of the copying and comparing of ﬁles by the back-up client is greatly reduced if the ﬁle system manages the quantity of altered blocks itself with the aid of the archive bit for blocks and the network back-up system can call this up via an interface. Unfortunately, the principle of archive bits cannot simply be combined with the principle of instant copies: if the ﬁle system copies uses instant copy to copy within the disk subsystem for back-up (Figure 7.11), the network back-up system sets the archive bit only on the copy of the ﬁle system. In the original data the archive bit thus remains cleared even though the data has been backed up. Consequently, the network back-up system backs this data up at the next incremental back-up because the setting of the archive bit has not penetrated through to the original data.

2008-04-01T04:44:00.000-07:00

KNOW MORE ABOUT BACK-UP OF FILE SYSTEMS AND BACKUP OF FILE SERVER

Almost all applications store their data in ﬁle systems or in databases. Therefore, in this section we will examine the back-up of ﬁle servers (Section 7.9) and in the next section we will look more closely at that of databases (Section 7.10). The chapter concludes with organizational aspects of network back-up (Section 7.11). This section ﬁrst of all discusses fundamental requirements and problems in the backup of ﬁle servers (Section 7.9.1). Then a few functions of modern ﬁle systems will be introduced that accelerate the incremental back-up of ﬁle systems (Section 7.9.2). Limitations in the back-up of NAS servers will then be discussed (Section 7.9.3). We will then introduce the Network Data Management Protocol (NDMP), a standard that helps to integrate the back-up of NAS servers into an established network back-up system

Back-up of ﬁle servers

We use the term ﬁle server to include computers with a conventional operating system such as Windows or Unix that exports part of its local ﬁle systems via a network ﬁle system or makes it accessible as service (Novell, FTP, HTTP). The descriptions in this section can be transferred to all types of computers, from user PCs through classical ﬁle servers to the web server. File servers store three types of information:

• data in the form of ﬁles;

• metadata on these ﬁles such as ﬁle name, creation date and access rights; and

• metadata on the ﬁle servers such as any authorized users and their groups, size of the individual ﬁle systems, network conﬁguration of the ﬁle server and names, components and rights of ﬁles or directories exported over the network. Depending upon the error situation, different data and metadata must be restored. The restoring of individual ﬁles or entire ﬁle systems is relatively simple: in this case only the ﬁle contents and the metadata of the ﬁles must be restored from the back-up server to the ﬁle server. This function is performed by the back-up clients introduced in

Restoring an entire ﬁle server is more difﬁcult. If, for example, the hardware of the ﬁle

server is irreparable and has to be fully replaced, the following steps are necessary:

1. Purchasing and setting up of appropriate replacement hardware.

2. Basic installation of the operating system including any necessary patches.

3. Restoration of the basic conﬁguration of the ﬁle server including LAN and storage network conﬁguration of the ﬁle server.

4. If necessary, restoration of users and groups and their rights.

5. Creation and formatting of the local ﬁle systems taking into account the necessary ﬁle system sizes.

6. Installation and conﬁguration of the back-up client.

7. Restoration of the ﬁle systems with the aid of the network back-up system.

This procedure is very labour-intensive and time-consuming. The methods of so-called Image Restore (also known as Bare Metal Restore) accelerate the restoration of a complete computer: tools such as 'mksysb' (AIX), 'Web Flash Archive' (Solaris) or various disk image tools for Windows systems create a complete copy of a computer (image). Only a boot diskette or boot CD and an appropriate image is needed to completely restore a computer without having to work through steps 2–7 described above. Particularly advantageous is the integration of image restore in a network back-up system: to achieve this the network back-up system must generate the appropriate image. Furthermore, the boot diskette or boot CD must create a connection to the network back-up system.

2008-04-01T04:38:00.001-07:00

Learn more on Data protection using remote mirroring

Instant copies help to quickly restore data in the event of application or operating errors;however, they are ineffective in the event of a catastrophe: after a ﬁre the fact that there

are several copies of the data on a storage device does not help. Even a power failure

can become a problem for a 24 × 7 operation.

The only thing that helps here is to mirror the data by means of remote mirroring

on two disk subsystems, which are at least separated by a ﬁre protection barrier. The

protection of applications by means of remote mirroring has already been discussed in

detail in Sections 6.3.3 and 6.3.5.Nevertheless, the data still has to be backed up: in remote mirroring the source disk

and copy are always identical. This means that if data is destroyed by an application or

operating error then it is also immediately destroyed on the copy. The data can be backed

up to hard disk by means of instant copy or by means of classical network back-up to

tapes. Since storage capacity on hard disks is more expensive than storage capacity on

tapes, only the most important data is backed up using instant copy and remote mirroring.

For most data, back-up to tapes is still the most cost effective.

2008-04-01T04:36:00.001-07:00

Free Tutors on Tape library sharing and Back-up using instant copies

Server-free back-up and LAN-free back-up can signiﬁcantly reduce the load upon the back-up server. However, the problem remains that a large number of objects to be backed up can break the metadata database. The only effective remedy is to distribute the load amongst several back-up servers (Section 7.7.2). All back-up servers can share a large tape library via the storage network by means of tape library sharing. An alternative would be to purchase each back-up server its own smaller tape library. However, many small tape libraries are more expensive to purchase and more difﬁcult to manage than one large one. Figure 7.10 shows the use of tape library sharing for network back-up: one back-up server acts as library master, all others as library clients. If a back-up client backs up data to a back-up server that is conﬁgured as a library client, then this ﬁrst of all requests a free tape from the library master. The library master selects the tape from its pool of free tapes and places it in a free drive. Then it notes in its metadata database that this tape is now being used by the library client and it informs the library client of the drive that the tape is in. Finally, the back-up client can send the data to be backed up via the LAN to the back-up server, which is conﬁgured as the library client. This then writes the data directly to tape via the storage network..

Back-up using instant copies

Instant copies can practically copy even terabyte-sized data sets in a few seconds, and thus freeze the current state of the production data and make it available via a second access path. The production data can still be read and changed over the ﬁrst access path, so that the operation of the actual application can be continued, whilst at the same time the frozen state of the data can be backed up via the second access path. Instant copies can be realized on three different levels:

1. Instant copy in the block layer (disk subsystem or block-based virtualization) Instant copy in the disk subsystem was discussed in detail in Section intelligent disk subsystems can practically copy all data of a hard disk onto a second hard diskwithin a few seconds. The frozen data state can be accessed and backed up via the second hard disk.

2. Instant copy in the ﬁle layer (local ﬁle system, NAS server or ﬁle-based virtualization) Many ﬁle systems also offer the possibility of creating instant copies. Instant copies on ﬁle system level are generally called snapshots (Section 4.1.3). In contrast to instant copies in the disk subsystem the snapshot can be accessed via a special directory path.

3. Instant copy in the application Finally, databases in particular offer the possibility of freezing the data set internally for back-up, whilst the user continues to access it (hot back-up, online back-up). Instant copies in the local ﬁle system and in the application have the advantage that they

can be realized with any hardware. Instant copies in the application can utilize the internal data structure of the application and thus work more efﬁciently than ﬁle systems. On the other hand, applications do not require these functions if the underlying ﬁle system already provides them. Both approaches consume system resources on the application server that one would sometimes prefer to make available to the actual application. This

is the advantage of instant copies in external devices (e.g., disk subsystem, NAS Server, network-based virtualization instance): although it requires special hardware, application server tasks are moved to the external device thus freeing up the application server. Back-up using instant copy must be synchronized with the applications to be backed up. Databases and ﬁle systems buffer write accesses in the main memory in order to increase

their performance. As a result, the data on the hard disk is not always in a consistent state. Data consistency is the prerequisite for restarting the application with this data set and being able to continue operation. For back-up it should therefore be ensured that an instant copy with consistent data is ﬁrst generated. The procedure looks something like this:

1. Shut down the application.

2. Perform the instant copy.

3. Start up the application again.

4. Back up the data of the instant copy.

Despite the shutting down and restarting of the application the production system is back in operation very quickly. Data protection with instant copies is even more attractive if the instant copy is con- trolled by the application itself: in this case the application must ensure that the data on disk is consistent and then initiate the copying operation. The application can then continue operation after a few seconds. It is no longer necessary to stop and restart the application. Instant copies thus make it possible to back-up business-critical applications every hour with only very slight interruptions. This also accelerates the restoring of data after application errors ('accidental deletion of a table space'). Instead of the time-consuming restore of data from tapes, the frozen copy that is present in the storage system can simply be put back. With the aid of instant copies in the disk subsystem it is possible to realize so-called application server-free back-up. In this, the application server is put at the side of a second server that serves exclusively for back-up (Figure 7.11). Both servers are directly connected to the disk subsystem via SCSI; a storage network is not absolutely necessary. For back-up the instant copy is ﬁrst of all generated as described above:

(1) shut down application;

(2) generate instant copy; and

(3) restart application.

The instant copy can then be accessed from the second computer and the data is backed up from there without placing a load on the application server. If the instant copy is not deleted in the dis subsystem, the data can be restored using this copy in a few seconds in the event o an error.

2008-04-01T03:36:00.001-07:00

Know more on LAN-free back-up

LAN-free back-up dispenses with the necessity for the 3rd-Party SCSI Copy Command by realizing comparable functions within the back-up client

back-up, metadata is sent via the LAN. File contents, however, no longer go through the back-up server: for back-up the back-up client loads the data from the hard disk into the main memory via the appropriate buses and from there writes it directly to the back-up medium via the buses and the storage network. To this end, the back-up client must be able to access the back-up server's back-up medium over the storage network. Furthermore,

back-up server and back-up client must synchronize their access to common devices. This is easier to realize than server-free back-up and thus well proven in production environments. In LAN-free back-up the load on the buses of the back-up server is reduced but not the load on those of the back-up client. This can impact upon other applications (databases, ﬁle and web servers) that run on the back-up client at the same time as the back-up.

LAN-free back-up is already being used in production environments. However, the manufacturers of network back-up systems only support LAN-free back-up for certain applications (databases, ﬁle systems, e-mail systems), with not every application being supported on every operating system. Anyone wanting to use LAN-free back-up at the moment must take note of the manufacturer's support matrix (see Section 3.4.6). It can be assumed that in the course of the next one to two years the number of the applications and operating systems supported will increase signiﬁcantly.7.8.3 LAN-free back-up with shared disk ﬁle systems Anyone wishing to back up a ﬁle system now for which LAN-free back-up is not supported can sometimes use shared disk ﬁle systems to rectify this situation (Figure 7.9). Shared disk ﬁle systems are installed upon several computers. Access to data is synchronized over the LAN; the individual ﬁle accesses, on the other hand, take place directly over the storage network (Section 4.3). For back-up the shared disk ﬁle system is installed on the ﬁle server and the back-up server. The prerequisite for this is that a shared disk ﬁle system is available that supports the operating systems of back-up client and back-up server. The back-up client is then started on the same computer on which the back-up

server runs, so that back-up client and back-up server can exchange the data via Shared Memory (Unix) or Named Pipe or TCP/IP Loopback (Windows). In LAN-free back-up using a shared disk ﬁle system, the performance of the back- up server must be critically examined. All data still has to be passed through the buses of the back-up server; in addition, the back-up client and the shared disk ﬁle system run on this machine. LAN data trafﬁc is no longer necessary within the network back- up system; however, the shared disk ﬁle system now requires LAN data trafﬁc for the

synchronization of simultaneous data accesses. The data trafﬁc for the synchronization of the shared disk ﬁle system is, however, comparatively light. At the end of the day, you have to measure whether back-up with a shared disk ﬁle system increases performance for each individual case.

Although the performance of LAN-free back-up with the aid of a shared disk ﬁle system is not as good as the performance of pure LAN-free back-up, it can be signiﬁcantly better than that of back-up over the LAN. Therefore, this approach has proved its worth in production environments, so that it can be viewed as an interesting transitional solution until LAN-free (or even server-free) back-up becomes available.

2008-04-01T02:57:00.001-07:00

Know more on LAN-free back-up and LAN-free back-up with shared disk ﬁle systems

LAN-free back-up dispenses with the necessity for the 3rd-Party SCSI Copy Command by realizing comparable functions within the back-up client

LAN-free back-up with shared disk ﬁle systems

Anyone wishing to back up a ﬁle system now for which LAN-free back-up is not supported can sometimes use shared disk ﬁle systems to rectify this situation (Figure 7.9). Shared disk ﬁle systems are installed upon several computers. Access to data is synchronized over the LAN; the individual ﬁle accesses, on the other hand, take place directly over the storage network (Section 4.3). For back-up the shared disk ﬁle system is installed on the ﬁle server and the back-up server. The prerequisite for this is that a shared disk ﬁle system is available that supports the operating systems of back-up client and back-up server. The back-up client is then started on the same computer on which the back-up

2008-04-01T02:51:00.001-07:00

Free tutors about Next Generation Backups, Server-free back-up

Storage networks open up new possibilities for getting around the performance bottlenecks of network back-up described above. They connect servers and storage devices, so that during back-up production data can be copied directly from the source hard disk to the back-up media, without passing it through a server (server-free back-up, Section 7.8.1). LAN-free back-up (Section 7.8.2) and LAN-free back-up with shared disk ﬁle systems

(Section 7.8.3) are two further alternative methods of accelerating back-up using storage networks. The introduction of storage networks also has the side-effect that several back- up servers can share a tape library (Section 7.8.4). The use of instant copies (Section 7.8.5) and remote mirroring (Section 7.8.6) provide further possibilities for accelerating back-up and restore operations.

Server-free back-up

The ultimate goal of back-up over a storage network is so-called server-free back-up (Figure 7.7). In back-up, the back-up client initially determines which data has to be backed up and then sends only the appropriate metadata (ﬁle name, access rights, etc.) over the LAN to the back-up server. The ﬁle contents, which make up the majority of the data quantity to be transferred, are then written directly from the source hard disk

to the back-up medium (disk, tape, optical) over the storage network, without a server being connected in between. The network back-up system co-ordinates the communication between source hard disk and back-up medium. A shorter transport route for the back-up of data is not yet in sight with current storage techniques. The performance of server-free back-up is predominantly determined by the performance of the underlying storage systems and the connection in the storage network. Shifting the transport route for the majority of the data from the LAN to the storage net- work without a server being involved in the transfer itself means that the internal buses and the I/O buses are freed up on both the back-up client and the back-up server. The cost of co-ordinating the data trafﬁc between source hard disk and back-up medium is comparatively low.

A major problem in the implementation of server-free back-up is that the SCSI commands have to be converted en route from the source hard disk to the back-up medium. For example, different blocks are generally addressed on source medium and back-up medium. Or, during the restoration of a deleted ﬁle in a ﬁle system, this ﬁle has to be restored to a different area if the space that was freed up is now occupied by other ﬁles In the back-up from hard disk to tape, even the SCSI command sets are slightly different. Therefore, software called 3rd-Party SCSI Copy Command is necessary for the protocol conversion. It can be realized at various points: in a SAN switch, in a box specially connected to the storage network that is exclusively responsible for the protocol conversion, or in one of the two participating storage systems themselves. Server-free back-up is running in the laboratories and demo centres of many manufacturers. Many manufacturers claim that their back-up products already support server-free back-up. In our experience, however, server-free back-up is almost never used in production environments, although it has now been available for some time (2003). In our opinion this proves that server-free back-up is still very difﬁcult to conﬁgure and operate at the current level of technology.

2008-04-01T02:48:00.001-07:00

Free tutors on Back-up server and application server on the same physical computer

The third possible way of increasing performance is to install the back-up server and application server on the same physical computer (Figure 7.6). This results in the back-up client also having to run on this computer. Back-up server and back-up client communicate over Shared Memory (Unix), Named Pipe or TCP/IP Loopback (Windows) instead of via LAN. Shared Memory has an inﬁnite bandwidth in comparison to the buses, which

means that the communication between back-up server and back-up client is no longer the limiting factor. However, the internal buses continue to get clogged up: the back-up client now loads the data to be backed up from the hard disk into the main memory via the buses. The back-up server takes the data from the main memory and writes it, again via the buses, to the back-up medium. The data is thus once again driven through the internal bus twice. Tape reclamation and any copying operations within the storage hierarchy of the back-up server could place an additional load on the buses. Without further information we cannot more precisely determine the change to the CPU load. Shared Memory communication (or Named Pipe or TCP/IP Loopback) dispenses with the CPU-intensive operation of the network card. On the other hand, a single computer must now bear the load of the application, the back-up server and the back-up client. This computer must incidentally possess sufﬁcient main memory for all three applications. One problem with this approach is the proximity of production data and copies on the back-up server. SCSI permits a maximum cable length of 25 m. Since application and back-up server run on the same physical computer, the copies are a maximum of 50 m away from the production data. In the event of a ﬁre or comparable damage, this is disastrous. Therefore, either a SCSI extender should be used or the tapes taken from the tape library every day and placed in an off-site store. The latter goes against the requirement of largely automating data protection.

2008-04-01T02:45:00.001-07:00

Know more how to get the opportunities for increasing performance on Several back-up servers

Back-up is a resource-intensive application that places great demands upon storage devices, CPU, main memory, network capacity, internal buses and I/O buses. The enormous amount of resources required for back-up is not always sufﬁciently taken into account during the planning of IT systems. A frequent comment is 'the back-up is responsible for the slow network' or 'the slow network is responsible for the restore operation taking so long'. The truth is that the network is inadequately dimensioned for end user data trafﬁc and back-up data trafﬁc. Often, data protection is the application that requires the most network capacity. Therefore, it is often sensible to view back-up as the primary application for which the IT

infrastructure in general and the network in particular must be dimensioned. In every IT environment, most computers can be adequately protected by a network back-up system. In almost every IT environment, however, there are computers – usually only a few – for which additional measures are necessary in order to back them up quickly enough or, if necessary, to restore them. In the server-centric IT architecture there are three approaches to taming such data monsters: the installation of a separate LAN for the network back-up between back-up client and back-up server the installation of several back-up servers and the installation of back-up client and back-up server on the same physical computer (Section 7.7.3). 7.7.1 Separate LAN for network back-up The simplest measure to increase back-up performance more of heavyweight back-up clients is to install a further LAN between back-up client and back-up server in addition to the existing LAN and to use this exclusively for back-up (Figure 7.4). An expensive, but powerful, transmission technology such as ATM, FDDI or Gigabit Ethernet can also help here. The concept of installing a further network for back-up in addition to the existing LAN is comparable to the basic idea of storage networks. In contrast to storage networks, however, in this case only computers are connected together; direct access to all storage devices is not possible. All data thus continues to be passed via TCP/IP and through

application server and back-up server which leads to a blockage of the internal bus and the I/O buses. Individual back-up clients can thus beneﬁt from the installation of a separate LAN for network back-up. This approach is, however, not scalable at will: due to the heavy load on the back-up server this cannot back up any further computers in addition to the back-up of one individual heavyweight client. Despite its limitations, the installation of a separate back-up LAN is sufﬁcient in many environments. With Fast-Ethernet you can still achieve a throughput of over 10 MByte/s.The LAN technique is made even more attractive by Gigabit Ethernet, 10Gigabit Ethernet and the above-mentioned TCP/IP ofﬂoad engines that free up the server CPU signiﬁcantly with regard to the TCP/IP data trafﬁc

Several back-up servers

Installing multiple back-up servers distributes the load of the back-up server over more hardware. For example, it would be possible to assign every heavyweight back-up client a special back-up server installed exclusively for the back-up of this client (Figure 7.5). Furthermore, a further back-up server is required for the back-up of all other back- up clients. This approach is worthwhile in the event of performance bottlenecks in the metadata database or in combination with the ﬁrst measure, the installation of a separate LAN between the heavyweight back-up client and back-up server.

The performance of the back-up server can be signiﬁcantly increased by the installation of multiple back-up servers and a separate LAN for back-up. However, from the point of view of the heavyweight back-up client the problem remains that all data to be backed up must be passed from the hard disk into the main memory via the buses and from there must again be passed through the buses to the network card. This means that back-up still heavily loads the application server. The resource requirement for back-up could be in conﬂict with the resource requirement for the actual a application. A further problem is the realization of the storage hierarchy within the individual backup server since every back-up server now requires its own tape library. Many small tape libraries are more expensive and less ﬂexible than one large tape library. Therefore, it would actually be better to buy a large tape library that is used by all servers. In a server- centric IT architecture it is, however, only possible to connect multiple computers to the same tape library to a very limited degree.

2008-04-01T02:37:00.001-07:00

Learn more on How to take performance bottlenecks of networks backup

At some point, however, the technical boundaries for increasing the performance of backup are reached. When talking about technical boundaries, we should differentiate between application-speciﬁc boundaries (Section 7.6.1) and those that are determined by server- centric IT architecture

Application-speciﬁc performance bottlenecks

Application-speciﬁc performance bottlenecks are all those bottlenecks that can be traced back to the 'network back-up' application. These performance bottlenecks play no role for other applications. The main candidate for application-speciﬁc performance bottlenecks is the metadata

database. A great deal is demanded of this. Almost every action in the network back- up system is associated with one or more operations in the metadata database. If, for example, several versions of a ﬁle are backed up, an entry is made in the metadata database for each version. The back-up of a ﬁle system with several hundreds of thousands of ﬁles can thus be associated with a whole range of database operations. A further candidate for application-speciﬁc performance bottlenecks is the storage hierarchy: when copying the data from hard disk to tape the media manager has to load the data from the hard disk into the main memory via the I/O bus and the internal buses, only to forward it from there to the tape drive via the internal buses and I/O bus. This means that the buses can get clogged up during the copying of the data from hard disk to tape. The same applies to tape reclamation.

Performance bottlenecks due to server-centric

IT architecture In addition to these two application-speciﬁc performance bottlenecks, some problems crop up in network back-up that are typical of a server-centric IT architecture. Let us mention once again as a reminder the fact that in a server-centric IT architecture storage devices only exist in relation to servers; access to storage devices always takes place via the computer to which the storage devices are connected. The performance bottlenecks described in the following apply for all applications that are operated in a server-centric IT architecture.

Let us assume that a back-up client wants to back data up to the back-up server . The back-up client loads the data to be backed up from the hard disk into the main memory of the application server via the SCSI bus, the PCI bus and the system bus, only to forward it from there to the network card via the system bus and the PCI bus. On the back-up server the data must once again be passed through the buses twice. In back-up, large quantities of data are generally backed up in one go. During back-up, therefore, the buses of the participating computers can become a bottleneck, particularly

if the application server also has to bear the I/O load of the application or the back-up server is supposed to support several simultaneous back-up operations. The network card transfers the data to the back-up server via TCP/IP and Ethernet. Previously the data exchange via TCP/IP was associated with a high CPU load. However, the CPU load caused by TCP/IP data trafﬁc can be disregarded with the increasing use of TCP/IP ofﬂoad engines (TOE) (Section 3.5.2 'TCP/IP and Ethernet as an I/O technology').

2008-04-01T02:36:00.001-07:00

Learn more on How to BACK-UP clients , (PERFORMANCE GAINS AS A RESULT OF NETWORK BACK-UP Application-speciﬁc performance bottlenecks)

BACK-UP CLIENTS

A platform-speciﬁc client (back-up agent) is necessary for each platform to be backed up. The base client can back up and archive ﬁles and restores them if required. The term platform is used here to mean the various operating systems and the ﬁle systems that they support. Furthermore, some base clients offer HSM for selected ﬁle systems. The back-up of ﬁle systems takes place at ﬁle level as standard. This means that each changed ﬁle is completely retransferred to the server and entered there in the metadata database. By using back-up at volume level and at block level it is possible to change the granularity of the objects to be backed up. When back-up is performed at volume level, a whole volume is backed up as an

individual object on the back-up server. We can visualize this as the output of the Unix command 'dd' being sent to the back-up server. Although this has the disadvantage that free areas, on which no data at all has been saved, are also backed up, only very few metadata database operations are necessary on the back-up server and on the client side it is not necessary to spend a long time comparing which ﬁles have changed since the last

back-up. As a result, back-up and restore operations can sometimes be performed more quickly at volume level than they can at ﬁle level. This is particularly true when restoring large ﬁle systems with a large number of small ﬁles. Back-up on block level optimizes back-up for members of the external sales force, who only connect up to the company network now and then by means of a laptop via a dial-up line. In this situation the performance bottleneck is the low transmission capacity of modem or ISDN connections. If only one bit of a large ﬁle is changed, the whole ﬁle

must once again be forced down the dial-up connection. When backing up on block level the back-up client additionally keeps a local copy of every ﬁle backed up. If a ﬁle has changed, it can establish which parts of the ﬁle have changed. The back-up client sends only the changed data fragments (blocks) to the back-up server. This can then reconstruct the complete ﬁle. As is the case for back-up on ﬁle level, each ﬁle backed up is entered

in the metadata database. Thus, when backing up on block level the quantity of data to be transmitted is reduced at the cost of storage space on the local hard disk. In addition to the standard client for ﬁle systems, most network back-up systems pro- vide special clients for various applications. For example, there are special clients for MS Exchange or Lotus Domino that make it possible to back up and restore individual documents. We will discuss the back-up of ﬁle systems and NAS servers and databases (Section 7.10) in more detail later on.

PERFORMANCE GAINS AS A RESULT OF NETWORK BACK-UP

The underlying hardware components determine the maximum throughput of network back-up systems. The software components determine how efﬁciently the available hard- ware is actually used. At various points of this chapter we have already discussed how network back-up systems can help to better utilize the existing infrastructure:

• Performance increase by the archiving of data: deleting data that has already been archived from hard disks can accelerate the daily back-up because there is less data to back up. For the same reason, ﬁle systems can be restored more quickly.

• Performance increase by hierarchical storage management (HSM): by moving ﬁle contents to the HSM server, ﬁle systems can be restored more quickly. The directory entries of ﬁles that have been moved can be restored comparatively quickly; the majority of the data, namely the ﬁle contents, do not need to be fetched back from the HSM server.

• Performance increase by the incremental-forever strategy: after the ﬁrst back-up, only the data that has changed since the last back-up is backed up. On the back-up server the metadata database is used to calculate the latest state of the data from the ﬁrst back-up and all subsequent incremental back-ups, so that no further full back-ups are necessary. The back-up window can thus be signiﬁcantly reduced.

• Performance increase by reducing tape mounts: the media manager can ensure that data that belongs together is only distributed amongst a few tapes. The number of time-consuming tape changes for the restoring of data can thus be reduced.

• Performance increase by streaming: the efﬁcient writing of tapes requires that the data is transferred quickly enough to the tape drive. If this is not guaranteed the back-up server can ﬁrst temporarily store the data on a hard drive and then send the data to the tape drive in one go.

• Performance increase by back-up on volume level or on block level: as standard, ﬁle systems are backed up on ﬁle level. Large ﬁle systems with several hundreds of thousands of ﬁles can sometimes be backed up more quickly if they are backed up at volume level. Laptops can be backed up more quickly if only the blocks that have changed are transmitted over the modem to the back-up server.

2008-03-31T22:17:00.001-07:00

Learn more on Network backup ,Server components(Job scheduler, Error handling, Metadata database, Media manager,)

SERVER COMPONENTS

Back-up servers consist of a whole range of component parts. In the following we will discuss the main components:

ü Job scheduler

ü Error handler metadata database

ü Media manager

Job scheduler

The job scheduler determines what data will be backed up when. It must be carefully conﬁgured; the actual back-up then takes place automatically. With the aid of job schedulers and tape libraries many computers can be backed up overnight without the need for a system administrator to change tapes on site. Small tape libraries have a tape drive, a magazine with space for around ten tapes and a media changer that can automatically move the various tapes back and forth between magazine and tape drive. Large tape libraries have several dozen tape drives, space for several thousands of tapes and a media changer or two to insert the tapes in the drives.

Error handling

If a regular automatic back-up of several systems has to be performed, it becomes difﬁcult to monitor whether all automated back-ups have run without errors. The error handler helps to prioritize and ﬁlter error messages and generate reports. This avoids the situation in which problems in the back-up are not noticed until a back-up needs to be restored.

Metadata database

The metadata database and the media manager represent two components that tend to be hidden. The metadata database is the brain of a network back-up system. It contains the following entries for every back-up up object: name, computer of origin, date of last change, date of last back-up, name of the back-up medium, etc. For example, an entry is made in the metadata database for every ﬁle to be backed up. The cost of the metadata database is worthwhile: in contrast to back-up tools provided by operating systems, network back-up systems permit the implementation of the incremental-forever strategy in which a ﬁle system is only fully backed up in the ﬁrst back- up. In subsequent back-ups, only those ﬁles that have changed since the previous back-up are backed up. The current state of the ﬁle system can then be calculated on the back-up server from database operations from the original full back-up and from all subsequent incremental back-ups, so that no further full back-ups are necessary. The calculations in the metadata database are generally performed faster than a new full back-up. Even more is possible: if several versions of the ﬁles are backed up on the back-up server, a whole ﬁle system or a subdirectory dated three days ago, for example, can be restored (point-in-time restore) – the metadata database makes it possible.

Media manager

Use of the incremental-forever strategy can considerably reduce the time taken by the back-up in comparison to the full back-up. The disadvantage of this is that over time the backed up ﬁles can become distributed over numerous tapes. This is critical for the

restoring of large ﬁle systems because tape mounts cost time. This is where the media manager comes into play. It can ensure that only ﬁles from a single computer are located on one tape. This reduces the number of tape mounts involved in a restore process, which means that the data can be restored more quickly. A further important function of the media manager is so-called tape reclamation. As a result of the incremental-forever strategy, more and more data that is no longer needed is located on the back-up tapes. If, for example, a ﬁle is deleted or changed very frequently over time, earlier versions of the ﬁle can be deleted from the back-up medium. The gaps on the tapes that thus become free cannot be directly overwritten using current techniques. In tape reclamation, the media manager copies the remaining data that is still required from several tapes, of which only a certain percentage is used, onto a common new tape. The tapes that have thus become free are then added to the pool of unused tapes. There is one further technical limitation in the handling of tapes: current tape drives can

only write data to the tapes at a certain speed. If the data is transferred to the tape drive too slowly this interrupts the write process, the tape rewinds a little and restarts the write process. The repeated rewinding of the tapes costs performance and causes unnecessary wear to the tapes so they have to be discarded more quickly. It is therefore better to send the data to the tape drive quickly enough so that it can write the data onto the tape in one go (streaming). The problem with this is that in network back-up the back-up clients send the data to be

backed up via the LAN to the back-up server, which forwards the data to the tape drive. On the way from back-up client via the LAN to the back-up server there are repeated ﬂuctuations in the transmission rate, which means that the streaming of tape drives is repeatedly interrupted. Although it is possible for individual clients to achieve streaming by additional measures (such as the installation of a separate LAN between back-up client and back-up server) (Section 7.7), these measures are expensive and technically not scalable at will, so they cannot be realized economically for all clients. The solution: the media manager manages a storage hierarchy within the back-up server. To achieve this, the back-up server must be equipped with hard disks and tape libraries. If a client cannot send the data fast enough for streaming, the media manager ﬁrst of all stores the data to be backed up to hard disk. When writing to a hard disk it makes no

difference what speed the data is supplied at. When enough of the data to be backed up has been temporarily saved to the hard disk of the back-up server, the media manager automatically moves large quantities of data from the hard disk of the back-up server to its tapes. This process only involves recopying the data within the back-up server, so that streaming is guaranteed when writing the tapes. This storage hierarchy is used, for example, for the back-up of user PCs Many user PCs are switched off overnight, which means that back-up cannot be guaranteed overnight. Therefore, network back-up systems often use the midday period to back up user PCs. Use of the incremental-forever strategy means that the amount of data to be backed up every day is so low that such a back-up strategy is generally feasible. All user PCs are ﬁrst of all backed up to the hard disk of the back-up server in the time window from 11 : 15 to 13 : 45. The media manager in the back-up server then has a good twenty hours to move the data from the hard disks to tapes. Then the hard disks are once again free so that the user PCs can once again be backed up to hard disk in the next midday break. In all operations described here the media manager checks whether the correct tape has been placed in the drive. To this end, the media manager writes an unambiguous signature to every tape, which it records in the metadata database. Every time a tape is inserted the media manager compares the signature on the tape with the signature in the metadata database. This ensures that no tapes are accidentally overwritten and that the correct data is written back during a restore operation. Furthermore, the media manager monitors how often a tape has been used and how old it is, so that old tapes are discarded in good time. If necessary, it ﬁrst copies data that is still required to a new tape. Older tape media formats also have to be wound back and forwards now and then so that they last longer; the media manager can also automate the winding of tapes that have not been used for a long time. A further important function of the media manager is the management of data in a so-called off-site store. To this end, the media manager keeps two copies of all data to be backed up. The ﬁrst copy is always stored on the back-up server, so that data can be quickly restored if it is required. However, in the event of a large-scale disaster (ﬁre in the data centre) the copies on the back-up server could be destroyed. For such cases the media manager keeps a second copy in an off-site store that can be several kilometres away. The media manager supports the system administrator in moving the correct tapes back and forwards between back-up server and off-site store. It even supports tape reclamation for tapes that are currently in the off-site store and it.

2008-03-31T22:05:00.001-07:00

Free tutors on Network Back-up administration ,GENERAL CONDITIONS FOR BACK-UP, NETWORK BACK-UP SERVICES

(Network back-up systems such as Arcserve (Computer Associates), NetBackup (Veritas), Networker (EMC/Legato) and Tivoli Storage Manager (IBM):)

Network back-up systems can back up heterogeneous IT environments incorporating several thousands of computers largely automatically. In the classical form, network back-up systems move the data to be backed up via the LAN; this is where the name 'network back-up' comes from. This chapter explains the basic principles of network back-up and shows typical performance bottlenecks for conventional server-centric IT architectures. Finally, it shows how storage networks and intelligent storage systems help to overcome these performance bottlenecks. Before getting involved in technical details, we will ﬁrst discuss a few general conditions that should be taken into account in back-up . Then the back-up, archiving and hierarchical storage management services will be discussed and we will show which components are necessary for their implementation. This is followed by a summary of the measures discussed up to this point that are available to network back-up systems to increase performance . Then, on the basis of network back-up, further technical boundaries of server-centric IT architectures will be described that are beyond the scope of and we will explain why these performance bottlenecks can only be overcome to a limited degree within the server-centric IT architecture. Then we will show how data can be backed up signiﬁcantly more efﬁciently with a storage-centric IT architecture (Section 7.8). Building upon this, the protection of ﬁle servers and databases using storage networks and network back-up systems will be discussed. Finally, organizational aspects of data protection will be considered The consideration of network back-up concludes the use of storage networks.

GENERAL CONDITIONS FOR BACK-UP

Back-up is always a headache for system administrators. Increasing amounts of data have to be backed up in ever shorter periods of time. Although modern operating systems come with their own back-up tools, these tools only represent isolated solutions, which are completely inadequate in the face of the increasing number and heterogeneity of systems to be backed up. For example, there may be no option for monitoring centrally whether all back-ups have been successfully completed overnight or there may be a lack of overall management of the back-up media. Changing preconditions represent an additional hindrance to data protection. There are three main reasons for this: 1. As discussed in Chapter 1, installed storage capacity doubles every four to twelve

months depending upon the company in question. The data set is thus often growing more quickly than the infrastructure in general (personnel, network capacity).

1.Nevertheless, the ever-increasing quantities of data still have to be backed up.

2. Nowadays, business processes have to be adapted to changing requirements all the time. As business processes change, so the IT systems that support them also have to be adapted. As a result, the daily back-up routine must be continuously adapted to the ever-changing IT infrastructure.

3. As a result of globalization, the Internet and e-business, more and more data has to be available around the clock: it is no longer feasible to block user access to applications and data for hours whilst data is backed up. The time window for back-ups is becoming ever smaller. Network back-up can help us to get to grips with these problems.

NETWORK BACK-UP SERVICES

Network back-up systems such as Arcserve (Computer Associates), NetBackup (Veritas), Networker (EMC/Legato) and Tivoli Storage Manager (IBM) provide the following services:

• back-up

• archive

• hierarchical storage management.

The main task of network back-up systems is to back data up regularly. To this end,at least one up-to-date copy must be kept of all data, so that it can be restored after a hardware or application error ('ﬁle accidentally deleted or destroyed by editing', 'error in the database programming').

The purpose of archiving is to freeze a certain version of the data so that this precise version can be restored later on. For example, after the conclusion of a project its data can be archived on the back-up server and then deleted from the local hard disk. This saves local disk space and accelerates back-up and restore processes, since only the data that is actually being worked with has to be backed up or restored. Hierarchical storage management (HSM) ﬁnally leads the end user to believe that any desired size of hard disk is present. HSM moves ﬁles that have not been accessed for a long time from the local disk to the back-up server; only a directory entry remains in the local ﬁle server. The entry in the directory contains meta information such as ﬁle name, owner, access rights, date of last modiﬁcation and so on. The metadata takes up hardly any space in the ﬁle system compared to the actual ﬁle contents, so space is actually gained by moving the ﬁle content from the local disk to the back-up server. If a process accesses the content of a ﬁle that has been moved in this way, HSM blocks the accessing process, copies the ﬁle content back from the back-up server to the local ﬁle system and only then gives clearance to the accessing process. Apart from the longer access time, this process remains completely hidden to the accessing processes and thus also to end users. Older ﬁles can thus be automatically moved to cheaper media (tapes) and, if necessary, fetched back again without the end user having to alter his behaviour. Strictly speaking, HSM and back-up and archive are separate concepts. However, HSM is a component of many network back-up products, so the same components (media, software) can be used both for back-up, archive and also for HSM. When HSM is used, the back-up software used must at least be HSM-capable: it must back up the metadata of the moved ﬁles and the moved ﬁles themselves, without moving the ﬁle contents back to the client. HSM-capable back-up software can speed up back-up and restore processes because only the meta-information of the moved ﬁles has to be backed up and restored, not their ﬁle contents. A network back-up system realizes the above-mentioned functions of back-up, archive and hierarchical storage management by the co-ordination of back-up server and a range of back-up clients. The server provides central components such as the management of back-up media that are required by all back-up clients. However, different back-up clients are used for different operating systems and applications. These are specialized in the individual operating systems or applications in order to increase the efﬁciency of data protection or the efﬁciency of the movement of data. The use of terminology regarding network back-up systems is somewhat sloppy: the main task of network back-up systems is the back-up of data. Server and client instances of network back-up systems are therefore often known as the back-up server and back-up client, regardless of what tasks they perform or what they are used for. A particular server instance of a network back-up system could, for example, be used exclusively for HSM, so that this instance should actually be called a HSM server – nevertheless this instance would generally be called a back-up server. A client that provides the back-up function usually also supports archive and the restore of back-ups and archives – nevertheless this

client is generally just known as a back-up client. In this book we follow the general, untidy conventions, because the phrase 'back-up client' reads better than 'back-up-archive- HSM and restore client'. The two following sections discuss details of the back-up server) and the back-up client We then turn our attention to the performance and the use of network back-up systems.

2008-03-31T21:47:00.001-07:00

Free Tutors on APPLICATION OF STORAGE NETWORKS in Web applications based upon the 'travel portal' case study

This section uses the 'travel portal' case study to demonstrate the implementation of a so called web application. The case study is transferable to web applications for the support of business processes. It thus shows the possibilities opened up by the Internet and highlights the potential and the change that stand before us with the transformation to e-business. Furthermore, the example demonstrates once again how storage networks, server clusters and the ﬁve tier architecture can fulﬁl requirements such as the fault-tolerance, adaptability and scalability of IT systems.

Figure 6.36 shows the realization of the travel portal in the form of a web application. Web application means that users can use the information and services of the travel portal from various end devices such as PC, PDA and mobile phone if these are connected to the Internet. The travel portal initially supports only editorially prepared content (including ﬁlm reports, travel catalogues, transport timetable information) and content added by the users themselves (travel tips and discussion forums), which can be called up via conventional web browsers. Figure 6.37 shows the expansion of the travel portal by further end devices such as mobile phones and PDAs and by further services. To use the travel portal, users ﬁrst of all build up a connection to the representation server by entering the URL. Depending upon its type, the end device connects to a web server (HTTP server) or, for example, to a WAP server. The end user only perceives the web server as being a single web server. In fact, a cluster of representation servers is working in the background. The load balancer of the representation server accepts the request to build up a connection and passes it on to the computer with the lowest load. Once a connection has been built up the web browser transfers the user identiﬁer, for example, in the form of a cookie or the mobile number, and the properties of the end device (for example, screen resolution). The web server calls up the user proﬁle from the user management. Using this information the web server dynamically generates websites (HTML, WML or iMode) that are optimally oriented towards the requirements of the user. Thus the representation of content can be adjusted to suit the end device in use at

for the implementation of a three-tier architecture, since the clients (representation layer) communicate only with the application servers the time. Likewise, content, adverts and information can be matched to the preferences of the user; one person may be interested in the category of city tips for good restaurants,

whilst another is interested in museums. The expansion of the travel portal to include the new 'hotel tips' application takes place by the linking of the existing 'city maps' and 'hotel directory' databases (Figure 6.37). The application could limit the selection of hotels by a preferred price category stored in the user proﬁle or the current co-ordinates of the user transmitted by a mobile end device equipped with GPS. Likewise, the ﬁve-tier architecture facilitates the support of new end devices, without the underlying applications having to be modiﬁed. For example, in addition to the con- ventional web browsers and WAP phones shown in Figure 6.37 you could also implement mobile PDAs (low resolution end devices) and a pure voice interface for car drivers.

	Representation Client (Web browser) Ø Represents graphical user interface generated by the Representation tier Ø Passes on user interaction to Representation servers

Representation server (Web server) Ø Generates graphical user interface to External systems Ø Converts user interactions into application calls
Applications Ø Converts block oriented storage into tables(databases)or file systems Ø Reads ,Processes and deletes data that the data tier stores
Data Ø Converts block oriented storage into tables(databases)or file systems
Storage Ø Stores application data permanently on block- oriented storage devices

Figure In the ﬁve-tier architecture the representation layer is split up into representation server and representation client and the data layer is split into data management and storage devices All server machines are connected together via a fast network. Today primarily Gigabit Ethernet is used; in future InﬁniBand will presumably also be used. With the aid of appropriate cluster software, applications can be moved from one computer to another. Further computers can be added to the cluster if the overall performance of the cluster is not sufﬁcient. Storage networks bring with them the ﬂexibility needed to provide the travel portal with the necessary storage capacity. The individual servers impose different requirements on the storage network:

• Databases

The databases require storage space that meets the highest performance requirements. To simplify the administration of databases the data should not be stored directly upon raw devices, instead it should be stored within a ﬁle system in ﬁles that have been specially formatted for the database. Nowadays (2003), only disk subsystems connected via Fibre Channel are considered storage devices. NAS servers cannot yet be used in this situation due to the lack of availability of standardized high-speed network ﬁle systems such as RDMA enabled NFS. In future, it will be possible to use storage virtualization on ﬁle level here.

• Representation server and media servers

The representation servers augment the user interfaces with photos and small ﬁlms. These are stored on separate media servers that the end user's web browser can access directly over the Internet. As a result, the media do not need to travel through the internal buses of the representation servers, thus freeing these up. Since the end users access the media over the Internet via comparatively slow connections, NAS servers are very suitable. Depending upon the load upon the media servers, shared-nothing or shared-everything NAS servers can be used. Storage virtualization on ﬁle level again offers itself as an alternative here.

• Replication of the media servers

The users of the travel portal access it from various locations around the globe. Therefore, it is a good idea to store pictures and ﬁlms at various sites around the world so that the large data quantities are supplied to users from a server located near the user (Figure 6.38). This saves network capacity and generally accelerates the transmission of the data. The data on the various cache servers is synchronized by appropriate replication software. Incidentally, the use of the replication software is independent of whether the media servers at the various sites are conﬁgured as shared-nothing NAS servers, shared-everything NAS servers, or as a storage virtualization on the ﬁle-level.

In the ﬁrst part of the book the building blocks of storage networks were introduced. Building upon these, this chapter has explained the fundamental principles of the use of storage networks and shown how storage networks help to increase the availability and the adaptability of IT systems.

As an introduction to the use of storage networks, we elaborated upon the characteristics of storage networks by illustrating the layering of the techniques for storage networks, investigated various forms of storage networks in the I/O path and deﬁned storage networks in relation to data networks and voice networks. Storage resource sharing was introduced as a ﬁrst application of storage networks. Individually, disk storage pooling, tape library partitioning, tape library sharing and data sharing were considered. We described the redundant I/O buses and multipathing software, redundant server and cluster software, redundant disk subsystems and volume manager mirroring or disk sub- system remote mirroring to increase the availability of applications and data, and ﬁnally redundant storage virtualization. Based upon the case study 'protection of an important database' we showed how these measures can be combined to protect against the failure of a data centre. With regard to adaptability and scalability, the term 'cluster' was expanded to include the property of load distribution. Individually, shared-null conﬁgrations, shared-nothing clusters, enhanced shared-nothing clusters and shared-everything clusters were introduced. We then introduced the ﬁve-tier architecture – a ﬂexible and scalable architecture for IT systems. Finally, based upon the case study 'travel portal', we showed how clusters and the ﬁve-tier architecture can be used to implement ﬂexible and scalable web applications. As a further important application of storage networks, the next chapter discusses network back-up (Chapter 7). A ﬂexible and adaptable architecture for data protection is introduced and we show how network back-up systems can beneﬁt from the use of disk subsystems and storage networks.

2008-03-31T05:03:00.001-07:00

Know more about ADAPTABILITY AND SCALABILITY OF IT SYSTEMS and Clustering for load distribution, Web architecture

A further requirement of IT systems is that of adaptability and scalability: successful companies have to adapt their business processes to new market conditions in ever shorter cycles. Along the same lines, IT systems must be adapted to new business processes so that they can provide optimal support for these processes. Storage networks are also required to be scalable: on average the storage capacity required by a company doubles in the course of each year. This means that anyone who has 1 terabyte of data to manage today will have 32 terabytes in ﬁve years time. A company with only 250 gigabytes today

will reach 32 terabytes in seven years time. This section discusses the adaptability and scalability of IT systems on the basis of clusters for load distribution (Section 6.4.1), the ﬁve-tier architecture for web application servers (Section 6.4.2) and the case study 'structure of a travel portal' (Section 6.4.3).

Clustering for load distribution

The term 'cluster' is very frequently used in information technology, but which is not clearly deﬁned. The meaning of the term 'cluster' varies greatly depending upon context. As the greatest common denominator we can only state that a cluster is a combination of components or servers that perform a common function in one form or another. This section expands the cluster concept for protection against the failure of a server introduced in Section 6.3.2 to include clustering for load distribution. We discuss three different forms of clusters based upon the example of a ﬁle server. The three different forms of cluster are comparable to the modes of multipathing software. The starting point is the so-called shared-null conﬁguration (Figure 6.24). The components are not designed with built-in redundancy. If a server fails, the ﬁle system itself is no longer available, even if the data is mirrored on two different disk subsystems and

redundant I/O buses are installed between server and disk subsystems (Figure 6.25) In contrast to the shared-null conﬁguration, shared-nothing clusters protect against the failure of a server. The basic form of the shared-nothing cluster was discussed in Section 6.3.2 in relation to the protection of a ﬁle server against the failure of a server. Figure 6.26 once again shows two shared-nothing clusters each with two servers. Shared-nothing clusters can be differentiated into active/active and active/passive con- ﬁgurations. In the active/active conﬁguration, applications run on both computers; for example, the computers 'server 1' and 'server 2' in Figure 6.26 each export a ﬁle sys- tem. If one of the two computers fails, the other computer takes over the tasks of the failed computer in addition to its own (Figure 6.27, top). This taking over of the applications of the failed server can lead to performance bottlenecks in active/active conﬁgurations. The active/passive conﬁguration can help in this situation. In this approach the application runs only on the primary server, the second computer in the cluster (stand-by server) does nothing in normal operation. It is exclusively there to take over the applications of the primary server if this fails. If the primary server fails, the stand-by server takes over its tasks (Figure 6.27, bottom). The examples in Figures 6.26 and 6.27 show that shared-nothing clusters with only two servers are relatively inﬂexible. More ﬂexibility is offered by shared-nothing clusters with more than two servers, so-called enhanced shared-nothing clusters. Current shared-nothing cluster software supports shared-nothing clusters with several dozens of computers. Figures 6.28 and 6.29 show the use of an enhanced shared-nothing cluster for static load distribution: during the daytime when the system is busy, three different servers each export two ﬁle systems (Figure 6.28). At night, access to the data is still needed; however, a single server can manage the load for the six ﬁle systems (Figure 6.28). The two otherservers are freed up for other tasks in this period (data mining, batch processes, back-up, maintenance). One disadvantage of the enhanced shared-nothing cluster is that it can only react to load peaks very slowly. Appropriate load balancing software can, for example, move the ﬁle system '/fs2' to one of the other two servers even during the day if the load on the ﬁle system '/fs1' is higher. However, this takes some time, which means that this process

is only worthwhile for extended load peaks. A so-called shared-everything cluster offers more ﬂexibility in comparison to enhanced shared-nothing clusters. For ﬁle servers, shared disk ﬁle systems are used as local ﬁle systems here, so that all servers can access the data efﬁciently over the storage network. Figure 6.30 shows a ﬁle server that is conﬁgured as a shared-everything cluster with three servers. The shared disk ﬁle system is distributed over several disk subsystems. All three servers export this ﬁle system to the clients in the LAN over the same virtual IP address by means of a conventional network ﬁle system such as NFS or CIFS. Suitable load balancing software distributes new incoming accesses on the network ﬁle system equally amongst all three servers. If the three servers are not powerful enough, a fourth server can simply be linked to the cluster. The shared-everything cluster also offers advantages in the event of the failure of a

single server. For example, the ﬁle server in Figure 6.30 is realized in the form of a distributed application. If one server fails, as in Figure 6.31, recovery measures are only necessary for those clients that have just been served by the failed computer. Likewise, recovery measures are necessary for the parts of the shared disk ﬁle system and the network ﬁle system have just been managed by the failed computer. None of the other clients of the ﬁle server notice the failure of a computer apart from a possible reduction in performance. Despite their advantages, shared-everything clusters are very seldom used. The reason

for this is quite simply that this form of cluster is the most difﬁcult to realize, so most cluster products and applications only support the more simply realized variants of shared- nothing or enhanced shared-nothing.

Web architecture

In the 1990s the so-called three-tier architecture established itself as a ﬂexible architecture for IT systems (Figure 6.32). The three-tier architecture isolates the tasks of data management, applications and representation into three separate layers. Figure 6.33 shows a possible implementation of the three-tier architecture. Individually the three layers have the following tasks:

• Data Information in the form of data forms the basis for the three-tier architecture. Databases and ﬁle systems store the data of the applications on block-oriented disks or disk subsystems. In addition, the data layer can provide interfaces to external systems and legacy applications.

• Applications Applications generate and process data. Several applications can work on the same databases or ﬁle systems. Depending upon changes to the business processes, existing applications are modiﬁed and new applications added. The separation of applications and databases makes it possible for no changes, or only minimal changes, to have to be made to the underlying databases or ﬁle systems in the event of changes to applications.

• Representation The representation layer provides the user interface for the end user. In the 1990s the user interface was normally realized in the form of the graphical interface on a PC.

The corresponding function calls of the application are integrated into the graphical interface so that the application can be controlled from there. Currently, the two outer layers can be broken down into sublayers so that the three-tier architecture is further developed into a ﬁve-tier architecture Figure 6.34 and Figure 5: • Splitting of the representation layer

In recent years the representation layer has been split up by the World Wide Web into web servers and web browsers. The web servers provide statically or dynamically generated websites that are represented in the browsers. Websites with a functional scope comparable to that of conventional user interfaces can currently be generated using Java and various script languages. The arrival of mobile end devices such as mobile phones and PDAs has meant that

web servers had to make huge modiﬁcations to websites to bring them into line with the properties of the end devices. In future there will be user interfaces that are exclusively controlled by means of the spoken word – for example navigation systems for use in the car, that are connected to the Internet for requesting up-to-date trafﬁc data. • Splitting of the data layer In the 1990s, storage devices for data were closely coupled to the data servers (storage centric IT architecture). In the previous chapters storage networks were discussed in detail, so at this point of the book it should be no surprise to learn that the data layer

is split into the organization of the data (databases, ﬁle servers) and the storage space for data (disk subsystems).

Representation

Ø Graphical user interface and interfaces to external Systems

Ø Converts user Interactions into applications calls

Ø Represents the returns of the applications

Applications

Ø Carries Out operations initiated by users in the Representation tier

Ø Reads processes and deletes dara that the data tier stores

Ø Stores applications data permanently on block-oriented storage devices

Data

Ø Converts block oriented storage into tables(databases)or file systems

Ø Stores applications data permanently on block-oriented storage devices

2008-03-31T04:28:00.001-07:00

Free Storage tutors Failure of virtualization in the storage network and case study on the Failure of a data centre

Failure of virtualization in the storage network

Virtualization in the storage network is currently (2003) treated as the solution for the consolidation of storage resources in large storage networks, with which the storage resources of several disk subsystems can be centrally managed (Chapter 5). However, it is necessary to be clear about the fact that precisely such a central virtualization instance represents a single point of failure. Even if the virtualization instance is protected against the failure of a single component by measures such as clustering, the data of an entire data centre can be lost as a result of conﬁguration errors or software errors in the virtualization instance,

since the storage virtualization aims to span all the storage resources of a data centre. Therefore, the same considerations apply for the protection of a virtualization instance positioned in the storage network (Section 5.7, 'Symmetric and Asymmetric Storage Virtualization in the Network') against the failure as the measures to protect against the failure of a disk subsystem discussed in the previous section. Therefore, the mirroring of important data from the server via two virtualization instances should also be considered in the case of virtualization in the storage network.

Failure of a data centre based upon the case study

'protection of an important database' The measures of server clustering, redundant I/O buses and disk subsystem mirroring (volume manager mirroring or remote mirroring) discussed above protect against the failure of a component within a data centre. However, these measures are useless in the event of the failure of a complete data centre (ﬁre, water damage). To protect against the failure of a data centre it is necessary to duplicate the necessary infrastructure in a

back-up data centre for the operation of the most important applications. Figure 6.22 shows the interaction between the primary data centre and back-up data

centre based upon the case study 'protection of an important database'. In the case study, all the measures discussed in this section for protection against the failure of a component are used. In the primary data centre all components are designed with built-in redundancy. The primary server is connected via two independent Fibre Channel SANs (Dual SAN) to two disk subsystems, on which the data of the database lies. Dual SANs have the advantage that even in the event of a serious fault in a SAN (defective switch, which corrupts the SAN with corrupt frames), the connection via the other SAN remains intact. The redundant paths between servers and storage devices are managed by appropriate multipathing software. Each disk subsystem is conﬁgured using a RAID procedure so that the failure of individual physical disks within the disk subsystem in question can be rectiﬁed. In addition, the data is mirrored in the volume manager so that the system can withstand the failure of a disk subsystem. The two disk subsystems are located at a distance from one another in the primary data centre. They are separated from one another by a ﬁre protection wall. Like the disk subsystems, the two servers are spatially separated by a ﬁre protection

wall. In normal operation the database runs on one server; in the meantime the second server is used for other, less important tasks. If the primary server fails, the cluster software automatically starts the database on the second computer. It also terminates all other activities on the second computer, thus making all its resources fully available to the main application. Remote mirroring takes place via an IP connection. Mirroring utilizes knowledge of the

data structure of the database: in a similarmanner to journaling in ﬁle systems (Section 4.1.2), databases write each change into a log ﬁle before then integrating it into the actual data set. In the example, only the log ﬁles are mirrored in the back-up data centre. The complete data set was only transferred to the back-up data centre once at the start of mirroring. Thereafter this data set is only ever adjusted with the aid of the log ﬁles. This has two advantages: the powerful network connection between the primary data centre and the remote back-up data centre is very expensive. The necessary data rate for this connection can be halved by only transferring the changes to the log ﬁle. This cuts costs. In the back-up data centre the log ﬁles are integrated into the data set after a delay of

two hours. As a result, a copy of the data set that is two hours old is always available in the back-up data centre. This additionally protects against application errors: if a table space is accidentally deleted in the database then the user has two hours to notice the error and interrupt the copying of the changes in the back-up data centre. A second server and a second disk subsystem are also operated in the back-up data centre, which in normal operation can be used as a test system or for other, less time- critical tasks such as data mining. If the operation of the database is moved to the back-up data centre, these activities are suspended (Figure 6.23). The second server is conﬁgured as a stand-by server for the ﬁrst server in the cluster; the data of the ﬁrst disk subsystem is mirrored to the second disk subsystem via the volume manager. Thus a completely redundant system is available in the back-up data centre. The realization of the case study discussed here is possible with current technology. However, it comes at a price; for most applications this cost will certainly not be justiﬁed. The main point of the case study is to highlight the possibilities of storage networks. In practice you have to decide how much failure protection is necessary and how uch this may cost. At the end of the day, protection against the loss of data or the temporary non-availability of applications must cost less than the data loss or the temporary non- availability of applications itself.