Free Tutors the snia shared storage model Graphical representations An elementary overview and The components

Free Tutors the snia shared storage model  Graphical representations An elementary overview and The components

The SNIA Shared Storage Model further defines how storage architectures can be graphically illustrated. Physical components are always represented as three-dimensional objects, whilst functional units should be drawn in two-dimensional form. The model itself also defines various colours for the representation of individual component classes. In the black and white format of the book, we have imitated these using shades of grey. A coloured version of the illustrations to this chapter can be found on our home page Thick lines in the model represent the data transfer,

whereas thin lines represent the metadata flow between the components.

 An elementary overview

The SNIA Shared Storage Model first of all defines four elementary parts of a shared storage environment (Figure 10.1):

1. File/record layer The file/record layer is made up of database and file system.

2. Block layer The block layer encompasses the storage devices and the block aggregation. The SNIA Shared Storage Model uses the term 'aggregation' instead of the often ambiguously used term 'storage virtualization'. In Chapter 5, however, we used the term 'storage virtualization' to mean the same thing as 'aggregation' in the SNIA model, in order to avoid ambiguity.

3. Services subsystem The functions for the management of the other components are defined in the services subsystem.

4. Applications Applications are not discussed further by the model. They will be viewed as users o the model in the widest sense.

 The components

The SNIA Shared Storage Model defines the following components:

• Interconnection network The interconnection network represents the storage network, i.e. the infrastructure, that connects the individual elements of a shared storage environment with one another. The interconnection network can be used exclusively for storage access, but it can

also be used for other communication services. Our definition of a storage network (Section 1.2) is thus narrower than the dentitions of the interconnection network in the SNIA model. The network must always provide a high-performance and easily scalable connection for the shared storage environment. In this context, the structure of the interconnection network – for example redundant data paths between two components to increase fault-tolerance – remains just as open as the network techniques used. It is therefore a prerequisite of the model that the components of the shared storage environment are connected over a network without any definite communication protocols or transmission techniques being specified.

In actual architectures or installations, Fibre Channel, Fast Ethernet, Gigabit Ethernet, InfiniBand and many other transmission techniques are used (Chapter 3). Communication protocols such as SCSI, Fibre Channel FCP, TCP/IP, RDMA, CIFS or NFS are based upon these.

• Host computer Host computer is the term used for computer systems that draw at least some of their storage from the shared storage environment. According to SNIA, these systems were often omitted from classical descriptive approaches and not viewed as part of the environment. The SNIA shared storage model, however, views these systems as part of the entire shared storage environment because storage-related functions can be implemented on them. Host computers are connected to the storage network via host bus adapters or network

cards, which are operated by means of their own drivers and software. Drivers and software are thus taken into account in the SNIA Shared Storage Model. Host computers can be operated fully independently of one another or they can work on the resources of the storage network in a compound, for example, a cluster

• Physical storage resource All further elements that are connected to the storage network and are not host computers are known by the term 'physical storage resource'. This includes simple hard disk drives, disk arrays, disk subsystems and controllers plus tape drives and tape libraries. Physical storage resources are protected against failures by means of redundant data paths (Section 6.3.1), replication functions such as snapshots and mirroring (Section 2.7) and RAID (Section 2.5).

\• Storage device A storage device is a special physical storage resource that stores data.

• Logical storage resource The term 'logical storage resource' is used to mean services or abstract compositions of physical storage resources, storage management functions or a combination of these. Typical examples are volumes, files and data movers.

• Storage management functions The term 'storage management function' is used to mean the class of services that monitor and check (Chapter 8) the shared storage environment or implement logical storage resources. These functions are typically implemented by software on physical

storage resources or host computers.

Free tutors on the SNIA Shared Storage Model and The functional approach

Free tutors on the SNIA Shared Storage Model and The functional approach

The fact that there is a lack of any unified terminology for the description of storage architectures has already become apparent at several points in previous chapters. There are thus numerous components in a storage network which, although they do the same thing, are called by different names. Conversely, there are many systems with the same name, but fundamentally different functions. A notable example is the term' data mover' relating to server-free back-up (Section 7.8.1) in storage networks. When this term is used it is always necessary to check whether the component in question is one that functions in the sense of the 3rd-party SCSI Copy Command for, for example, a software component of back-up software on a special server, which implements the server-free back-up without 3rd-party SCSI. This example shows that the type of product being offered by a manufacturer and the functions that the customer can ultimately expect from this product are often unclear. This makes it difficult for customers to compare the products of individual manufacturers and find out the differences between the alternatives on offer. There is no unified model for this with clearly defined descriptive terminology. For this reason, in 2001 the Technical Council of the Storage Networking Industry Association (SNIA) introduced the so-called Shared Storage Model in order to unify the terminology and descriptive models used by the storage network industry. Ultimately, the SNIA wants to use the SNIA Shared Storage Model to establish a reference model, which will have the same importance for storage architectures as the seven-tier OSI model has for computer networks. In this chapter, we would first like to introduce the disk-based Shared Storage Model (Section 10.1) and then show, based upon examples (Section 10.2), how the model can be used for the description of typical disk storage architectures. In Section 10.3 we introduce the extension of the SNIA model to the description of tape functions. We then discuss examples of tape-based back-up architectures (Section 10.4). Whilst describing the SNIA Shared Storage Model we often refer to text positions in this book where the subject in question is discussed in detail, which means that this chapter also serves as a summary of the entire book.

THE MODEL

In this book we have spoken in detail about the advantages of the storage-centric architecture in relation to the server-centric architecture. The SNIA sees its main task as being to communicate this paradigm shift and to provide a forum for manufacturers and developers so that they can work together to meet the challenges and solve the problems in this field. In the long run, an additional reason for the development of the Shared Storage Model by SNIA was the creation of a common basis for communication between the manufacturers who use the SNIA as a platform for the exchange of ideas with other manufacturers. Storage-centric IT architectures are called shared storage environments by the SNIA. We will use both terms in the following. First of all, we will describe the functional approach of the SNIA model (Section 10.1.1) and the SNIA conventions for graphical representation (Section 10.1.2). We will then consider the model (Section 10.1.3), its components (Section 10.1.4) and the layers' file/record layer' and 'block layer' in detail (Section 10.1.5 to Section 10.1.8). Then we will introduce the definitions and representation of concepts from the SNIA

model, such as access paths (Section 10.1.9), caching (Section 10.1.10), access control (Section 10.1.11), clustering (Section 10.1.12), data (Section 10.1.13) and resource and data sharing (Section 10.1.14). Finally, we will take a look at the service subsystem (Section 10.1.15).

The functional approach

The SNIA Shared Storage Model first of all describes functions that have to be provided in a storage-centric IT architecture. This includes, for example, the block layer or the file/record layer. The SNIA model describes both the tasks of the individual functions and also their interaction. Furthermore, it introduces components such as server ('host computer') and storage networks ('interconnection network'). Due to the separation of functions and components, the SNIA Shared Storage Model is suitable for the description of various architectures, specific products and concrete

installations. The fundamental structures, such as the functions and services of a shared storage environment, are highlighted. In this manner, functional responsibilities can be assigned to individual components and the relationships between control and data flows in the storage network worked out. At the same time, the preconditions for interoperability between individual components and the type of interoperability can be identified. In addition to providing a clear terminology for the elementary concepts, the model should be simple to use and, at the same time, extensive enough to cover a large number of possible storage network configurations. The model itself describes, on the basis of examples, possible practicable storage architectures and their advantages and disadvantages. We will discuss these in Section 10.2  without evaluating them or showing any preference for specific architectures. Within the model definition, however, only a few selected examples will be discussed in order to highlight how the model can be applied for the description of storage-centred environments and further used.

Know more about THE IEEE 1244 STANDARD FOR REMOVABLE MEDIA MANAGEMENT Operational characteristics of the media manager

Know more about THE IEEE 1244 STANDARD FOR REMOVABLE MEDIA MANAGEMENT  Operational characteristics of the media manager

From the point of view of the client, the media manager works as a server that waits for MMP commands, which the client sends via a TCP/IP connection. The media manage executes these commands, generates appropriate responses and sends these back to the clients. All commands are given unambiguous task identifiers. The responses contain the task identifier of the command in question. The response takes place in two stages. First, the successful receipt of the command is acknowledged. In a second response, the application is informed whether the command has been successfully executed and which responses the system has supplied. Example 1 An application wants to mount the volume with the name back-up-1999-12 31. To this end it, sends the following command to the media manager: mount task[' '1' '] volname [' 'back-up-1999-12-31' ']

report [MOUNTLOGICAL.' 'MountLogicalHandle' ']; The media manager has recognized the command and accepted it for processing and

therefore sends the following response: response task[' '1' '] accepted; Now the media manager will transport the cartridge containing the volume into a drive to which the application has access. Once the cartridge has been successfully inserted, a response is generated that could look like this:

response task[' '1' '] success text [' '/dev/rmt0' ']; The media manager stores all commands in a task queue until all resources required for execution are available. Once all the resources are available, the media manager removes the command from the task queue and executes it. If several commands are present that require the same resources, the media manager selects the next command to be carried out on the basis of priorities or on a first come, first served basis. All other commands remain in the task queue until the resources in question become free again. In this manner

libraries, drives and also cartridges can be shared. Commands that are in the task queue can be removed again using the Cancel command

Operational characteristics of the library and drive managers The library manager receives the media manager's commands via the library manage

ment protocol (LMP) and converts these into the specific commands for the hardware in question. From the point of view of the media manager, a unified abstract interface tha conceals the properties of the hardware in question thus exists for all libraries. New hard ware can thus be integrated into the management system using a suitable library manager without having to make changes to the whole system. Accordingly, drive manager implementations of the abstract drive management protocol (DMP) are interfaces for a certain drive hardware. However, drive management

must also take into account the specific properties of the various client platforms upon which the applications that want to use the media management system run. If such an application is running on a UNIX-compatible platform, the drive manager must provide the corresponding names of a device special file for access to the drive. Under Windows, such a drive manager must supply a windows-specific file name, such as \\.\TAPE0.

Privileged and non-privileged clients The media manager carries out requests from clients that want to take advantage of the media management services. From the point of view of the media manager there are privileged and non-privileged clients:

• Non-privileged clients, such as back-up systems, can only handle objects for which they have been granted an appropriate authorization.

• Privileged clients, usually administrative applications, may perform all actions and manipulate all objects. They serve primarily to include non-privileged applications in the system and to establish suitable access controls.  The IEEE 1244 data model (Table 9.1) In addition to the architecture and the logs for communication, the standard also describes a complete data model, which includes all objects, and their attributes, that are necessary for the representation of the media management system. Objects can be provided with additional application-specific attributes. The object model can thus be dynamically and flexibly adapted to the task at hand, without changes being necessary to the underlying management system. 9.5.3 Media Management Protocol (MMP) The media management protocol (MMP) is used by the applications to make use of

the media management services of an IEEE 1244-compatible system. MMP is a text based protocol, which exchanges messages over TCP/IP. The syntax and semantics of the individual protocol messages are specified in the MMP specification IEEE 1244.3. MMP permits applications to allocate and mount volumes, read and write metadata and to manage and share libraries and drives platform-independently. Due to the additional bstraction levels, the application is decoupled from the direct control of the hardware. Thus, applications can be developed independently of the capability of the connected hard- ware and can be made available to a large number of different types of removable media.

Table 9.1 The most important objects of the IEEE 1244 data model

Object Description

APPLICATION Authorized client application. Access control is performed on the basis of applications. User management is not part of this standard, since it is assumed that it is not individual users, but applications that already manage their users, that will

use the services of the media management system.  AI Authorized instances of a client application. All instances of an application have unrestricted access to resources that are assigned to the application. LIBRARY Automatic or manually operated libraries.

LM Library managers know the details of a library. The library manager protocol serves as a hardware-independent interface between media

manager and library manager. BAY Part of a LIBRARY (contains DRIVES and SLOTS). SLOT Individual storage space for CARTRIDGEs within a

BAY. SLOTGROUP Group of SLOTS to represent a magazine, for example, within a LIBRARY. SLOTTYPE Valid types for SLOTs, for example 'LTO',

'DL-Tor3480', 'QIC' or 'CDROM'. DRIVE Drives, which can accept CARTRIDGEs for writing or reading. DRIVEGROUP Groups of drives.

DRIVEGROUPAPPLICATION This object makes it possible for applications to access drives in a DRIVEGROUP. This connection can be assigned a priority so that several DRIVEGROUPs with different priorities are available. The media manager selects a suitable drive according to priority.

DM Drive manager. Drive managers know the details of a drive and make this available to the media manager. The drive manager protocol serves as a hardware-independent interface between media manager and drive manager. CARTRIDGE Removable data carrier; media. CARTRIDGEGROUP Group of CARTRIDGEs. CARTRIDGEGROUPAPPLICATION This object makes it possible for applications to access CARTRIDGEs in a CARTRIDGEGROUP.

This connection can be assigned a priority so that several CARTRIDGEGROUPs with different priorities are available. The media manager selects a

suitable CARTRIDGE according to priority in order to allocate a VOLUME, if no further entries are made. (continued overleaf )

FREE TUTORS ABOUT THE IEEE 1244 STANDARD FOR REMOVABLE MEDIA MANAGEMENT SYSTEM ARCHITECTURE

FREE TUTORS ABOUT THE IEEE 1244 STANDARD FOR REMOVABLE MEDIA MANAGEMENT SYSTEM ARCHITECTURE

As early as 1990, the IEEE Computer Society set up the 1244 project for the development of standards for storage systems. The Storage System Standards Working Group was also established with the objective of developing a reference model for mass storage systems (Mass Storage System Reference Model/MSSRM). This reference model has significantly influenced the design of some storage systems that are in use today. The model was then revised a few times and in 1994 released as the IEEE Reference Model for Open Storage Systems Interconnection (OSSI). Finally, in the year 2000, after further revisions, the 1244 Standard for Media Management Systems was released. This standard consists of a series of documents that describe a platform-independent,distributed management system for removable media. It also defines both the architecture for a removable media management system and its interfaces towards the outside world. The architecture makes it possible for software manufacturers to implement very

scalable, distributed software systems, which serve as generic middleware between application software and library and drive hardware. The services of the system can thus be consolidated in a central component and from there made available to all applications. The specification paid particular attention to platform-independence and the heterogeneous environment of current storage networks was thus taken into account mazingly early on. Systems that build upon this standard can manage different types of media. In addition to the typical media for the computer field such as magnetic tape, CD, DVD or optical media, audio and video tapes, files and video disks can also be managed. In actual fact, there are no assumptions about the properties of a medium in IEEE 1244- compliant systems. Their characteristic features (number of sides, number of partitions, etc.) must be defined for each media type that the system is to support. There is a series of predefined types, each with their own properties. This open design makes it possible to specify new media types and their properties at any time and to add them to the current system. In addition to neutrality with regard to media types, the standard permits the management of both automatic and manually-operated libraries. An operator interface, which is

also documented, and with which messages are sent to the appropriate administrators of a library, serves this purpose. In the following sections we wish to examine more closely the architecture and functionality of a system based upon the IEEE standard.

 Media management system architecture

The IEEE 1244 standard describes a client/server architecture (Figure 9.9). Applications such as network back-up systems take on the role of the client that makes use of the services of the removable media management system. The following components are individually defined:

 • a media management component, which serves as a central repository for the metadata and provides mechanisms for controlling and co-ordinating the use of media, libraries and drives;

• a library manager component, which controls the library hardware on behalf of the media manager and transmits the properties and the content of the library to the media manager;

• a drive manager component, which manages the drive hardware on behalf of the media manager and transmits the properties of the drives to the media manager. In addition, the standard defines the interfaces for the communication with these components:

• the Media Management Protocol (MMP) for the communication between application (client) and media manager (server);

• the Library Management Protocol (LMP) for the communication between library manager and media manager;

• the Drive Management Protocol (DMP) for the communication between drive manager and media manager.

These protocols use TCP/IP as the transport layer. As in the popular Internet applications HTTP, FTP or SMTP, commands are sent via TCP/IP in the form of text messages. These protocols can be implemented and used just as simply on different platforms. The advantage of this approach is that the media manager can implement components as a generic application, i.e. independently of the specific library and drive hardware used. The differences, in particular with the control, are encapsulated in the library manager or drive manager for the hardware in question. For a new tape library, therefore, only a new library manager component needs to be implemented that converts the specific interface of the library into the library management protocol, so that this can be linked into an existing media manager installation. The next sections describe how communication takes place between clients and servers and how the media manager processes the commands.

Learn more on Life cycle management

Learn more on  Life cycle management

The life cycle of a cartridge describes a series of states that a cartridge can take on over the course of time. Essentially, the following events, which lead to state transitions, can be observed in the life cycle:

• Initialization Cartridges are announced to the system.

• Allocation of access rights Applications are permitted to access certain cartridges.

• Use Cartridges are used for reading and writing.

• deallocation The data on a cartridge is no longer needed; the storage space can once again be made available to other applications.

• Retirement The cartridge has reached the end of its life cycle and is removed from the system. These events directly yield a series of states for cartridges. States of media In addition to the location of the media, it is also important that storage administrators are aware of the state of the media. For example, a cartridge may not be removed from the system until no more logical volumes have been allocated to it. Otherwise, there is a

danger of data loss, because back-up software can no longer access this volume. During its life cycle (Figure 9.8) a cartridge can take on the following states:

• Undefined (unknown) No further information is known about a cartridge. This is the initial state of a cartridge before it is taken into a management system.

• Defined A cartridge is announced to the system. Information about the type, cartridge label, etc. are given, and the cartridge is thus described.

• Available In addition to the information that a cartridge exists in the system, information about whether, and where, data can be still written to the cartridge is also important. If this information is known, the cartridge is available for applications

• Allocated In order that an application can use a cartridge, suitable storage space on a cartridge

must be allocated by the system. This allocation of storage space generally leads to the placing of a volume on a partition of a cartridge. The application should be able to freely choose the identification of the volume. In general, the state of the first volume placed determines the state of the entire cartridge. To this end, as soon as the first volume has been placed on a cartridge, the state of the cartridge is set to 'allocated'.

• deallocated If the storage space (the volume) is no longer required by an application, the volume can be de-allocated. The system should then delete all information about the volume.

The application should continue to be able to reallocate the same storage space. If the

storage space is to be made available to other applications, a cartridge must be recycled.

• Recycled Depending upon system configuration, once all volumes have been removed from a cartridge the entire storage space can be made available to other applications.

• Purged A cartridge, and all information about this cartridge, is completely removed from the system. In general, this state is reached at the end of a cartridge's life cycle. Typically, this state exists only for a short time, since the cartridge is immediately placed in the undefined state.

Policy-based life cycle management Certain tasks should be automated so that as little manual intervention as possible is required during the management of the data carriers. In life cycle management, some tasks positively demand to be performed automatically. These include:

• the monitoring of retention periods;

• transportation to the next storage location (movement or rotation);

• the copying of media when it reaches a certain age;

• the deletion of media at the end of the storage period;

• the recycling or automatic removal of the cartridge from the system.

The individual parameters for the automated tasks are specified by suitable policies.

Individual cartridges, or groups of cartridges, are assigned suitable policies.

Tutors on Monitoring management of removable media

Tutors on Monitoring management of removable media

The large number of devices and media that have to be monitored in a data centre makes it almost impossible for monitoring to be performed exclusively by administrators. Automatic control of the system, or at least of parts of it, is therefore absolutely necessary for installations above a certain size. For removable media, in particular, it is important that monitoring is well constructed because in daily operation there is too little time to verify every back-up. If errors creep in whilst the system is writing to tape this may not be recognized until the data needs to be restored – when it is too late. If there is no second copy, the worst conceivable incident for the datacenter has occurred: data loss! Modern tape drives permit a very good monitoring of their state. This means that the number of read-write errors that cannot be rectified by the built-in firmware, and also the

number of load operations, are stored in the drive. Ideally, this data will be read by the management system and stored so that it is available for further evaluations. A further step would be to have this data automatically analyzed by the system. If certain error states are reached, actions can be triggered automatically so that at least no further error states are permitted. Under certain circumstances, errors can even be rectified automatically, for example by switching a drive off and back on again. In the worst case, it is only possible to mark the drive as defective so that it is not used further. In these tasks, too, a mechanism controlled by means of rules can help and significantly take the pressure off the administrator.

The data stored on the drives not only provides information on the drives themselves, but also on the loaded tapes. This data can be used to realize a tape quality management, which, for example, monitors the error rates when reading and writing and, if necessary, copies the data to a new tape if a certain threshold is exceeded.

Reporting

In addition to media management and drive and library sharing a powerful system requires the recording of all actions. For certain services it is even a legal requirement that so-called security audits are performed. Therefore, all actions must be precisely logged. In addition, he log data must be protected against manipulation in an appropriate manner. With the aid of a powerful interface, it should be possible to request data including

the following:

• When was a cartridge incorporated into the system?

• Who allocated which volume to which cartridge when?

• Who accessed which volume when?

• Was this volume just read or also written?

• Which drive was used?

• Was this an authorized access or was access refused? The following requirements should be fulfilled by the reporting module of a removable

media management system:

• Audit trails As already mentioned, it should be possible to obtain a complete list of all accesses to

a medium. Individual entries in this list should give information about who accessed a

medium, for how long, and with what access rights.

• Usage statistics Data about when the drives were used, and for how long they were used, is important

in order to make qualitative statements about the actual utilization of all drives. At

any point in time, were sufficient drives available to carry out all mount requests? Are

more drives available than the maximum amount needed at the same time over the last

twelve months? The answers to such questions can be found in the report data. Like

the utilization of the drives, the available storage space is, of course, also of interest.

Was enough free capacity available? Were there bottlenecks?

• Error statistics Just like the data on the use of resources, data regarding the errors that occurred during use is also of great importance for the successful use of removable media. Have the storage media of manufacturer X caused less read-write errors in the drives of manufacturer Y than the media of manufacturer Z? Appropriate evaluations help considerably in the optimization of the overall performance of a system.

• Future planning  Predictions for the future can be made from the above-mentioned statistics. How will the need for storage grow? How many drives will be used in twelve months? And  MANAGEMENT OF REMOVABLE MEDIA how many slots and cartridges? A management system should be able to help in the search for answers to these questions. In order that future changes can also be simply carried out, the addition of further drives or cartridges must also be possible without any problems and must not require any changes to the existing applications that use the management services.

Free tutors on Media tracking Grouping, pooling Drive pools

Free tutors on Media tracking Grouping, pooling Drive pools

As an integral part of a disaster recovery solution, a management system must ensure that all removable media plus the appropriate metadata remain in the system until they are deleted by an appropriately authorized user. Thus, under no circumstances may tapes be 'lost' or removed from the system without authorization. Furthermore, it must be possible to determine the storage location of each medium at all times. If access is available to the media online, for example, in automatic tape libraries in which tapes can be automatically identified by the reading of a barcode label by a scanner, a suitable audit can be performed at any time. In such an audit, the content of the inventory is compared with the real existing tapes. Such libraries also automatically report the opening of a door. After the door has been closed an audit should once again be automatically performed in order to ensure that no media has been removed from the system without authorization. If a part of the media is withdrawn from direct access, either to a well-protected safe or to another manually-operated library, this storage place must be managed with appropriate care by the responsible administrators. Ideally, the management software provides an interface for this vaulting. In order to increase the reliability of a disaster recovery concept and to fulfil statutory provisions, a two-stage or multi-stage strategy made up of online and offline storage is often performed. Storage media are first written in automatic libraries, then stored offline for a certain period of time and subsequently either taken out of circulation or reused (Figure 9.5). Media that are in transit from an online library to an offline storage place must be identified. A management system for removable media should serve both as a central repository for all resources and as a universal interface for applications. It should not be possible for any application to withdraw itself from the control of the system and access or move media in an uncontrolled manner. Only thus is it actually guaranteed that all media in the system can be located at any time. Such a central interface is always in danger of becoming a single point of failure. It is therefore very wise to use appropriate measures to guarantee a high level of availability for the entire solution. Consequently, care must be taken to ensure that all components of the entire system, from the hardware through the operating systems used with the media management to the back-up software, are designed to have a high level of availability. The hardware often offers suitable options. Drives and media changers are available with a redundant power supply and redundant access paths. Modern operating systems such as AIX or Solaris can automatically use such redundantly designed access paths in the  event of a fault.

Grouping, pooling

Systems for the management of removable media must be able to deal with a great many media and facilitate access to many applications. In order to plan and execute access control in a sensible manner and to guarantee its effective use, it is necessary to combine cartridges and drives into groups. Grouping also allows budgets for storage capacities or drives to be created, which are made available to the applications. Scratch pools

A scratch pool contains unused cartridges that are available to authorized applications so that they can place volumes upon them. As soon as an application has placed a volume upon a cartridge from such a scratch pool, this cartridge is no longer available to all other applications and is thus removed from the scratch pool. If several scratch pools are available, they cannot only be grouped for different media, it is also possible to define groups for certain application purposes. For example, an administrator should be able to make a separate pool of cartridges available for important

back-up jobs, whilst cartridges from a different scratch pool are used for 'normal' back-up jobs. The back-up application can choose which pool it would like to have a cartridge from. If it is possible to assign priorities to scratch pools on the basis of which the management system can decide from which pool a new cartridge will be provided, then such a request can be automated to a certain degree without the back-up application

having to know all available pools. To this end, the request for a new tape must be given an appropriate priority, whereupon the management system searches for a cartridge from a scratch pool with a suitably high priority. In addition to the requirement that all cartridges can be located at all times, a management system for removable media should be capable of offering free storage space to an application at any time. Only thus can back-up windows actually be adhered to. As is the case for media tracking, in order to fulfil this requirement a high-availability solution

covering all levels, from the hardware to the application software, should be pursued. In addition, scratch pools can help to contain the cartridges from two or more libraries (Figure 9.6). They also offer the guarantee that, even in the event of the failure of indi- vidual libraries, cartridges in the other libraries will remain usable. It is precisely in this case that the advantages of a storage network, together with an intelligent management

system for removable media, fully come to bear in the optimal utilization of resources that are distributed throughout the entire system.

In order to be able to react flexibly to changes, scratch pools should be dynamically expandable. To this end, an administrator must make additional storage space available to the system dynamically, whether by the connection of a new library or by the addition of previously unused cartridges. Ideally, this can be achieved without making changes to the applications that have previously accessed the scratch pool. An adjustable minimum size (low water mark) makes the management of a scratch pool easier. If this threshold is reached, measures must be taken to increase the size of the

pool, as otherwise there is the danger that the system will cease to be able to provide free storage space in the foreseeable future. The management system can help here by flexibly offering more options. Many actions are possible here, from the automatic enlargement of the scratch pools – as long as free media are available in the libraries – to the 'call home' function, in which an administrator is notified. If a cartridge has several partitions, then it would occasionally be desirable to collect just the free partitions – rather than the complete cartridges – into a scratch pool. Then

it would be possible to manage free storage capacity with a finer granularity and thus achieve an optimal utilization of the total amount of available storage capacity. Since, owever, the individual partitions of a medium cannot be accessed at the same time, a cartridge is currently generally managed and allocated to an application as the smallest unit of a scratch pool. As capacity increases, however, the additional use of partitions for

this may also be required.

Drive pools

The mere fact that cartridges are available does not actually mean that the storage space can be used. In addition, drives must be available that can mount the cartridges for reading or writing. Similarly to cartridges, it is also possible to combine drives into pools. A pool of high- priority drives can, for example, always be kept to fulfil mount requests if all other drives are fully utilized. In order to save the applications from having to know and request all drive pools, it is a good idea to have a priority attribute that is used by the management system to automatically locate a drive with an appropriate priority. If several libraries are available, drive pools should include drives from several libraries (Figure 9.7). This ensures that drives are still available even if one library has failed. At the very least, this helps when writing new data if free cartridges are still available.