Learn Increasing complexity in the administration of large storage networks and Proposed solution: storage virtualization

Learn  Increasing complexity in the administration of large storage networks and Proposed solution: storage virtualization

 

The introduction of storage networks is a first step towards the efficient management of storage resources, but it alone cannot solve the problems associated with the data explosion. In large environments with several dozens of servers, each working on several disks, a storage administrator has to deal with hundreds of virtual and even more physical disks. In the following we wish to describe some difficulties associated with the administration of storage resources and the requirements of data and data users in such large environments. Storage networks offer the possibility of realizing requirements such as scalability, availability, performance, data protection and migration. Often, however, this calls for complex configurations, the administration of which again requires additional time and expense. Furthermore, requirements regarding availability, performance, data protection or migration are different for different data. For example, certain data has to be available even in the event of the failure of its storage resources, whereas it is possible to temporarily manage without other data. Some data has to be available to applications quickly, for example in order to keep the response times of a database low. For other data the response time is not so critical, but it may have to be backed up frequently. Still other data tends to change so seldom that frequent back-ups are not required. All this has to be taken into consideration when distributing the data on the resources. Data that requires a high throughput should therefore be stored on faster – and thus more expensive – media such as hard disks, whereas other data can be moved to slower – and thus cheaper – tapes. A storage administrator is not able to match the requirements for the data to the physical storage devices. In large environments it would be too much to deal with the needs of every single virtual disk which is assigned to a server. Therefore, corresponding data profiles have to be created that specify the requirements of the data. An entity is required here that creates such data profiles automatically based particularly upon the data usage, and realizes these accordingly. In this manner load peaks in the use of individual data can be recognized so that this data can be moved to fast media or can be distributed over several resources in order to achieve a balanced resource utilization. Furthermore, users of storage do not want to think about the size, type and location of the storage media when using their applications. They demand an intuitive handling of the storage media. The actual processes that are necessary for the administration of storage should remain hidden and run so that they are invisible to users. Users of storage space are, however, very interested in response times, data throughput and the availability of their applications. In short: users only think about what is related to their applications and not to the physical aspects of their data. The creation of this user- friendliness and the respective service level agreements for storage resources imposes additional requirements in storage administrators.

 Proposed solution: storage virtualization

To sum up, we can say that the implementation of a storage network alone does not meet the requirements for the management of large quantities of data. This requires additional mechanisms that simplify administration and at the same time make it possible to make full use of the storage resources. The use of storage virtualization software offers the appropriate possibilities for, on the one hand, simplifying the administration of data and storage resources and, on the other, making their use by the users easier. The objectives of storage virtualization can be summed up by the following three points: • simplification of the administration and access of storage resources; • full utilization of the possibilities of a storage network: the possibilities of a storage network should be fully utilized with regard to the efficient use of resources and data, the improvement of performance and protection in the event of failures by a high levelof data availability;  • realization of advanced storage functions that are oriented towards the data profiles and run automatically, such as data back-ups and archiving, data migration, data integrity, access controls and data sharing.

 DEFINITION OF STORAGE VIRTUALIZATION

The term 'storage virtualization' is generally used to mean the separation of the storage  into the physical implementation level of the storage devices and the logical representation level of the storage for use by operating systems, applications and users. In the following we will also use the term 'virtualization', i.e. dropping the word 'storage'. This is always used in the sense of the above definition of storage virtualization. Various uses of the term 'storage virtualization' and 'virtualization' are found in the literature depending upon which level of the storage network the storage virtualization takes place on. The various levels of the storage network here are the server, the storage devices and the network. Some authors only speak of storage virtualization if they explicitly mean storage virtualization within the network. They use the term virtualization, on the other hand, to mean the storage virtualization in the storage devices (for example, in the disk subsystems) or on servers (such as in a volume manager). However, these different types of storage virtualization are not fundamentally different. Therefore, we do not differentiate between the two terms and always use 'storage virtualization' and 'virtualization' in the sense of the above definition.

In Section 5.1 ('Once Again: Virtualization in the I/O Path') we revised various types of storage virtualization on the various levels of the storage network and we will pick these up again later. First of all, however, we want to deal in detail with the conceptual realization of storage virtualization. Storage virtualization inserts – metaphorically speaking – an additional layer between storage devices and storage users (Figure 5.11). This forms the interface between virtual and physical storage, by mapping the physical storage onto the virtual and conversely the virtual storage onto the physical. The separation of storage into the physical implementation level and the logical representation level is achieved by abstracting the physical storage to the logical storage by aggregating several physical storage units to form one or more logical, so-called virtual, storage units. The operating system or applications no longer have direct access to the physical storage devices, they use exclusively the virtual storage. Storage accesses to the physical storage resources take place independently and separately from the storage accesses to the virtual storage resources. For example, the physical hard disks available on a disk stack (JBOD) are brought together by the volume manager of a server to form a large logical volume. In this manner the volume manager thus forms an additional layer between the physical disks of the disk stack and the logical and thus virtual volume with which the applications (e.g. filesystems and databases) of the server work. Within this layer, the mapping of physical hard disks onto logical volumes and vice versa is performed. This means that storage virtualization always calls for a virtualization entity that maps

from virtual to physical storage and vice versa. On the one hand it has to make the virtual storage available to the operating system, the applications and the users in usable form and, on the other, it has to realize data accesses to the physical storage medium. This entity can be implemented both as hardware and software on the various levels in a storage network. It is also possible for several virtualization entities to be used concurrently. For example, an application can use the virtualized volume of a volume manager on server level, which for its part is formed from a set of virtualized volumes which are exported by one or  ore disk subsystems (Section 5.1).