Know more on The Network Data Management Protocol (NDMP)

Know more on  The Network Data Management Protocol (NDMP)

The Network Data Management Protocol (NDMP) defines an interface between NAS servers and network back-up systems that makes it possible to back up NAS servers with- out providing a specific back-up client for them. More and more manufacturers – both of NAS servers and network back-up systems – are supporting NDMP. The current version of NDMP is Version 4; Version 5 is in preparation. NDMP uses the term 'data management operations' to describe the back-up and restoration of data. A so-called data management application (DMA) – generally a back-up system – initiates and controls the data management operations, with the execution of a data management operation generally being called an NDMP session. The DMA cannot directly access the data; it requires the support of so-called NDMP services (Figure 7.13). NDMP services manage the current data storage, such as file systems, back-up media and tape libraries. The DMA creates an NDMP control connection for the control of every participating NDMP service; for the actual data flow between source medium and back-up medium a so-called NDMP data connection is established between the NDMP services in question. Ultimately, the NDMP describes a client-server architecture, with the DMA taking on the role of the NDMP client. An NDMP server is made up of one or more NDMP services. Finally, the NDMP host is the name for a computer that accommodates one or more NDMP servers. NDMP defines different forms of NDMP services. All have in common that they only manage their local state. The state of other NDMP services remains hidden to an NDMP service. Individually, NDMP Version 4 defines the following NDMP services: • NDMP Data Service The NDMP data service forms the interface to primary data such as a file system on a NAS server. It is the source of back-up operations and the destination of restore operations. To back-up a file system, the NDMP Data Service converts the content of the file system into a data stream and writes this in an NDMP data connection, which is generally created by means of a TCP/IP connection. To restore a file system it reads the data stream from an NDMP data connection and from this reconstructs the content of a file system. The Data Service only permits the back-up of complete file systems;

it is not possible to back up individual files. By contrast, individual files or directories can be restored in addition to complete file systems.

The restoration of individual files or directories is also called 'direct access recovery'. To achieve this, the Data Service provides a so-called file history interface, which it uses to forward the necessary metadata to the DMA during the back-up. The file history stores the positions of the individual files within the entire data stream. The DMA  cannot read this so-called file locator data, but it can forward it to the NDMP tape

service in the event of a restore operation. The NDMP tape service then uses this information to wind the tape to the appropriate position and read the files in question. • NDMP Tape Service The NDMP Tape Service forms the interface to the secondary storage. Secondary storage, in the sense of NDMP, means computers with connected tape drive, connected tape library or a CD burner. The Tape Service manages the destination of a back-up or the source of a data restoration operation. For a back-up, the Tape Service writes an incoming data stream to tape via the NDMP data connection; for a restoration it reads the content of a tape and writes this as a data stream in a NDMP data connection. The Tape Service has only the information that it requires to read and write, such as tape size or block size. It has no knowledge of the format of the data stream. It requires the assistance of the DMA to change tapes in a tape library. • NDMP SCSI Pass Through Service

The SCSI Pass Through Service makes it possible for a DMA to send SCSI commands to a SCSI device that is connected to a NDMP server. The DMA requires this service, for example, for the changing of tapes in a tape library. The DMA holds the threads of an NDMP session together: it manages all state information of the participating NDMP services, takes on the management of the back-up media and initiates appropriate recovery measures in the event of an error. To this end the DMA maintains an NDMP control connection to each of the participating NDMP

services, which – like the NDMP data connections – are generally based upon TCP/IP. Both sides – DMA and NDMP services – can be active within an NDMP session. For example, the DMA sends commands for the control of the NDMP services, whilst the NDMP services for their part send messages if a control intervention by the DMA is required. If, for example, an NDMP Tape Service has filled a tape, it informs the DMA. This can then initiate a tape change by means of an NDMP SCSI Pass Through Service. The fact that both NDMP control connections and NDMP data connections are based upon TCP/IP means that flexible configuration options are available for the back-up of data using NDMP. The NDMP architecture supports back-up to a locally connected tape drive (Figure 7.14) and likewise to a tape drive connected to another computer, for example a second NAS server or a back-up server (Figure 7.15). This so-called remote back-up has the advantage that smaller NAS servers do not need to be equipped with a tape library. Further fields of application of remote back-up are the replication of file systems (disk-to-disk remote back-up) and of back-up tapes (tape-to-tape remote back-up). In remote back-up the administrator comes up against the same performance bottlenecks as in conventional network back-up over the LAN (Section 7.6). Fortunately, NDMP local back-up and LAN-free back-up of network back-up systems complement each other excellently: a NAS server can back up to a tape drive available in the storage network, with the network back-up system co-ordinating access to the tape drive outside of NDMP by means of tape library sharing In Version 5, NDMP will have further functions such as multiplexing, compressing and encryption. To achieve this, NDMP Version 5 expands the architecture to include the so-called translator service (Figure 7.17). Translator services process the data stream (data stream processor): they can read and change one or more data streams. The implementation of translator services is in accordance with that of previous NDMP services. This means that the control of the translator service lies with the DMA; other participating NDMP

services cannot tell whether an incoming data stream was generated by a translator service or a different NDMP service. NDMP Version 5 defines the following translator services:

• Data stream multiplexing The aim of data stream multiplexing is to bundle several data streams into one data stream (N:1-multiplexing) or to generate several data streams from one (1:M-multi- plexing). Examples of this are the back-up of several small, slower file systems onto a

faster tape drive (N:1-multiplexing) or the parallel back-up of a large file system onto several tape drives (1:M-multiplexing).

• Data stream compression In data stream compression the translator service reads a data stream, compresses it and sends it back out. Thus the data can be compressed straight from the hard disk, thus freeing up the network between it and the back-up medium.

• Data stream encryption Data stream encryption works on the same principle as data stream compression, except that it encrypts data instead of compressing it. Encryption is a good idea, for example, for the back-up of small NAS servers at branch offices to a back-up server in a data centre via a public network. NDMP offers many opportunities to connect NAS servers to a network back-up system. The prerequisite for this is NDMP support on both sides. NDMP data services cover approximately the functions that back-up clients of network back-up systems provide. One weakness of NDMP is the back-up of the NAS server metadata, which makes the restoration of a NAS server after the full replacement of hardware significantly more difficult (Section 7.9.1). Furthermore, there is a lack of support for the back-up of file

systems with the aid of snapshots or instant copies. Despite these missing functions NDMP has established itself as a standard and so we believe that it is merely a matter of time before NDMP is expanded to include these functions.