DiskShadow TM - V4.24 User Manual

INTRODUCTION

DiskShadowing or mirroring is a technique used to ensure that data is secure on two disk surfaces. This product performs disk mirroring functions. DiskShadow manages two disk directories in order to maintain a 'mirror' image that is updated with each disk access. Should one of the disks fail, there will still be an up to date copy of the data on the other disk. To achieve this duplication of data, without the DiskShadow resource manager, an application process would have to duplicate all I/O operations to two directories. Disk shadowing or mirroring is the first level of 'raid' disk data security and integrity.

This DiskShadowing software allows I/O operations to be performed on two disks (or two directories on the same disk) while referring only to a single file. When this driver is started it registers a prefix that becomes the DiskShadow directory. The I/O operations performed on this prefixed directory will be duplicated on two (real) nominated directories (the directories may reside on different disks and on different nodes on the network). Use of a directory as the DiskShadow drive makes DiskShadowing accessible to any process (on any node ) via any normal I/O function.

Most commercially available shadow products offer a range of solutions. The most common solution involves an additional disk drive that has exactly the same physical characteristics as the original drive and hardware to ensure the synchronization of the two disks. Other solutions involve modifications to the Operating System or special drivers. The DiskShadow adds to the available range, like everything it has advantages and disadvantages. The most important advantages are :

  • Network transparency (the primary and secondary directories/drives can be anywhere on the network).

  • Simplicity (no extra disks)

  • Transparent to applications, utilities and QNX

  • Versatility. Disks or directories on any node may be the primary and/or secondary shadow drives. DiskShadowing across the network provides additional hardware independence (ie. if the Primary PC dies, you have a "hot" standby machine).

  • Ability to shadow only critical directories. In most cases, shadowing mostly static directories, such as /bin, /etc and so forth doesn't make a great deal of sense. Shadowing of critical data files in a single directory may be all that is required. Multiple instances of DiskShadow allow shadowing of multiple critical directories.

  • The main disadvantage is that shared with most other shadow/mirror products: a reduction overall in I/O speed. DiskShadow must perform two I/O operations for each application I/O request. However, due to the caching and other performance enhancinng features of the QNX File System Manager (Fsys), this theoretical loss of performance is rarely noticed

    Since this DiskShadow driver utilises the standard QNX file system, there is no need for the primary and secondary disks to be physically similar. Indeed they may be on completely different systems (ram/floppy, ide/scsi, ide/ram etc) and different nodes.

    For I/O requests that use file names, two intermediate names are used by DiskShadow, one for the primary and another for the secondary drive. These names are created by appending the name in the I/O request to each directory name. The request is then done for the two names and the 'file descriptors' are maintained by the DiskShadow driver. By appending the original names for each I/O operation a directory structure of the shadow disk is easily maintained. Subsequent I/O requests will refer to the calling tasks 'file descriptor' (fd), this has been kept in the driver and is used to retrieve the fd's of the DiskShadows files that reflect the real file.

    The release file set for this product consists of :

    • DiskShadow - The DiskShadow server executable.
    • DiskCtrl - DiskShadow management utility.
    • DiskSh_Error.h - Definitions of DiskShadow report error numbers.
    • DiskCtrl.h - DiskShadow/DiskCtrl management and reporting structures.
    • demo.c - Sample coding that demonstrates the use of DiskCtrl structures.
    • user423b.doc - MS-Word6 copy of the DiskShadow User manual.
    • tech423b.doc - MS-Word6 copy of the DiskShadow Technical manual.

    Installation

    DiskShadow is distributed by floppy disk. To install it on your system simply ‘install’ from the floppy drive.

    Read and understand the license agreement then install DiskShadow. You will be prompted to enter your agreement to the license conditions. DiskShadow will not run and the license not apply until you agree. DiskShadow will create a license file ".ds_license" on the root directory. This file will hold any extra licenses and the license agreement record. See the Technical Manual for more detail.

    Startup

    DiskShadow may be started on any PC running QNX 4.23A, 4.24, or 4.25

    The minimum startup is :

    DiskShadow /test1 /test2 &

    This startup will cause DiskShadow to create and manage a new directory "/shadow" (default), "/shadow" acts exactly as a ‘real’ directory it responds to ‘ls’, ‘cp’ as would the ‘real’ directories ‘/test1’ and ‘/test2’. "/test1" is the Primary directory.

    Additional parameters are available that provide more detailed control of DiskShadow’s startup and define its actions during normal and abnormal conditions.

    Use

    To demonstrate the use of DiskShadow simply create a file containing data on the ‘/shadow’ directory by :

    date >/shadow/date_data

    See that the file has been created and the data written.

    cat </shadow/date_data

    Verify that the file has been duplicated by :

    ls -l /test1/date_data /test2/date_data /shadow/date_data

    NB: the files in /test1 and 2 are accessible equal. It is recommended that the files in ‘/test1’ and ‘/test2’ are not changed by directly accessing them. If the two files become ‘out of step’ then DiskShadow will unpredictably process these files.

    DiskShadow Startup Options

    Any number of DiskShadow processes may be started on a node. Each process is able to adopt only a single DiskShadow directory and only two disk paths are available for DiskShadowing. The process must be started with root privileges in order to adopt the DiskShadow (/shadow) prefix.

    OPTIONS: As part of the initial setup procedures the driver may attempt to ensure that the two paths are in step. These procedures are done prior to the DiskShadow directory adoption. These options are available to the operator.

    DiskShadow [-d ff|-] [DwuUS] [-a0|1|2] [-L0|1|2] [-bnn] [-m/DiskShadow_dir] [-I 0|1]
    [-T nn] [-h ff] [-s ff] /dir1/ /dir2/ &

    -d ff: Starts the DiskShadow software in 'debug' mode. In this mode every I/O request will result in a report being printed to stdout. The output file must be specified (this is a change from the V4.22 release). A file may be named or a ‘-’ used to indicate stdout. The report will be in the following form :

  • the name of the message type e.g. IO_READ,

  • the pid of the requesting task,

  • related details from the request (fd, modes and names),

  • Response data and

  • the errno string.

  • -D : Displays errors and insecure messages to the active console using 'display_msg'

    -w : Warnings off. Reduces the number of errors and status changed that are reported.

    -m name : DiskShadow Dir Name. The '-m/name' option will allow an operator to select a directory name to be used as the adopted prefix path. If the name cannot be adopted then the process will terminate. '/shadow' is the default name.

    -u : At process startup the Secondary directory will be updated from the Primary. Files and directories are copied to the second path using 'cp -Rnsvp'. All sub-directories and files will be copied if the dates are newer. Also the file ownership and permissions will be preserved.

    -U : As per the ‘-u’ option except that this option will ensure that the second path is exactly the same as the Primary path, this is achieved by at process startup, deleting all files and directories in the second path then copying the Primary path over the second path. This update option should be used with CAUTION as the Secondary directory is completely cleared before the updating is started.
    With this and ‘-u’ DiskShadow will not mirror until the updating process is complete.

    -L n: At DiskShadow startup ‘-L n’ specifies that the secondary will be updated by the primary using method ‘n’. n=0, copies only the newest files and directories. n=1, copies all primary files and directories, and n=2 ensures that the secondary exactly matches the primary by first removing all secondary files and directories before copying. The -L option performs the same function as ‘-u’ and ‘-U’.

    -a n: Automatic Path recovery ON. The parameter ‘n’ determines the update method to be used. For the update method see "-L". This option is selected when one or both of the mirrored directories is on another node and DiskShadow is required to adjust for failure of the node. DiskShadow will test and compensate for node failures and also is able to rebuild a node when it becomes available. (See -T).

    -T nn: Defines the time interval (seconds) that DiskShadow will wait between tests to determine if a failed node has recovered. This time only applies when ‘auto recovery’ has been selected and a node has failed. Every ‘nn’ seconds DiskShadow will test the node, if it is now available and has a viable file system then DiskShadow will restore it using the update method defined in ‘-a n’.

    -b nn: Internal buffer size to be used for read and write requests. The default is 16K. The minimum is 1K. A small buffer may be used if memory is in short supply.

    -I n: Inhibit the Primary or Secondary path at process start up. (n=0=Primary, n=1=Scondary).

    -S: Strict error testing. An error is returned if either the primary or secondary file accesses fail. Normally DiskShadow will use the available data source, in strict mode all errors are returned to the application without attempt at continued operation.

    -s ff: Write status information to ‘ff’ every 30 seconds. See ‘Status Information’.

    -h ff: Write history information to ‘ff’. This option is turned off by ‘-w’.

    /dir1 The Primary path.

    /dir2 The Secondary path. Both must be specified and must exist.

    DiskCtrl - Controlling DiskShadow.

    A utility is provided that permits the user to modify the operation of DiskShadow while DiskShadow is running. DiskCtrl will let a user

    DiskCtrl [-n nn] [-N path_name] [-L update_level] [-w] [-p pid]

    request [Option (On|Off| nn)]

    Options :

    -n node: Node on which DiskShadow is running.

    -N path: The name of a new path to use during the open request. If this name isn’t provided then DiskShadow uses the existing name.

    -L update_level: The update level to be used during open or update requests.

    -w: Wait until DiskShadow has completed the requested operation.

    -p pid: Pid of DiskShadow. Usually not necessary as DiskShadow registers its name locally at startup.

    Request Options:

    REPORT Reports DiskShadow statistics.
    Prints a report indicating the status of currently open files, including the number of links to file. This is useful for determining if any files have been incorrectly left open. The status of the primary and secondary paths is reported, enabled and if they are accessible over the network is reported. The number of I/O requests is reported.

    FLUSH Flushes all open files to disk

    DEBUG Enable/disable debug message display. Options On/Off.
    Turns debugging On or Off.

    SEC Enable/disable shadowing to secondary path. Options ON/OFF

    TERM Terminate DiskShadow. All open files are closed and the task terminates.

    SHADOW Enable/disable requests to shadow directory. Options ON/OFF. Stops all DiskShadowing operations. All I/O requests return ENOSYS.

    UPDATE Update the secondary path using the primary. Options 0 | 1 | 2
    0 = update newer files only
    1 = cp
    2 = remove all files in secondary path before cp.

    STRICT Enable/disable strict error checking. Options On/Off.

    CLOSE Close the primary or secondary path. Options Pri/Sec.

    OPEN Open a new primary or secondary path. Options Pri/Sec.
    Closes the current primary or secondary path then Re-opens it using the name supplied in the "-N" startup parameter. The "-L" parameter specifies the update type.

    NOTIFY Informs DiskShadow that this process (DiskCtrl) is be Triggered when there is a change in the status of either DiskShadows primary or secondary nodes.

    What DiskShadow Can’t do.

    Hard Disk Failures

    DiskShadow is not able to detect failures in hard disks. There are no error conditions that only report on a disk failure therefore DiskShadow is unable to perform auto recovery functions when a disk fails.

    Symbolic links

    DiskShadow is capable of handling symbolic links within its own file space and links into the shadowed directory. A problem exists when a link in the DiskShadow file space refers to a file in Fsys (or other) file space. As part of its normal processing DiskShadow will create 2 copies of the link. Both copies will refer to the same physical file in Fsys file space. Therefore if for example, a write is performed on the file then two copies of the data will be written to the destination file. DiskShadow cannot easily detect that the symbolic link starts in its space but ends in Fsys space. To test for this would introduce much unnecessary overhead for normal operations.

    Pipes

    DiskShadow is not yet able to correctly process pipes created in directories under its prefix. If pipes are used then the solution is to run the ‘Pipe’ manager prior to using DiskShadow.

    BACK

     

    _______________________________________________________