IBM General Parallel File System for AIX: Administration and Programming Reference

gpfsDataShipMap_t Structure

Name

gpfsDataShipMap_t - Indicates which agent nodes are to be used for data shipping.

Structure

#define GPFS_MAX_DS_AGENT_NODES 2048
typedef struct
{
  int  structLen;
  int  structType;
  int  partitionSize;
  int  agentCount; 
  int  agentNodeNumber[GPFS_MAX_DS_AGENT_NODES]  
 } gpfsDataShipMap_t;

Description

GPFS recognizes which agent nodes to use for data shipping:

• This directive can only appear in a gpfs_fcntl( ) call that also gives the GPFS_DATA_SHIP_START directive.
• If any of the participating threads include an explicit agent mapping with this directive, all threads must provide the same agent mapping, or else GPFS will return EINVAL in errno. If this directive is not used, the agents will be exactly the nodes on which the GPFS_DATA_SHIP_START directive was given. The order of these nodes in the mapping will be random. Once the order is set, when all instances have issued the GPFS_DATA_SHIP_START directive, the partitioning of the blocks will be round robin among the agent nodes.
• All of the nodes named in the data shipping mapping must also be data shipping clients that have issued the GPFS_DATA_SHIP_START directive. The reason for this is that GPFS, like most file systems, does not guarantee that data is written through to disk immediately after a write call from an application, or even after a close returns. Thus, cached data can be lost if a node crashes. Data loss can only occur, however, if the node that crashes is the node that wrote the data.

  With data shipping, this property is no longer true. Any node crash in the collective of nodes can cause loss of data. An application running with a file in data shipping mode will write data by shipping it to the GPFS cache on an agent node. That agent node may later crash before writing the data to disk. The originating node may not receive, pay attention to, or realize the severity of an error message. Presumably, a distributed application would notice a crash of one of the nodes on which it was running and would take corrective action, such as rolling back to an earlier stable checkpoint or deleting a corrupt output file. By requiring that all agent nodes also have at least one data shipping client, GPFS ensures that at least one of the nodes of a distributed application will crash if there is the potential for data loss because of an agent node crash. If any of the data shipping client nodes suffers a node or GPFS crash, the file will be taken out of data shipping mode.

The value for partitionSize must be a multiple of the number of bytes in a single file system block.

Members

structLen

Length of the gpfsDataShipMap_t structure.

structType

The directive identifier GPFS_DATA_SHIP_MAP.

partitionSize

The number of contiguous bytes per server.

This value must be a multiple of the number of bytes in a single file system block.

agentCount

The number of entries in the agentNodeNumber array.

agentNodeNumber array

The data ship agent node numbers as listed in the SDR or the global ODM

Error status

EINVAL

Not all participating threads have provided the same agent mapping.

ENOMEM

The available data space in memory is not large enough to allocate the data structures necessary to run in data shipping mode.

EPERM

An attempt to open a file in data shipping mode that is already open in write mode by some thread that did not issue the GPFS_DATA_SHIP_START directive.

ESTALE

A node in the data shipping collective has gone down.