Installation

User Authorization

POE User Authorization

POE supports the following user authentication methods, which are based on the SP security services methods, and are set using the PSSP chauthts command:

• compatibility

  AIX authentication will be used, based on entries in the /etc/hosts.equiv or .rhosts files. This is the default mechanism.

• dce

  DCE authentication will be used, which requires the following:

  1. You must have a valid DCE ID and principal, with which you perform a dce_login.
  2. The system administrator must have set up the PMD service principal.
• DCE and compatibility

  DCE authentication is attempted first. If this attempt is not successful, AIX authentication is tried.

• none

  No security methods are enabled. POE defaults to use AIX authentication, as it does with compatibility.

Note:

If you are using LoadLeveler to submit POE jobs, which includes all user space applications, LoadLeveler is responsible for the security authentication. The security function in POE is not invoked when POE runs under LoadLeveler.

When POE is used in a standalone pSeries or RS/6000 workstation environment without the ssp.clients fileset installed, only AIX authentication can be used. See DCE User Authorization on a Standalone pSeries or RS/6000 Workstation for more information.

The lsauthts command can be used to check the authentication method in use. See IBM Parallel System Support Programs for AIX: Command and Technical Reference for more information on the chauthts and lsauthts commands.

DPCL User Authorization

DPCL supports the following user authentication methods, which are based on the SP security services methods, and are set using the PSSP chauthts command:

• compatibility

  AIX authentication will be used, based on entries in the /etc/hosts.equiv or .rhosts files. This is the default mechanism.

• dce

  DCE authentication will be used, which requires the following:

  1. You must have a valid DCE ID and principal on both the client and daemon nodes, with which you perform a dce_login.
  2. The system administrator must have set up the DPCL service principal.
• DCE and compatibility

  DCE authentication is attempted first. If this attempt is not successful, AIX authentication is tried.

• none

  AIX authentication will be used, based on entries in the /etc/hosts.equiv or .rhosts files.

The /etc/hosts.equiv file, the .rhosts file, or both, are used to specify DPCL user authorization when AIX authentication is used.

If the combination of the home node machine and user name:

• is authorized in /etc/hosts.equiv on the remote node, the user is authorized to run parallel tasks there.
• is disallowed in /etc/hosts.equiv on the remote node, the user is not able to run parallel tasks there.
• does not appear in /etc/hosts.equiv, the combination is checked in the .rhosts file in the user's home directory on the remote node. If the user name and the home node machine combination appears in .rhosts, the user is authorized to run parallel tasks on the remote node.

For more information on .rhosts and /etc/host.equiv, see the chapter on managing jobs in IBM AIX 5L Version 5.1 Files Reference.

Using AIX User Authorization

If AIX user authorization, or compatibility, (the default) is used as a security mechanism on the system, each node needs to be set up so that each userid is authorized to access that node or remote link from the initiating home node. Use the /etc/hosts.equiv file and/or the .rhosts file to specify this user ID authorization, as explained below.

If the combination of the home node machine and user name:

• is authorized in /etc/hosts.equiv on the remote node, the user is authorized to run parallel tasks there.
• is disallowed in /etc/hosts.equiv on the remote node, the user is not able to run parallel tasks there.
• does not appear in /etc/hosts.equiv, the combination is checked in the .rhosts file in the user's home directory on the remote node. If the user name and the home node machine combination appears in .rhosts, the user is authorized to run parallel tasks on the remote node.

For more information on .rhosts and /etc/host.equiv, see the chapter on managing jobs in IBM AIX 5L Version 5.1 Files Reference.

Using DCE User Authorization

POE User Authorization in a DCE Environment

When you enable DCE authentication as the SP security method of choice, POE will expect a valid set of DCE credentials in order to submit parallel jobs.

When both DCE and compatibility methods are enabled, POE attempts DCE authentication first. If DCE authentication is unsuccessful, POE will then use AIX authentication.

In order to use DCE with POE, you need the following:

1. A valid set of DCE credentials created by a dce_login to a valid principal.
2. A valid set of Kerberos Version 5 principals, created by klogin or an entry in a .k5login file in the user's home directory.
3. The system administrator must have properly set up the PMD service principal as part of the SP security administration and configuration steps.

   POE requires that a DCE service principal is defined for the partition manager daemon (PMD), to control the use of the service with DCE. The PMD service is defined in the spsec_defaults file as ppe/pmdv3, with these attributes:

   • it runs on a workstation
   • it is not partitioned
   • it uses key management

   You need to define this PMD service principal for each host. This means that every POE remote node must have a PMD service principal defined for its host name.

   The SP security installation and configuration should set up the PMD service principal automatically. See PSSP Administration Guide for more information.

For parallel jobs that involve a large number of tasks, see PSSP Planning for information about optimizing the replication of DCE servers.

DPCL User Authorization in a DCE Environment

In order to use DCE with DPCL, the system administrator must have properly set up the DPCL service principal as part of the SP security administration and configuration steps.

DPCL requires that a DCE service principal is defined for the DPCL daemon to control the use of the service with DCE. The DPCL service is defined in the spsec_defaults file as ppe/dpcl, with these attributes:

• it runs on a workstation
• it is not partitioned
• it uses key management

This DPCL service principal needs to be defined for each host; that is, every DPCL remote node must have a DPCL service principal defined for its host name.

The SP security installation and configuration should set up the DPCL service principal automatically. See PSSP Administration Guide for more information.

User Authorization in a Mixed Environment

POE User Authorization in a Mixed AIX and DCE Environment

The POE remote nodes are the locations from where the parallel job was ultimately submitted. Therefore, it is on the remote nodes that it is decided which authentication method is to be used.

POE supports a mixed environment of DCE and non-DCE nodes only under certain conditions. When running on a mix of nodes with and without DCE enabled, the following conditions apply:

1. If the DCE security method is being used on the POE home node, and POE remote nodes are enabled for compatibility only, the job will run without DCE, and the remote node security will default to AIX.
2. If the compatibility method only is enabled on the POE home node, and the remote nodes are enabled with DCE only, DCE authentication will be attempted with no credentials, which may ultimately fail.
3. When the remote nodes are enabled with DCE compatibility, DCE authentication will be tried first, and then AIX authentication will be used.

   In this case, some nodes may fail to successfully authenticate with DCE, in which case the job will try to authenticate with AIX. When the parallel job starts, it may be running with some nodes authenticated to DCE and not others. This may result in a problem if the application needs to use resources under DCE control.

   If it is absolutely critical that an application start with successful DCE authentication, enable nodes with DCE only as the authentication method, to ensure that DCE will be used.

4. When the SP authentication method is not set on a node (the lsauthts command returns no value), no methods are enabled. In this case, POE is disabled and no job will be allowed to run on that node.

The destination remote node (server) sets the security policy, and ultimately enforces the security method. The following table shows the various combinations of supported methods, and how POE security will work:

DPCL User Authorization in a Mixed AIX and DCE Environment

When a DPCL client is run, it is the remote nodes where the DPCL daemons run that determine which authentication method is to be used on that node.

DPCL supports a mixed environment of DCE and non-DCE nodes. As the DPCL client goes through the authentication process as part of connecting to each DPCL daemon, it will use the authentication process required by that DPCL daemon. In a mixed environment of DCE and non-DCE nodes, the following conditions apply:

1. If the security model on the DPCL client node is DCE and the security model on the DPCL daemon node is compatibility, AIX authentication (.rhosts and /etc/hosts.equiv) will be used.
2. If the security model on the DPCL client node is compatibility and the security model on the DPCL daemon node is DCE only, DCE authentication on the daemon node will be attempted with no credentials, and the attempt may fail.
3. When the DPCL daemon node's security model is DCE and compatibility, DCE authentication will be attempted first, and if that fails, AIX authentication will be used.

   In this case, some nodes may fail to successfully authenticate with DCE, in which case the job will try to authenticate with AIX. When the parallel job starts, it may be running with some nodes authenticated to DCE and not others. This may result in a problem if the application needs to use resources under DCE control.

   If it is absolutely critical that an application start with successful DCE authentication, nodes should be enabled with DCE only as the authentication method, to ensure that DCE will be used.

4. When the SP authentication method is not set on a DPCL daemon node (the lsauthts command returns no value), DPCL will assume that the compatibility security model is set on that node.

The following table shows the various combinations of supported methods and how security will work.

DCE User Authorization on a Standalone pSeries or RS/6000 Workstation

In order to have DCE authentication for POE users on standalone RS/6000 workstations, either as a cluster or connected to an SP, the following must be in place on each workstation:

1. The system administrator must properly configure the workstations as part of the DCE cell.
2. The ssp.clients fileset needs to be installed.
3. The SP security commands config_spsec and create_keyfiles must be used to configure the SP security services and the key table entries on which POE depends. See PSSP Administration Guide for more information on these commands, and on setting up DCE security on the SP.

DCE Credentials Lifetime

When DCE is used to submit a parallel job, it will obtain a new set of credentials prior to submitting the job. This is important for long-running applications, because once a job is submitted, the credentials cannot be refreshed or renewed. As a result, the credentials lifetime should be long enough to outlast the longest-running application.

Careful planning is recommended to avoid premature expiration of DCE credentials before a job can complete. The PSSP Administration Guide describes how to alter the default credentials lifetime.

Once credentials have expired, they should be removed using the rmxcred command, to prevent filling up the /var file system.

Port Numbers

When POE is installed, it modifies entries in /etc/services and in /etc/inetd.conf to install the partition manager daemon. In doing so, it requires an available port number that must be the same number on all nodes on which POE is to be installed and running. You need to ensure such a port number is available.

Running Large POE Jobs and IP Buffer Usage

A POE application may require additional IP buffers (mbufs) under any of the following circumstances:

• Partition size is larger than 128 nodes.
• Large amounts of STDIO (stdin, stdout, or stderr) are generated.
• The home node is running many POE jobs simultaneously, or there is significant additional IP traffic via mounted file system activity (or other sources), or both.
• Many large messages are passed via the UDP implementation of the Message Passing Library.

The need for additional IP buffers is usually evident when repeated requests for memory are denied. Using the netstat -m command can tell you when such a condition exists. In such a case, it may be necessary to use the no command to change the network option system parameters on the home node or on the SP nodes being used in the partition. You can use the no command to initially check the values as well.

The number of IP buffers allocated in the kernel is controlled by the thewall parameter of the no command. Increasing the value of the thewall parameter increases the number of IP buffers.

Notes:

1. You must have root authority to change options with the no command, and the setting applies to all processes running on the node on which it is executed.

2. In AIX Version 5L 5.1, the thewall default value is 16384.

On SP nodes, you can use the dsh command to execute the no command on each node of an SP. See the section on tuning in IBM Parallel System Support Programs for AIX: Administration Guide for more information on dsh.

For non-SP nodes, you can also set the values at system boot time by adding the appropriate call to the no command in either /etc/rc.net or /etc/rc.tcpip.

For more information on mbufs, see IBM AIX 5L Version 5.1 Performance Management Guide.

Running Multiple Versions of POE

POE Version 3 and POE Version 2 have limited compatibility. POE Version 3 can run on a combination of POE Version 3 and POE Version 2.4 remote nodes as long as the home node (where POE is started from) is running POE Version 3. Earlier versions of POE, such as Version 2.2 and 2.3, are not supported in a mixed environment. You must also be at the supported level of AIX and PSSP for the particular POE version within a partition to submit PE jobs. When the POE home node is at a Version 2.4 level, it can only work with remote nodes that are also running POE Version 2.4. POE Version 2.4 home nodes cannot run with POE Version 3.2 remote nodes.

When POE Version 3.2 is installed on top of a POE Version 2 node, POE Version 3 completely replaces the previous version of POE. All Version 2 commands and executables are removed and replaced with Version 3 levels.

See Chapter 3, Migrating to PE 3.2 for more information.

Partition Manager Daemon Services and Installation

As part of the Version 3 installation, the Partition Manager daemon (pmd) and POE executables have different names than their Version 2 counterparts. Also, different TCP/IP port numbers and daemon service names are used.

The following table summarizes the differences and can be used to tell which version of POE you have if you are not sure.