This chapter contains the man pages for the Parallel Utility functions. These user-callable, thread-safe functions exploit features of the IBM Parallel Environment for AIX. Included are functions for:
There is a C version and a Fortran version for most of the functions.
The Parallel Utility functions are:
For more information on the Program Marker Array, or on controlling STDIN and STDOUT using POE, refer to IBM Parallel Environment for AIX: Operation and Use, Volume 1.
Purpose
Starts user-initiated checkpointing.
Version
libmpi.a
C Synopsis
#include <pm_util.h> int mp_chkpt(int flags);
Fortran Synopsis
i = MP_CHKPT(%val(j))
Parameters
In C, flags can be set to MP_CUSER, which indicates complete user-initiated checkpointing.
In Fortran, j should be set to 0 (zero), which is the value of MP_CUSER.
Description
MP_CHPKT initiates complete user-initiated checkpointing. When this function is reached, the program's execution is suspended. At that point, the state of the application is captured, along with all data, and saved to a file pointed to by the MP_CHECKFILE and MP_CHECKDIR environment variables.
Only POE/MPI applications submitted under LoadLeveler in batch mode are able to call this function. LoadLeveler is required for programs to call this function. Checkpointing of interactive POE applications is not allowed.
Notes
In complete user-initiated checkpointing, all instances of the parallel program must call MP_CHKPT. After all instances of the application have issued the MP_CHKPT call and have been suspended, a local checkpoint is taken on each node, with or without saving the message state, depending on the stage of the implementation.
Upon returning from the MP_CHKPT call, the application continues to run. It may, however, be a restarted application that is now running, rather than the original.
There are certain limitations associated with checkpointing an application. See "Checkpoint/Restart Limitations" for details.
For general information on checkpointing and restarting programs, refer to IBM Parallel Environment for AIX: Operation and Use, Volume 1.
For more information on the use of LoadLeveler and checkpointing, refer to Using and Administering LoadLeveler.
Return Values
Purpose
Disables message arrival interrupts on a node.
Version
libmpi.a
C Synopsis
#include <pm_util.h> int mpc_disableintr();
Fortran Synopsis
MP_DISABLEINTR(INTEGER RC)
Parameters
In Fortran, rc contains the values as described below in Return Values.
Description
This Parallel Utility function disables message arrival interrupts on the individual node on which it is run. Use this function to dynamically control masking interrupts on a node.
Notes
Return Values
Examples
C Example
/*
* Running this program, after compiling with mpcc,
* without setting the MP_CSS_INTERRUPT environment variable,
* and without using the "-css_interrupt" command-line option,
* produces the following output:
*
* Interrupts are DISABLED
* About to enable interrupts..
* Interrupts are ENABLED
* About to disable interrupts...
* Interrupts are DISABLED
*/
#include "pm_util.h"
#define QUERY if (intr = mpc_queryintr()) {\
printf("Interrupts are ENABLED\n");\
} else {\
printf("Interrupts are DISABLED\n");\
}
main()
{
int intr;
QUERY
printf("About to enable interrupts...\n");
mpc_enableintr();
QUERY
printf("About to disable interrupts...\n");
mpc_disableintr();
QUERY
}
Fortran Example
Running the following program, after compiling with mpxlf, without setting the MP_CSS_INTERRUPT environment variable, and without using the "-css_interrupt" command-line option, produces the following output:
Interrupts are DISABLED
About to enable interrupts..
Interrupts are ENABLED
About to disable interrupts...
Interrupts are DISABLED
PROGRAM INTR_EXAMPLE
INTEGER RC
CALL MP_QUERYINTR(RC)
IF (RC .EQ. 0) THEN
WRITE(6,*)'Interrupts are DISABLED'
ELSE
WRITE(6,*)'Interrupts are ENABLED'
ENDIF
WRITE(6,*)'About to enable interrupts...'
CALL MP_ENABLEINTR(RC)
CALL MP_QUERYINTR(RC)
IF (RC .EQ. 0) THEN
WRITE(6,*)'Interrupts are DISABLED'
ELSE
WRITE(6,*)'Interrupts are ENABLED'
ENDIF
WRITE(6,*)'About to disable interrupts...'
CALL MP_DISABLEINTR(RC)
CALL MP_QUERYINTR(RC)
IF (RC .EQ. 0) THEN
WRITE(6,*)'Interrupts are DISABLED'
ELSE
WRITE(6,*)'Interrupts are ENABLED'
ENDIF
STOP
END
Related Information
Functions:
Purpose
Enables message arrival interrupts on a node.
Version
libmpi.a
C Synopsis
#include <pm_util.h> int mpc_enableintr();
Fortran Synopsis
MP_ENABLEINTR(INTEGER RC)
Parameters
In Fortran, rc contains the values as described below in Return Values.
Description
This Parallel Utility function enables message arrival interrupts on the individual node on which it is run. Use this function to dynamically control masking interrupts on a node.
Notes
Return Values
Examples
C Example
/*
* Running this program, after compiling with mpcc,
* without setting the MP_CSS_INTERRUPT environment variable,
* and without using the "-css_interrupt" command-line option,
* produces the following output:
*
* Interrupts are DISABLED
* About to enable interrupts..
* Interrupts are ENABLED
* About to disable interrupts...
* Interrupts are DISABLED
*/
#include "pm_util.h"
#define QUERY if (intr = mpc_queryintr()) {\
printf("Interrupts are ENABLED\n");\
} else {\
printf("Interrupts are DISABLED\n");\
}
main()
{
int intr;
QUERY
printf("About to enable interrupts...\n");
mpc_enableintr();
QUERY
printf("About to disable interrupts...\n");
mpc_disableintr();
QUERY
}
Fortran Example
Running this program, after compiling with mpxlf, without setting the MP_CSS_INTERRUPT environment variable, and without using the "-css_interrupt" command-line option, produces the following output:
Interrupts are DISABLED
About to enable interrupts..
Interrupts are ENABLED
About to disable interrupts...
Interrupts are DISABLED
PROGRAM INTR_EXAMPLE
INTEGER RC
CALL MP_QUERYINTR(RC)
IF (RC .EQ. 0) THEN
WRITE(6,*)'Interrupts are DISABLED'
ELSE
WRITE(6,*)'Interrupts are ENABLED'
ENDIF
WRITE(6,*)'About to enable interrupts...'
CALL MP_ENABLEINTR(RC)
CALL MP_QUERYINTR(RC)
IF (RC .EQ. 0) THEN
WRITE(6,*)'Interrupts are DISABLED'
ELSE
WRITE(6,*)'Interrupts are ENABLED'
ENDIF
WRITE(6,*)'About to disable interrupts...'
CALL MP_DISABLEINTR(RC)
CALL MP_QUERYINTR(RC)
IF (RC .EQ. 0) THEN
WRITE(6,*)'Interrupts are DISABLED'
ELSE
WRITE(6,*)'Interrupts are ENABLED'
ENDIF
STOP
END
Related Information
Functions:
Purpose
Version
libmpi.a
C Synopsis
#include <pm_util.h> int mpc_flush(int option);
Fortran Synopsis
MP_FLUSH(INTEGER OPTION)
Parameters
Description
This Parallel Utility function flushes output buffers from all of the parallel tasks to STDOUT at the home node. This is a synchronizing call across all parallel tasks.
If the current STDOUT mode is ordered, then when all tasks have issued this call or when any of the output buffers are full:
If current STDOUT mode is unordered and all tasks have issued this call, all output buffers are flushed and put out to the user screen (or redirected).
If the current STDOUT mode is single and all tasks have issued this call, the output buffer for the current single task is flushed and put out to the user screen (or redirected).
Notes
Return Values
In C and C++ calls, the following applies:
Examples
C Example
The following program uses poe with the -labelio yes option and three tasks:
#include <pm_util.h>
main()
{
mpc_stdout_mode(STDIO_ORDERED);
printf("These lines will appear in task order\n");
/*
* Call mpc_flush here to make sure that one task
* doesn't change the mode before all tasks have
* sent the previous printf string to the home node.
*/
mpc_flush(1);
mpc_stdout_mode(STDIO_UNORDERED);
printf("These lines will appear in the order received by the home node\n");
/*
* Since synchronization is not used here, one task could actually
* execute the next statement before one of the other tasks has
* executed the previous statement, causing one of the unordered
* lines not to print.
*/
mpc_stdout_mode(1);
printf("Only 1 copy of this line will appear from task 1\n");
}
Running the above C program produces the following output (task order of lines 4-6 may differ):
0 : These lines will appear in task order.
1 : These lines will appear in task order.
2 : These lines will appear in task order.
1 : These lines will appear in the order received by the home node.
2 : These lines will appear in the order received by the home node.
0 : These lines will appear in the order received by the home node.
1 : Only 1 copy of this line will appear from task 1.
Fortran Example
CALL MP_STDOUT_MODE(-2)
WRITE(6, *) 'These lines will appear in task order'
CALL MP_FLUSH(1)
CALL MP_STDOUT_MODE(-3)
WRITE(6, *) 'These lines will appear in the order received by the
xhome node'
CALL MP_STDOUT_MODE(1)
WRITE(6, *) 'Only 1 copy of this line will appear from task 1'
END
Related Information
Functions:
Purpose
Passes numeric and text data to the Program Marker Array.
Version
libmpi.a
C Synopsis
#include <mp_marker.h> void mpc_marker(int light, int color, char *string);
Fortran Synopsis
MP_MARKER(INTEGER LIGHT, INTEGER COLOR,
CHARACTER STRING)
Parameters
If the value of light is out of range, the parameter is ignored. No light is colored by the subroutine. Setting the light to a negative number lets you update the string only.
If the value of color is out of range, the parameter is ignored. The subroutine does not give the light a new color.
Description
This Parallel Utility function requests that the numeric and text data passed in the call be forwarded to the Program Marker Array for display. This call waits for a specific acknowledgement from the Partition Manager if the number of lights (specified by the MP_PMLIGHTS environment variable) is positive. The program returns only after the message has been acknowledged by the POE home node. Hence, this call will slow down the user's application and synchronize it approximately with the Program Marker Array.
If MP_PMLIGHTS is set to 0, no message is sent.
Notes
Examples
C Example
The C statement:
#include <mp_marker.h> mpc_marker(2, 0, "Starting task 2");
gives the third light of the calling task the color black and after the lights on that task's row, prints the string "Starting task 2".
Fortran Example
The Fortran statement:
CALL MP_MARKER(2, 0, 'Starting task 2')
gives the third light of the calling task the color black and after the lights on that task's row, prints the string 'Starting task 2'.
Related Information
Commands: pmarray
Functions: MP_NLIGHTS, mpc_nlights
Purpose
Gets the number of Program Marker Array lights defined for this session.
Version
libmpi.a
C Synopsis
#include <mp_marker.h> int mpc_nlights();
Fortran Synopsis
MP_NLIGHTS(INTEGER NLIGHT)
Parameters
In Fortran, the number of Program Marker Array lights defined for this session is returned in the variable NLIGHT.
Description
This Parallel Utility function returns the number of Program Marker Array lights defined for this session.
Notes
Return Values
In C, the current number of the Program Marker Array lights is the return value.
Examples
C Example
The following program uses poe with the -pmlights 3 option and 1 task:
#include <mp_marker.h>
main()
{
printf("Number of Program Marker Array lights for this session is %d\n",
mpc_nlights());
}
Running the above C program produces the following output:
Number of Program Marker Array lights for this session is 3.
Fortran Example
The following program uses poe with the -pmlights 3 option and 1 task:
INTEGER LIGHTS
CALL MP_NLIGHTS(LIGHTS)
WRITE(6, *) 'Number of Program Marker Array lights for this
xsession is ', LIGHTS
END
Running the above produces the following output:
Number of Program Marker Array lights for this session is 3.
Related Information
Commands: pmarray
Functions: MP_MARKER, mpc_marker
Purpose
Returns the state of interrupts on a node.
Version
libmpi.a
C Synopsis
#include <pm_util.h> int mpc_queryintr();
Fortran Synopsis
MP_QUERYINTR(INTEGER RC)
Parameters
In Fortran, rc contains the values as described below in RETURN VALUES.
Description
This Parallel Utility function returns the state of interrupts on a node.
Notes
Return Values
Examples
C Example
/*
* Running this program, after compiling with mpcc,
* without setting the MP_CSS_INTERRUPT environment variable,
* and without using the "-css_interrupt" command-line option,
* produces the following output:
*
* Interrupts are DISABLED
* About to enable interrupts..
* Interrupts are ENABLED
* About to disable interrupts...
* Interrupts are DISABLED
*/
#include "pm_util.h"
#define QUERY if (intr = mpc_queryintr()) {\
printf("Interrupts are ENABLED\n");\
} else {\
printf("Interrupts are DISABLED\n");\
}
main()
{
int intr;
QUERY
printf("About to enable interrupts...\n");
mpc_enableintr();
QUERY
printf("About to disable interrupts...\n");
mpc_disableintr();
QUERY
}
Fortran Example
Running this program, after compiling with mpxlf, without setting the MP_CSS_INTERRUPT environment variable, and without using the "-css_interrupt" command-line option, produces the following output:
Interrupts are DISABLED
About to enable interrupts..
Interrupts are ENABLED
About to disable interrupts...
Interrupts are DISABLED
PROGRAM INTR_EXAMPLE
INTEGER RC
CALL MP_QUERYINTR(RC)
IF (RC .EQ. 0) THEN
WRITE(6,*)'Interrupts are DISABLED'
ELSE
WRITE(6,*)'Interrupts are ENABLED'
ENDIF
WRITE(6,*)'About to enable interrupts...'
CALL MP_ENABLEINTR(RC)
CALL MP_QUERYINTR(RC)
IF (RC .EQ. 0) THEN
WRITE(6,*)'Interrupts are DISABLED'
ELSE
WRITE(6,*)'Interrupts are ENABLED'
ENDIF
WRITE(6,*)'About to disable interrupts...'
CALL MP_DISABLEINTR(RC)
CALL MP_QUERYINTR(RC)
IF (RC .EQ. 0) THEN
WRITE(6,*)'Interrupts are DISABLED'
ELSE
WRITE(6,*)'Interrupts are ENABLED'
ENDIF
STOP
END
Related Information
Functions:
Purpose
Returns the current interrupt delay time.
Version
libmpi.a
C Synopsis
#include <pm_util.h> int mpc_queryintrdelay();
Fortran Synopsis
MP_QUERYINTRDELAY(INTEGER RC)
Parameters
In Fortran, rc contains the values as described below in RETURN VALUES.
Description
This Parallel Utility function returns the current interrupt delay time in microseconds.
Notes
Return Values
The current interrupt delay time in microseconds.
Examples
C Example
/*
* Running this program, after compiling with mpcc,
* without setting the MP_INTRDELAY environment variable,
* and without using the "-intrdelay" command-line option,
* produces the following output:
*
* Current interrupt delay time is 35
* About to set interrupt delay time to 100...
* Current interrupt delay time is 100
*/
#include "pm_util.h"
main()
{
printf("Current interrupt delay time is %d\n", mpc_queryintrdelay());
printf("About to set interrupt delay time to 100...\n");
mpc_setintrdelay(100);
printf("Current interrupt delay time is %d\n", mpc_queryintrdelay());
}
Fortran Example
Running this program, after compiling with mpxlf, without setting the MP_INTRDELAY environment variable, and without using the "-intrdelay" command-line option, produces the following output:
Current interrupt delay time is 35
About to set interrupt delay time to 100...
Current interrupt delay time is 100
PROGRAM INTRDELAY_EXAMPLE
INTEGER DELAY, RC
CALL MP_QUERYINTRDEALY(DELAY)
WRITE(6,*)'Current interrupt delay time is', delay
WRITE(6,*)'About to set interrupt delay time to 100...'
DELAY = 100
CALL MP_SETINTRDELAY(DELAY, RC)
CALL MP_QUERYINTRDELAY(DELAY)
WRITE(6,*)'Current interrupt delay time is', delay
STOP
END
Related Information
Functions:
Purpose
Version
libmpi.a
C Synopsis
#include <pm_util.h> int mpc_setintrdelay(integer val);
Fortran Synopsis
MP_SETINTRDELAY(INTEGER VAL, INTEGER RC)
Parameters
Description
This Parallel Utility function sets the delay parameter to the value specified in val. This call can be made multiple times in a program with different values being passed to it each time.
You can use the environment variable MP_INTERDELAY to set an integer value before running your program. In this way, you can tune your delay parameter without having to recompile existing applications.
The cost of servicing an interrupt is quite high. For an application with few nodes exchanging small messages, it will help latency if the interrupt delay is kept small. For an application with a large number of nodes or one which exchanges large messages, keeping the delay parameter large will help the bandwidth. This allows multiple read transmissions to occur in a single read cycle. You should experiment with different values functions to achieve the desired performance depending on the communication pattern.
Notes
Return Values
Examples
C Example
/*
* Running this program, after compiling with mpcc,
* without setting the MP_INTRDELAY environment variable,
* and without using the "-intrdelay" command-line option,
* produces the following output:
*
* Current interrupt delay time is 35
* About to set interrupt delay time to 100...
* Current interrupt delay time is 100
*/
#include "pm_util.h"
main()
{
printf("Current interrupt delay time is %d\n", mpc_queryintrdelay());
printf("About to set interrupt delay time to 100...\n");
mpc_setintrdelay(100);
printf("Current interrupt delay time is %d\n", mpc_queryintrdelay());
}
Fortran Example
Running this program, after compiling with mpxlf, without setting the MP_INTRDELAY environment variable, and without using the "-intrdelay" command-line option, produces the following output:
Current interrupt delay time is 35
About to set interrupt delay time to 100...
Current interrupt delay time is 100
PROGRAM INTRDELAY_EXAMPLE
INTEGER DELAY, RC
CALL MP_QUERYINTRDELAY(DELAY)
WRITE(6,*)'Current interrupt delay time is', delay
WRITE(6,*)'About to set interrupt delay time to 100...'
DELAY = 100
CALL MP_SETINTRDELAY(DELAY, RC)
CALL MP_QUERYINTRDELAY(DELAY)
WRITE(6,*)'Current interrupt delay time is', delay
STOP
END
Related Information
Functions:
Purpose
Version
libmpi.a
C Synopsis
#include <pm_util.h> int mpc_stdout_mode(int mode);
Fortran Synopsis
MP_STDOUT_MODE(INTEGER MODE)
Parameters
Description
This Parallel Utility function requests that STDOUT be set to single, ordered, or unordered mode. In single mode, only one task output is displayed. In unordered mode, output is displayed in the order received at the home node. In ordered mode, each parallel task writes output data to its own buffer. When a flush request is made all the task buffers are flushed, in order of task ID, to STDOUT home node.
Notes
Return Values
In C and C++ calls, the following applies:
Examples
C Example
The following program uses poe with the -labelio yes option and three tasks:
#include <pm_util.h>
main()
{
mpc_stdout_mode(STDIO_ORDERED);
printf("These lines will appear in task order\n");
/*
* Call mpc_flush here to make sure that one task
* doesn't change the mode before all tasks have
* sent the previous printf string to the home node.
*/
mpc_flush(1);
mpc_stdout_mode(STDIO_UNORDERED);
printf("These lines will appear in the order received by the home node\n");
/*
* Since synchronization is not used here, one task could actually
* execute the next statement before one of the other tasks has
* executed the previous statement, causing one of the unordered
* lines not to print.
*/
mpc_stdout_mode(1);
printf("Only 1 copy of this line will appear from task 1\n");
}
Running the above C program produces the following output (task order of lines 4-6 may differ):
0 : These lines will appear in task order.
1 : These lines will appear in task order.
2 : These lines will appear in task order.
1 : These lines will appear in the order received by the home node.
2 : These lines will appear in the order received by the home node.
0 : These lines will appear in the order received by the home node.
1 : Only 1 copy of this line will appear from task 1.
Fortran Example
CALL MP_STDOUT_MODE(-2)
WRITE(6, *) 'These lines will appear in task order'
CALL MP_FLUSH(1)
CALL MP_STDOUT_MODE(-3)
WRITE(6, *) 'These lines will appear in the order received by the
xhome node'
CALL MP_STDOUT_MODE(1)
WRITE(6, *) 'Only 1 copy of this line will appear from task 1'
END
Running the above program produces the following output (task order of lines 4-6 may differ):
0 : These lines will appear in task order.
1 : These lines will appear in task order.
2 : These lines will appear in task order.
1 : These lines will appear in the order received by the home node.
2 : These lines will appear in the order received by the home node.
0 : These lines will appear in the order received by the home node.
1 : Only 1 copy of this line will appear from task 1.
Related Information
Functions:
Purpose
Queries the current STDOUT mode setting.
Version
libmpi.a
C Synopsis
#include <pm_util.h> int mpc_stdoutmode_query(int *mode);
Fortran Synopsis
MP_STDOUTMODE_QUERY(INTEGER MODE)
Parameters
Description
This Parallel Utility function returns the mode to which STDOUT is currently set.
Notes
Return Values
In C and C++ calls, the following applies:
Examples
C Example
The following program uses poe with one task:
#include <pm_util.h>
main()
{
int mode;
mpc_stdoutmode_query(&mode);
printf("Initial (default) STDOUT mode is %d\n", mode);
mpc_stdout_mode(STDIO_ORDERED);
mpc_stdoutmode_query(&mode);
printf("New STDOUT mode is %d\n", mode);
}
Running the above program produces the following output:
Initial (default) STDOUT mode is -3
New STDOUT mode is -2
Fortran Example
The following program uses poe with one task:
INTEGER MODE
CALL MP_STDOUTMODE_QUERY(mode)
WRITE(6, *) 'Initial (default) STDOUT mode is', mode
CALL MP_STDOUT_MODE(-2)
CALL MP_STDOUTMODE_QUERY(mode)
WRITE(6, *) 'New STDOUT mode is', mode
END
Running the above program produces the following output:
Initial (default) STDOUT mode is -3
New STDOUT mode is -2
Related Information
Functions:
Purpose
Determines whether a device is a terminal on the home node.
Version
libmpi.a
C Synopsis
#include <pm_util.h> int mpc_isatty(int FileDescriptor);
Fortran Synopsis
A Fortran version of this function is not available.
Parameters
Description
This Parallel Utility function determines whether the file descriptor specified by the FileDescriptor parameter is associated with a terminal device on the home node. In a Parallel Operating Environment partition, these three file descriptors are implemented as pipes to the Partition Manager Daemon. Therefore, the AIX isatty() function will always return FALSE for each of them. This function is provided for use by remote tasks that may want to know whether one of these devices is actually a terminal on the home node, for example, to determine whether or not to output a prompt.
Notes
Return Values
In C and C++ calls, the following applies:
Examples
C Example
/*
* Running this program, after compiling with mpcc,
* without redirecting STDIN, produces the following output:
*
* isatty() reports STDIN as a non-terminal device
* mpc_isatty() reports STDIN as a terminal device
*/
#include "pm_util.h"
main()
{
if (isatty(STDIN)) {
printf("isatty() reports STDIN as a terminal device\n");
} else {
printf("isatty() reports STDIN as a non-terminal device\n");
if (mpc_isatty(STDIN)) {
printf("mpc_isatty() reports STDIN as a terminal device\n");
} else {
printf("mpc_isatty() reports STDIN as a non-terminal device\n");
}
}
}