Using and Administering

Task Assignment Considerations

You can use the following keywords to specify how LoadLeveler assigns tasks to nodes. With the exception of unlimited blocking, each of these methods prioritizes machines in an order based on their MACHPRIO expressions. Various task assignment keywords can be used in combination, and others are mutually exclusive.

Table 6. Valid Combinations of Task Assignment Keywords

Keyword	Valid Combinations
total_tasks	X	X
tasks_per_node			X	X
node = <min, max>			X
node = <number>	X			X
min_processors					X		X
max_processors						X	X
task_geometry								X
blocking		X

The following examples show how each allocation method works. For each example, consider a 3-node SP with machines named "N1," "N2," and "N3". The machines' order of priority, according to the values of their MACHPRIO expressions, is: N1, N2, N3. N1 has 4 initiators available, N2 has 6, and N3 has 8.

node and total_tasks

When you specify the node keyword with the total_tasks keyword, the assignment function will allocate all of the tasks in the job step evenly among however many nodes you have specified. If the number of total_tasks is not evenly divisible by the number of nodes, then the assignment function will assign any larger groups to the first node(s) on the list that can accept them. In this example, 14 tasks must be allocated among 3 nodes:

# @ node=3
# @ total_tasks=14
 
 

Table 7. node and total_tasks

Machine	Available Initiators	Assigned Tasks
N1	4	4
N2	6	5
N3	8	5

The assignment function divides the 14 tasks into groups of 5, 5, and 4, and begins at the top of the list, to assign the first group of 5. The assignment function starts at N1, but because there are only 4 available initiators, cannot assign a block of 5 tasks. Instead, the function moves down the list and assigns the two groups of 5 to N2 and N3, the assignment function then goes back and assigns the group of 4 tasks to N1.

node and tasks_per_node

When you specify the node keyword with the tasks_per_node keyword, the assignment function will assign tasks in groups of the specified value among the specified number of nodes.

# @ node = 3
# @ tasks_per_node = 4
 

blocking

When you specify blocking, tasks are allocated to machines in groups (blocks) of the specified number (blocking factor). The assignment function will assign one block at a time to the machine which is next in the order of priority until all of the tasks have been assigned. If the total number of tasks are not evenly divisible by the blocking factor, the remainder of tasks are allocated to a single node. The blocking keyword must be specified with the total_tasks keyword. For example:

# @ blocking = 4
# @ total_tasks = 17
 

Where blocking specifies that a job's tasks will be assigned in blocks, and 4 designates the size of the blocks. Here is how a blocking factor of 4 would work with 17 tasks:

Table 8. blocking

Machine	Available Initiators	Assigned Tasks
N1	4	4
N2	6	5
N3	8	8

The assignment function first determines that there will be 4 blocks of 4 tasks, with a remainder of one task. Therefore, the function will allocate the remainder with the first block that it can. N1 gets a block of four tasks, N2 gets a block, plus the remainder, then N3 gets a block. The assignment function begins again at the top of the priority list, and N3 is the only node with enough initiators available, so N3 ends up with the last block.

unlimited blocking

When you specify unlimited blocking, the assignment function will allocate as many jobs as possible to each node; the function prioritizes nodes primarily by how many initiators each node has available, and secondarily on their MACHPRIO expressions. This method allows you to allocate tasks among as few nodes as possible. To specify unlimited blocking, specify "unlimited" as the value for the blocking keyword. The total_tasks keyword must also be specified with unlimited blocking. For example:

# @ blocking = unlimited
# @ total_tasks = 17
 

Table 9. unlimited blocking

Machine	Available Initiators	Assigned Tasks
N3	8	8
N2	6	6
N1	4	3

The assignment function begins with N3 (because N3 has the most initiators available), and assigns 8 tasks, N2 takes six, and N1 takes the remaining 3.

task_geometry

The task_geometry keyword allows you to specify which tasks run together on the same machines, although you cannot specify which machines. In this example, the task_geometry keyword groups 7 tasks to run on 3 nodes:

# @ task_geometry = {(5,2)(1,3)(4,6,0)}
 

The entire task_geometry expression must be enclosed within braces. The task IDs for each node must be enclosed within parentheses, and must be separated by commas. The entire range of task IDs that you specify must begin with zero, and must end with the task ID which is one less than the total number of tasks. You can specify the task IDs in any order, but you cannot skip numbers (the range of task IDs must be complete). Commas may only appear between task IDs, and spaces may only appear between nodes and task IDs.