Using and Administering

Step 1: Specify Machine Stanzas

The information in a machine stanza defines the characteristics of that machine. You do not have to specify a machine stanza for every machine in the LoadLeveler cluster but you must have one machine stanza for the machine that will serve as the central manager.

If you do not specify a machine stanza for a machine in the cluster, the machine and the central manager still communicate and jobs are scheduled on the machine but the machine is assigned the default values specified in the default machine stanza. If there is no default stanza, the machine is assigned default values set by LoadLeveler.

Any machine name used in the stanza must be a name which can be resolved to an IP address. This name is referred to as an interface name because the name can be used for a program to interface with the machine. Generally, interface names match the machine name, but they do not have to.

By default, LoadLeveler will append the DNS domain name to the end of any machine name without a domain name appended before resolving its address. If you specify a machine name without a domain name appended to it and you do not want LoadLeveler to append the DNS domain name to it, specify the name using a trailing period. You may have a need to specify machine names in this way if you are running a cluster with more than one nameserving technique. For example, if you are using a DNS nameserver and running NIS, you may have some machine names which are resolved by NIS which you do not want LoadLeveler to append DNS names to. In situations such as this, you also want to specify name_server keyword in your machine stanzas.

Under the following conditions, you must have a machine stanza for the machine in question:

Machine stanzas take the following format. Default values for keywords appear in bold:

Figure 25. Format of a Machine Stanza

label: type = machine
adapter_stanzas = stanza_list
alias = machine_name
central_manager = true | false | alt
cpu_speed_scale = true | false
dce_host_name = dce hostname
machine_mode = batch | interactive | general
master_node_exclusive = true | false
max_adapter_windows = [all | none | <+> n | -n ]
max_jobs_scheduled = number
name_server = list
pvm_root = pathname
pool_list = pool_numbers
resources = name(count) name(count) ... name(count)
schedd_fenced = true | false
schedd_host = true | false
spacct_excluse_enable = true | false
speed = number
submit_only = true | false

You can specify the following keywords in a machine stanza:

adapter_stanzas = stanza_list
where stanza_list is a blank-delimited list of one or more adapter stanza names which specify adapters available on this machine. All adapter stanzas you define must be specified on this keyword.

alias = machine_name

where machine_name is a blank-delimited list of one or more machine names. Depending upon your network configurations, you may need to add alias keywords for machines that have multiple interfaces.

Note: In general, if your cluster is configured with machine hostnames which match the hostnames corresponding to the IP address configured for the LAN adapters which LoadLeveler is expected to use, you will not have to specify the alias keyword. For example, if all of the machines in your cluster are configured like this sample machine, you should not have to specify the alias keyword.

Machine porsche.kgn.ibm.com

However, if any machine in your cluster is configured like either of the following two sample machines, then you will have to specify the alias keyword for those machines:

  1. Machine yugo.kgn.ibm.com
    • The hostname command returns yugo.kgn.ibm.com.
    • The Ethernet adapter address 129.40.8.21 resolves to hostname chevy.kgn.ibm.com.
    • No adapter address resolves to yugo.

    You need to code the machine stanza as:

    chevy: type = machine
    alias = yugo
    
  2. Machine rover.kgn.ibm.com
    • The hostname command returns rover.kgn.ibm.com.
    • The FDDI adapter address 129.40.9.22 resolves to hostname rover.kgn.ibm.com.
    • The Ethernet adapter address 129.40.8.22 resolves to hostname bmw.kgn.ibm.com.
    • No route exists via the FDDI adapter to the clusters central manager machine.
    • A route exists from this machine to the central manager via the Ethernet adapter.

    You need to code the machine stanza as:

    bmw:   type = machine
    alias = rover
    

central_manager =  true | false |  alt 

where true designates this machine as the LoadLeveler central manager host, where the negotiator daemon runs. You must specify one and only one machine stanza identifying the central manager. For example:
machine_a: type = machine
central_manager = true

false specifies that this machine is not the central manager.

alt specifies that this machine can serve as an alternate central manager in the event that the primary central manager is not functioning. For more information on recovering if the primary central manager is not operating, refer to What Happens if the Central Manager Isn't Operating?. Submit-only machines cannot have their machine stanzas set to this value.

If you are going to select machines to serve as alternate central managers, you should look at the following keywords in the configuration file:

For information on setting these keywords, see Step 10: Specify Alternate Central Managers.

cpu_speed_scale =  true | false
where true specifies that CPU time (which is used, for example, in setting limits, in accounting information, and reported by the llq -x command), is in normalized units for each machine. false specifies that CPU time is in native units for each machine. For an example of using this keyword to normalize accounting information, see Task 5: Specifying Machines and Their Weights.

dce_host_name = dce hostname

where dce hostname is the dce hostname of this machine. Execute either "SDRGetObjects Node dcehostname," or "llextSDR" to obtain a listing of DCE hostnames of nodes on an SP system.

machine_mode =  batch  |  interactive  | general
Specifies the type of job this machine can run. Where:

batch
Specifies this machine can run only batch jobs.

interactive
Specifies this machine can run only interactive jobs. Only POE is currently enabled to run interactively.

general
Specifies this machine can run both batch jobs and interactive jobs.

master_node_exclusive =  true | false
where true specifies that this machine is used only as a master node for parallel jobs.

max_adapter_windows = [ all | none | <+>n | -n ]
This keyword specifies how many of a machine's available adapter windows LoadLeveler can use. The default value is all, which specifies that LoadLeveler can reserve all of the windows which are not already reserved by other applications. The value none indicates that LoadLeveler can not use any windows (consequently, no user space jobs will be dispatched to that machine). A positive number (specified, with or without the plus sign), means that LoadLeveler can use no more than the specified number of windows; however, LoadLeveler may use less than the specified number if fewer windows are actually available on the machine's adapter. A negative number means that LoadLeveler will use all but the specified number of the available windows (e.g., -n means that LoadLeveler will reserve n windows for use by other applications).

max_jobs_scheduled = number
where number is the maximum number of jobs submitted from this scheduling (schedd) machine that can run (or start running) in the LoadLeveler cluster at one time. If number of jobs are already running, no other jobs submitted from this machine will run, even if resources are available in the LoadLeveler cluster. When one of the running jobs completes, any waiting jobs then become eligible to be run. The default is -1, which means there is no maximum.

name_server = list
where list is a blank-delimited list of character strings that is used to specify which nameserver(s) are used for the machine. Valid strings are DNS, NIS, and LOCAL. LoadLeveler uses the list to determine when to append a DNS domain name for machine names specified in LoadLeveler commands issued from the machine described in this stanza.

If DNS is specified alone, LoadLeveler will always append the DNS domain name to machine names specified in LoadLeveler commands. If NIS or LOCAL is specified, LoadLeveler will never append a DNS domain name to machine names specified in LoadLeveler commands. If DNS is specified with either NIS or LOCAL, LoadLeveler will always look up the name in the administration file to determine whether to append a DNS domain name. If the name is specified with a trailing period, it doesn't append the domain name.

pvm_root = pathname
Where pathname specifies the location of the directory in which PVM is installed. The default pathname is $HOME/pvm3.

pool_list = pool_numbers
Where pool_numbers is a blank-delimited list of non-negative numbers identifying pools to which the machine belongs. These numbers may be any positive integers including zero. This keyword provides compatability with function that was previously part of the Resource Manager.

resources = name(count) name(count) ... name(count)

Specifies quantities of the consumable resources initially available on the machine. Where name(count) is an administrator-defined name and count, or could also be ConsumableCpus(count), ConsumableMemory(count units), or ConsumableVirtualMemory(count units). ConsumableMemory and ConsumableVirtualMemory are the only two consumable resources that can be specified with both a count and units. The count for each specified resource must be an integer greater than or equal to zero, with three exceptions: ConsumableCpus, and ConsumableMemory must be specified with a value which is greater than zero, and ConsumableVirtualMemory must be specified with a value greater than 0, and greater than or equal to the image_size. The allowable units are those normally used with LoadLeveler data limits:

b bytes
w words
kb kilobytes (2** 10 bytes)
kw kilowords (2** 10 words)
mb megabytes (2** 20 bytes)
mw megawords (2**20 words)
gb gigabytes (2** 30 bytes)
gw gigawords (2** 30 words)
 
 
ConsumableMemory and ConsumableVirtualMemory values are stored in mb (megabytes) and rounded up. Therefore, the smallest amount of ConsumableMemory or ConsumableVirtualMemory which you can request is one megabyte. If no units are specified, then megabytes are assumed. Resources defined here that are not in the SCHEDULE_BY_RESOURCES list in the global configuration file will not effect the scheduling of the job.

schedd_fenced =  true  | false
where true specifies that the central manager ignores connections from the schedd daemon running on this machine. Use the true setting in conjunction with the llctl -h host purgeschedd command when you want to attempt to recover resources lost when a node running the schedd daemon fails. A true setting prevents conflicts from arising when a schedd machine is restarted while a purge is taking place. For more information, see "How Do I Recover Resources Allocated by a schedd Machine?" in the LoadLeveler Diagnosis and Messages Guide.

schedd_host =  true  | false
where true designates this as a public scheduling machine used to receive job submissions from submit-only machines, or for accepting jobs from machines which run stard but not schedd daemons. Submit-only machines do not run LoadLeveler jobs.

spacct_excluse_enable =  true  | false
Where true specifies that the accounting function on an SP system is informed that a job step has exclusive use of this machine. Note that your SP system must have exclusive user accounting enabled in order for this keyword to have an effect. For more information on SP accounting, see Parallel System Support Programs for AIX: Administration Guide, GC23-3899.

speed = number
where number is a floating point number that is used for machine scheduling purposes in the MACHPRIO expression. For more information on machine scheduling and the MACHPRIO expression, see Step 7: Prioritize the Order of Executing Machines Maintained by the Negotiator. In addition, the speed keyword is also used to define the weight associated with the machine. This weight is used when gathering accounting data on a machine basis. The default is 1.0.

The following example illustrates how the speed keyword can be used for assigning weights to machines.

If your cluster consisted of five RISC System/6000 machines that you want to have the same weight, you would not have to specify this keyword in the administration file. By default, all machines would have a weight of 1.0. If, however, you add an SP system to your cluster for parallel job processing, you may want to update the local configuration file for each node of the SP system to charge differently for resource consumption on those nodes. You would need to set the speed keyword to something other than 1.0 to make the SP nodes have a different weight.

For information on how the speed keyword can be used to schedule machines, refer to Step 7: Prioritize the Order of Executing Machines Maintained by the Negotiator.

submit_only =  true | false
where true designates this as a submit-only machine. If you set this keyword to true, in the administration file set central_manager and schedd_host to false.

Examples of Machine Stanzas

Example 1

In this example, the machine is being defined as the central manager.

#
machine_a: type = machine
central_manager = true    # central manager runs here

Example 2

This example sets up a submit-only node. Note that the submit-only keyword in the example is set to true, while the schedd_host keyword is set to false. You must also ensure that you set the schedd_host to true on at least one other node in the cluster.

#
machine_b: type = machine
central_manager = false   # not the central manager
schedd_host = false       # not a scheduling machine
submit_only = true        # submit only machine
alias = machineb          # interface name

Example 3

In the following example, machine_c is the central manager, has an alias associated with it, and can run parallel PVM jobs:

#
machine_c: type = machine
central_manager = true    # central manager runs here
schedd_host = true        # defines a public scheduler
alias = brianne
pvm_root = /u/brianne/loadl/1.2.0/aix32/pvm3


[ Top of Page | Previous Page | Next Page | Table of Contents | Index ]