AXIS GridLink - Controller and Utility - Farm Profile Management Features

GridLink Utility:
Farm Profile Management Features

A farm profile is a collection of configurations including the list of server computers on the processor farm and settings for GridLink controllers running on the servers. The user can perform the following actions under "Farm Profile" menu.

Select Current Farm

The user can create more than one farm profiles and can switch between them by selecting the current farm.

There is a "Farm List" combo box on the tool bar. The user can use it as a short-cut to quickly switch to another farm.

Create Farm Profile

There are 3 ways to create a new farm profile.

Create a new farm with empty server list and default settings.
Create a new farm based on an existing farm profile.
Create a new farm based on a farm profile retrieved from an existing server.

After a farm profile has been created, the user can add servers into the farm, sort server list, and configure all options using the utility.

Delete Farm Profile

The user can delete the current farm profile. A confirmation is required to perform this action.

Rename Farm Profile

The user can rename the current farm profile.

Copy Farm Settings To

The feature provides an easy way to synchronize farm settings (AXIS Path, Authorized User list, etc.) between multiple Farm Profiles.

If you have multiple Farm Profiles with different server lists but the same settings, after you have made some changes on the farm settings in one profile, you can select this menu and select one or more farm profiles to update them with the same changes.

Please note that the server list is not copied.

Add Server to Farm

To add a new server to an existing farm, the user can either type in a server name or select one from a computer browse dialog.

Remove All Servers

To remove all servers from the current farm.

Sort Server List by Name

The user may sort the server list in the current farm by server name in ascending order.

Farm Settings

The use can see all the current farm settings shown in the following tabs.

Authorized Users

Only authorized AXIS users can use features provided by GridLink module, e.g. submitting jobs to the farm, monitoring remote job status, etc. There are 3 types of users: basic, regular and advanced users.

Basic users can submit jobs to the farm and monitor/manage their own jobs. If the company doesn't allow a user to view jobs submitted by other people, the company should assign this user as a "basic" user.

Regular users can submit jobs to the farm, monitor/manage their own jobs and monitor all jobs in the queue. If the company allows a user to view jobs submitted by other people but doesn't allow him/her to manipulate them, the company should assign this user as a "regular" user.

Advanced users can submit jobs to the farm and monitor/manage all jobs in the queue.

You can add new users into the list or remove users from the list by clicking on "Add" or "Remove" buttons. When adding a new user, you can specify a number of user properties; You can also modify the properties for an existing user by selecting a user and clicking on "Modify" button.

In "Advanced" tab, the farm administrator can specify more properties for the user.

By default, users are able to work with all licenced controllers on the farm. However, consider the following case: a user A from Department A and another user B from Department B are sharing the same farm with 2 licenced controllers. For security reasons, it would be preferable to designate one controller to user A, and the other controller to user B. While both of them will be able to utilize the whole farm (e.g. if there's only one job submitted by user A is running on the farm, this job will use the whole farm), user A will not be able to view the information of jobs submitted by user B, and vice versa. Controller designation can be done by editing user's information.

If you wish to limit the resource priority for jobs submitted by a user, you may specify a "Maximum Resource Priority" for this user. e.g. once you specify it to be "low", all jobs submitted by this user will not get a priority higher than "low". If you don't want to have any priority limit on a user, simply specify this setting as "urgent". It is "high" by default.

If a user wishes his/her remote batch jobs to always use a particular printer, you can specify the printer for this user in the advanced user properties dialog.

If this is a local printer (attached directly to the server) or a network printer but installed as a local printer, then the printer name is like "MYPRINTER".

If this is a network printer shared from a server, then the printer name is like "\\MYSERVER\MYPERINTER".

The printer name is not case-sensitive and can contain spaces. If the printer has not been specified or is not available when AXIS is running, it will use the default printer on the server.

Please note that AXIS versions prior to 11.5.02.001 don't support the user authentication feature. The user may want to enforce security for older versions so that users using older AXIS versions will not be able to connect to a controller to submit a job, monitor the job status, and manage the job queue.

AXIS Installation Path

In order to launch the AXIS master of the correct version, the controller needs to know the path name where AXIS is installed. When installing AXIS on the server machine, the user needs to append the AXIS version string to the path name (e.g. C:\WinAXIS11401001) so that more than one AXIS version can coexist on the same machine.

The user can choose the format of the AXIS version, which can either have dots (e.g. C:\WinAXIS11401001) or not (e.g. C:\WinAXIS11.4.01.001).

The user can also specify a temporary files path for AXIS master and helpers running on this server. By default the path is "C:\Spare". The controller is responsible of clearing temporary files that may be left by a previous AXIS crash so that they will not accumulate and eventually use up the disk space.

Network Settings

Exclude IP Addresses - Suppose that the computer where the AXIS master is running has 2 network cards for 2 separate sub-nets, and all helpers are running on the same sub-net, then the network card on the master computer for the other sub-net will not be used to connect to any helper computer participating the distributed processing. Same situation also applies to a helper computer with more than one network card. However, IP addresses of these unused network cards are still exposed to other computers during the distributed processing. Therefore, one computer may try to connect to an IP address on a different sub-net and will report an error message after the attempt fails. When this happens, a helper will take longer time to join the distributed processing than it should.

The user can specify the IP addresses associated with these unused network cards to make the distributed processing more efficient. The user can either add a new address into this list or remove one from it.

Network Services - the network services settings (UDP Ports, IPX Sockets, TCP port, etc) should not be changed unless they conflict with existing network services. If it becomes necessary to change these settings, please ensure that all masters and helpers that will perform distributed processing together have the same UDP/IPX/NetBIOS settings, and all GridLink Controllers have the same TCP port. Otherwise, they will not be able to communicate with each other.

The TCP port used by controllers can be either changed remotely through the utility, or locally on the controller by using "FirstAid.exe".

Network Drive Mapping

If the AXIS user refers to a shared network folder in his/her batch job (e.g. dataset path, Import/Export database link, a file opened in AXIS script, etc.) with a mapped drive letter, GridLink service needs to have the mapping information in order to access the correct folder. This is be done by specifying the drive mapping information in this tab.

Please note that AXIS remote job will not start if any of the drive mapping applicable to the user didn’t succeed. In the case that there are a few Access Groups created and a mapped drive is only to be used by one of the groups, this drive can be associated with this group, so that this drive will not be mapped when a job is launched under another group.

Advanced (GridLink)

Resource Priority Ratio

When a job is submitted, it is given a "resource priority" ("urgent", "high", "normal", "low", or "no helpers") which is used to balance helper resources for jobs between 2 queues. Currently, the default ratios for high/normal/low priorities are 100:50:25:5. You can specify your own ratios here.

For more information about "Resource Priority", please click here.

Windows Firewall

Computers might be protected by Windows Firewall, which blocks some communication required to manage the distributed processing feature of AXIS. The user may choose to let the controller to automatically unblock AXIS if it is blocked.

Please note that this feature works for Windows XP SP2 only and the account under which the controller is run requires administrator's right to unblock AXIS.

AXIS Remote Master Failure Detection

The controller can check the CPU usage of AXIS remote masters to make sure they are working properly. If a master has been idling for more than 10 minutes, the job status will be changed from "Running" to "Idling" and the user should check the job status immediately. If a master's idling time is longer than the specified time out period, the master will be terminated so that the CPU can be released to run other jobs. The job status is then changed to "Hung".

Before the master is terminated, the controller will try to retrieve the batch status dialog and save it in the job record. The user can view this information by right-clicking on a hung job in the Job Monitor and select the menu "Show Job Status Retrieved When It Hung".

The user can specify the time out period from 30 minutes to 30000 minutes. The user can also choose to disable the detection.

Advanced (AXIS)

The following advanced parameters are parts of AXIS System Parameters. When AXIS jobs are running on a GridLink farm, these parameters are controlled by GridLink, and can be edited through GridLink Utility.

Subnet

If the user's servers are not sitting on the same subnet, the user needs to turn on "Subnet support" option to ensure the communication between AXIS master and helpers. This setting will override the setting in AXIS system parameters.

Failure Detection Time-out period - A helper that has stopped responding for a certain time period is deemed to have failed. The default value of the time-out period is 1 minute. The user may want to use a longer period if he/she has slow machines. It can be adjusted from 1 to 10 minutes.

Failure Recovery ensures that the master will recover cells that this failed helper has taken but not finished and will redistribute these cells to other machines. Please note that Failure Recovery is not supported in Batch Generate.

Stochastic Processing Memory Usage

Specify the maximum amount of memory (RAM) that can be allocated for each instance of AXIS in Block calculations. For more details, click here.

Formula Table Accelerator

This option has 3 choices:

1. Use the setting received from the job submitter (still allows the user to control this setting. Default.)

2. Enable Formula Table Accelerator for all jobs

3. Disable Formula Table Accelerator for all jobs

Job Start-up

Number of Job Slots (Previously called "Number of Remote Jobs Running Simultaneously")

In each job queue, GridLink can run one distributable job and multiple non-distributable jobs in parallel. The number of job slots is the maximum number of jobs that can run simultaneously in one job queue.

The number of job slots in one job queue is determined by a number specified by the user and the number of CPU cores on the master server. (e.g. if the user specifies 4 job slots for all job queues, the number of job slots in a quad-core server's job queue will be 2.)

The user can use the same number of job slots for all queues, or a different number for each job queue.

The maximum number of Job Slots is 4.

By default, a new job is started when a job slot is available AND there's no other distributable job running in this queue. The user can specify to start a new job as long as a job slot is available, even if a distributable job is already running. In this case, GridLink controller will need to shutdown one helper to release a CPU core to start the new job.

Moving Remote Job Between Queues

If the current job queue is busy, then the job may be moved to another available queue so that it can be started right away. When this happens, the job status is shown as "Moved". When the job is running, the user can be redirected to the other job queue to view the batch status dialog. When the job has finished, the original job record will be updated with the final status ("Done", "Aborted", etc.) and the runtime information.

Job Status

Update Speed

When a batch job is running on the server machine, the controller constantly checks its status so that the next job in the queue can start as soon as the current one is finished and the machine becomes available. The controller may need to retrieve detailed information on the batch status dialog from the AXIS master and send it back to a user who is monitoring the batch status from his/her machine.

The user can specify how often the controller refreshes the batch status in seconds. The range is 1 - 60 seconds.

Windows of the AXIS Remote Master/Helper

When a controller runs as Windows service, by default, AXIS master and helper launched by the controller run on an invisible desktop and cannot receive user input. If the user wants to interact with the AXIS window, he/she needs to turn on the option "Make the window of the AXIS remote master/helper visible". (Please note that this option is not supported by AXIS versions prior to 11601001. Also, the local administrator's right is required for the service account.)

To view the window of AXIS that was remotely launched by GridLink service, the user needs to log on to the console session (session 0) of the server. The command line syntax is shown below:

mstsc -v:servername -console

Once the user has logged on, he/she should not log off if he/she sees any AXIS windows (i.e. a job is running). It is safe to log off if there's no AXIS windows shown.

If the server has move than 4 CPU cores, the user need to either turn on this option or modify the registry to allocate more memory for non-interactive desktop heap. If the user doesn't do so but GridLink service detects that the operating system may run out of non-interactive desktop heap, it will enforce AXIS to be launched on a visible desktop. If it failed to do so, e.g. due to lack of user rights, the non-interactive desktop heap may be used up and AXIS will crash. For more information, please refer to Microsoft KB article:

http://support.microsoft.com/default.aspx?scid=kb;en-us;184802

Also, please note that in order to use this feature, the service account must have the local administrator's right.

Job Records & Logs

Remote Job Records

By default, an authorized user can delete job records if these jobs are not currently running. The farm administrator may stop the user from doing this by disabling "Allow job records to be deleted" option.

Also, the farm administrator can schedule the automatic historic records clean-up if they have finished more than a specified number of days.

Log Controller Activity Messages

When "Log controller activity messages" option is enabled, the activities of clients connecting to the controller will be logged and saved into a text file "Log.txt", which is located under the utility installation directory. Otherwise, only error messages will be saved.

Please note that this file is limited under 1 MB. Old messages will be removed automatically once it reaches the limit in order to log new messages.

Backup

Starting from version 2.30, the GridLink Utility has introduced a new feature which allow users to backup a farm. For more details, click here.

Access Group

Access Group feature allows remote jobs submitted by a user to run under this user's credential. This enables different users to have controlled access to different directories when running AXIS jobs. This can be particularly useful when both development and production jobs are being run on the same farm by different users and special protection and restricted access rights are necessary to protect the production datasets and associated directories. For more details, click here.

CPU Limit

On a GridLink farm, one CPU core is used to run one AXIS copy. A distributable job involves many copies of AXIS running on many CPU cores.

There is no limit on the size of a GridLink farm, as long as you have enough GridLink licences to cover the CPU cores. However, the number of CPU cores participating in one distributable job on a farm is limited.

For AXIS versions prior to 12.1.02.001, the maximum number of CPU cores participating in one distributable job is limited up to 64.

For AXIS 12.1.02.001 and later versions, GridLink supports up to 256 CPU cores per job. The user can choose to use less cores per job if it is necessary. The user can either specify a new limit which applies to all AXIS versions, or different limits for different AXIS versions.

Enterprise Grid

GridLink now has been integrated with DataSynapse GridServer and Platform EGO. If you are currently using one of these Grids and wish to have GridLink working with it, you can select "DataSynapse GridServer" or "Platform EGO" from this page.

A number of parameters (e.g. the environment variable DSDRIVER_DIR) need to be specified so that GridLink knows how to connect to the Grid and request CPU cores when AXIS job starts and release them when the job finishes.

We will continue to work with more enterprise grid providers and integrate GridLink with them upon requests from users.

For more information about Grid Computing, please click here.