AXIS GridLink - Overview

AXIS GridLink Overview

AXIS GridLink is a Grid solution to managing the processor farms used in the distributed processing of AXIS batch calculation runs. It offers a comprehensive set of tools that help you create and manage a more efficient and secure production environment.

AXIS GridLink vs. Standard Distributed Processing

The standard AXIS license allows you to run an AXIS batch job using the Distributed Processing option with up to 15 Helpers. This Standard Distributed Processing feature is our user-managed solution for reducing AXIS runtimes when complex processes on large volumes of data are involved. Helpers used in Distributed Processing may run on the same machine as the master, or on different machines in any combination.

With Standard Distributed Processing, there is quite a bit of manual intervention required to prepare and launch the batch process that is to be distributed. You need to install the correct version of AXIS on each computer, start the helper instances of AXIS using this version on each computer, retarget each helper to monitor the desired dataset paths and then launch the batch job on the designated master version of AXIS. If the master resides on your local PC it will be fully occupied with both processing the job and coordinating the helpers, and there will not likely be enough available resources to handle any other applications. If, on the other hand, the master is a remote server, you need full access to that server in order to log on as a user and run the AXIS application. You will also need full access rights to every helper machine.

Standard Distributed Processing works very well for a single batch job especially when only a few helpers are used and access rights are not a concern. When long, complex production jobs are being run on a regular basis, however, it is advantageous to set up a "farm" of connected processors dedicated to such jobs. It is difficult to efficiently share a farm among a group of users, and there is too much work involved in utilizing such a farm as a production environment with AXIS standard distributed processing.

To address these concerns, we have developed a Grid solution to managing your farm. AXIS GridLink is designed for use with dedicated farms, and gives you a more efficient test and production environment, especially when there is more than one user sharing the farm resources.

With AXIS GridLink you can:

Control your farm without logging on to the farm servers.
Initiate AXIS batch jobs from any AXIS workstation.
When the job is submitted to the farm it is not running on the user's workstation anymore so it becomes free to do other work - even run AXIS and work in a different dataset
Automatically start the appropriate Master and Helpers for the current job on the farm, and control the maximum number of helpers.
Monitor the batch job running on the farm and the queue of jobs submitted by multiple users in multiple AXIS versions, from your local workstation or from home (using VPN).
Move pending jobs up and down in the queue, or cancel them, if required.
Perform multiple non-distributed jobs (e.g. DataLink macros) in parallel with distributed ones
Review the history of runs with detailed documentation
Install new versions of AXIS on all computers in the farm with just one button click
Authorize users to perform runs on the processor farm and specify what actions are allowed for each user
Create groups with different access permissions for extra security in a production environment
Perform dataset level batch jobs as well as system batch jobs such as system formula
Schedule and execute safe backups of your data that will not interfere with active jobs

How does AXIS Gridlink work?

The following diagrams illustrate how AXIS Gridlink can help manage the use of the processor farm and optimise your access to its computing resources.

As shown in the above scenario, the AXIS user can monitor the remote job status screen and control the job from any AXIS machine without logging on to the farm server. At any time the user can close the monitor and even turn off the local machine. He/she can reconnect to AXIS Gridlink Controller later and reopen the monitor as long the job is still running. More than one user can monitor and control the job at the same time.

How AXIS GridLink can help efficiently share the farm?

There are many scenarios where the processor farm can be shared between multiple users.

You may, for example, want to allow the processor farm to be shared by two or more departments or have it organized in such a way so you can perform development work regularly and high priority production runs at the end of each month or quarter.

Each GridLink license allows you to control one job queue for one Master with up to 15 Helpers per Master. But if you have a batch job to process on your farm which does not use distributed processing (for example a DataLink batch job), you will be able to run this at the same time as it processes another distributed batch job. This means you won't have to tie up the whole farm just to run a DataLink batch job.

The most flexible setups, however, can be achieved with multiple AXIS GridLink licenses installed on the farm. If you have two or more AXIS GridLink licenses you can have two or more queues with multiple Masters splitting the pool of Helpers on a cooperative basis for both distributable and non-distributable jobs. When submitting the job to each of these queues GridLink allows you to specify the resource priority as "High", "Normal", "Low" and "No Helpers". GridLink will then automatically determine whether the job needs helpers or it is a non-distributable job and balance the resources between the queues according to the resource priorities of the jobs that are currently running in such a way that the farm processing power is always 100% utilized.

With two GridLink licenses, for example, if you have 32 processor cores available on the farm then you can run two distributed jobs in parallel with 15 Helpers on each if they have the same resource priority, or one job using only 3 Helpers and another using the remaining 27 if the first job had Low resource priority and the second one - High.

If there is only one job running on the farm from one of the queues and there are no more jobs in other queues, the job will use all 31 Helpers regardless of its resource priority (except if it is "No Helpers"). But as soon as a new job appears in another queue - the processor cores will be split according to the jobs respective resource priorities.

The following diagrams illustrate how AXIS GridLink provides the ability to balance processing resources between multiple users and jobs and achieve full utilization of the computing power of your farm:

Who can use AXIS GridLink?

AXIS GridLink offers powerful User Management feature. The administrator of the processor farm can define the list of users who can connect and perform jobs on the GridLink farm. Each user can be granted one of three levels of control: Basic, Regular and Advanced. These levels define what actions (e.g. monitor, cancel, change order of the jobs) a given user may perform on the farm.

For additional production environment security, AXIS GridLink implements a feature called "Access Groups" that allows the administrator to set up a farm in such a way that jobs submitted by different users will be ran under specific windows accounts using the accounts permissions and security settings.

What if I need to add more Helpers to my processor farm?

You may increase the maximum number of Helpers (currently 15) by buying additional Helper licenses to enhance your GridLink license.

How do I perform safe backups of my data if a farm is constantly working?

AXIS GridLink now has built-it Active Backup functionality that allows you to schedule and execute backups of your data without interfering with calculation jobs currently running on the farm.

Another way to backup your datasets and other related files is to perform the backup at the system level batch - such as system macro or system formula - these types of batch jobs are now supported on the GridLink too.