Home | Downloads | Support | About GGY | About AXIS

Article Title:
Recommended Hardware for a GridLink Farm
Article Type:
General System  
Article ID #:
GGY Contacts :
Robert Sweezie
Last Modified:
01 Oct 15
Article Summary:

You may be considering a farm with a certain number of cores and job queues to satisfy your current processing requirements. We strongly recommend you consider your future needs before setting up even a small farm, so that you can continue to use the hardware you choose now as part of the larger farm you may need tomorrow.

(Click here for printable version)

Article Detail:

Quick Recommendations:

Click on the quick recommendation section title ("Processors, "Networking", etc.) for detailed discussion on that topic:

Processors: Fewer cores with higher clock speeds are preferable to a larger number of cores with lower clock speeds.  Using higher clock speed cores also saves money as GridLink is licensed per core. Current generation Intel E5-26xx V3 processors with up to 14 cores per processor are recommended  (16 and 18 core E5 processor options are not recommended).   Servers containing 4 or more processors (Xeon E5-46xx or similar) are also not recommended.
Networking: Minimum 1 Gigabit networking connecting all GridLink servers.   All servers should be located in a single physical location on a single subnet without routers or firewalls between them.
Virtual Servers: Virtualization of GridLink is not recommended.  In cases where there is no other choice (such as public cloud environments), only specific node types and configurations should be used.  See detailed discussion for further details.
Memory: 4 GB of memory per processor core - a 16 core server would need 64 GB ram, 24 cores would need 96 GB, etc.
Operating System: Windows Server 2012 R2 Standard is recommended, 2012 R1 and 2008 R2 Enterprise are also supported.   2008 R2 Standard is not recommended due to limitations on maximum amount of memory per server.
Hard Drives: Local disk performance on all servers should be equivalent to 3 x 15k drives in raid 0 for servers up to 24 cores. 4 x 15k drives in raid 0 are required in servers with more than 24 cores.  The use of raid 10 for redundancy is recommended on master servers but is not required on helper servers due to automatic recovery features in GridLink.  Using NAS or SAN devices for GridLink local storage is not recommended.
Antivirus, Backups, etc.: If antivirus is required, proper exclusions must be in place for all servers.  In addition, other applications (backup, deduplication, etc.) which can interfere with disk access should not be run on GridLink servers.
Master and Helper Servers: Each queue in your GridLink farm requires an individual server - it is not possible to have more than one queue on a single server.  As a result, the number of servers in your farm must be greater than or equal to the number of queues.
Backups: No data is stored permanently on the GridLink servers.  As a result, only the GridLink farm profile needs to be back up.  GridLink offers a built in option to backup this profile to a location outside the Grid environment.
AXIS EnterpriseLink: GGY recommends deploying AXIS EnterpriseLink along with GridLink to provide frontend access to the GridLink environment

Sample Configuration:

Most major vendors (IBM, HP, Dell etc.) offer rack servers designed for the Xeon E5-2600 V3 series

The Dell PowerEdge R730 is a good example of a 2U server that supports two Xeon E5-2600 V3 series chips and up to eight 2.5" 15K SAS drives.

Dell R730 Configuration 1 - 16 Cores Configuration 2 - 24 Cores Configuration 3 - 28 Cores
Processor: Xeon E5-2667 v3 (3.2 Ghz) Xeon E5-2690 v3 (2.6 Ghz) Xeon E5-2697 v3 (2.6 Ghz)
Memory: 64 GB 96 GB 112 GB
Drives - Master: 6x300 GB 15k SAS, RAID 10 6x300 GB 15k SAS, RAID 10 8x300 GB 15k SAS, RAID 10
Drives - Helpers: 3x300 GB 15k SAS, RAID 0 3x300 GB 15k SAS, RAID 0 4x300 GB 15k SAS, RAID 0

Sample Configuration Notes:

  • Configure additional servers to get to the required farm size - i.e. 6 x Configuration 1 would produce a farm of 96 cores. 

  • A minimum of one server per queue is required - a farm with 4 queues, for example, would require a minimum of 4 servers.  See "Master and Helper Servers" for more details.

  • It is generally recommended to use the same configuration for all nodes in a farm.  If mixing different server configurations, use the fastest (highest clock speed) nodes as masters.

  • The hard drives specifications are minimums and performance may be improved with faster drives, SSD drives or more drives per RAID array.

  • Blade servers can be used, but tend to have limitations on local storage.  See "Blade Servers" for more details.

Detailed Recommendations:


The Intel E5-26xx V3 processors carry our top recommendation for use with GridLink.  Processors are available with 4, 6, 8, 10, 12, 14, 16 and 18 cores per processor in two processor servers.   In general, the higher the core clock speed the better each core will perform.   Sixteen 3.4 GHz cores, for example, will outperform sixteen 2.6 GHz cores. 3.4 GHz clock speeds, however, are only available on 6 core processors while 2.6 GHz processors are available with 14 cores.  So you would need more than twice as many 3.4 Ghz processors to get the same number of cores. As a result, other factors such as server and operating system costs, licensing fees, and your datacenter density and power consumption requirements should be considered along with your speed requirements to determine the optimal processors for your environment.  

Note that we do not recommend 16 and 18 core processor options.  The maximum native clock speed of 2.3 GHz combined with the high price and power consumption of these chips makes them a poor choice for GridLink workloads. 

Our top recommendations are as follows: 

Xeon E5-2697 V3  2.60 GHz 14 cores
Xeon E5-2690 V3   2.60 GHz 12 cores
Xeon E5-2685 V3  2.60 GHz 12 cores
Xeon E5-2660 V3 2.60 GHz 10 cores
Xeon E5-2667 V3 3.20 GHz 8 cores

We are disappointed by the performance of 4-processor and 8-processor servers for high performance computing so at this time we will not recommend their use in a GridLink farm.


Connect up all the servers in your GridLink farm on the same subnet using at least 1 Gbps networking. This applies even if the rest of your data centre is using a 100Mbps network. InfiniBand or 10G Ethernet is preferred but not required and will have the most benefit on the Master servers.

At this time we require a registry entry to disable SMB 2.0 which automatically downgrades to SMB1 level. Click here: AXIS GridLink Advisory - Windows 2008 with SMB 2.0 Severely Impacting AXIS GridLink for more information.

Virtual Servers

We do not recommend virtualizing GridLink servers as the hypervisor layer reduces the performance of the hardware.  We do, however, support virtual machines in those environments where there is no direct access to “bare metal” hardware, such as public clouds.

This support is conditional on meeting the following minimum requirements:

  • There is one-to-one correspondence between physical (non-hyperthreaded) cores on the host and virtual cores allocated to the VM running GridLink.
  • There is a minimum 4GB per core of physical RAM on the host backing the RAM allocated to the VM.
  • The disk space allocated to the virtual machine meets the requirements outlined in the "Hard Drives" section of this document.
  • The performance of the disks in the host is on a par with our recommended configuration for a physical machine (min 3 way striping x 15 K, RAID 10)
  • Minimum 1Gbps network connectivity (single subnet)
  • Performance and stability can only be guaranteed if a single VM is hosted on a physical server with one-to-one correspondence between the size of the host and the size of the VM. If multiple VM’s are running on the same host the effect of their load on the GridLink VM is unpredictable. Note: this requirement from GGY is consistent with the approach chosen by all major cloud providers for compute intensive (HPC) applications like AXIS.
Virtual servers are appropriate for EnterpriseLink Servers or Front-End Servers if adequate processing power and memory as per GGY requirements for each concurrent user session is backed up by the identical number of CPU cores and the amount of RAM in the host machine.

Memory and O/S

You will need at least 4GB of memory per processor core.

Please note that each Windows version has its own memory limit. For details, please review Microsoft KB article "Memory Limits for Windows Releases".

For new installations we recommend Windows Server 2012 R2 Standard.   Windows Server 2012 Standard and Windows Server 2008 R2 Enterprise are also supported.   We no longer recommend Windows Server 2008 R2 Standard as it is limited to 32 GB of memory per machine.   With modern 24+ core servers, this is not enough to meet our minimum requirement of 4 GB of memory per core.   Window Server 2008 R2 Enterprise supports 2 TB of ram per machine and 2012 (R2) Standard supports up to 4 TB. 

Hard Drives

GridLink servers require access to large amounts of high speed storage.  This storage is temporary - it is used during the calculations, but no data is stored there permanently.  We recommend the larger of either:

600GB of capacity


<# of cores in the server> multiplied by <the uncompressed size of your largest dataset> + 25% extra space as a buffer

Minimum disk performance should be the equivalent to 3 x 15k drives in Raid 0, with 4 x 15k drives in Raid 0 being recommended when using servers with more than 24 cores.   If a disk failure occurs on a helper server, the work is automatically redistributed to other nodes, so disk redundancy on helper servers is optional.  On master servers, however, it is recommended to provision 6 or 8 disks in Raid 10 so that disk failure does not cause running jobs to fail. You may also provide separate drives (RAID 1) for the O/S on both Master and Helpers servers although this is not a GridLink requirement or a GGY recommendation. You can save money and drive bays by introducing an O/S partition of say 100GB on the main RAID array.

We recommend that this storage be provided via local disks.   For master servers only, you can alternately provision the drives via a direct fiber connection to a suitable SAN.  The SAN, however, needs to be able to sustain performance equivalent to the local disk configurations noted above.   Helper servers should not be provisioned with disks residing on a SAN. Given the temporary nature of the data stored on the helper servers, and the automatic recovery mechanisms, there is no advantage associated with the added expense of SAN based disks.  In addition, we have seen situations where the combined throughput of a large number of helper servers has caused congestion and performance problems even on enterprise grade SAN devices. 
We get a lot of questions about SSD devices instead of traditional spinning disks. They have improved a lot in terms of reliability
and affordability. They are very fast for random access reads and writes, but this does not necessarily translate into improved AXIS performance since for most of the time we rely on  sequential reads and writes where they offer little advantage. They are a good solution in situations like blade servers where space is limited and, as they continue to evolve, may eventually replace spinning disks as our recommended solution.  At this time, however, we still do our development and testing against the performance characteristics of the spinning disk configurations listed above.

Important - Hard Disk Performance and Reliability

Both master and helper servers perform a large number of time sensitive disk reads and writes. As a result, it is vital that you not use any software which interferes with native disk access. This software includes (but is not limited to):

  • Data deduplication software
  • Disk compression software
  • Real-time defragmentation software
  • Real time backup software (e.g. Microsoft Volume Shadow Copy Service)
  • Real-time anti-virus scanners (please see the "Anti-Virus" section of this document for more info on anti-virus exclusions)

These types of software will cause severe instability and performance issues.

Master Servers and Helper Servers

You can now choose how many cores you want in a server farm and how many Queues you want on that farm. Note that each Queue requires a separate server. The number of Queues should correspond to how you wish to use the farm. For example, you may wish to set up separate Queues for Pricing, Testing and Valuation. You can buy GridLink Core licenses ($1000 per core) and GridLink Queue licenses ($4000 per Queue) to support the configuration you require. Special pricing is available for Large Farms (256 cores or greater).

A server with a GridLink Queue license is a Master server, and without it, it's a Helper server.  In most cases the requirements for master and helpers servers are the same except that we recommend redundant disk configurations on the masters but redundancy is optional on the helpers.   If mixing different server configurations, the GridLink queue licenses should always be attached to the servers with the fastest processors and best disk subsystems in the farm.

We strongly recommend that the datasets you run should be stored on a network share with fast and reliable connection between the server hosting this share and the Master servers in your farm. The GridLink service will automatically copy the Dataset from its current location to the drive of the Master server before the job starts, and copy it back once the job is complete. This is a change from our earlier recommendation to store the datasets on the drive of the Master server. The reason for this change is that the end user may be opening datasets located on the Master Server’s drives and running jobs or performing other disk intensive tasks locally on his/her desktop or front-end/EnterpriseLink server before or after the GridLink run This may impose an additional load on the GridLink server's drive system that can cause a significant deterioration in performance or in some cases instability on both sides: the GridLink runs as well as in the users’ interactive AXIS sessions.

Blades or Rack Mounted Servers

Some Blade servers offer excellent performance and flexible storage options, but many Blade servers provide very limited disk capacity, speed, and do not support the fast processors.  As a result, you need to be careful that the recommended storage, memory and processors are available in the Blade chassis that you are considering. Some 1U rack servers are also limited to two 3.5" drives or four 2.5" drives. We  recommend 2U rack servers for their storage abilities. If you are using Blade or other small servers you may get the disk speed you need by using SSD drives or Fusion IO cards.


If you implement our current recommendations for storing the datasets outside of the GridLink farm you no longer have any data residing on the GridLink master’s or helper’s drives that needs backing up. Taking a snapshot or image of the server to be able to recover the operating system and installed executable files should be quite sufficient. Even this may not be needed as re-installing GridLink is usually a very quick process. You should consider backing up the farm profile after every configuration change for faster recovery. See this page for more information: https://www.ggy.com/gridlink/farm_recovery.htm

For the partitions outside of the GridLink farm that house the datasets and other user data files we now provide intelligent, safe and fully automated backup functionality built into the AXIS EnterpriseLink module. For more information please contact GGY System support.

Do not perform backups using third party tools on GridLink Master or Helper servers while AXIS is running under any circumstances. The backup you perform may lock files that AXIS needs and produce errors and the backup will also be unusable since it may be made while the dataset is changing.

For farms that still house the datasets on the local drives of the Masters we have built an intelligent backup facility into GridLink itself. It allows you to schedule backups like other backup software but it will not attempt to back up a dataset that is running - it will wait for a suitable time. The backup it produces is simply a zip file for each Dataset or Database located in a safe place for you to backup later using your normal backup software.

AXIS will store only temporary data on the Helper servers so there is no need to back these up.

You do not need to backup the various AXIS versions installed on the GridLink servers because they are always readily available for download from our website.

AXIS EnterpriseLink

We recommend that you set up one or more AXIS EnterpriseLink front end servers to support the interactive clients who are using your farm, and also for those who do not require farm access. EnterpriseLink is included in your AXIS license.

For more information please visit AXIS EnterpriseLink web site or contact GGY.


Contact | Client Login / File Upload | E-mail GGY   Search