 |
|
|||||||||||||||||||||||||||||||||
Article Title: |
Recommended Hardware for a GridLink Farm - May 2010 |
Article Type: |
General System |
Article ID #: |
677 |
GGY Contact : |
Phil Gold |
Last Modified: |
04 May 10 |
| Article Summary: You may be considering an 8 core farm or a 2048 core farm, or something in between. We strongly recommend you to consider your future needs before setting up even a small farm, so that you can continue to use the hardware you choose now as part of the larger farm you may need tomorrow. |
|
(Click here for printable version) Article Detail: Principles to Consider: Multi-core Processors Minimize the number of processors. You can best achieve this today by buying the highest speed multi-core processors available. Currently that means the Xeon 5600 series, and for larger servers, the Xeon 7500 series. If your equipment maintenance costs depend on the number of servers and are high, then you may prefer quad-processor quad-core boxes (32 cores per box). Otherwise you will probably find that dual-processor six-core boxes (12 cores per box) represent the best value. Memory and O/S You will need 1 GB of memory per processor core available for AXIS. Standard 32 bit versions of Windows do not support more than 4GB of memory. To support more than 4GB, therefore, we recommend 64 bit operating systems. We now recommend Microsoft HPC Server 2008 operating system for all GridLink farms. This 64 bit O/S is a subset of Widows Server 2008 64-bit edition and offers top performance and inexpensive licensing. Master Servers and Helper Servers You can now choose how many cores you want in a server farm and how many Queues you want on that farm. Note that each Queue requires a separate server. The number of Queues should correspond to how you wish to use the farm. For example, you may wish to set up separate Queues for Pricing, Testing and Valuation. You can buy GridLink Core licences ($1000 per core) and GridLink Queue licenses ($4000 per Queue) to support the configuration you require. Special pricing is available for Large Farms (256 cores or greater). A server with a GridLink Queue license is a Master server, and without it, it's a Helper server. The two types of server have different requirements, especially regarding hard drives. The GridLink Queue licenses should always be attached to the servers with the fastest processors in the farm. We strongly recommend that the datasets you run should be as close to the Master server as possible. We now provide the option of automatically copying the Dataset from its current location to a folder specified for optimal performance (typically the Master server's local drive or the SAN directly attached to the Master server) before the job starts, and copying it back once the job is complete. Hard Drives for Master Servers On a Master server you need large capacity, high speed and redundancy. The drives need to support the largest job you will run on that farm as the Master may be receiving massive simultaneous input from all the helpers working on that job. We currently support up to 256 cores per job, and if your farm is 256 cores or larger you need a minimum of six 15K 300GB drives in a RAID 10 configuration per server (eight drives are preferred). You may also need separate drives (RAID 1) for the O/S, although this is not a GridLink requirement.
You can set up drive arrays like this using local disks, or via a fibre connection to a suitable SAN. Hard Drives for Helper Servers AXIS / GridLink supports full failure recovery for Helpers so we do not advise you to provide expensive redundant storage. The capacity requirements are lower - we recommend 146GB for up to 16 cores, and 300GB for larger servers. For speed we recommend 15K RAID 0. You may also need separate drives (RAID 1) for the O/S, although this is not a GridLink requirement.
We recommend against the use of a SAN for Helper servers. The data is temporary and connecting to a SAN adds expense and adds a large workload to the SAN, potentially slowing down other operations on the SAN.
Network Speed Connect up all the boxes in your farm using at least Gigabit networking. This applies even if the rest of your data centre is using a 100Mbps network. InfiniBand is preferred but not required. Blades or Rack Mounted Servers We do not normally recommend Blade servers because they often provide very limited disk capacity and speed, and sometimes they do not support the fast processors we recommend. Some 1U rack servers are also limited to two 3.5" drives or four 2.5" drives. We therefore recommend 2U rack servers for 12-core boxes and 4U rack servers for 32-core boxes. Anti-Virus Allow the drives to work unimpeded. This means you should not be running other disk intensive applications on these drives or this SAN while AXIS is processing and you must exclude the specific directories AXIS is writing to from any real time antivirus checking. Failure to do this will result in severe performance deterioration and will lead to disk errors. See: www.ggy.com/support/kbase/kbdetails.asp?searchterm=&articleid=271 Backups Do not perform backups on GridLink Master servers while AXIS is running under any circumstances. The backup you make may lock files that AXIS needs and produce errors and it will also be unusable since it may be made while the dataset is changing. We have built an intelligent backup facility into GridLink itself. It allows you to schedule backups like other backup software but it will not attempt to backup a dataset that is running - it will wait for a suitable time. The backup it produces is simply a zip file for each Dataset or Database located in a safe place for you to backup later using your normal backup software. AXIS will store only temporary data on the Helper servers so there is no point to backing these up.
You do not need to backup the various AXIS versions installed on the GridLink servers because they are always readily available for download from our website. Current Hardware Recommendations: Most major vendors (IBM, HP, Dell etc) offer rack servers designed for the Xeon 5600 series and Xeon 7500 series chips. The Dell PowerEdge C210 is a good example of a 2U server that supports two Xeon 5600 series chips (up to 6-core 3.33GHz) and up to twelve 3.5" 15K SAS drives. The Dell PowerEdge R910 is a good example of a 4U server that supports four Xeon 7500 series chips (up to 8 core 2.27GHz) and up to sixteen 2.5" 15K SAS drives We strongly recommend the C210 over the R910, since the fastest processor speed is 50% faster (performance is proportional in this case) and it supports 3.5" drives rather than 2.5" drives. An R910 with four Xeon 7560 processors (32 cores per box) is roughly as fast as two C210s with two Xeon 5680 processors each (12 cores per box) and occupies the same amount of space in a rack. Sample Configurations: Here are some typical configurations (without SAN): 2U Configurations: 12 Core farm - 1 Queue One 2U server (e.g. Dell C210 with 2 six-core 3.33GHz processors (Xeon X5680), 12 GB memory, 4x 300GB 15K SAS drives RAID 10. 24 Core farm - 1 Queue Two 2U servers (e.g. Dell C210) each with 2 six-core 3.33GHz processors (Xeon X5680), 12 GB memory. One Master server with 4x 300GB 15K SAS drives RAID 10, one Helper server with 2 x 146GB 15K SAS drives, RAID 0. 24 Core farm - 2 Queues Two 2U servers (e.g. Dell C210) each with 2 six-core 3.33GHz processors (Xeon X5680), 12 GB memory, 4x 300GB 15K SAS drives RAID 10. 48 Core farm - 2 Queues Four 2U servers (e.g. Dell C210) each with 2 six-core 3.33GHz processors (Xeon X5680), 12 GB memory. Two Master servers with 4x 300GB 15K SAS drives RAID 10, two Helper servers with 2 x 146GB 15K SAS drives, RAID 0. 72 Core farm - 3 Queues Six 2U servers (e.g. Dell C210) each with 2 six-core 3.33GHz processors (Xeon X5680), 12 GB memory. Three Master servers with 4x 300GB 15K SAS drives RAID 10, three Helper servers with 2 x 146GB 15K SAS drives, RAID 0. 132 Core farm - 4 Queues Eleven 2U servers (e.g. Dell C210) each with 2 six-core 3.33GHz processors (Xeon X5680), 12 GB memory. Four Master servers with 6x 300GB 15K SAS drives RAID 10, 2 x 72GB 10K SAS drives RAID 1 for O/S, seven Helper servers with 2 x 146GB 15K SAS drives, RAID 0, 2 x 72GB 10K SAS drives RAID 1 for O/S. 264 Core farm - 8 Queues Twenty two 2U servers (e.g. Dell C210) each with 2 six-core 3.33GHz processors (Xeon X5680), 12 GB memory. Eight Master servers with 10x 300GB 15K SAS drives RAID 10, 2 x 72GB 10K SAS drives RAID 1 for O/S, fourteen Helper servers with 2 x 146GB 15K SAS drives, RAID 0, 2 x 72GB 10K SAS drives RAID 1 for O/S. 528 Core farm - 8 Queues Forty four 2U servers (e.g. Dell C210) each with 2 six-core 3.33GHz processors (Xeon X5680), 12 GB memory. Eight Master servers with 10x 300GB 15K SAS drives RAID 10, 2 x 72GB 10K SAS drives RAID 1 for O/S, thirty six Helper servers with 2 x 146GB 15K SAS drives, RAID 0, 2 x 72GB 10K SAS drives RAID 1 for O/S. 4U Configurations: 32 Core farm - 1 Queue One 4U server (e.g. Dell R910) with 4 eight-core 2.27 GHz processors (Xeon X7560), 32 GB memory, 4x 300GB 10K SAS drives RAID 10, 2 x 72GB 10K SAS drives RAID 1 for O/S. 64 Core farm - 2 Queues Two 4U servers (e.g. Dell R910) each with 4 eight-core 2.27 GHz processors (Xeon X7560), 32 GB memory, 4x 300GB 10K SAS drives RAID 10, 2 x 72GB 10K SAS drives RAID 1 for O/S. 128 Core farm - 4 Queues Four 4U servers (e.g. Dell R910) each with 4 eight-core 2.27 GHz processors (Xeon X7560), 32 GB memory, 6x 300GB 10K SAS drives RAID 10, 2 x 72GB 10K SAS drives RAID 1 for O/S. 256 Core farm - 8 Queues Eight 4U servers (e.g. Dell R910) each with 4 eight-core 2.27 GHz processors (Xeon X7560), 32 GB memory, 12x 300GB 10K SAS drives RAID 10, 2 x 72GB 10K SAS drives RAID 1 for O/S. Expanding an Existing Farm When expanding an existing farm you do not have to match the processor speed or make, but the disk drives in the original farm may need to be replaced or augmented, and you may need to move the licensed controllers between machines. Please contact us for advice. We're here to help These guidelines should help you to plan your farm. We will be very happy to review your plans in detail. Please contact Phil Gold (phil.gold@ggy.com) or Victor Rubinstein (victor.rubinstein@ggy.com). |
|
 |
Contact | | | Send a File to GGY | | | E-mail GGY | Search | |||