For those big production runs, there are never enough hours in the day, and deadlines are tight. Maybe you are running millions of seriatim policies or perhaps you are running some Variable Annuities through thousands of scenarios, or perhaps a combination of both. You may be using, or thinking of using, the distributed processing features in AXIS or the GridLink module. You may even consider sending your datasets to GGY to run on our farm.
GGY devotes a great deal of resources to improving the scalability of AXIS on Processor Farms. This article is designed to keep you up to date with the latest developments in all these areas.
Changes in Hardware
The first way in which we can help you is by keeping you up to date on the latest new processors from Intel and AMD. This used to be a fairly steady process, but the last year or so has seen breakthroughs from both companies, with Intel’s Quad-Core chips currently claiming the high ground. The running order may well be reversed later in the year when AMD introduces its next generation Quad-Core Opteron processor.
If you are sitting on computers over two years old the latest Quad-Core Intel processor has probably at least five times the processing power. We do our best to keep you informed by commenting on and benchmarking these new processors on a timely basis. For example, a Xeon X5355 (2.6GHz) can process in 55 seconds what an older 3.6GHz Xeon (2MB cache) can process in 350 seconds.
Check out our Recommended Equipment page:
www.ggy.com/support/reqequip.asp
I also invite you to call me personally for up to the minute advice for any purchasing decisions, especially if it involves setting up a processor farm. We have been able to save our clients a lot of money and perhaps we can do the same for you, since we have an intimate understanding of which components have the biggest impact on AXIS performance, scalability and robustness, and we do not charge for this advice.
You can reach me at 416 250 6777 x 224 or by email at Phil.Gold@ggy.com
Grid Computing
We introduced AXIS GridLink in late 2005 to manage processor farms. Although we are the Developers of AXIS GridLink, we are also one of its largest users. We have a 16-core Opteron farm and a new 64-core Xeon farm, which is, if you can believe it, less than 8 inches high. This new Xeon farm has over 6 times the capacity of our Opteron farm, in exactly the same space.
Why do we need it? We set up both farms for the same reason, to help us develop and improve GridLink. We run many large scale tests on these farms, and these experiments allow us to improve the scalability and performance of AXIS for our users, and to make sure it runs reliably. We can merge the farms together to produce an 80-core farm. And we can also easily add another 48 cores from our developers’ workstations to create a single 128-core farm when the need arises. We also have separate farms for nightly testing and automated developer testing, all of which run under GridLink.
Recent Enhancements to GridLink
We have added some major features to GridLink recently. Here are the highlights:
- We have changed GridLink from an application to a service, enabling simpler administration and recovery.
- GridLink can now easily deploy AXIS versions onto all or specified machines in your farm.
- GridLink now features Access Groups, allowing higher levels of Security between users on the farm.
- GridLink can now schedule safe farm backups through a feature called Active Backup.
- GridLink can now process System Batch Jobs as well as Dataset batch jobs.
More details can be found in these two links:
- Major enhancements to AXIS GridLink (April - December 2006)
www.ggy.com/support/kbase/kbdetails.asp?articleid=569
- More features added to GridLink (February 2007)
www.ggy.com/support/kbase/whatsnewdetail.asp?artid=574
Industry Standard Grids
Some of our clients have set up Enterprise Grid Solutions using technology from Platform Computing or DataSynapse. GGY has been working to integrate AXIS GridLink with both of these Enterprise Grid Solutions. This integration means that one or more AXIS GridLink farms can happily coexist with other Grid-enabled software. This can allow you to leverage your hardware costs over a number of different applications. More details will be available on the GGY website shortly.
Renting the GGY Farm
We can rent out space and time on our two Processor Farms on a first come first served basis, and you can also reserve capacity at peak periods in advance. So you can consider renting time on our farm as a convenient spillover option.
We are usually able to be flexible and responsive to our clients’ needs by rescheduling our own testing to non-peak periods. Now that we have a total of seven times our original capacity, we can tackle jobs that are too big to run in your own offices, and we can often provide a quicker turnaround than would be available in your own production areas.
Recent changes in the AXIS Stochastic Processing module allow you to run unlimited numbers of scenarios within the required memory of 1 GB per processor core. If you would like to process more scenarios than before, but do not have the capacity to do so, this is another good reason to check out the GGY Processor Farms.
More details are available online here:
www.ggy.com/support/kbase/kbdetails.asp?searchterm=&articleid=433
Transferring Large Amounts of Data
Along with this increased capacity we have also increased our internet bandwidth by a factor of 6 and redesigned the GGY web file upload utility to handle larger files. The combination of more bandwidth and the new upload utility mean that you can now securely upload files of up to 1 gigabyte in size at speeds of up to 1.25 megabytes per second. You can then download your results from GGY with the same speed and security. We will provide you with a personalized download page that has a unique URL address known only to you.
While there are virtually no restrictions anymore on our side in terms of being able to receive large files, there may be limitations on the uploads established at your end. To help you send us large files we have added an option to the dataset backup tool in AXIS that allows you to span the ZIP file over multiple smaller files while backing up the dataset. More details can be found online:
www.ggy.com/support/enhancebug/upddetail.asp?id=10443&sequence=0
The Output Manager
We have also developed a very powerful tool in AXIS called the Output Manager. This allows you to get a very quick overview of all the outputs configured in the dataset and easily re-target them to new destinations. This helps us as well. When we receive the dataset from you we can quickly re-configure all required outputs to our farm drives. This way you do not need to do any extra preparations before sending us your data to run.
The Future
Our Systems Design team is working flat out to bring you the highest performance software and to make it as easy as possible to use. Much of the development work involves the support of processor farms and we have built up a large amount of experience and expertise. We are very happy to share that with you when it comes to setting up your own production environments and we look forward to hearing from you.
Feel free to call Victor Rubinstein [416 250 6777 x237], Jeffrey Wei [x236] or myself Phil Gold [x224] at any time.
|