The Armari Magnetar X64T Workstation OC Review: 128 Threads at 4.0 GHz, Sustained!
by Dr. Ian Cutress on September 9, 2020 12:00 PM EST- Posted in
- Desktop
- Systems
- AMD
- OC Systems
- ThreadRipper
- 3990X
- Armari
- Magnetar
- X64T
- Rendering
Blitzing around a race track in a fast car only ever convinces you of one thing: I need to go around the track even faster. I need a better car, I need a better engine, better brakes, or better tires. I need that special go faster juice, and I need to nail the perfect run. The world of professional computing works the same, whether it comes down to rendering, rapid prototyping, scientific compute, medical imaging, weather modelling, or something like oil and gas simulation, the more raw horsepower there is, the more can be done. So enter the new Armari Magnetar X64T – an overclocked 64-core Threadripper 3990X that holds the new SPECworkstation3 world record. We got hold of one. It’s really fast.
Playing with Performance
AMD’s Threadripper 3990X is one of those crazy processors. It comes at you with some of the best of any processor statistics: it has 64 cores and 128 threads, it has 256 MB of L3 cache, it has a TDP of 280 W, which allows for a 2.9 GHz base frequency up to a 4.3 GHz turbo. It is overclockable, and so with the right system those frequencies can go even higher. With the best binned 7nm chiplets, paired with quad-channel DDR4-3200 memory, for multithreaded workloads it is one of the ultimate powerhouses anyone can build in a single socket with a socketable processor.
In our initial review of the Threadripper 3990X, it blitzed any software that could take advantage of all those threads – the nearest competitors were the 32-core Threadrippers, or Intel’s 28-core Xeon-W processors. We even put it up against two of Intel’s $10000 28-core Xeons, and it won pretty much everything by a large margin.
So what happens when we overclock it? There are those that want more, and not just those overclocking for fun – workstation customers, like animation studios, are always looking for ways in which they can rapidly render frames for upcoming projects. If a cooling system can be built to withstand it, and the power is available, then there’s always scope to get more out of the hardware that comes from the big players. This is what the Armari Magnetar X64T Workstation is designed to do – get more.
To that end, today AMD and SPEC is announcing that the Magnetar X64T workstation, a system that you can buy, will off-the-shelf give the best performance in SPECworkstation3 ever seen.
The Magnetar X64T: Performance Reimagined
The key highlight from this review, should you not read any further, is that this system is built to blitz workloads. The Threadripper 3990X is usually fast enough in its own right, but Armari have gone above and beyond. The goal of this system is to be an off-the-shelf powerhouse that requires very little setup from its customers.
Armari, perhaps a lesser well known system integrator, is a company that has in recent years focused on building systems for 3D animation, video editing, and scientific research. With over 20 years of experience, Armari’s hardware has gone into high performance computing solutions and clusters that have featured in the TOP500 lists, as well as rendering server farms for the top animation, VFX, and CGI studios in Soho, London.
These are clients who want the best performance, and Armari positions itself not so much as a boutique system builder, but something between the big OEMs (like Dell/HP) and the main retailers to offer custom solutions by leveraging its network of cooling and hardware contacts around the world. This enables the company to build custom chassis, obtain optimized memory, order power supplies with custom connector configurations, and ensure consistency from batch-to-batch when ordering from its partners. In speaking to Armari’s Technical Director Dan Goldsmith, he mentioned that working with partner companies for so long has enabled them to get access to rapid prototyping and component consistency with continual feedback with partners such as EKWB, ASRock, Tyan, and many other ODM companies that Armari leverages on a regular basis.
The Magnetar X64T, I was told, leverages the strong relationship Armari has with AMD. The Opteron was a popular range a decade ago, and that partnership has been maintained through today. The goal of the Magnetar project was to create a system that offers the best that Threadripper has to offer while still enabling the under-the-desk workstation platform. This project has been slightly delayed due to COVID, and AMD now has Threadripper Pro, but those processors are not overclockable – for those that want raw performance, AMD and Armari believe they are on a winner.
The key to the system is in how Armari is cooling the processor, and the choice of components. The Magnetar X64T features a custom water cooling loop, which is perhaps not anything new in its own right, however the company has created a component chain to ensure consistency in its design, as well as using some of the most powerful options available.
The water block is probably the best place to start, because this is a completely custom-for-Armari design built in partnership with EK Water Blocks. This block is specifically built for this one motherboard, the ASRock TRX40 Taichi, and applies cooling to both the processor and the power delivery. The block works in conjunction with the highest-quality thermal paste pads on the market, to ensure a flat connection with the water block. As it also covers the power delivery, Armari worked with ASRock to enable a consistent z-height of all the power delivery components, something that can vary during manufacturing, and maintain that consistency on a batch-by-batch basis. Pair this up with Armari’s custom FWL liquid cooling pump, reservoir, tubing, 3x140mm radiator, and fan combinations (many of which are custom from their respective ODMs), and we have a cooling capacity in excess of 700 W. The coolant is a special long-life coolant designed for 24/7 over three years, and the standard warranty comes with service during those three years, including collection and return, at no extra cost.
Now, the ASRock TRX40 Taichi isn’t the top Threadripper motherboard on the market, and Armari fully admits that, however it points out that the best motherboard available costs twice as much. In working with ASRock, they were able to co-ordinate what was needed within the discrete motherboard component lists as well as enable a custom BIOS implementation for additional control. One of the tradeoffs I was told about is that a cheaper motherboard might mean slightly cheaper components, however Armari says that their cooling system and setup were co-operatively tuned to meet its customers’ demands.
With this cooling arrangement, Armari have fired up the overclock. In our initial review of the Threadripper 3990X, we were observing ~3450 MHz during our sustained running with the CPU reaching its full 280 W. For the Armari Magnetar X64T, we have an all-core frequency from 3950-4100 MHz, depending on the workload. Users might scoff at the +400-550 MHz lift, but bearing in mind this is across all of the 64 cores simultaneously, and the cooling is built such that this frequency is sustained for renders or simulations that might take days. Further details of frequency and power later in the review.
While having the overclocked CPU is great, the Magnetar X64T system we were delivered also had memory, graphics, and storage.
Armari Magnetar X64T as shipped (X64T-RD1600G3-FWL) |
|
Processor | AMD Ryzen Threadripper 3990X Overclocked to ~4.0 GHz All-Core Turbo |
Cooling | Custom Armari FWLv2 Liquid Cooling Loop Custom CPU+VRM Monoblock 420x45mm EK Coolsense Radiator 3 x EK-Vardar 140ER EVO 140mm fans High Performance Pump Clear Coolant, Designed for 3yr operation |
Graphics | PNY NVIDIA Quadro RTX 6000 24 GB |
Motherboard | ASRock TRX40 Taichi |
Memory | 256 GB of DDR4-3200 |
Power Supply | 1600W 80PLUS Gold 93%, rated to 50ºC 0% fan under 40% load 9x PCIe connections |
Storage | ASRock Hyper Quad M.2 PCIe 4.0 x16 add-in card 1 x Corsair MP600 PCIe 4.0 x4 1 TB Boot Drive 2 x Corsair MP600 PCIe 4.0 x4 1TB Striped Array |
Networking | Realtek RTL8125 2.5 GbE (motherboard) Intel I211-AT 1 GbE (motherboard) Intel AX201 Wi-Fi 6 module (motherboard) |
Audio | Onboard Realtek ALC1220 + ALC4050H |
Fans | 3 x EK 140mm for radiator 2 x Noctua 140mm for internal airflow 1 x SanAce 80mm low noise for DRAM |
Price as Built | £10790 + tax (~$14200 + tax) Special launch price for September |
UK Warranty | 1 Year RTB 3 Year Parts+Labor One service/coolant replacement, inc collection/pickup Loaner systems available if bigger issues occur |
The system as shipped came with an PNY NVIDIA RTX6000 graphics card, which is essentially an RTX 2080 Ti on steroids with 24 GiB of GDDR6, and the system can be configured with two of them. As Threadripper is not an ECC-qualified platform, the X64T comes with the peak configuration supported, 256 GB, but with custom SPD profiles to run up to DDR4-3600. Unfortunately due to how quickly this system was rebuilt for this review, the system I was sent was using DDR4-3200 at CL20, as some of the original memory was accidentally splashed with coolant, and Armari wanted to ensure I wouldn’t have any issues with the system.
Storage comes in two forms, both of which are PCIe 4.0. As shipped, we were specified with a boot drive to the tune of a Corsair MP600 1 TB PCIe 4.0 x4 drive. Another two of these drives were provided inside an ASRock Hyper M.2 PCIe 4.0 card, plugged into one of the PCIe 4.0 slots. Armari says that as newer and bigger PCIe 4.0 drives come to market beyond the Phison E16 solutions, this should expand to higher capacity drives or faster drives as required.
The power supply is a fully custom 1600W 80PLUS Gold unit, rated to run at 50 ºC with 93% efficiency. It has a custom fan profile directly from the OEM, and is set to only stir up the fans if the power required goes above 40% (640 W). The fully modular PSU has nine 8-pin connections and five 6-pin connections, providing 14 total, should any customer want to go above and beyond. The PSU on its own has a 10-year warranty.
The motherboard has a 2.5 GbE wired network port and a 1 GbE wired network port, and Armari does offer a 10G upgrade (space permitting based on the PCIe slots). Wi-Fi 6 support comes as standard, as does the ALC1220 audio configuration.
The chassis is the last custom part to discuss, with the system featuring the Magnetar naming on the front with the Armari logo. The chassis is big, but quite standard for a high-end workstation platform: 53cm x 22cm x 57cm (20.9-in x 8.7-in x 22.4-in), with a typical single GPU weight of 18 kg (39.7 lbs).
The chassis comes with handles on top that fold away, making the system easy to move around as required. I love these.
Inside there is lots of ‘space’ for directed airflow. The pump and reservoir is found in the bottom of the case, underneath the standard drive-bays, while the 3x140mm double thick radiator is at the top built into the side of the chassis. This is a special hinged mount, which makes the side panel easy to remove and the cooling apparatus easy to inspect.
There is a PCIe retention bracket for any add-in card installed, and in the base of the chassis is the power supply, hidden away. The insides weren’t necessarily built to look aesthetically pleasing, however the system as provided by Armari has a nice clean look.
Due to a couple of issues with arranging this system for review, I was told that normally Armari adds in some custom sealant to help with the liquid cooling loop, however as it requires 24 hours to set, they weren’t able to in this instance. The liquid cooling loop is pre-tested for every system they build at over 1 bar of pressure, along with full stability testing and thermal testing before shipping. For any reason if a system needs to be returned for warranty, Armari can supply a loaner system if required. As mentioned above, the standard warranty includes one full service and inspection, and the coolant can be replaced in order to give the customer another 3 years of ‘hassle free’ operation.
The News Today: World Records
Today AMD and Armari are announcing that the new Magnetar X64T has set a new world record in the SPECworkstation 3 benchmark. The system that achieved this test is, by and large, the system I am testing today (it was stripped down and rebuilt with an updated water block). For the customers that Armari typically services this one of the primary benchmarks they care about, and so getting a new world record for a commercially available system should put Armari’s offerings high on their list.
Our testing, as shown over the next few pages, is similarly impressive. We already saw that the Threadripper 3990X with no overclock was a formidable beast in almost every one of our rendering and compute workloads. The only real comparison point we have to compare against is our W-3175X workstation that was provided when we reviewed that system.
The Magnetar X64T-RD1600G3 FWL (the full name) system in our testing is ~£10790 ($14200) excluding tax . This includes a Windows 10 Professional 64-bit license, and Armari’s 3 year premium workstation warranty, with 1-year on site and 2/3rd year parts and labor, along with a loaner system for the duration of any repairs.
Read over the next few pages for our testing on Performance and Power.
96 Comments
View All Comments
WaltC - Wednesday, September 9, 2020 - link
Very impressive box!...;) Great write-up, too! Great job, Ian--your steady diet of metal & silicon is really producing obvious positive results! I would definitely want to go with a different motherboard, though. Even the GB x570 Aorus Master has received ECC ram support with the latest bios featuring the latest couple of AGESA's from AMD--so it seems like a shoe-in for a TR motherboard. I agree it's kind of an odd exclusion from AMD for TR. But, I suppose if you want ECC support and lots more ram support you'll need to step up to EPYC and its associated motherboards. This is a real pro-sumer offering and the price--well, everything about it--seems right on the money, imo. I really like the three-year warranty and the included service to change out coolant fluid every three years--very nice! If I used water cooling that is the only kind of fluid I would want to use--some of these el cheapo concoctions will eat up a radiator in a year or less! Enjoyed the article--thanks again...;)Makaveli - Wednesday, September 9, 2020 - link
"Ideally AMD would need a product that pairs the 8-channel + ECC support with a processor overclock."But why?
I maybe wrong here but when you are spending 10k+ on a build isn't stability more important than overclocking?
MenhirMike - Wednesday, September 9, 2020 - link
Overclocking doesn't have to be unstable - and for some workloads, the extra performance is worth the effort to find the limits and beef up the cooling solution.Ian Cutress - Thursday, September 10, 2020 - link
Had a chat with Armari. The system was built with the OC requirements in mind, and customized to support that. They're using a PBO-based overclock as well, and they've been really impressed with how AMD's latest variation of PBO can optimize the DVFS of the chip to keep the system stable regardless of workload (as long as heat is managed). In my testing, there was zero instability. I was told by Armari that they can build 10, 20, or 50 systems in a row without having any stability issues coming from the processor, and that binning the CPU is almost virtually non-existant.Everett F Sargent - Wednesday, September 9, 2020 - link
So, such a waste of power, time and money.Anyone can build TWO 3990X systems for less then half the price AND less then half the power consumption with each of them easily getting ~90% of the benchmarks.
That is with a top of the line titanium PSU, top of the line MB, top of the line 2TB SSD, top of the line Quadro RTX 4000 (oh damn paying 5X for the 6000 for only less then 2X the performance, what a b1tch, not), top of the line ... everything.
So ~1.8X of the total performance at ~1.8X the cost (sale pricing for all components otherwise make that ~1.9X the cost) and ~0.9X of the total power consumption.
But, you say, an ~15KUS system pays for itself, in less then 1E−44 seconds, even. Magnetarded indeed. /:
TallestGargoyle - Thursday, September 10, 2020 - link
That's a pretty disingenuous stance. Yes, you can split the performance across multiple systems, likely for cheaper, but that ignores general infrastructure requirements, like having multiple locations to set up a system in, having enough power outlets to run them all, being capable of splitting the workload across multiple systems.Not every workload can support distributed processing. Not every office has the space for a dedicated rack of systems churning away. Not every workload can sufficiently run on a half-performing GPU, even if the price is only a fifth of the one used here.
If your only metric is price-to-performance, then yes, this isn't the workstation for you. But this clearly isn't hitting a price-to-performance metric. It's focusing on the performance.
Tomatotech - Thursday, September 10, 2020 - link
Something something chickens and oxen.Or was it something something sports cars and 49-ton trucks? I’ll go yell at clouds instead.
Spunjji - Friday, September 11, 2020 - link
Perish the thought that someone might order one of these with a Quadro 4000 and obviate his biggest gripe (not that it would solve the potential problem of fitting a dataset into 8GB of RAM instead of 24GB)Everett F Sargent - Saturday, September 12, 2020 - link
Well there is that 20% VAT that pushes the actual system price to ~$17K US so dividing by two ~$8.5K per system minus the cost of a RTX 4000 at ~1K US gives one ~$6K US sans graphics card (remember I said build two systems with RTX 4000 at ~$7K US).So I have about ~$2.5K per system for a graphics card. Which easily gives one RTX 5000 per build at ~$2K US. Whiich is 2/3 of the RTX 6000.
But, if the GPU benchmarks are all eventually CPU bound, which appears to be the case here in this review, then yes I get two systems at ~$8K US or ~$16K US for two systems, in other words, I get to build a 3900X system for about ~$1K US to boot..
So no, I would rather see benchmarks at 4K, mind you (as these were at standard HD) with different GPU options. Basically, I would want to use my money wisely (assuming at a minimum a 3990X CPU at ~$4K US as the entry point).
Therefore, this review is totally useless to anyone interested in efficiency (which should normally include everyone) and costs (ditto).
TallestGargoyle - Monday, September 14, 2020 - link
That still doesn't take into account the issues I brought up initially with owning and running two systems.