Unlimited freedom for edge servers

Intel Xeon D processors in COM-HPC server modules

The integration of Intel Xeon D processors into COM-HPC server modules from manufacturers such as congatec allows edge server installations to be freed from the tight thermal constraints of air-conditioned server rooms. For the first time, they can be deployed anywhere massive data throughput is required with the lowest possible latencies, all the way to deterministic real-time.

Edge servers process data at the edge of communication networks, rather than in the central clouds. This makes it possible to interact with clients of all kinds without delay or in real time, but it poses great challenges for manufacturers of server, network and storage technologies.

Until now, they used to develop standardized rack solutions for their systems, with active ventilation concepts and powerful climate control technology to control rack thermal management and server room climate control. However, this approach is often no longer suitable for today's cutting-edge server technology.

The American Society of Heating, Refrigerating and Air-Conditioning Engineers, or ASHRAE for short, has thoroughly studied the question of how to best install edge server performance in harsh environments. So, from the perspective of HVAC partners, there are already quite plausible recommendations for how to design edge data centers with high-performance HVAC and the best possible insulation to protect them from heat and wind. cold.

Freeing edge servers from the shackles of HVAC

However, ASHRAE proposes a maximum allowable temperature fluctuation of 20°C in one hour and a maximum of 5°C in 15 minutes for edge data centers. This requires complex air conditioning technology and is therefore very difficult to apply. But not only that; Meeting these guidelines is nearly impossible, especially during maintenance work on edge data centers that are smaller than a phone booth, because these solutions must be open for maintenance at any ambient temperature. It is simply not possible to break into these systems through an HVAC chamber and quickly re-close the door before servicing the fully climate-controlled edge server room.

Edge servers and data centers operating in harsh environments therefore require system designs that can handle greater temperature fluctuations and a much wider temperature range than the typical 0-40°C in IT. inside. In industrial settings, embedded system designs can be exposed to ambient temperatures ranging from an arctic -40°C to a searing +85°C. Which means that each component must be reinforced.

Robust designs reduce cooling costs

The most critical point in the design of edge servers, networking, and storage technologies is the choice of processor technology. The decision that goes back and forth with this choice is whether to follow ASHRAE recommendations and invest massively in HVAC and insulation technology, which leads to high investment and operating costs for secondary energy. Or whether to develop systems that don't need any of that because they work reliably even in extreme temperatures and can therefore be deployed in harsh environments much more cheaply: from factory sites to outdoor communications, video surveillance and other critical infrastructure equipment. , passing through servers in mobile systems ranging from trains and planes to autonomous shuttle buses in smart cities.

Thanks to the new Intel Xeon D processors, there is now a very powerful server technology that is qualified for use in extreme temperature ranges, from -40°C to +85°C. Even ultra-high performance server designs are no longer limited by the strict thermal restrictions of air-conditioned server rooms. Ultimately, they can be deployed wherever massive, zero-latency data throughput is required at the edge of the Internet of Things and in Industry 4.0 factories.

However, a server processor alone does not make an edge server robust. Meeting the demands of system design for harsh environments also requires extensive technical knowledge. Each component used must be qualified for this environment, and special requirements also apply to circuit board and card design. Some examples are special coatings that protect against condensation water and other environmental influences, or a high level of protection against extraneous high-frequency and electromagnetic signals that could hinder the performance of the device.

Figure 2. The modular Server-on-Module approach makes it easy to develop dedicated edge servers with application-specific interface designs using custom-designed carrier boards.

Developers of embedded system technologies, such as congatec, have decades of experience designing such systems. They have long been integrating standard PC technologies, such as Intel Core processors, into embedded systems in a way suitable for industrial use. They are well versed in the requirements and certification standards of a wide range of industries and are used to designing their systems to be available for the long term, in order to meet industry requirements and to be able to supply OEM solutions with configurations of identical plate for 7, 10 or 15 years. They also know that industrial applications differ significantly from standard system designs for the office environment, as industrial applications always require a greater or lesser degree of customization, making modular designs that implement COM modules the ideal way. to develop plaques. They have also learned that standardization is key, which is why they have contributed to creating globally recognized standards for this type of module.

Get to the finish line faster with standards

With the new COM-HPC server specification and the release of Intel Xeon D processors, this combined expertise has now been carried over to cutting-edge industrial server designs. For the first time, developers have access to real products. The advantage of these new COM-HPC standardized server modules is that developers can integrate them into their custom carrier boards as application-ready embedded computer logic. This means that they don't have to worry about the basics of processor technology, but only have to deal with the application-specific positioning of the components on the board and execute the interfaces in the right place on the carrier board. To this end, the PICMG standardization committee has recently published the COM-HPC Carrier Design Guide. It provides essential guidelines for building customer-specific, interoperable, and scalable embedded computing platforms based on the new standard, and also makes it easier for developers to understand the logic behind the standard.

Knowledge is power

To enable developers to quickly, easily and efficiently immerse themselves in the new design standards, congatec has opened an online and face-to-face training academy for COM-HPC client and server designs. Here, developers can get an expert-led introduction to the new world of high-end edge and embedded system designs based on the new COM (Computer-on-Module) standard. The training program covers all required and recommended design basics, as well as best practice guidance for the design of COM-HPC carrier boards and accessories, such as high-end fanless cooling solutions for server designs. up to 100 watts or more. COM-HPC Server Module Evaluation Carrier Boards serve as a reference platform for learning how to implement Intel Xeon D processors. They take advantage of the full feature set of the standard, and developers can use them as platforms for further application development.

The congatec academy aims to train developers in all the basics of COM-HPC design, from PCB layer principles, power management rules and signal integrity requirements to component selection. Sessions with a special focus on communication interfaces provide guidance on avoiding the pitfalls in the challenging design of high-speed serial communications: from PCIe Gen 4 and 5 to USB 3.2 Gen 2 and USB 4 with Thunderbolt on USB- C up to 100 Gigabit Ethernet, and also including sideband signal management for 10G / 25G / 40G / 100G KR Ethernet interfaces, which in COM-HPC must be deserialized on the carrier board. During these sessions it is also explained how best practice designs use interface standards such as eSPI, I²C and GPIOs.

Figure 3. Standard motherboard/motherboard designs typically only support standard onboard interfaces that run on the back of the board (rear I/O). As this does not take industrial requirements into account, their suitability as edge servers for the Internet of Things is limited. And, as a rule, they are not designed for the wide temperature range from -40°C to +85°C nor do they guarantee long-term availability of 7 to 15 years. However, with server modules it is possible to use the mechanics of these form factors and design a carrier board that runs the desired interfaces where they are required.

An introduction to x86 firmware implementation—covering everything from the embedded BIOS to board management controller and module management controller functions—complements design training. And last but not least, there are sessions on testing and verification strategies that address all challenges from initial motherboard design verification to mass production testing. With such a comprehensive training program, the congatec academy aims to make designing robust edge server technology as easy as possible. Needless to say, the company can also provide interested OEM customers with complete system designs that take advantage of its new COM-HPC server modules and extensive partner network.

Reference Design for Machine Learning AI Clustering

COM-HPC edge server designs are not limited to single module concepts. The standard also explicitly supports multi-module support with heterogeneous configurations of COM-HPC modules integrating, for example, FPGAs or GPGPU accelerators. A mix of COM-HPC server and COM-HPC client modules on one board is also possible. For example, congatec is currently working with the University of Bielefeld and Christmann IT on an edge server design that combines different COM-HPC modules on one motherboard to process extreme real-time workloads in a multi-system design for AI clustering. machine learning of high-dimensional data (self-organizing maps).

Edge server with three COM-HPC modules for extreme real-time workloads.

Figure 4. The difference between the Intel Xeon D in COM-HPC Server Size E and Size D is in the number of possible RAM sockets, which also determines the size of the modules.

Acceleration of edge server workloads

However, the new E-size and D-size COM-HPC server modules with BGA-mounted Intel Xeon D processors (reference name Ice Lake D) impress not only with their compatibility with the extended temperature range from -40°C to +85°C. They also break many of the previous bottlenecks caused by edge server constraints and will significantly accelerate the next generation of real-time microserver workloads in harsh environments and extended temperature ranges. Enhancements include up to 20 cores, up to 1TB of memory in up to 8 DRAM sockets at 2933MT/sg, up to 47 total PCIe lanes per module and 32 PCIe Gen 4 lanes with twice the performance per lane, and up to 100GbE connectivity and TCC/TSN support with optimized energy consumption thanks to 10nm manufacturing. Video analytics and storage servers also benefit from built-in support for Intel AVX-512, VNNI, and OpenVINO for AI-based data analytics.

A milestone for real-time edge server designs

In fact, the market launch of Ice Lake D-based COM-HPC modules marks a triple milestone: First, because extended temperature range support means Intel Xeon D server modules are no longer limited to standard industrial applications, but are also targeted at outdoor and automotive environments. Second, the world's first COM-HPC server modules increase the number of available cores to 20 for the first time; With up to 8 DRAM sockets, this provides much higher memory bandwidth than server modules based on other PICMG specifications. Third, these new server modules are capable of real-time operation, both in terms of processor cores and TCC/TSN-enabled real-time Ethernet, which is essential for digitized IIoT and Industry 4.0 projects.

In order to implement server balancing and consolidation services for deterministic real-time edge server deployments, where multiple real-time applications operate independently on a single edge server, it is useful for platforms to support time-capable virtual machines. real, as does, for example, the RTS hypervisor from Real-Time Systems. This enables Industry 4.0 factories to host heterogeneous real-time applications on a single edge server platform of their private 5G networks, and allocate dedicated system resources to individual processes. congatec's server modules are pre-qualified for this type of service. Customized installations with all the necessary parameterizations can be included in the standard services that congatec offers for the new COM-HPC modules.

The modules further impress with a comprehensive set of server-grade features: For mission-critical designs, they offer powerful hardware security features such as Intel Boot Guard, Intel Total Memory Encryption – Multi-Tenant (Intel TME-MT) and Intel Software Guard Extensions (Intel SGX). For the best RAS capabilities, the processor modules integrate the Intel ME Manageability Engine and support remote hardware management features such as IPMI and redfish. In fact, there is another PICMG specification that guarantees the interoperability of such implementations, and the congatec academy training program also covers this aspect.

Server module options for Intel Xeon D

The new modules will come as an HCC (High Core Count) and an LCC (Low Core Count) variant with different types of the Intel Xeon D series of processors.

The conga-HPC/sILH COM-HPC Size E server modules will be available with 5 different Intel Xeon D 27xx HCC processors with a choice of 4 to 20 cores, 8 DIMM sockets for up to 1 TByte of fast 4 MT/sg DDR2933 memory with ECC, 32x PCIe Gen 4 and 16x PCIe Gen 3, as well as 100 GbE performance plus real-time 2,5 Gbit/s Ethernet with TSN and TCC support with 65 to 118 watt base processor power.

COM-HPC Size D and COM Express Type 7 server modules will come with 5 different Intel Xeon D 17xx LCC processors with a choice of 4 to 10 cores. While the conga-B7Xl COM Express Server Module supports up to 128 GB of DDR4 2666 MT/sg RAM via up to 3 SODIMM sockets, the conga-HPC/SILL COM-HPC Server Size D module offers 4 DIMM sockets for up to 256 GB of fast 4 MT/s DDR2933 RAM or 128 GB with UDIMM ECC RAM. Both module families offer 16 PCIe Gen 4 lanes and 16 PCIe Gen 3 lanes. For fast networking, they offer up to 50 GbE throughput and TSN/TCC support over 2,5 Gbit/sg Ethernet with a base power of the processor from 40 to 67 watts.

Modules can already be ordered in advance and application-ready test samples - with robust cooling solutions that match the TDP of the processor - are immediately available. Cooling solutions range from powerful active cooling with heat pipe adapter to fully passive cooling solutions for higher mechanical resistance to vibration and shock. The latter also alleviate thermal stress in applications that must withstand short bursts of extreme temperature fluctuations. Software-wise, the new modules come with full board support packages for Windows, Linux, and VxWorks, and RTS hypervisor technology.

COM-HPC based multi-module designs

Module	conga-B7Xl	conga-HPC/sILL	conga-HPC/sILH
Form factor	COM Express Type 7	COM-HPC Server Size D	COM-HPC Server Size E
Dimensions	125 mm x 95 mm	160 mm x 160 mm	200 mm x 160 mm
Processor	Intel Ice Lake D (LCC)	Intel Ice Lake D (LCC)	Intel Ice Lake D (HCC)
Nuclei	4 to 10	4 to 10	4 to 20
RAM	Up to 128GB DDR4 (up to 4x S0-DIMMs)	Up to 256GB DDR4 (4x RDIMM/UDIMM)	Up to 1TB GB DDR4 (8x RDIMM/UDIMM)
TDP	40 67-W	40 67-W	65 118-W
Interfaces
PCIe	16x PCIe Gen 4 16x PCIe Gen 3	16x PCIe Gen 4 16x PCIe Gen 3	32x PCIe Gen 4 16x PCIe Gen 3
USB	4 USB 3.1 Gen 1 4x USB 2.0	4 USB 3.1 Gen 1 4x USB 2.0	4 USB 3.1 Gen 1 4x USB 2.0
Ethernet	1x 2.5GbE with TSN 4x 10GbE (KR support)	1x 2.5GbE with TSN 2x 25, 4x 10, 8x 2.5 GbE (supports KR or SFI)	1x 2.5GbE with TSN 1x 100, 2x 50, 4x 25, … GbE (KR or SFI support)
SATA	2x SATA II	2x SATA II	2x SATA II

Figure 5. COM Express Type 7 and COM-HPC Size D server modules based on the Intel Xeon D processor differ not only in size, but also in pinout.

Figure 6. Up to 20 cores offer a huge variety of secure options for a wide variety of real-time applications when using Real-Time Systems virtualization technology.

Figure 7. The Intel Xeon D 27xx HCC processor configurations of the congatec COM-HPC Size E (200mm x 160mm) server modules

Figure 8, The Intel Xeon D 17xx LCC processor configurations of the congatec COM-HPC Size D (160mm x 160mm) and COM Express Type 7 (95mm x 120mm) server modules