Most top-of-rack (ToR) data center switches are installed where their name says, on the top of the server rack. This means their 10G downlink ports are connected to servers that are within a few meters of the switch. In data center applications where low latency is critical, SFP+ direct attach (DA) copper cables can be used to connect servers at up to 7 meters from the switch. But this requires high quality PHYs inside the ToR switch.
Switching ASICs used in ToR switches can have up to 72 10G ports on a single piece of silicon. But high quality 10G SerDes are difficult to design, and many of these large chips are designed with the assumption that the SerDes only needs to drive a locally connected PHY chip that will take on the burden of driving DA copper cables or backplanes.
At Intel, we took a different approach. With the knowledge that many of our customers are designing low-latency ToR switches using DA copper cabling, we chose to embed high quality 10G PHYs within our Intel Ethernet FM6000 series switch silicon. These PHYs can drive up to 7m of SFP+ DA copper on 10GbE ports or up to 5m of QSFP DA copper on 40GbE ports. With up to 72 10G SerDes on the FM6000, this eliminates up to 18 external quad PHY chips that must be used when lower quality SerDes are used within the switch ASIC. Elimination of these external PHYs saves cost, power and board area, which are critical in today’s large, flat data center installations.