The boxed CPU ships with a regular fan heatsink like this:
You're gonna need a 1U heatsink to use the CPU in a 2U chassis. See Intel® Xeon® Processor 3400 Series and LGA1156 Socket Thermal and Mechanical Design Guide for more details.
Some rack chassis will ship with CPU heatsink included. If not you'll need to get one from market.