The second SCM gives you redundant SAS connection. But MPIO driver needs to be installed on your OS. See Special instructions for installing dual Storage Control Modules for details.
With the second ESM, you can connect your compute modules to two seperate physical networks, or you may configure SFT (Switch Fault Tolerance) teaming for redundancy.
Btw please take note that one Dual Gigabit Ethernet I/O Expansion Mezzanine Card AXXGBIOMEZ is required for each compute module that needs to access the second switch. See FAQs about the Intel® Gigabit Ethernet Switch AXXSW1GB for details.