Check out www.vm-help.com and look at the thread "Customising OEM.TGZ"
I have the DQ57TM board with 8Gig n a i3 530.
I have, using the methods on that page got a non standard driver to load for the intel Nic.
It's possible that your hardware is a bit too new of ESX 3.5, I'd recommend either 4.0 or best alternative 4.1. As a workaround to get ESX 4.0 installed and then load the appropriate drivers after you have console access, add a PCIe 100/1000 NIC that's on VMware's hardware list. We do a lot of beta work in my area and often the motherboards we work with don't have supported NICs. I've had a good success with the Intel Pro 1000/PT adapter and getting ESX to load correctly. Hope this helps out - Chris
PS: You should also probably invest in an i5/7 and another 2-6GB of RAM if you're going to run more than 1-2 VMs in this box, depending on your use case and software stack.