I have already updated the firmware some days ago (with the RMM4 plugged in). Link speed was set to auto in ESXi, here i could see the 100mbit/s. I tried to set it to 1gbit/s manually, but it wasn't able. Data transfer was about 10-12MB/s. I tried some reboots, but still the same issue. I pulled the RMM4 lite module out and everything goes up to 1gibt/s, data transfer about 80-100MB/s.
However, I just plugged the RMM4 module back in again to try some other OS. Issue is solved! Everything at 1gibt/s! I can start a remote session, seeing the bootscreen/desktop and the OS still gets 1gbit. I wasn't able to reproduce the issue.
Apperently pulling the module out and plugging it in again helped somehow.