Well, if you are going to continue to use 4K, a proper HDMI 2.0-compliant switch is going to help, as will HDMI 2.0-compliant cables. There are some other possibilities, however. One is that this switch is one-way and will not allow the necessary return signals. Another is that one of the cables is one-way. There are other possibilities as well. Cables are a good place to start.
Hope this helps,