Connect second monitor directly via display port, according to product page it should have it
I guess your rig is confused by two HDMI handshakes at the same time or by the DP/HDMI adapter?
In theory it should work though as seen at 5th Generation Intel® Core™ Processor Family Datasheet Vol. 1 page 29
Did you already install the latest BIOS update 0365 from Downloads for Intel® NUC Kit NUC5i7RYH ?
Thanks! I have a MDP to DP cable on the way!
Now I have to figure out what happened to my Bluetooth overnight, but that is for a different thread.
OK, the new cable arrived today (Thanks Amazon!). It solved the problem! I now get 2560x1440 from both monitors.
All is well with the world again, the sun will come up in the morning and world peace will reign!