If you are reaching the point where throttling is occurring, then yes, you are likely having some thermal degradation occurring and, in the long term, this will shorten the life of the processor and the system. I suggest that you go into the BIOS and modify the fan speed control configuration to make it more aggressive at lower temperatures and ensure that the fan is running at 100% at all temperatures above ~85 degrees. As I have said many times, I am NOT a fan of the preset configurations that Intel is providing.
I am travelling currently and thus do not have access to my NUCs to grab a screen shot of the configuration that I use. You can find examples of it in some of my previous posts regarding thermal (and noise) issues, however (try a search using key word "Tcontrol")...