While Intel technically considers the use of XMP profiles that go above the processor's maximum supported bus speed to be a form of overclocking, this does not void your warranty. Of course, if the system doesn't work while configured for these profiles, don't expect Intel to help you solve this problem. In most cases, this is the fault of the motherboard, not the processor, and Intel isn't going to debug support for individual 3rd-party motherboards.
And what about XMP profile in range of CPU official supported memory clock speed (e.g. 2400 MHz for Kaby Lake)? Is this also considered as overclocking?
If it is within the range of clock speeds that are formally supported, then no, this wouldn't be considered overclocking.
The "over" and "clock" in "overclocking" infer that you are operating some portion of the processor at clock speeds that are above what Intel intended and above what Intel has validated the processor to support without issue. What happens in this case is an unknown. It may work without issue and without any long-term consequences. I may work but have longer-term consequences (for example, thermals-based silicon degradation that shortens the processor's lifetime). It may work but have temporary or permanent short-term consequences (for example, thermal overrun and potentially resulting silicon damage). Finally, it may just not work at all.
I hope I have explained this in terms you will understand.