The size of the SPI_Flash memory is 8MB so you must take care of the packages you are going to install. The perl module uses a lot of modules and packages. Have you tried to use only perl without perl-base? Are you adding more packages or not?
I tried it using both:
IMAGE_INSTALL_append = " perl"
IMAGE_INSTALL_append = " perl-base"
It will compile ok just simply using perl, but then the image is too big to use. My assumption was the perl-base was the smallest possible version of perl without any of the other fluff. But I'm still not sure if that's even the right way to add perl to my image
Ive also downloaded the meta-perl layer from openembedded and added it to my layers, but it doesn't seem to change the outcome of the build whatsoever.