I’m not sure how or if it can be improved. I’d suggest you to check these posts http://stackoverflow.com/questions/21997523/how-to-speed-up-libjpeg-decompression, https://communities.intel.com/thread/60846 . I hope you find them helpful.
If you can switch to using an 800x600 imaage, that should be 4 times faster, as it's a quarter of the pixels, getting closer to the 100ms you need.
Thank you, I will check that link.
My algorithm is base on the detail in picture, the basic size is 1600*1200. The favorite size is more larger(2048*2048). thanks for your reply.