The processor speed is not the problem. The I/O speed is the problem.
There have been several people looking into the I/O speed problem, and some people have made suggestions as to how to speed it up.
The current reality is that the I/O speeds on Galileo are 100x slower (at least maybe as bad as 1000x) than most Arduinos.
In the future, I think there might be some software tricks to reduce this time, but I those tricks will probably not beat the I/O speeds of most Arduino devices. Maybe the next version of the board will get the speeds on par with other Arduinos.