Speed up seems to work somewhat incorrectly:
When just starting building up the latest levels I see that monsters pass to my gems most of the time when I hold the speed up while exactly the same build of towers successfully defend from the same waves of monsters most of the time in the normal mode (without speed up).