No way, just the opposite. Folding bikes with small wheels can actually accelerate faster than large wheel bikes and, given a well distributed gearing, go just as fast.
Take my Brompton folding bike for instance. It has 16 inch wheels and yet I can go faster on this folding bike than my full sized road bike. Why? Because it has a very high gearing. “What does that mean”, you ask. Well it’s like this:
(This paragraph is a bit technical sounding. Do you have your protractor and slide rule? Just kidding. You can skip it if you want, it’s not critical.)
Bikes have gears to make it easier to go up hill and easier to go really fast. How is this accomplished? By switching between gears of larger and smaller sizes you are actually changing what you might call your folding bike’s “theoretical wheel size.” Or more simply put, when you shift into a high-speed gear you can imagine your wheel gets much larger and therefor spins much faster on the road relative to the speed at its center. When shift into a hill climbing gear you can imagine that your wheel gets smaller and spins slower than the speed being put in. That ability to increase or decrease your speed via gears is called mechanical advantage.
Basically by raising the overall gearing on a folding bicycle with small wheels you can make it feel just like riding a large wheel folding bike (or regular bicycle for that matter). It can sometimes be difficult to get the same top speed on a folding bike without the help of an internal gear hub because the really small gears required are expensive to manufacture. High quality folding bikes should be able to go just as fast or faster than your average hybrid bike.
That said the simple simple answer is: You can go just as fast on a folding bike as you can on a regular bike. It’s all a matter of gearing.