I set my ACC speed slightly above not because I want to arrive a few seconds earlier but to facilitate smoother passing of other vehicles that are often just a few clicks below the speed limit. It seems to work.
The VW software, not just for the speed limit "reading", is pretty pathetic when compared to American systems. I shouldn't have to envy my Fiat (ie Chrisler) driving friends' older generation systems.
I pretty well always drive with ACC and have never experienced any slowing down based on the speed limit reading system. If it slows unexpectedly without a car ahead it is usually a mis-reading of the situation, typically a slower vehicle in the exit lane or in a curve. Maybe it's just a speck of dirt flying up to the camera.
I have had the speed display jump up and down because the navigator mistakenly had me following a nearby road instead of the highway I was on and of course it is unable to interpret qualifying signs such as the ones that limit the speed for lorries.
There is a slight hope that a future software update could deliver a miracle but I won't be holding my breath waiting for it.