Problem 6.2 - Optimization

[mvalsania]

Is there a reason to believe that using SGD on the full dataset would not always be a better idea compared to sampling a smaller dataset and using 2GD on it?

Isn't it true that "[in the realm of big data], approximate optimization can achieve better expected risk because more training examples can be processed within the allowed time"?

I feel like there is something I am missing.