OOM errors for large datasets

If we load a sufficiently big dataset (using `tf.data.dataset` ==> TFDS in "not all in memory mode"), the instance crashes with an OOM error. Since we are iteratively using `TFDS` in batches, this should not be the case, right ... ? 

Thus, we can conclude that the model tries to load the entire dataset into memory. Is this behavior normal?
How can we scale this to big-data usage ?

THNX!