Local docker runs should honor resource specifications in config

[pankajroark]

We don't enforce any limits right now, but we should to mimic production deployments and let one tune these settings well locally.

[pankajroark]

We'll need to pass the right settings in https://github.com/basetenlabs/truss/blob/main/truss/truss_handle.py#L194
https://docs.docker.com/config/containers/resource_constraints/

[bolasim]

What's the expected behavior around the use_gpu flag and the new accelerator spec?

[pankajroark]

What's the expected behavior around the use_gpu flag and the new accelerator spec?

Good point, this needs more thought. It's unlikely that someone would have the same accelerator locally as they have in a deployed environment. My worry is, in trying to enforce strict use of specs we may make local dev harder. I think we should honor CPU and memory, as those seem like easy pickings. We can revisit use_gpu and accelerator at a later point, in the meanwhile we could show a message indicating any mismatch of gpu/accelerator settings in the docker environment used.