Simple guide to setting up an appropriate VM on EC2 or Google Cloud? #299
Description
🚀 The feature, motivation and pitch
Sadly my old NVIDIA GeForce GT470 is way below spec, so I've been looking into provisioning an appropriate VM with GPU to do my 1000 or so pages of poorly scanned document.
Of course, I'd happily pay for an online service from yourselves, but I'm guessing something about the AllenAI organisation prevents that?
UPDATE: Baptism of fire over the last 24 hours, finally ended up with success on runpod. Struggled initially until I found that you need to add: { "cmd": ["3600"], "entrypoint": ["sleep"] } to the pod startup script to actually be able to ssh in.
The only downside now is that it takes a few minutes to download the docker image each time it starts, but I've figured out how to store the cached downloaded models to a persistent disk between sessions so it doesn't have to download 20Gb each time I want to do a couple of PDFs.
Not sure if this is the best way to do it, but for my small needs, $0.35 per hour for a RTX 4090 spot instance suits me fine.
With a bit of help from chatGPT, I've got a script which starts a pod, pulls some docs from s3, processes them, saves them to persistent storage, and then optionally terminates the pod.
Might put together a document to help others once I've sanitized my notes and improved things a bit. There is definitely a market for having a "pay as you go" service for small users like myself!