How to use minio to serve the model?

[photoszzt]

Is there any example using minio to serve model? I modify the tensorflow example to let it serve resnet50 using minio. I try to modify s3's endpoint url to use minio's cluster ip. Currently the container log shows:

2023-02-07 06:46:49.962252: I tensorflow_serving/model_servers/server.cc:89] Building single TensorFlow model file config:  model_name: resnet50v2tf model_base_path: /mnt/models
2023-02-07 06:46:49.962484: I tensorflow_serving/model_servers/server_core.cc:465] Adding/updating models.
2023-02-07 06:46:49.962502: I tensorflow_serving/model_servers/server_core.cc:591]  (Re-)adding model: resnet50v2tf
2023-02-07 06:46:49.963431: W tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:268] No versions of servable resnet50v2tf found under base path /mnt/models. Did you forget to name your leaf directory as a number (eg. '/1/')?

Does it mean the model is not downloaded?

[jagadeeshi2i]

Tf is looking for a version sub-folder.

you can download the model from tf example and and use that as reference..

https://github.com/kserve/kserve/tree/master/docs/samples/v1beta1/tensorflow

gsutil cp -r gs://kfserving-examples/models/tensorflow/flowers .

Refer below links on using minio with kserve:

https://github.com/kubeflow/pipelines/blob/master/samples/contrib/pytorch-samples/mino-secret.yaml
https://github.com/kubeflow/pipelines/blob/master/samples/contrib/pytorch-samples/Pipeline-Cifar10.ipynb

[Figjam69697]

Maybe just stop using it

[photoszzt]

@jagadeeshi2i I have been following the instruction. The pod is stuck in init which is the storage initializer. I dump the log of it and it doesn't print any error. Using the local mc client, I can also see the files are uploaded.

$ kubectl logs resnet50v2tf-predictor-default-00001-deployment-874d6c79f-c2298 -c storage-initializer
2023-02-13 18:48:50.527 1 root INFO [<module>():13] Initializing, args: src_uri [s3://zzt-models/tf/resnet50v2] dest_path[ [/mnt/models]
2023-02-13 18:48:50.527 1 root INFO [download():63] Copying contents of s3://zzt-models/tf/resnet50v2 to local
2023-02-13 18:48:50.555 1 botocore.credentials INFO [load():1120] Found credentials in environment variables.
$ kubectl get pods
NAME                                                              READY   STATUS     RESTARTS        AGE
minio-deployment-c5d9fb859-pjq7g                                  1/1     Running    0               45m
resnet50v2tf-predictor-default-00001-deployment-874d6c79f-c2298   0/2     Init:0/1   1 (4m40s ago)   9m47s**
$ mc ls miniodev/zzt-models/tf/resnet50v2/
[2023-02-13 12:53:44 CST]     0B 0001/
$ kubectl get svc
NAME                                           TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                                              AGE
kubernetes                                     ClusterIP   10.96.0.1      <none>        443/TCP                                              62m
minio-service                                  ClusterIP   10.96.0.46     <none>        9000/TCP                                             45m
resnet50v2tf-predictor-default-00001           ClusterIP   10.106.56.22   <none>        80/TCP,443/TCP                                       10m
resnet50v2tf-predictor-default-00001-private   ClusterIP   10.97.127.21   <none>        80/TCP,443/TCP,9090/TCP,9091/TCP,8022/TCP,8012/TCP   10m

This is the S3 spec:

apiVersion: v1
kind: Secret
metadata:
  name: s3secret
  annotations:
     serving.kserve.io/s3-endpoint: "10.96.0.46" # replace with your s3 endpoint e.g minio-service.kubeflow:9000
     serving.kserve.io/s3-usehttps: "0" # by default 1, if testing with minio you can set to 0
     serving.kserve.io/s3-region: "us-east-1"
     serving.kserve.io/s3-useanoncredential: "false" # omitting this is the same as false, if true will ignore provided credential and use anonymous credentials
type: Opaque
data: # use `stringData` for raw credential string or `data` for base64 encoded string
  AWS_ACCESS_KEY_ID: "bWluaW8K"
  AWS_SECRET_ACCESS_KEY: "bWluaW8xMjMK"
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: sa
secrets:
- name: s3secret

I'm not sure what's the problem.

[jagadeeshi2i]

Try using minio endpoint with port and region as minio - https://github.com/kubeflow/pipelines/blob/master/samples/contrib/pytorch-samples/mino-secret.yaml