Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Need Help]When I upload a file directly to the server's mount path of kadalu pvc, the file cannot be identified in the corresponding pod. #1032

Open
et21w opened this issue Dec 1, 2023 · 5 comments
Labels
on-user pending on user

Comments

@et21w
Copy link

et21w commented Dec 1, 2023

I have a single-node Kubernetes cluster and am using Kadalu as my storage. When creating the Kadalu storage, I set the type to replica1 and configured the storage to a path created on the server. Then I defined my own PVC for use in my application. When my pod starts, I obtain the detailed mount path of subvol/xxx/xxx from the PV on the server. However, when I directly upload files to the server's mount path, the application cannot recognize them. Examining the file type in the pod shows something abnormal.Like this,
ls: cannot access 'nlp_test.csv': No data available
-????????? ? ? ? ? ? nlp_test.csv
But it's normal for me to upload via the frontend.
Thanks.

@amarts
Copy link
Member

amarts commented Dec 1, 2023

This is not a use case which is supported by Gluster. Gluster has some internal data which gets added along with the file when created using mountpoint. They will be missing when the file gets added directly.

@et21w
Copy link
Author

et21w commented Dec 1, 2023

This is not a use case which is supported by Gluster. Gluster has some internal data which gets added along with the file when created using mountpoint. They will be missing when the file gets added directly.

Besides Gluster's own support, is this issue related to the type of kadaluStorage? Because in another k8s cluster, I created kadaluStorage using the replica2 and replica3 types, and I directly uploaded files to the mount paths of these two types of servers. They were able to be read normally in the pod.Or, this issue is only caused by the characteristics of Glusterfs when a single node uses Gluster to store files?

@vatsa287
Copy link
Member

vatsa287 commented Dec 4, 2023

This is not a use case which is supported by Gluster. Gluster has some internal data which gets added along with the file when created using mountpoint. They will be missing when the file gets added directly.

Besides Gluster's own support, is this issue related to the type of kadaluStorage? Because in another k8s cluster, I created kadaluStorage using the replica2 and replica3 types, and I directly uploaded files to the mount paths of these two types of servers. They were able to be read normally in the pod.Or, this issue is only caused by the characteristics of Glusterfs when a single node uses Gluster to store files?

Yes it might be because of type of kadalustorage selected.
Replica1 is equivalent to pure distribute type in glusterfs, and requires the data to be set through client side for it to be placed in a algo based(through dht translator) predictable path to store and retrieve from nodes, which might not be the case with replicas as all data is stored in all nodes.

But anyway as @amarts stated, here storage use case is to store the data from client side and retrieve from server nodes.

@et21w
Copy link
Author

et21w commented Dec 7, 2023

This is not a use case which is supported by Gluster. Gluster has some internal data which gets added along with the file when created using mountpoint. They will be missing when the file gets added directly.

Besides Gluster's own support, is this issue related to the type of kadaluStorage? Because in another k8s cluster, I created kadaluStorage using the replica2 and replica3 types, and I directly uploaded files to the mount paths of these two types of servers. They were able to be read normally in the pod.Or, this issue is only caused by the characteristics of Glusterfs when a single node uses Gluster to store files?

Yes it might be because of type of kadalustorage selected. Replica1 is equivalent to pure distribute type in glusterfs, and requires the data to be set through client side for it to be placed in a algo based(through dht translator) predictable path to store and retrieve from nodes, which might not be the case with replicas as all data is stored in all nodes.

But anyway as @amarts stated, here storage use case is to store the data from client side and retrieve from server nodes.

Okay, thank you very much

@amarts
Copy link
Member

amarts commented Jan 16, 2024

This is not a use case which is supported by Gluster. Gluster has some internal data which gets added along with the file when created using mountpoint. They will be missing when the file gets added directly.

Besides Gluster's own support, is this issue related to the type of kadaluStorage? Because in another k8s cluster, I created kadaluStorage using the replica2 and replica3 types, and I directly uploaded files to the mount paths of these two types of servers. They were able to be read normally in the pod.Or, this issue is only caused by the characteristics of Glusterfs when a single node uses Gluster to store files?

That works because replica2 (or replica3) does a self-heal logic). In case of replica1 (or is called distribute), which doesn't come with any self-heal logic, and hence in that case, the backend writing is not supported.

But in general, its a good practise to never write directly to backend. (think of you updaing a xfs / ext4 block directly while FS is mounted, it would corrupt the system).

@leelavg leelavg added the on-user pending on user label Apr 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
on-user pending on user
Projects
None yet
Development

No branches or pull requests

4 participants