[docs] Add documentation for file-based metastore

[steveburnett]

Description

Add how to configure Presto to use a file-based hms to installation/deployment.rst.

Motivation and Context

@ethanyzhang suggested this would be a good addition to the Presto documentation in an internal discussion that @nmahadevuni contributed the configuration in. I discussed where such information would fit best in the Presto documentation with @tdcmeehan.

Impact

Documentation. Readers wanting to try out Presto quickly can bypass the need for the steps in Configure Hive MetaStore.

Test Plan

Local doc build. Screenshot with existing text above and below included for context.

Contributor checklist

• Please make sure your submission complies with our contributing guide, in particular code style and commit standards.
• PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
• Documented new properties (with its default value), SQL syntax, functions, or other functionality.
• If release notes are required, they follow the release notes guidelines.
• Adequate tests were added if applicable.
• CI passed.

Release Notes

Please follow release notes guidelines and fill in the release notes below.

== RELEASE NOTES ==

General Changes
* Add documentation for file-based Hive metastore to :doc:`/connector/file-based-metastore`.

[majetideepak]

@steveburnett I have an issue here with some more details
#19112
We need to list the restrictions as well. file-based metastore does not support partitioning for example. @nmahadevuni should confirm

[imjalpreet]

file-based meta store does not support partitioning

@majetideepak I haven't used the file-based Hive metastore much myself, but did you encounter any issues when trying to create partitioned tables?

Ideally, it should be possible and I can see that partition-specific metadata calls are implemented even for the FileHiveMetastore.

https://github.com/prestodb/presto/blob/master/presto-hive-metastore/src/main/java/com/facebook/presto/hive/metastore/file/FileHiveMetastore.java

[ethanyzhang]

@imjalpreet should this be in the hive.properties or config.properties?

[imjalpreet]

should this be in the hive.properties or config.properties?

@ethanyzhang These should be in the catalog properties file. I also added a review comment above.

[steveburnett]

@steveburnett I have an issue here with some more details #19112 We need to list the restrictions as well. file-based metastore does not support partitioning for example. @nmahadevuni should confirm

Thanks for the additional information @majetideepak!

With the new information in #19112, I am going to move this topic from where I initially put it in this PR as a small topic in Deploying Presto.

I thought about moving it into the Hive Connector doc, but as it is relevant to "Hive and Lakehouse Connectors (Iceberg, Delta, and Hudi)" I think I will move it to its own page in /installation and include Deepak's instructions how to use it, which will be a big help to readers.

[imjalpreet]

Would it make more sense to include this in the connectors section?

Right now, metastore properties are spread across different parts of the Hive Connector documentation, mainly in Hive Configuration Properties, Metastore Configuration Properties, and Glue Configuration Properties.

Perhaps we could extract the metastore-related documentation into a separate subsection, similar to Hive Security. This would make sense since metastore properties are relevant not just to Hive, but also to other connectors like Iceberg, Delta, and Hudi.

[majetideepak]

Ideally, it should be possible and I can see that partition-specific metadata calls are implemented even for the FileHiveMetastore.

@imjalpreet I remember seeing some issue with partitions and file metastore. It could be due to my setup. I think its good to test this once before documenting.

[hantangwangd]

I use file bases HMS for development environment, just confirmed again that it supports Hive partitioned tables. Or maybe you encountered some specific problems when using partitioned tables with file bases HMS @majetideepak?

[majetideepak]

@hantangwangd If you use it, then it's good. My issue was likely related to my setup.

[nmahadevuni]

@majetideepak I have used partitioned tables too recently with file based HMS. I didn't have any issues.

[steveburnett]

Hi everyone, thanks for your feedback! I took @imjalpreet's suggestion and moved this to a separate page in /connector, corrected the directory path that @nmahadevuni noted, added text describing the supported connectors and corrected the instructions for the properties file to the connector property file, and added usage examples from @majetideepak's #19112.

I welcome everyone to review again and comment with new corrections and additions.

[majetideepak]

@steveburnett, @hantangwangd, @nmahadevuni
The file-based metastore approach is also useful for reading existing files.
I tested all the content in #19112
It does not cover partitioning. We could cover partitioning in a separate PR.

[nmahadevuni]

Since we are mentioning one can provide .prestoSchema and .prestoPermissions for existing data, is there any format or steps on how to create these? @majetideepak

[steveburnett]

Hi everyone! I revised based on the feedback from @nmahadevuni, @majetideepak, and @hantangwangd. PTAL when you can.

Specific open questions I haven't addressed in this update:

• what I should say or remove from the current draft about partitioning. Should I add a sentence in the Overview "Partitioning with file-based metastores is not supported."?

• what I should add about "provide .prestoSchema and .prestoPermissions files", or remove that line entirely?

[majetideepak]

@steveburnett Let's remove Reading Existing Data Files with a File-based Metastore from this PR since both the open questions are related to that.
I believe @imjalpreet mentioned that partitioning works when you create the table with file based metastore.

[steveburnett]

@steveburnett Let's remove Reading Existing Data Files with a File-based Metastore from this PR since both the open questions are related to that. I believe @imjalpreet mentioned that partitioning works when you create the table with file based metastore.

Done and done, thanks! PTAL.

[hantangwangd]

Thanks for the work @steveburnett.

[nmahadevuni]

Thank you @steveburnett.

[imjalpreet]

@steveburnett thanks for the PR, LGTM!

[ethanyzhang]

@steveburnett is this ready to go? We can ping @yingsu00 to merge

[steveburnett]

@steveburnett is this ready to go? We can ping @yingsu00 to merge

@ethanyzhang yes, this is ready to merge.