CMR-10210: Fix 502 gateway error on cmr-stac index.html/documentation & CMR-10208: Add content type in stac response to match service-desc (#384)

* CMR-10210: Try regenerating documentation with redocly

* CMR-10210: Clean up more markdown linter warnings; add documentation on using the stac validator

* CMR-10210: Add note about using the stac-validator

* CMR-10210: Update the type field for the rootcatalog service-desc response

* CMR-10208: Update mimetype for documentation endpoint with the correct Iana spec

* EDSC-10210: Update media type for `service-desc` to `application/yaml`

Author: eudoroolivares2016

README.md

@@ -1,4 +1,5 @@
 # NASA CMR STAC
+
 NASA's [Common Metadata Repository (CMR)](https://cmr.earthdata.nasa.gov/search) is a metadata
 catalog of NASA Earth Science data. [STAC, or SpatioTemporal Asset Catalog](https://stacspec.org/), is a
 [Specification](https://github.com/radiantearth/stac-spec) for describing geospatial data with
@@ -7,6 +8,7 @@ catalog of NASA Earth Science data. [STAC, or SpatioTemporal Asset Catalog](http
 for searching and browsing STAC catalogs.
 
 ## CMR-STAC
+
 CMR-STAC acts as a proxy between the CMR repository and STAC API queries.
 The goal is to expose CMR's vast collections of geospatial data as a STAC-compliant API.
 Even though the core metadata remains the same, a benefit of the CMR-STAC proxy is the ability
@@ -20,11 +22,15 @@ CMR-STAC follows the STAC API 1.0.0-beta.1 specification, see the
 [OpenAPI Documentation](https://api.stacspec.org/v1.0.0-beta.1/index.html).
 
 ## Usage
+
 ### Endpoints
+
 - [CMR-STAC](https://cmr.earthdata.nasa.gov/stac): The entire catalog of NASA CMR data, organized by provider.
+
 - [CMR-CLOUDSTAC](https://cmr.earthdata.nasa.gov/cloudstac): Also organized by provider, this API only contains STAC Collections where the Item Assets are available "in the cloud" (i.e., on S3).
 
 ### Navigating
+
 CMR-STAC can be navigated manually using the endpoints provided above, or you can utilize available STAC software to browse and use the API.  
 
 A common STAC utility is Radiant Earth's `stac-browser` to use this tool against your development server navigate to
@@ -33,13 +39,17 @@ A common STAC utility is Radiant Earth's `stac-browser` to use this tool against
 See the [Usage Documentation](docs/usage/usage.md) for examples of how to interact with the API and search for data.  
 
 ### Limitations
+
 While CMR-STAC provides some advantages over the CMR, there are some limitations that you should be aware of:  
+
 - Limited search functionality: CMR-STAC does not support all of the search capabilities that CMR provides. For example, with CMR, you can search for data based on temporal and spatial criteria, as well as specific parameters such as platform, instrument, and granule size. However, with CMR-STAC, you can only search based on the STAC standard.
 - Limited metadata availability: CMR-STAC only provides metadata that follows the STAC specification. While this metadata is very rich and comprehensive, it may not provide all of the information that you need for your specific use case.
 
 ## For Developers
+
 [Developer README](docs/README.md)
 
 ## License
+
 NASA Open Source Agreement v1.3 (NASA-1.3)
 See [LICENSE.txt](./LICENSE.txt)

docs/README.md

@@ -1,10 +1,10 @@
-## CMR-STAC Development
+# CMR-STAC Development
 
 CMR-STAC is a Node.js application built on the [Express.js framework](https://expressjs.com/) and deployed as an AWS serverless application using API Gateway + Lambda. This README is intended for developers who want to contribute to CMR-STAC, or set up a development environment for it.
 
 The remainder of this README is documentation for developing, testing, and deploying CMR-STAC. See the [Usage documentation](../docs/usage/usage.md) if you are interested in using the CMR-STAC API.
 
-### Repository Structure
+## Repository Structure
 
 | Directory            | Description  |
 | -------------------- | ------------ |
@@ -14,6 +14,7 @@ The remainder of this README is documentation for developing, testing, and deplo
 | [usage](../docs/usage/usage.md)       | Documentation on usage of the CMR-STAC endpoint(s) |
 
 ## Getting Started
+
 ### Setup
 
 Set the correct NodeJS version (specified in [.nvmrc](../.nvmrc) required
@@ -49,11 +50,38 @@ npm run dev
 
 This will run the process in the current terminal session, the local server will be available at:
 
-```
+```bash
 http://localhost:3000/stac
 http://localhost:3000/cloudstac
 ```
 
+### Creating index.html from Swagger.json
+
+To Create the index.html located in docs/index we can use the `redocly` service
+the most straightforward way to do this is to use the cli tool against our `swagger.json` file
+
+```bash
+  npx @redocly/cli build-docs swagger.json
+```
+
+### Testing STAC validation
+
+We can test our API both locally and on deployed instance against the on a stac-validation service using the <https://github.com/stac-utils/stac-api-validator> tool
+
+The tool can be installed using pip and requires a Python runtime
+
+```bash
+  pip install stac-api-validator
+```
+
+```bash
+   stac-api-validator\
+    --root-url http://localhost:3000/stac/CMR_ONLY \
+    --conformance core
+```
+
+this can be extended to validate against additional conformance APIs
+
 ### Deploying
 
 The deployment is handled via the [Serverless Framework](https://serverless.com). Each service has a
@@ -85,4 +113,4 @@ npm run deploy:docs -- --stage <sit|uat|prod>
 ## License
 
 NASA Open Source Agreement v1.3 (NASA-1.3)
-See [LICENSE.txt](../LICENSE.txt)
+See [LICENSE.txt](../LICENSE.txt)

docs/index/index.html

@@ -16,8 +16,8 @@ deployed to production. UAT CMR-STAC uses the UAT CMR environment which has a mu
 
 Due to the vast number of collections contained within CMR, a user cannot query across all granules at once.
 [Attempting to do so returns an error message](https://cmr.earthdata.nasa.gov/search/granules.json)
-indicating that at least one of a set of fields must be provided. 
-Because a STAC API allows for searching across all Items (i.e., granules), instead of a single API, 
+indicating that at least one of a set of fields must be provided.
+Because a STAC API allows for searching across all Items (i.e., granules), instead of a single API,
 CMR-STAC provides multiple APIs, one for each provider. Upon hitting the root endpoint (https://cmr.earthdata.nasa.gov/stac)
 the response will include a list of CMR providers as a series of links
 with a "rel" field equal to "child".
@@ -34,7 +34,7 @@ with a "rel" field equal to "child".
 Navigating to a provider yields a STAC Catalog, but one that has some additional links and a
 `conformsTo` field, indicating that it is a STAC API. These links include:
 
-| rel | path | Decription |
+| rel | path | Description |
 | --- | ---- | ---------- |
 | self | /{providerId} | this provider catalog |
 | root | / | the root CMR-STAC catalog |
@@ -55,10 +55,10 @@ is given here. The table shows the different endpoints available from the root c
 | ---- | ----------- |
 | /    | Root endpoint. Returns a list of provider Catalogs |
 | /{provider-id} | Returns singular provider Catalog |
-| /{provider-id}/collections | Returns a Catalog of all Collections that belong to the indicated provider | 
-| /{provider-id}/collections/{collection-id} | Returns the individual Collection indicated in the path | 
-| /{provider-id}/collections/{collection-id}/items | Returns an ItemCollection of all Items contained in the indicated Collection | 
-| /{provider-id}/collections/{collection-id}/items/{item-id} | Returns the individual Item indicated in the path | 
+| /{provider-id}/collections | Returns a Catalog of all Collections that belong to the indicated provider |
+| /{provider-id}/collections/{collection-id} | Returns the individual Collection indicated in the path |
+| /{provider-id}/collections/{collection-id}/items | Returns an ItemCollection of all Items contained in the indicated Collection |
+| /{provider-id}/collections/{collection-id}/items/{item-id} | Returns the individual Item indicated in the path |
 
 For example, if a user wanted to get all the collections under the ASF provider, the user would hit:
 
@@ -88,7 +88,8 @@ Each Item returned from CMR-STAC includes a series of links as well:
 | via | The original CMR metadata used to generate the STAC metadata returned |
 
 ## Browsing
-The first thing to do when getting familiar with CMR-STAC is to browse through the catalog and get a sense for its structure, and what types of data (Collections and Items) are available. 
+
+The first thing to do when getting familiar with CMR-STAC is to browse through the catalog and get a sense for its structure, and what types of data (Collections and Items) are available.
 The easiest way to browse is to install a JSON extension for your browser of choice (e.g., [JSON Formatter](https://chrome.google.com/webstore/detail/json-formatter/bcjindcccaagfpapjjmafapmmgkkhgoa/related) for Chrome).
 
 ![](https://s3.gifyu.com/images/cmr-stac-browse-opt.gif)
@@ -139,6 +140,7 @@ JSON Body:
 ```
 
 ## Searching Collections
+
 Similar to searching for Items, CMR-STAC provides endpoints to search for Collections. Both GET and POST requests are supported for collection searches.
 
 | Parameter | Type | Description |
@@ -152,7 +154,6 @@ Similar to searching for Items, CMR-STAC provides endpoints to search for Collec
 
 GET:
 
-
 https://localhost:3000/stac/LARC_ASDC/collections?bbox=-180,-90,180,90&datetime=2000-01-01T00:00:00Z/2022-01-01T00:00:00Z&keyword=atmosphere
 
 **To sort the results:**

docs/usage/usage.md

@@ -109,7 +109,7 @@ describe("GET /:provider", () => {
 
       const link: Link = catalog.links.find((l: Link) => l.rel === "service-desc");
       expect(link).to.have.property("rel", "service-desc");
-      expect(link).to.have.property("type", "application/vnd.oai.openapi;version=3.0");
+      expect(link).to.have.property("type", "application/yaml");
       expect(link).to.have.property("href", "https://api.stacspec.org/v1.0.0-beta.1/openapi.yaml");
       expect(link).to.have.property("title", "OpenAPI Doc");
     });

src/__tests__/providerCatalog.spec.ts

@@ -97,7 +97,7 @@ const extractLicense = (_collection: Collection) => {
 };
 
 /**
- * Examing a collections related URLs to see if it contains a reference to a STAC catalog.
+ * Examining a collections related URLs to see if it contains a reference to a STAC catalog.
  * If the collection has a RelatedURL of type: "GET CAPABILITIES",
  * and subtype: "STAC" then that URL should be placed in the href for the items link.
  *

src/domains/collections.ts

@@ -51,7 +51,7 @@ const generateSelfLinks = (req: Request, nextCursor?: string | null, count?: num
     {
       rel: "service-desc",
       href: "https://api.stacspec.org/v1.0.0-beta.1/openapi.yaml",
-      type: "application/vnd.oai.openapi;version=3.0",
+      type: "application/yaml",
       title: "OpenAPI Doc",
     },
     {

src/routes/catalog.ts

@@ -25,9 +25,9 @@ const selfLinks = (req: Request): Link[] => {
     },
     {
       rel: "service-desc",
-      href: `${stacRoot}/resources/swagger.json`,
-      title: "OpenAI Documentation",
-      type: "application/vnd.oai.openapi+json;version=3.0",
+      href: `https://api.stacspec.org/v1.0.0-beta.1/openapi.yaml`,
+      title: "OpenAPI Documentation",
+      type: "application/yaml",
     },
     {
       rel: "service-doc",