Consuming Atlassian Jira API with new OpenAPI handler #3852

wwerner · 2022-04-20T14:37:21Z

wwerner
Apr 20, 2022

Hey there,

as discussed on Discord (starting here), we're trying to consume the Jira OAS in mesh. After some clean-up of the spec, we were able to use it with graphql-to-openapi.

It did not work our right away with the newOpenapi handler.

Here are some findings, perhaps the list will grow. I can chop it into several issues later on if required/helpful.

Was tested w/

@graphql-mesh/cli ^0.67.5
@graphql-mesh/new-openapi ^0.4.7
graphql 16.3.0

1

Atlassian provides the response samples as strings, which handleOperationResponseConfig. If it sees a string, it assumes it points to a file containing the sample, if I read it correctly. Applying the 👇 patch makes the 👇👇 error go away.

Reproduction

In an empty folder with the following .meshrc.yaml, run yarn add @graphql-mesh/[email protected] @graphql-mesh/[email protected] [email protected] && DEBUG=1 yarn mesh dev
Copy the patch below to 1.patch and run patch node_modules/@omnigraph/json-schema/index.js 1.patch && yarn mesh dev. (GH won't let me attach yaml or patch files, sorry.)

.meshrc.yaml

sources:
  - name: Jira
    handler:
      newOpenapi:
        oasFilePath: https://developer.atlassian.com/cloud/jira/platform/swagger-v3.v3.json

1.patch

109c109,112
<         const sample = typeof operationResponseConfig.responseSample === 'object'
---
>         const sample = (
>             typeof operationResponseConfig.responseSample === 'object' 
>             || typeof operationResponseConfig.responseSample === 'string'
>         )

Errors

before patching

Jira: Failed to generate the schema 
    at /Users/.../mesh-jira-oas/node_modules/@omnigraph/json-schema/index.js:116:23
    at async  (/Users/.../mesh-jira-oas/node_modules/@omnigraph/json-schema/index.js:112:15)
  Error: responseSample - ENOENT: no such file or directory, open '/Users/.../mesh-jira-oas/{"errorMessages":["Connect and Forge app users with Administer Jira global permission can override screen security."],"errors":{}}'
   at async getReferencedJSONSchemaFromOperations (/Users/.../mesh-jira-oas/node_modules/@omnigraph/json-schema/index.js:168:49)
    at async getDereferencedJSONSchemaFromOperations (/Users/.../mesh-jira-oas/node_modules/@omnigraph/json-schema/index.js:255:34)
    at async Object.createBundle (/Users/.../mesh-jira-oas/node_modules/@omnigraph/json-schema/index.js:1649:32)
    at async Object.getWithSet (/Users/.../mesh-jira-oas/node_modules/@graphql-mesh/store/index.js:148:38)
    at async OpenAPIHandler.getMeshSource (/Users/...mesh-jira-oas/node_modules/@graphql-mesh/new-openapi/index.js:38:24)
    at async /Users/.../mesh-jira-oas/node_modules/@graphql-mesh/runtime/index.js:39:28

after patching

🕸️ - Jira: Failed to generate the schema 
 TypeError: Cannot read properties of undefined (reading 'trim')
    at Object.sanitizeNameForGraphQL (/private/tmp/x/node_modules/@graphql-mesh/utils/index.js:524:36)
    at getValidTypeName (/private/tmp/x/node_modules/@omnigraph/json-schema/index.js:522:33)
    at getTypeComposer (/private/tmp/x/node_modules/@omnigraph/json-schema/index.js:1552:35)
    at mutateFn (/private/tmp/x/node_modules/@omnigraph/json-schema/index.js:1599:24)
    at visitJSONSchema (/private/tmp/x/node_modules/json-machete/index.js:192:41)
    at async visitJSONSchema (/private/tmp/x/node_modules/json-machete/index.js:163:62)
    at async visitJSONSchema (/private/tmp/x/node_modules/json-machete/index.js:120:43)
    at async visitJSONSchema (/private/tmp/x/node_modules/json-machete/index.js:163:62)
    at async visitJSONSchema (/private/tmp/x/node_modules/json-machete/index.js:163:62)
    at async Object.visitJSONSchema (/private/tmp/x/node_modules/json-machete/index.js:163:62)
🕸️: An error occured while writing the schema file: Error: Schemas couldn't be generated successfully. Check for the logs by running Mesh with DEBUG=1 environmental variable to get more verbose output.
    at Object.getMesh (/private/tmp/x/node_modules/@graphql-mesh/runtime/index.js:69:15)
🕸️: An error occurred while building the artifacts: Error: Schemas couldn't be generated successfully. Check for the logs by running Mesh with DEBUG=1 environmental variable to get more verbose output.
    at Object.getMesh (/private/tmp/x/node_modules/@graphql-mesh/runtime/index.js:69:15)
🕸️ - Server: Error: Schemas couldn't be generated successfully. Check for the logs by running Mesh with DEBUG=1 environmental variable to get more verbose output.
    at Object.getMesh (/private/tmp/x/node_modules/@graphql-mesh/runtime/index.js:69:15)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

2

DELETE operations returning 204 do not seem to be supported. openapi-to-graphql had a switch called fillEmptyResponses for that.
The new handler does not seem to support operations of that kind.

Reproduction

Run yarn mesh dev 2>&1 >/dev/null | grep -B 1 '204' in the folder created in 1
You'll see 58 type creation errors

Details

Example error (full debug log attached below)

🕸️ - Jira: GraphQL Type cannot be created for this JSON Schema definition;
subSchema: { $resolvedRef: "/paths/~1rest~1api~13~1workflowscheme~1{id}/delete/responses/204/content/application~1json/schema", title: "deleteWorkflowScheme_204_response" }

3

sanitizeNameForGraphQL fails as the unsafeName is undefined in places in the spec. I think the place in the code this happens is here. Replacing undefined unsafeNames w/ "MISSING" makes this error go away.

Reproduction

Run yarn mesh dev in the same folder after reproducing the error in 1 and applying the 1.patch
Apply 2.patch shown below: patch node_modules/@graphql-mesh/utils/index.js 2.patch

2.patch

523a524
>     if(unsafeName === undefined) unsafeName = "MISSING"

Details

error before patching

 TypeError: Cannot read properties of undefined (reading 'trim')
    at Object.sanitizeNameForGraphQL (/Users/.../mesh-jira-oas/node_modules/@graphql-mesh/utils/index.js:524:36)
    at getValidTypeName (/Users/.../mesh-jira-oas/node_modules/@omnigraph/json-schema/index.js:522:33)
    at getTypeComposer (/Users/.../mesh-jira-oas/node_modules/@omnigraph/json-schema/index.js:1552:35)
    at mutateFn (/Users/.../mesh-jira-oas/node_modules/@omnigraph/json-schema/index.js:1599:24)
    at visitJSONSchema (/Users/.../mesh-jira-oas/node_modules/json-machete/index.js:192:41)
    at async visitJSONSchema (/Users/.../mesh-jira-oas/node_modules/json-machete/index.js:163:62)
    at async visitJSONSchema (/Users/.../mesh-jira-oas/node_modules/json-machete/index.js:120:43)
    at async visitJSONSchema (/Users/.../mesh-jira-oas/node_modules/json-machete/index.js:163:62)
    at async visitJSONSchema (/Users/.../mesh-jira-oas/node_modules/json-machete/index.js:163:62)
    at async Object.visitJSONSchema (/Users/.../mesh-jira-oas/node_modules/json-machete/index.js:163:62)

error after patching

🕸️ - Jira: Failed to generate the schema 
 TypeError: Cannot read properties of undefined (reading 'getFields')
    at getTypeComposer (/private/tmp/x/node_modules/@omnigraph/json-schema/index.js:988:73)
    at mutateFn (/private/tmp/x/node_modules/@omnigraph/json-schema/index.js:1599:24)
    at visitJSONSchema (/private/tmp/x/node_modules/json-machete/index.js:192:41)
    at async visitJSONSchema (/private/tmp/x/node_modules/json-machete/index.js:163:62)
    at async visitJSONSchema (/private/tmp/x/node_modules/json-machete/index.js:163:62)
    at async visitJSONSchema (/private/tmp/x/node_modules/json-machete/index.js:120:43)
    at async visitJSONSchema (/private/tmp/x/node_modules/json-machete/index.js:163:62)
    at async visitJSONSchema (/private/tmp/x/node_modules/json-machete/index.js:120:43)
    at async visitJSONSchema (/private/tmp/x/node_modules/json-machete/index.js:163:62)
    at async visitJSONSchema (/private/tmp/x/node_modules/json-machete/index.js:163:62)
🕸️: An error occured while writing the schema file: Error: Schemas couldn't be generated successfully. Check for the logs by running Mesh with DEBUG=1 environmental variable to get more verbose output.
    at Object.getMesh (/private/tmp/x/node_modules/@graphql-mesh/runtime/index.js:69:15)
🕸️: An error occurred while building the artifacts: Error: Schemas couldn't be generated successfully. Check for the logs by running Mesh with DEBUG=1 environmental variable to get more verbose output.

4

There are additional errors in the logs that didn't seem to break the schema generation right away.
Examples below, in sum there are 184 similar occurrences in the attached log, but several of them look very similar.
Complete log attached.
debug.log

🕸️ - Jira Sanitized: GraphQL Type cannot be created for this JSON Schema definition; subSchema: { readOnly: true, title: "query_getAttachment_additionalProperties", $resolvedRef: "/definitions/query_getAttachment_additionalProperties" } path: "/properties/query/properties/getAttachment/properties/properties/additionalProperties"
🕸️ - Jira Sanitized: GraphQL Type cannot be created for this JSON Schema definition; subSchema: { description: "The value of the property. Required on create and update.", title: "query_getCommentProperty_value", $resolvedRef: "/definitions/query_getCommentProperty_value" } path: "/properties/query/properties/getCommentProperty/properties/value"
subSchema: { $resolvedRef: "/paths/~1rest~1api~13~1configuration~1timetracking/get/responses/204/content/application~1json/schema", title: "getSelectedTimeTrackingImplementation_204_response" } path: "/properties/query/properties/getSelectedTimeTrackingImplementation/oneOf/1"
🕸️ - Jira Sanitized: GraphQL Type cannot be created for this JSON Schema definition; subSchema: { description: "The default value of the field.", readOnly: true, title: "query_getCreateIssueMeta_projects_items_issuetypes_items_fields_additionalProperties_defaultValue", $resolvedRef: "/definitions/query_getCreateIssueMeta_projects_items_issuetypes_items_fields_additionalProperties_defaultValue" } path: "/properties/query/properties/getCreateIssueMeta/properties/projects/items/properties/issuetypes/items/properties/fields/additionalProperties/properties/defaultValue"
🕸️ - Jira Sanitized: GraphQL Type cannot be created for this JSON Schema definition; subSchema: { description: "The comment text in [Atlassian Document Format](https://developer.atlassian.com/cloud/jira/platform/apis/document/structure/).", title: "query_getComments_comments_items_body", $resolvedRef: "/definitions/query_getComments_comments_items_body" } path: "/properties/query/properties/getComments/properties/comments/items/properties/body"

Answered by wwerner

Apr 26, 2022

To wrap this up: We were able to use the new openapi handler from @omnigraph/[email protected] (which contains @ardatan's PR ^^^) on its own for stitching the Jira OAS API into an existing gateway.

Thanks so much for the support & patching right away! 🙌 🙏

View full answer

ardatan · 2022-04-20T15:01:48Z

ardatan
Apr 20, 2022
Maintainer

In order to helpyou better, we need a clear reproduction. Maybe an example openapi file for that.

0 replies

wwerner · 2022-04-20T20:49:14Z

wwerner
Apr 20, 2022
Author

Yeah, I realize now the findings were really not optimized for easy consumption. I updated the the original post with reproduction steps. Hope it's better now.

This is not so much a question as the report on how the new OAS handler did on the Jira OAS as prev. discussed.
Would be great to know if these are - in your opinion - things that are - um - weird in the Jira spec or actual issues in mesh. And if so, whether you're planning to address them and/or are open to accepting patches.

TL;DR: With my current knowledge, I feel making the new OAS handler able to deal with the Jira spec is quite some effort. But I'd love to learn I'm mistaken re my judgement.

Thanks in advance.

0 replies

ardatan · 2022-04-21T11:53:13Z

ardatan
Apr 21, 2022
Maintainer

Thank you for all those details which are REALLY helpful!
I did some fixes and improvements to make it work with JIRA's schema;
A working example with canaries;
https://codesandbox.io/s/unruffled-sky-xcu0wx?file=/.meshrc.yml
Related PR;
#3816

I haven't tested any of the queries or mutations and checked if the generated schema correctly represents what OpenAPI schema has.

Waiting for your feedback. Thanks again!

4 replies

wwerner Apr 21, 2022
Author

Awesome, thanks so much for addressing this so quickly! 🙏
I'll give it a spin later today, run some queries/mutations and let you know.

wwerner Apr 21, 2022
Author

That's awesome progress! I was pulling my hair out over the weird/massive Jira API spec. I honestly wasn't expecting (at least partial) success so rapidly. Respect.

Status TL;DR: Conversion works, queries work, issue w/ mutations due to deficient Jira OAS. My assumption is that as the input type in the spec is insufficiently specified, the content-type header is not getting set which makes the downstream server reject the call w/ 415

I was able to query the Jira API through mesh with the versions used in the sandbox you provided and the following .meshrc

sources:
  - name: Jira
    handler:
      newOpenapi:
        oasFilePath: https://developer.atlassian.com/cloud/jira/platform/swagger-v3.v3.json
        baseUrl: https://api.atlassian.com/ex/jira/{context.headers['cloud-id']}/
        operationHeaders:
          authorization: Bearer {context.headers['token']}

Trying to run a mutation, I hit an issue, though, but that's also b/c Atlassian's spec is deficient.
But perhaps we can find a workaround.

Many text blocks are defined as Atlassian Document Format, which is basically marked-up text as JSON
This format does not seem to be part of the spec but is defined separately ¯_(ツ)_/¯
The OAS does not specify a type for the input type but just refers to ADF in the comment ¯_(ツ)_/¯^2
I can send any object through graphiql, but it seems while it is getting serialized correctly the content-type is not set, as I get back a 415 without further messages from the downstream API. (the operation accepts application/json as per the docs)
This status is not propagated, as in json-schema's addExecutionLogicToComposer we receive an empty response text and try to parse it as JSON.
I have verified my token has all the required scopes, just in case
When looking at it in the debugger in addExecutionLogicToComposer I see that, indeed, the content-type header is not getting set. I can reproduce getting a 415 w/o that header using curl, too.

Jira OAS Comment Schema (body property)

      "Comment": {
        "type": "object",
        "properties": {
        ...
          "body": {
            "description": "The comment text in [Atlassian Document Format](https://developer.atlassian.com/cloud/jira/platform/apis/document/structure/)."
          },
       ...
      },

Test mutation

mutation addComment {
  addComment(
    issueIdOrKey: "PJIT-1"
    input: {
      visibility: {}
      body: {
        version: 1,
        type: "doc",
        content: [
          {
            type: "paragraph",
            content: [
              {
                type: "text",
                text: "This was added through the "
              },
              {
                type: "text",
                text: "GraphQL Gateway",
                marks: [
                  {
                    type: "strong"
                  }
                ]
              }
            ]
          }
        ]
      }
    }) {
    id
  }
}

ardatan Apr 21, 2022
Maintainer

Did you try to set Content-Type manually within operationHeaders?

wwerner Apr 21, 2022
Author

This sets the header. I'm hitting 401 now, but I guess that's on my side as the query went through fine.
🥳

Is this
a) just a 🤦 on my side?
b) an actual solution?
c) a workaround that could have ill side effects?

Thanks!

wwerner · 2022-04-26T15:45:16Z

wwerner
Apr 26, 2022
Author

To wrap this up: We were able to use the new openapi handler from @omnigraph/[email protected] (which contains @ardatan's PR ^^^) on its own for stitching the Jira OAS API into an existing gateway.

Thanks so much for the support & patching right away! 🙌 🙏

1 reply

ardatan Apr 26, 2022
Maintainer

Sounds awesome! Please feel free to share your feedback anytime!

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Consuming Atlassian Jira API with new OpenAPI handler #3852

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 4 comments 5 replies

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

Select a reply

Consuming Atlassian Jira API with new OpenAPI handler #3852

wwerner Apr 20, 2022

1

Reproduction

Errors

2

Reproduction

Details

3

Reproduction

Details

4

Replies: 4 comments · 5 replies

ardatan Apr 20, 2022 Maintainer

wwerner Apr 20, 2022 Author

ardatan Apr 21, 2022 Maintainer

wwerner Apr 21, 2022 Author

wwerner Apr 21, 2022 Author

ardatan Apr 21, 2022 Maintainer

wwerner Apr 21, 2022 Author

wwerner Apr 26, 2022 Author

ardatan Apr 26, 2022 Maintainer

wwerner
Apr 20, 2022

Replies: 4 comments 5 replies

ardatan
Apr 20, 2022
Maintainer

wwerner
Apr 20, 2022
Author

ardatan
Apr 21, 2022
Maintainer

wwerner Apr 21, 2022
Author

wwerner Apr 21, 2022
Author

ardatan Apr 21, 2022
Maintainer

wwerner Apr 21, 2022
Author

wwerner
Apr 26, 2022
Author

ardatan Apr 26, 2022
Maintainer