Getting Started Example does not fail on the callback-example

[mandybuchhold]

I tested the example https://github.com/RepreZen/KaiZen-OpenApi-Parser/blob/master/GettingStarted.md and wanted especially to test the callback-example(a file that should fail) so that I can see what error messages I get. But I don't get any errors.

In the code normaly i expect that model.isValid() is false, but it is always true when i checked it. I even checked the callback-example in https://editor.swagger.io/, if it is really not valid and it is not valid. I have a similar problem in my project with an other file, that i expected to be invalid, but there i got no error.

Does the validation from the KaiZen-OpenApi-Parser work different than in the Swagger Editor?

Reproduction:
I just run this example https://github.com/RepreZen/KaiZen-OpenApi-Parser/blob/master/GettingStarted.md in a new project and get no errors.

[jdiazgon]

I can confirm that it is possible to reproduce this error.

[ghillairet]

@mandybuchhold What are the errors you are seeing in swagger editor?
I just checked the callback-example (this one https://github.com/OAI/OpenAPI-Specification/blob/master/examples/v3.0/callback-example.yaml) in https://editor.swagger.io/ and I am not getting any errors.

[mandybuchhold]

I'm really sorry i think i made a mistake when i checked this in the swagger editor, there are no errors.
I think i mixed something up because normaly this example should fail like it says in the getting Started guide. Maybe something changed in the code, it would be nice if there could be a new example that fails in the parser.

Here I changed the example code so that it fails in the swagger editor but not in the kaizen parser:

info:
  title: Callback Example
  version: 1.0.0
paths:
  /streams:
    post:
      description: subscribes a client to receive out-of-band data
      parameters:
        - name: callbackUrl
          in: query
          required: true
          description: |
            the location where data will be sent.  Must be network accessible
            by the source server
          schema:
            type: string
            format: uri
            example: https://tonys-server.com
            properties:
              errorList:
                type: array
                items:
                  $ref: "#/components/schemas/ErrorResponse_errorList"
      responses:
        '201':
          description: subscription successfully created
          content:
            application/json:
              schema:
                description: subscription information
                required:
                  - subscriptionId
                properties:
                  subscriptionId:
                    description: this unique identifier allows management of the subscription
                    type: string
                    example: 2531329f-fb09-4ef7-887e-84e648214436
      callbacks:
        # the name `onData` is a convenience locator
        onData:
          # when data is sent, it will be sent to the `callbackUrl` provided
          # when making the subscription PLUS the suffix `/data`
          '{$request.query.callbackUrl}/data':
            post:
              requestBody:
                description: subscription payload
                content:
                  application/json:
                    schema:
                      properties:
                        timestamp:
                          type: string
                          format: date-time
                        userData:
                          type: string
              responses:
                '202':
                  description: |
                    Your server implementation should return this HTTP status code
                    if the data was received successfully
                '204':
                  description: |
                    Your server should return this HTTP status code if no longer interested
                    in further updates

I checked the code and model.isValid is still true, even if the yaml file is false in the swagger editor.
In the swagger editor I get the error

'Semantic error at paths./streams.post.parameters.0.schema.properties.errorList.items.$ref $refs must reference a valid location in the document'

, but with the kaizen parser I don't get any error.

[jdiazgon]

I checked the code and model.isValid is still true, even if the yaml file is false in the swagger editor.
In the swagger editor I get the error

Any news on this?

[tedepstein]

@jdiazgon , @mandybuchhold , I think it's likely that what you're seeing is incomplete validation by KaiZen Parser. In its current state, the parser has a limited set of rules that it enforces. Each rule is in a class that you'll find in this package:

https://github.com/RepreZen/KaiZen-OpenApi-Parser/tree/master/kaizen-openapi-parser/src/main/java/com/reprezen/kaizen/oasparser/val3

There are problems in the example you posted. I suspect that those problems are not detected by the validator. You might want to try modifying the example by introducing a violation of a specific rule that you can see is being enforced by its corresponding validator class in the above-referenced package.

If I have time later today, I'll try this myself. And yes, we should update the Getting Started guide so that it includes a failing example to test validation. Thanks!

[jdiazgon]

Thanks for your quick answer @tedepstein.

Ideally, it would be nice to have the same validation rules that we get from editor.swagger.io, just like you can see below:

I have been searching how Swagger validates it's files, and it seems they have not exposed any public API to validate files. It's a pitty and I understand it will be very hard from your side to implement all the needed validation rules.

[tedepstein]

@jdiazgon , the Swagger Editor uses a combination of JSON schema validation and semantic validation, each of these implemented as plugins to the core editor.

Implementing all of the essential validation rules from the OpenAPI specification is on our roadmap, and I don't think it's actually difficult, only time-consuming to build these rules into KaiZen Parser's validator. I cannot promise a time frame to complete this work, as our team is currently occupied on other priorities.

If you would like to do some of this work to improve validation in KaiZen Parser, we would welcome the contribution! Please let me know if you're interested, and I'll make sure to provide whatever guidance is needed, and a feedback loop with our team to help you along the way.

[jdiazgon]

Unfortunately, I also don't have enough time to implement this feature. Anyway, thanks for your help and I will stay up to date with your latest news 👍