Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Schema for Literal differs between version 2.6 and 2.7 #9402

Open
1 task done
vigneshmanick opened this issue May 6, 2024 · 4 comments
Open
1 task done

Schema for Literal differs between version 2.6 and 2.7 #9402

vigneshmanick opened this issue May 6, 2024 · 4 comments
Labels
bug V2 Bug related to Pydantic V2 pending Awaiting a response / confirmation

Comments

@vigneshmanick
Copy link

Initial Checks

  • I confirm that I'm using Pydantic V2

Description

The schema for the fields that are marked Literal is different between versions 2.6 and 2.7.

The version 2.7 has both enum and const as keys in the schema. See example below. Is this expected?

Example Code

from pydantic import BaseModel, Field
from typing import Literal

class Base(BaseModel):
    type: str = Field(
        ...,
    )

class MyModel(Base):
    type: Literal["modela"]

MyModel.model_json_schema()

#Pydantic v2.6
#{'properties': {'type': {'const': 'modela', 'title': 'Type'}},
# 'required': ['type'],
# 'title': 'MyModel',
# 'type': 'object'}

# Pydantic v2.7
#{'properties': {'type': {'const': 'modela',
#   'enum': ['modela'],
#   'title': 'Type',
#   'type': 'string'}},
# 'required': ['type'],
# 'title': 'MyModel',
# 'type': 'object'}

Python, Pydantic & OS Version

pydantic version: 2.7.1
pydantic-core version: 2.18.2
pydantic-core build: profile=release pgo=false
install path: /scratch/mv/myprograms/miniconda3/envs/pydantic2.7/lib/python3.11/site-packages/pydantic
python version: 3.11.9 | packaged by conda-forge | (main, Apr 19 2024, 18:36:13) [GCC 12.3.0]
platform: Linux-3.10.0-1160.45.1.el7.x86_64-x86_64-with-glibc2.17
related packages: typing_extensions-4.11.0
commit: unknown
@vigneshmanick vigneshmanick added bug V2 Bug related to Pydantic V2 pending Awaiting a response / confirmation labels May 6, 2024
@vigneshmanick vigneshmanick changed the title Schema for Literal differts between version 2.6 and 2.7 Schema for Literal differs between version 2.6 and 2.7 May 6, 2024
@lindapaiste
Copy link

The version 2.7 has both enum and const as keys in the schema. See example below. Is this expected?

Worth noting that @dmontagu raised concerns when this was introduced in #8944

However, I think we should be careful about merging this, as any change to the JSON schema usually comes with some complaints. In particular, it's probably a good idea to track down the initial commit adding the use of constant and determine if it might be a problem to include the duplicate information about the enum value.

Having both enum and const properties isn't valid under the OpenAPI spec and it breaks some of the tools that consume the JSON output.

I'm having similar problems but in my case I am not using the Literal type at all. The const property will get added to all Enum types which have only one value, whether or not Literal is used. Is that supposed to be happening?

result['enum'] = expected
if len(expected) == 1:
result['const'] = expected[0]

@vigneshmanick
Copy link
Author

thanks for the info. I feel this should not be the case? it should either be const or enum but not both for the same field.

In our case, our documentation generation script output the field type to have 2 available values modela and again enum: modela which is confusing for those reading the documentation. I feel it should either have one value only or atleast this be an opt in

@lindapaiste
Copy link

Consider the following code example:

from enum import Enum

from pydantic import BaseModel

from typing_extensions import Literal


class SpeciesEnum(str, Enum):
    CAT = 'cat'


class PetWithEnum(BaseModel):
    name: str
    species: SpeciesEnum


class PetWithLiteral(BaseModel):
    name: str
    species: Literal['cat']


print(PetWithEnum.model_json_schema())

print(PetWithLiteral.model_json_schema())

Here's the raw output across 3 different Pydantic versions.

Version 1.10 used 'enum': ['cat'], 'type': 'string'

PetWithEnum
{'title': 'PetWithEnum', 'type': 'object', 'properties': {'name': {'title': 'Name', 'type': 'string'}, 'species': {'$ref': '#/definitions/SpeciesEnum'}}, 'required': ['name', 'species'], 'definitions': {'SpeciesEnum': {'title': 'SpeciesEnum', 'description': 'An enumeration.', 'enum': ['cat'], 'type': 'string'}}}

PetWithLiteral
{'title': 'PetWithLiteral', 'type': 'object', 'properties': {'name': {'title': 'Name', 'type': 'string'}, 'species': {'title': 'Species', 'enum': ['cat'], 'type': 'string'}}, 'required': ['name', 'species']}

Version 2.6 used 'const': 'cat', 'type': 'string'

PetWithEnum
{'$defs': {'SpeciesEnum': {'const': 'cat', 'title': 'SpeciesEnum', 'type': 'string'}}, 'properties': {'name': {'title': 'Name', 'type': 'string'}, 'species': {'$ref': '#/$defs/SpeciesEnum'}}, 'required': ['name', 'species'], 'title': 'PetWithEnum', 'type': 'object'}

PetWithLiteral
{'properties': {'name': {'title': 'Name', 'type': 'string'}, 'species': {'const': 'cat', 'title': 'Species'}}, 'required': ['name', 'species'], 'title': 'PetWithLiteral', 'type': 'object'}

Version 2.7 uses 'const': 'cat', 'enum': ['cat'], 'type': 'string'

PetWithEnum
{'$defs': {'SpeciesEnum': {'const': 'cat', 'enum': ['cat'], 'title': 'SpeciesEnum', 'type': 'string'}}, 'properties': {'name': {'title': 'Name', 'type': 'string'}, 'species': {'$ref': '#/$defs/SpeciesEnum'}}, 'required': ['name', 'species'], 'title': 'PetWithEnum', 'type': 'object'}

PetWithLiteral
{'properties': {'name': {'title': 'Name', 'type': 'string'}, 'species': {'const': 'cat', 'enum': ['cat'], 'title': 'Species', 'type': 'string'}}, 'required': ['name', 'species'], 'title': 'PetWithLiteral', 'type': 'object'}

In every version there is no fundamental difference between the model which uses a Literal string 'cat' and the model which uses a string Enum that happens to have only one value.

IMO, there probably should be more of a difference between the two setups. I would think that the Literal version would have 'const': 'cat' and the Enum version would have 'enum': ['cat'].

There needs to be some way for me to tell Pydantic what form of output I want. Maybe it's an option, maybe it's by using Literal or not.


Here's how the SpeciesEnum looks across the versions. IMO 2.7 is an improvement over 2.6 because 2.6 feels like a mistake -- this is defined as an Enum so it should be an enum whether it has one value or two.

Considering the code:

class SpeciesEnum1(str, Enum):
    CAT = 'cat'

class SpeciesEnum2(str, Enum):
    CAT = 'cat'
    DOG = 'dog'

Version 1.10 - both have enum only

'SpeciesEnum1': {
  'title': 'SpeciesEnum1',
  'description': 'An enumeration.',
  'enum': ['cat'],
  'type': 'string'
},
'SpeciesEnum2': {
  'title': 'SpeciesEnum2',
  'description': 'An enumeration.',
  'enum': ['cat', 'dog'],
  'type': 'string'
}

Version 2.6 - one has const only and the other has enum only

'SpeciesEnum1': {
  'const': 'cat',
  'title': 'SpeciesEnum1',
  'type': 'string'
},
'SpeciesEnum2': {
  'enum': ['cat', 'dog'],
  'title': 'SpeciesEnum2',
  'type': 'string'
}

Version 2.7 - one has both const and enum and the other has enum only

'SpeciesEnum1': {
  'const': 'cat',
  'enum': ['cat'],
  'title': 'SpeciesEnum1',
  'type': 'string'
}, 
'SpeciesEnum2': {
  'enum': ['cat', 'dog'],
  'title': 'SpeciesEnum2',
  'type': 'string'
}

@austinyu
Copy link

austinyu commented May 9, 2024

According to the specification of json schema for enum and const,

"Use of this keyword is functionally equivalent to an enum with a single value."

From json schema perspective, it does not matter if the generated json schema is enum or const or both.
However, if there are considerable amount of consumers of the generated schema that distinguish between the two, we should distinguish enum with one value from const. Currently, we are kind of relying on Literal type hint for both cases because there is no type hint for const yet. If we need it, may be we can do something like this?

class DiffModel(BaseModel):
    const_field: CONST['this_is_constant']
    lit_field: Literal['this_is_constant']

Output of json schema:

'const_field': {
  'const': 'this_is_constant',
  'title': 'ConstField',
  'type': 'string'
}, 
'lit_field': {
  'enum': ['this_is_constant'],
  'title': 'LitField',
  'type': 'string'
}

Explicitly providing an annotation for people who actually need const json schema validation keyword in my opinion is a more scalable solution than keep mutating the behavior of Literal annotation,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug V2 Bug related to Pydantic V2 pending Awaiting a response / confirmation
Projects
None yet
Development

No branches or pull requests

3 participants