While handling marshmallow ValidationError containing other ValidationError(s), make_response fails

[tarunchhn]

If marshmallow dataclass definition has a field of union type like
a: B | C

if the input doesnt match either A or B, the ValidationError thrown contains other validationError(s), which apiflask fails to handle and throws internal server error instead.

ValidationError: {'orders': {0: {'destination': [ValidationError({'geometry': ['Missing data for required field.']}), ValidationError({'geometry': ['Missing data for required field.']}), ValidationError({'geofence_id': ['Missing data for required field.'], 'address': ['Unknown field.']})]}}}
  File "webargs/core.py", line 433, in parse
    data = self._process_location_data(
  File "webargs/core.py", line 387, in _process_location_data
    data = schema.load(preprocessed_data, **load_kwargs)
  File "__init__.py", line 768, in load
    all_loaded = super().load(data, many=many, **kwargs)
  File "marshmallow/schema.py", line 722, in load
    return self._do_load(
  File "marshmallow/schema.py", line 909, in _do_load
    raise exc

_ValidationError: Validation error
  File "flask/app.py", line 1823, in full_dispatch_request
    rv = self.dispatch_request()
  File "flask/app.py", line 1799, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "flask_httpauth.py", line 174, in decorated
    return self.ensure_sync(f)(*args, **kwargs)
  File "webargs/core.py", line 637, in wrapper
    parsed_args = self.parse(
  File "webargs/core.py", line 437, in parse
    self._on_validation_error(
  File "webargs/core.py", line 263, in _on_validation_error
    error_handler(
  File "apiflask/scaffold.py", line 46, in handle_error
    raise _ValidationError(

TypeError: Object of type ValidationError is not JSON serializable
  File "flask/app.py", line 2528, in wsgi_app
    response = self.full_dispatch_request()
  File "flask/app.py", line 1825, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "flask_cors/extension.py", line 176, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "flask/app.py", line 2156, in make_response
    rv = self.json.response(rv)
  File "flask/json/provider.py", line 309, in response
    f"{self.dumps(obj, **dump_args)}\n", mimetype=mimetype
  File "flask/json/provider.py", line 230, in dumps
    return json.dumps(obj, **kwargs)
  File "__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "json/encoder.py", line 200, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "json/encoder.py", line 258, in iterencode
    return _iterencode(o, 0)
  File "flask/json/provider.py", line 122, in _default
    raise TypeError(f"Object of type {type(o).__name__} is not JSON serializable")

It can be handled using @app.error_processor, but it still is a bug

Environment:

• Python version: 3.11
• Flask version: 2.2.3
• APIFlask version: 2.0.1

[greyli]

Hi, could you provide a minimal example to reproduce this issue?

[tarunchhn]

test.py

from apiflask import APIFlask
from marshmallow_dataclass import dataclass

app = APIFlask(__name__)


@dataclass
class Req:
    var1: int | str


@app.post("/")
@app.input(Req.Schema)
def index():
    return {"message": "lol"}

shell

flask --app test:app run

req

curl http://127.0.0.1:5000 -H 'Content-Type: application/json' -d '{"var1": true}'

output:

* Serving Flask app 'test:app'
 * Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on http://127.0.0.1:5000
Press CTRL+C to quit
[2024-02-05 12:37:31,799] ERROR in app: Exception on / [POST]
Traceback (most recent call last):
  File "/Users/tarun/workspace/feature-foundation/venv/lib/python3.11/site-packages/flask/app.py", line 2528, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tarun/workspace/feature-foundation/venv/lib/python3.11/site-packages/flask/app.py", line 1826, in full_dispatch_request
    return self.finalize_request(rv)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tarun/workspace/feature-foundation/venv/lib/python3.11/site-packages/flask/app.py", line 1845, in finalize_request
    response = self.make_response(rv)
               ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tarun/workspace/feature-foundation/venv/lib/python3.11/site-packages/flask/app.py", line 2156, in make_response
    rv = self.json.response(rv)
         ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tarun/workspace/feature-foundation/venv/lib/python3.11/site-packages/flask/json/provider.py", line 309, in response
    f"{self.dumps(obj, **dump_args)}\n", mimetype=mimetype
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tarun/workspace/feature-foundation/venv/lib/python3.11/site-packages/flask/json/provider.py", line 230, in dumps
    return json.dumps(obj, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
          ^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/encoder.py", line 200, in encode
    chunks = self.iterencode(o, _one_shot=True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/encoder.py", line 258, in iterencode
    return _iterencode(o, 0)
           ^^^^^^^^^^^^^^^^^
  File "/Users/tarun/workspace/feature-foundation/venv/lib/python3.11/site-packages/flask/json/provider.py", line 122, in _default
    raise TypeError(f"Object of type {type(o).__name__} is not JSON serializable")
TypeError: Object of type ValidationError is not JSON serializable
127.0.0.1 - - [05/Feb/2024 12:37:31] "POST / HTTP/1.1" 500 -

[uncle-lv]

I have found the reason.

When union fields raise errors, marshmallow_dataclass adds the error objects into errors directly.
But marshmallow hints errors should be dict[str, list[str]].
marshmallow_dataclass doesn't obey it.

[greyli]

Thanks for the example. After some investigation, I find the root cause of this issue. When handling an error with union type, the error message generated by marshmallow-dataclass will be a list of ValidationError objects.

https://github.com/lovasoa/marshmallow_dataclass/blob/d6396c18470582a4fe5f0f2bd29ac012da4f0f1f/marshmallow_dataclass/union_field.py#L80

[greyli]

We could report it to marshmallow-dataclass or help them fix it.