Skip to content

[BUG] RecursionError when attempting to capture a traceback from an error (Rich v14.0.0) #3682

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
2 tasks done
jjallaire opened this issue Mar 30, 2025 · 6 comments · May be fixed by #3683
Open
2 tasks done

[BUG] RecursionError when attempting to capture a traceback from an error (Rich v14.0.0) #3682

jjallaire opened this issue Mar 30, 2025 · 6 comments · May be fixed by #3683

Comments

@jjallaire
Copy link

Describe the bug

After updating to Rich v14.0.0 I get a RecursionError when attempting to capture a traceback from an error.

Here's my function along with the helper function to create the traceback:

def eval_error(
    exception: BaseException,
    exc_type: Type[Any],
    exc_value: BaseException,
    exc_traceback: TracebackType | None,
) -> EvalError:
    # create ansi traceback
    with open(os.devnull, "w") as f:
        console = Console(record=True, file=f, legacy_windows=True)
        console.print(rich_traceback(exc_type, exc_value, exc_traceback))
        traceback_ansi = console.export_text(styles=True)
   
    # return error
    return EvalError(
        message=exception_message(exception),
        traceback=traceback_text,
        traceback_ansi=traceback_ansi,
    )

def rich_traceback(
    exc_type: Type[Any], exc_value: BaseException, exc_traceback: TracebackType | None
) -> RenderableType:
    rich_tb = Traceback.from_exception(
        exc_type=exc_type,
        exc_value=exc_value,
        traceback=exc_traceback,
        suppress=[click, asyncio, tenacity, sys.modules[PKG_NAME]],
        show_locals=False,
        width=CONSOLE_DISPLAY_WIDTH,
    )
    return rich_tb

Here's the stack trace for the RecursionError:

      | 
      | Traceback (most recent call last):
      |   File "/home/parallels/UKGovernmentBEIS/inspect_ai/src/inspect_ai/_eval/run.py", line 280, in run_task
      |     result = await task_run(tasks[index])
      |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      |   File "/home/parallels/UKGovernmentBEIS/inspect_ai/src/inspect_ai/_eval/task/run.py", line 389, in task_run
      |     error = eval_error(ex, type, value, traceback)
      |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      |   File "/home/parallels/UKGovernmentBEIS/inspect_ai/src/inspect_ai/log/_log.py", line 649, in eval_error
      |     console.print(rich_traceback(exc_type, exc_value, exc_traceback))
      |                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      |   File "/home/parallels/UKGovernmentBEIS/inspect_ai/src/inspect_ai/log/_log.py", line 665, in rich_traceback
      |     rich_tb = Traceback.from_exception(
      |               ^^^^^^^^^^^^^^^^^^^^^^^^^
      |   File "/home/parallels/UKGovernmentBEIS/inspect_ai/.venv/lib/python3.12/site-packages/rich/traceback.py", line 381, in from_exception
      |     rich_traceback = cls.extract(
      |                      ^^^^^^^^^^^^
      |   File "/home/parallels/UKGovernmentBEIS/inspect_ai/.venv/lib/python3.12/site-packages/rich/traceback.py", line 466, in extract
      |     Traceback.extract(
      |   File "/home/parallels/UKGovernmentBEIS/inspect_ai/.venv/lib/python3.12/site-packages/rich/traceback.py", line 466, in extract
      |     Traceback.extract(
      |   File "/home/parallels/UKGovernmentBEIS/inspect_ai/.venv/lib/python3.12/site-packages/rich/traceback.py", line 466, in extract
      |     Traceback.extract(
      |   [Previous line repeated 2950 more times]
      |   File "/home/parallels/UKGovernmentBEIS/inspect_ai/.venv/lib/python3.12/site-packages/rich/traceback.py", line 491, in extract
      |     iter_locals: Iterable[Tuple[str, object]],
      |                  ~~~~~~~~^^^^^^^^^^^^^^^^^^^^
      |   File "/usr/lib/python3.12/typing.py", line 395, in inner
      |     return _caches[func](*args, **kwds)
      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      | RecursionError: maximum recursion depth exceeded

Platform

What platform (Win/Linux/Mac) are you running on? What terminal software are you using?

Running on Linux, running in the VS Code integrated terminal

I may ask you to copy and paste the output of the following commands. It may save some time if you do it now.

If you're using Rich in a terminal:

python -m rich.diagnose
pip freeze | grep rich
╭─────────────────────── <class 'rich.console.Console'> ───────────────────────╮
│ A high level console interface.                                              │
│                                                                              │
│ ╭──────────────────────────────────────────────────────────────────────────╮ │
│ │ <console width=80 ColorSystem.TRUECOLOR>                                 │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│                                                                              │
│     color_system = 'truecolor'                                               │
│         encoding = 'utf-8'                                                   │
│             file = <_io.TextIOWrapper name='<stdout>' mode='w'               │
│                    encoding='utf-8'>                                         │
│           height = 20                                                        │
│    is_alt_screen = False                                                     │
│ is_dumb_terminal = False                                                     │
│   is_interactive = True                                                      │
│       is_jupyter = False                                                     │
│      is_terminal = True                                                      │
│   legacy_windows = False                                                     │
│         no_color = False                                                     │
│          options = ConsoleOptions(                                           │
│                        size=ConsoleDimensions(width=80, height=20),          │
│                        legacy_windows=False,                                 │
│                        min_width=1,                                          │
│                        max_width=80,                                         │
│                        is_terminal=True,                                     │
│                        encoding='utf-8',                                     │
│                        max_height=20,                                        │
│                        justify=None,                                         │
│                        overflow=None,                                        │
│                        no_wrap=False,                                        │
│                        highlight=None,                                       │
│                        markup=None,                                          │
│                        height=None                                           │
│                    )                                                         │
│            quiet = False                                                     │
│           record = False                                                     │
│         safe_box = True                                                      │
│             size = ConsoleDimensions(width=80, height=20)                    │
│        soft_wrap = False                                                     │
│           stderr = False                                                     │
│            style = None                                                      │
│         tab_size = 8                                                         │
│            width = 80                                                        │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─── <class 'rich._windows.WindowsConsoleFeatures'> ────╮
│ Windows features available.                           │
│                                                       │
│ ╭───────────────────────────────────────────────────╮ │
│ │ WindowsConsoleFeatures(vt=False, truecolor=False) │ │
│ ╰───────────────────────────────────────────────────╯ │
│                                                       │
│ truecolor = False                                     │
│        vt = False                                     │
╰───────────────────────────────────────────────────────╯
╭────── Environment Variables ───────╮
│ {                                  │
│     'CLICOLOR': None,              │
│     'COLORTERM': 'truecolor',      │
│     'COLUMNS': None,               │
│     'JPY_PARENT_PID': None,        │
│     'JUPYTER_COLUMNS': None,       │
│     'JUPYTER_LINES': None,         │
│     'LINES': None,                 │
│     'NO_COLOR': None,              │
│     'TERM_PROGRAM': 'vscode',      │
│     'TERM': 'xterm-256color',      │
│     'TTY_COMPATIBLE': None,        │
│     'VSCODE_VERBOSE_LOGGING': None │
│ }                                  │
╰────────────────────────────────────╯
platform="Linux"
rich==14.0.0
Copy link

Thank you for your issue. Give us a little time to review it.

PS. You might want to check the FAQ if you haven't done so already.

This is an automated reply, generated by FAQtory

@jjallaire jjallaire changed the title [BUG] [BUG] RecursionError when attempting to capture a traceback from an error (Rich v14.0.0) Mar 30, 2025
@willmcgugan
Copy link
Collaborator

I will need an MRE from you, if you would like me to look in to this.

@jjallaire
Copy link
Author

Okay, will do! It appears to have something to do with Python 3.11's ExceptionGroup (as it doesn't occur in Python 3.10 where we import ExceptionGroup from the exceptiongroup backport package).

@shyam-ramani shyam-ramani linked a pull request Mar 31, 2025 that will close this issue
@Textualize Textualize deleted a comment Apr 10, 2025
@Arnatious
Copy link

I've reproduced this is a very specific context, which I'm having trouble isolating the cause of. The key symptom involves the continue block at

rich/rich/traceback.py

Lines 575 to 580 in 8c4d3d1

if cause and not getattr(exc_value, "__suppress_context__", False):
exc_type = cause.__class__
exc_value = cause
traceback = cause.__traceback__
is_cause = False
continue

When the bug this occurs, extract() recursively re-invokes itself on the same ExceptionGroup, where the ExceptionGroup contains an arbitrary other Exception within it. exc_value.__context__ of the Exception refers back to the original ExceptionGroup, creating a cycle. This seems to occur under very specific async conditions. Opening it in the debugger shows it's just calling extract, going through the rest of the function normally until it hits that continue, then looping back to the top of the while block and recursive calling Extract on the same ExceptionGroup again.

Apologies I can't provide a minimal example yet - I've tried for hours to reproduce this in isolation without success. However, it consistently reproduces in our production stack, which uses uvicorn.serve running in asyncio, and starlette with and a custom middleware that injects a few headers.

Summary: the stop condition in extract seems insufficient, and can't account for cases where the context of an Exception is the ExceptionGroup containing it (per exc_value.__context__ at line 574). This happens in ASGI situations where an unhandled exception in a task group bubbles up and is reported from call_exception_handler in asyncio.base_events.

@Arnatious
Copy link

Arnatious commented May 21, 2025

EDIT: stripped a few more lines out, verified it also occurs with using asyncio taskgroups directly. This is all spawning from "unhandled error in taskgroup" so it's likely specific to how taskgroups form ExceptionGroups

Update - managed to strip it down to a shareable example, it's still dependent on several external dependencies, so I'm going to see if I can capture it a bit better elsewhere.

Dependencies:

  • fastapi 0.115.12
  • anyio 4.9.0
  • starlette 0.46.2
  • Python 3.12
import anyio

import logging.config
import uvicorn
import fastapi

from starlette.middleware.base import BaseHTTPMiddleware

logger = logging.getLogger(__name__)

app = fastapi.FastAPI()


@app.get("/api/v1/foo")
async def root():
    raise Exception("original exception")


async def run():
    async with anyio.create_task_group() as tg:
        app.add_middleware(BaseHTTPMiddleware)
        server = uvicorn.Server(uvicorn.Config(app, host="0.0.0.0", port=12345, log_config=None, log_level=None))
        tg.start_soon(server.serve)


def main():
    anyio.run(run, backend="asyncio")


if __name__ == "__main__":
    # Configure logging with RichHandler
    logging.config.dictConfig({
        "version": 1,
        "disable_existing_loggers": False,
        "handlers": {
            "console": {
                "class": "rich.logging.RichHandler",
                "level": "INFO",
                "rich_tracebacks": True,
            }
        },
        "root": {"handlers": ["console"], "level": "INFO"},
        "incremental": False,
    })
    main()

Then, reach the API with curl (you can use httpx and call it from within the task group but I didn't want to add more confusion)

$ curl -H "BAZ: qux" localhost:12345/api/v1/foo

Running this will cause the issue - it'll log

[05/21/25 16:55:29] INFO     Started server process [1521717]                                                                                                                                                                                                                                                           server.py:83
                    INFO     Waiting for application startup.                                                                                                                                                                                                                                                               on.py:48
                    INFO     Application startup complete.                                                                                                                                                                                                                                                                  on.py:62
                    INFO     Uvicorn running on http://0.0.0.0:12345 (Press CTRL+C to quit)                                                                                                                                                                                                                            server.py:215
[05/21/25 16:55:41] INFO     127.0.0.1:54526 - "GET /api/v1/foo HTTP/1.1" 500     

before it hangs for a moment and then another error about maximum recursion depth blows up the console.

The middleware is necessary, removing it causes it to just log an error as normal. If using a debugger, placing it in Traceback.extract() will show the loop - it's failing when being handed Exception("original exception"), which has no cause but has __context__ set to ExceptionGroup('unhandled errors in a TaskGroup', [Exception('original exception')]).

@Arnatious
Copy link

Arnatious commented May 21, 2025

For convenience, here's a version that calls the api for you and therefore requires no further input to trigger the error

import anyio

import logging.config
import uvicorn
import fastapi
import httpx

from starlette.middleware.base import BaseHTTPMiddleware

logger = logging.getLogger(__name__)

app = fastapi.FastAPI()


@app.get("/api/v1/foo")
async def root():
    raise Exception("original exception")


async def call_api():
    await anyio.sleep(2) # wait for the server to start
    async with httpx.AsyncClient() as client:
        await client.get("http://localhost:12345/api/v1/foo")


async def run():
    async with anyio.create_task_group() as tg:
        app.add_middleware(BaseHTTPMiddleware)
        server = uvicorn.Server(uvicorn.Config(app, host="0.0.0.0", port=12345, log_config=None, log_level=None))
        tg.start_soon(server.serve)
        tg.start_soon(call_api)


def main():
    anyio.run(run, backend="asyncio")


if __name__ == "__main__":
    # Configure logging with RichHandler
    logging.config.dictConfig({
        "version": 1,
        "disable_existing_loggers": False,
        "handlers": {
            "console": {
                "class": "rich.logging.RichHandler",
                "level": "INFO",
                "rich_tracebacks": True,
            }
        },
        "root": {"handlers": ["console"], "level": "INFO"},
        "incremental": False,
    })
    main()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants