Skip to content

AI Auto sort causes server to crash #4285

@pantherale0

Description

@pantherale0

Tandoor Version

2.3.5

Setup

Docker / Docker-Compose

Reverse Proxy

Caddy

Other

No response

Bug description

When attempting to use the AI auto sort feature with Gemini, the request appears to timeout for some reason, which causes gunicorn to restart.

Tandoor is configured to run as a systemd service so I believe its systemd that is restarting the process rather than tandoor itself.

Relevant logs

[2025-12-05 16:28:43 +0000] [171] [DEBUG] POST /api/ai-step-sort/
[2025-12-05 16:29:14 +0000] [94] [CRITICAL] WORKER TIMEOUT (pid:171)
[2025-12-05 16:29:14 +0000] [171] [ERROR] Error handling request /api/ai-step-sort/?provider=1
Traceback (most recent call last):
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/gunicorn/workers/sync.py", line 134, in handle
    self.handle_request(listener, req, client, addr)
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/gunicorn/workers/sync.py", line 177, in handle_request
    respiter = self.wsgi(environ, resp.start_response)
  File "/opt/tandoor/recipes/wsgi.py", line 34, in application
    return _application(environ, start_response)
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/django/core/handlers/wsgi.py", line 124, in __call__
    response = self.get_response(request)
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/django/core/handlers/base.py", line 140, in get_response
    response = self._middleware_chain(request)
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/corsheaders/middleware.py", line 56, in __call__
    result = self.get_response(request)
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/django/utils/deprecation.py", line 120, in __call__
    response = response or self.get_response(request)
                           ~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/whitenoise/middleware.py", line 123, in __call__
    return self.get_response(request)
           ~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/django/utils/deprecation.py", line 120, in __call__
    response = response or self.get_response(request)
                           ~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/allauth/usersessions/middleware.py", line 18, in __call__
    response = self.get_response(request)
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/django/utils/deprecation.py", line 120, in __call__
    response = response or self.get_response(request)
                           ~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/django/utils/deprecation.py", line 120, in __call__
    response = response or self.get_response(request)
                           ~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/django/utils/deprecation.py", line 120, in __call__
    response = response or self.get_response(request)
                           ~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/django/utils/deprecation.py", line 120, in __call__
    response = response or self.get_response(request)
                           ~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/django/utils/deprecation.py", line 120, in __call__
    response = response or self.get_response(request)
                           ~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/django/utils/deprecation.py", line 120, in __call__
    response = response or self.get_response(request)
                           ~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/opt/tandoor/cookbook/helper/scope_middleware.py", line 71, in __call__
    return self.get_response(request)
           ~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/allauth/account/middleware.py", line 36, in middleware
    response = get_response(request)
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/django/views/decorators/csrf.py", line 65, in _view_wrapper
    return view_func(request, *args, **kwargs)
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/django/views/generic/base.py", line 105, in view
    return self.dispatch(request, *args, **kwargs)
           ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/rest_framework/views.py", line 512, in dispatch
    response = handler(request, *args, **kwargs)
  File "/opt/tandoor/cookbook/views/api.py", line 2641, in post
    ai_response = completion(**ai_request)
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/litellm/utils.py", line 1113, in wrapper
    result = original_function(*args, **kwargs)
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/litellm/main.py", line 2356, in completion
    response = vertex_chat_completion.completion(  # type: ignore
        model=model,
    ...<17 lines>...
        extra_headers=extra_headers,
    )
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py", line 1285, in completion
    response = client.post(url=url, headers=headers, json=data)  # type: ignore
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/litellm/llms/custom_httpx/http_handler.py", line 558, in post
    response = self.client.send(req, stream=stream)
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/httpx/_client.py", line 914, in send
    response = self._send_handling_auth(
        request,
    ...<2 lines>...
        history=[],
    )
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/httpx/_client.py", line 942, in _send_handling_auth
    response = self._send_handling_redirects(
        request,
        follow_redirects=follow_redirects,
        history=history,
    )
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/httpx/_client.py", line 979, in _send_handling_redirects
    response = self._send_single_request(request)
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/httpx/_client.py", line 1014, in _send_single_request
    response = transport.handle_request(request)
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/httpx/_transports/default.py", line 250, in handle_request
    resp = self._pool.handle_request(req)
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/httpcore/_sync/connection_pool.py", line 256, in handle_request
    raise exc from None
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/httpcore/_sync/connection_pool.py", line 236, in handle_request
    response = connection.handle_request(
        pool_request.request
    )
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/httpcore/_sync/connection.py", line 103, in handle_request
    return self._connection.handle_request(request)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/httpcore/_sync/http11.py", line 136, in handle_request
    raise exc
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/httpcore/_sync/http11.py", line 106, in handle_request
    ) = self._receive_response_headers(**kwargs)
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/httpcore/_sync/http11.py", line 177, in _receive_response_headers
    event = self._receive_event(timeout=timeout)
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/httpcore/_sync/http11.py", line 217, in _receive_event
    data = self._network_stream.read(
        self.READ_NUM_BYTES, timeout=timeout
    )
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/httpcore/_backends/sync.py", line 128, in read
    return self._sock.recv(max_bytes)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/root/.local/share/uv/python/cpython-3.13.9-linux-x86_64-gnu/lib/python3.13/ssl.py", line 1285, in recv
    return self.read(buflen)
           ~~~~~~~~~^^^^^^^^
  File "/root/.local/share/uv/python/cpython-3.13.9-linux-x86_64-gnu/lib/python3.13/ssl.py", line 1140, in read
    return self._sslobj.read(len)
           ~~~~~~~~~~~~~~~~~^^^^^
  File "/opt/tandoor/.venv/lib/python3.13/site-packages/gunicorn/workers/base.py", line 204, in handle_abort
    sys.exit(1)
    ~~~~~~~~^^^
SystemExit: 1
[2025-12-05 16:29:14 +0000] [171] [INFO] Worker exiting (pid: 171)
Running django-vite in production mode (no HMR)
[2025-12-05 16:29:14 +0000] [321] [INFO] Booting worker with pid: 321

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions