Python FastAPI state not exist on Vercel but does locally - lifespan

This follows on from Python typings packages breaks FastAPI on Vercel - #3 by brooke-tintunacom.

After I fixed that problem (using python package for Typing) and solving some other problems I now have a problem where the State I’m storing does not exist when the app runs on Vercel, however it works fine when running locally. Any ideas?

The code is:

$ cat requirements.txt
python-dotenv
openai
instructor
rich
pydantic
fastapi
mangum
uvicorn
atomic-agents

$ cat vercel.json
{
    "version": 2,
    "builds": [
     {
      "src": "app/main.py",
      "use": "@vercel/python"
     }
    ],
    "routes": [
     {
      "src": "/(.*)",
      "dest": "app/main.py"
     }
    ]
   }

$ cat app/lifespan.py
from contextlib import asynccontextmanager
from app.llm.atomic_agents_service import AtomicAgentsService

@asynccontextmanager
async def lifespan(app):
    # Load the LLM service - do it this way so that it is loaded once and not on every request
    atomic_agents_service = AtomicAgentsService()
    print(" - created atomic_agents_service: ", atomic_agents_service)
    app.state.atomic_agents_service = atomic_agents_service
    yield
    # Clean up the LLM service and release the resources
    app.state.atomic_agents_service = None

$ cat app/main.py
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from mangum import Mangum
from .routes import router as api_router
from app.lifespan import lifespan

app = FastAPI(lifespan=lifespan)

# CORS configuration
app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],  # Allows all origins
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

app.include_router(api_router)

@app.get("/")
async def read_root():
    return {"health": "up"}

$ cat app/routes.py
from app.atomic_agents_data import AtomicAgentMemoryModel
from app.llm.atomic_agents_service import AtomicAgentsService
from rich.console import Console
import json

router = APIRouter()

console = Console()

@router.post("/choice")
async def post_choice(request: Request, choice: str = None, memory: AtomicAgentMemoryModel = None):
    console.print("/choice")
    console.print(" - choice: ", choice)
    console.print(" - memory: ", memory.json()) if memory else console.print(" - memory: None")
    console.print("IN BETWEEN")
    console.print(" - request.app.state: ", request.app.state)
    console.print(" - request.app.state dir: ", dir(request.app.state))
    console.print(" - request.app.state state: ", request.app.state._state)
    console.print(" - atomic_agents_service: ", request.app.state.atomic_agents_service)
    # `memory` will automatically be deserialized into an AtomicAgentMemoryModel instance
    if (memory is None or memory.history is None or len(memory.history) == 0):
        response = request.app.state.atomic_agents_service.start()
    else:
        response = request.app.state.atomic_agents_service.post_choice(choice, memory)

    # Perform your logic here
    return {"request": choice, "response": response}

Locally the terminal shows:

AtomicAgentsService - init
 - created atomic_agents_service:  <app.llm.atomic_agents_service.AtomicAgentsService object at 0x1087fd940>
...
/choice
 - choice:  None
 - memory: None
IN BETWEEN
 - request.app.state:  <starlette.datastructures.State object at 0x1087cf0e0>
 - request.app.state state:
{'atomic_agents_service': <app.llm.atomic_agents_service.AtomicAgentsService object at 0x1087fd940>}
 - atomic_agents_service:  <app.llm.atomic_agents_service.AtomicAgentsService object at 0x1087fd940>

But the vercel runtime log shows (just request and message):

# NOTE --> " - created atomic_agents_service:  ..." is misssing <-- NOTE
/choice    127.0.0.1 - - [30/Nov/2024 04:44:52] "POST /choice HTTP/1.1" 500 -
/choice.   - request.app.state state: {}
/choice.  - request.app.state: <starlette.datastructures.State object at 0x7f5d466da810>
/choice.   IN BETWEEN
/choice    - memory: None
/choice.   - choice: None

It looks like the lifespan is not executing. There is no - atomic_agents_service: xxxxx in the Vercel runtime log.

Any ideas?

1 post - 1 participant

Read full topic