A deliberately broken Text-to-SQL API. Three planted bugs. One rule for the agent: no guessing — every fix must be backed by evidence from Okahu Cloud telemetry, fetched through the hosted Okahu MCP.
$ pytest test_analyst.py -v FAILED test_generate_sql_returns_select — openai.NotFoundError @analyst_v3 › querying hosted Okahu MCP… trace #a91f… request span: model="gpt-5.4-typo" → 404 @analyst_v3 › archiving to versions/analyst_v1.py, applying fix… fix analyst.py: model = "gpt-4o" $ pytest test_analyst.py -v PASSED 5/5 — all bugs healed. Final report with trace IDs ready. ✅
Most AI-coding demos show an agent building an app from scratch. This POC flips it: the agent inherits a broken production service and must repair it using the observability stack alone. Every LLM call in the app is auto-instrumented by Monocle and exported to Okahu Cloud, so the traces — not the terminal — are the single source of truth.
The agent cannot guess fixes. It must call /okahu:get_latest_traces on the hosted
Okahu MCP and quote the exact span evidence that reveals each root cause.
Observability arrives via a hosted MCP server. The agent queries the platform the same way it uses any other tool — no local log spelunking, no debug files.
Because the app calls the official openai SDK, Monocle captures every prompt,
response, model name, and error — with zero manual instrumentation code.
pytest test_analyst.py -v — watch them fail.
Query Okahu MCP for workflow text_to_sql_analyst_v3.
Find the failing request/response evidence in the trace.
Snapshot to versions/analyst_vN.py, patch the bug, record the trace ID.
Loop until 5/5 pass, then emit a report of fixes + trace IDs.
Each bug hides the next — the agent can't shortcut its way to green. Fixing the model name reveals the response-parsing bug; fixing that reveals the schema mismatch.
The analyst requests gpt-5.4-typo — a model that doesn't exist. Every single call
404s before a token is generated.
Trace clue: the request span's attributes show the bogus model string sent to OpenAI.
The code reads response.choices[0].text — the legacy Completions shape. Chat
completions keep content at .message.content.
Trace clue: the response span shows exactly where the content actually lives.
The system prompt teaches the LLM a customers/products schema — but
sales.db actually contains users and orders.
Trace clue: the captured LLM output shows SQL targeting tables that don't exist.
Monocle (a Linux Foundation project) auto-instruments supported GenAI SDKs and exports spans to Okahu Cloud. This only works if the app talks to the LLM through a supported SDK:
| Instrumented by Monocle ✅ | Invisible to Monocle ❌ |
|---|---|
| openai Python SDK | Raw requests.post() to LLM APIs |
| google-genai SDK | Direct httpx / aiohttp calls |
| langchain framework | Custom API wrappers w/o SDK instrumentation |
| llama-index framework |
from openai import OpenAI from monocle_apptrace import setup_monocle_telemetry setup_monocle_telemetry(workflow_name="text_to_sql_analyst_v3") # spans → Okahu Cloud client = OpenAI(api_key=os.getenv("OPENAI_API_KEY")) response = client.chat.completions.create( model="gpt-4o", messages=[...], ) answer = response.choices[0].message.content # not .text 😉
You'll need an OpenAI API key, an Okahu API key, and OpenCode with the hosted Okahu MCP configured. Full details in the README.
# 1. Clone + configure git clone https://github.com/tirth1263/telemetry-mcp-okahu.git cd telemetry-mcp-okahu cp .env.example .env # add your OpenAI + Okahu keys # 2. Install + seed python3 -m venv venv && source venv/bin/activate pip install -r requirements.txt python setup_db.py # 3. Reset to the buggy state (before every demo) python reset_demo.py # 4. Unleash the self-healing agent in OpenCode "@analyst_v3 Fix the buggy Text-to-SQL API ..."