Brun-E Session Report Not Returning — Root Cause Analysis

Date: 2026-04-06 Status: Analysis complete — awaiting decision Severity: High (core feature broken)

Problem Statement

When a Brun-E session ends, the final report is not being returned to the frontend. The session appears to have all the information needed to produce a report, but final_report arrives as null on the complete response.

Architecture Context

┌─────────────────────────────────────────────────────────────────────┐
│  FRONTEND (session-runtime.ts)                                      │
│  ┌────────────┐    ┌──────────────┐    ┌─────────────────────────┐ │
│  │ WebRTC DC  │───▶│ Bridge       │───▶│ Socket.IO Sideband      │ │
│  │ (OpenAI)   │◀───│ fn_call ↔    │◀───│ (Nest Gateway)          │ │
│  └────────────┘    │ fn_result    │    └─────────────────────────┘ │
│                    └──────────────┘                                  │
│                          │                                          │
│  runtime.end() ──▶ POST /brun-e/:id/complete ──▶ finalReport        │
└─────────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────────┐
│  BACKEND                                                            │
│  ┌──────────────────┐    ┌──────────────────────────────────┐      │
│  │ HTTP Controller   │    │ Sideband Gateway                 │      │
│  │ POST /complete    │    │ function_call → Dispatcher        │      │
│  │ (no report)       │    │ → EndSessionSidebandHandler       │      │
│  └────────┬─────────┘    │   (extracts report from args)     │      │
│           │               └────────────┬─────────────────────┘      │
│           ▼                            ▼                            │
│  ┌──────────────────────────────────────────────────────────┐      │
│  │ CompleteBrunESessionHandler                               │      │
│  │ session.complete(reason, report?) → save → outbox         │      │
│  └──────────────────────────────────────────────────────────┘      │
└─────────────────────────────────────────────────────────────────────┘

Root Causes (ordered by impact)

RC-1: `end_session` tool has NO report schema (PRIMARY)

File: src/modules/brun-e/infrastructure/sideband/schemas/brun_e_sideband_tool.schemas.ts

The end_session tool definition sent to OpenAI Realtime:

{
  "type": "function",
  "name": "end_session",
  "description": "Closes the coaching session. Call this when the conversation reaches a natural conclusion or the user explicitly asks to stop.",
  "parameters": {
    "type": "object",
    "additionalProperties": false,
    "required": ["reason"],
    "properties": {
      "reason": { "type": "string", "enum": ["user", "timeout", "disconnect", "system"] },
      "report": { "type": "object", "additionalProperties": true }
    }
  }
}

Problems: 1. report is optional — not in required, model may omit it entirely 2. report has NO inner properties — just { type: "object", additionalProperties: true }. The model has zero guidance on what fields to generate (title, summary, emap_projection, insights, recommendations, alerts, confidence, etc.) 3. Tool description says nothing about generating a report — only "Closes the coaching session"

RC-2: OpenAI Realtime API does NOT support `strict` mode (CONFIRMED)

Unlike Chat Completions API which supports strict: true on tool definitions (guaranteeing schema compliance), the Realtime API does NOT support strict mode. Attempting strict: true produces an error.

This means even if we define a detailed schema, the model's compliance is best-effort only — no hard guarantees on output shape.

Source: OpenAI community + API reference (confirmed April 2026). The Realtime models (gpt-realtime, gpt-realtime-2025-08-28) don't support the structured outputs feature that strict relies on.

RC-3: System instructions are too vague about report format

File: e-training-front/lib/brun-e/session-runtime.ts (line 452)

Injected instruction:

"When concluding the session, ALWAYS call end_session with a complete session report before saying goodbye."

This tells the model to include a report but doesn't specify: - What fields are required - The exact nested structure (emap_projection with affective/effective/perspective) - Score ranges (0-1 floats) - What each field means

RC-4: Frontend parser is strict about `emap_projection`

File: e-training-front/lib/brun-e/json-api-parse.ts (line 100)

function parseFinalReport(raw: Record<string, unknown>): FinalReport | null {
  const projection = raw.emap_projection as Record<string, unknown> | undefined;
  if (!isRecord(projection)) return null;  // ← returns null if missing
  // ...
}

Even if the model sends a report with title, summary, insights, etc., if it doesn't include emap_projection with the exact nested structure (affective, effective, perspective each with score, label, insight), the frontend discards the entire report.

RC-5: User-initiated close has no report generation path

File: e-training-front/lib/brun-e/session-runtime.ts (line 510)

When user clicks "leave": 1. runtime.end("user") → HTTP POST /complete with { reason: "user" } 2. Backend controller: CompleteBrunESessionCommand(sessionId, userId, orgId, "user") — no report 3. Session completes with finalReport = null

There is no mechanism to trigger report generation when the user initiates the close. The only path that produces a report is the model calling end_session via sideband with a report argument.

RC-6: Race condition on model-initiated close

When the model does call end_session: 1. Sideband completes session with report ✓ 2. Frontend receives function_result for end_session 3. Frontend calls finishAfterRemoteEnd() → HTTP POST /complete (idempotent) 4. Backend returns already-completed session with stored finalReport ✓

This path should work if the model sends a properly shaped report. The issue is RC-1 through RC-3 preventing the model from generating a valid report.

Evidence Summary

Factor	Current State	Impact
Tool `report` schema	`{ type: "object", additionalProperties: true }` — no properties	Model doesn't know what to generate
Tool `report` required	No (optional)	Model may omit entirely
Tool description	"Closes the coaching session" — no mention of report	Model doesn't prioritize report generation
Strict mode	Not available in Realtime API	Cannot enforce schema compliance
System instructions	"call end_session with a complete session report"	Vague — no field specification
Frontend parser	Returns null if `emap_projection` missing	Even partial reports get discarded
User-initiated close	HTTP only, no report param	Always produces null report

Proposed Solutions

Option A: Enrich tool schema + system instructions (Quick fix, medium reliability)

Effort: ~2-4h | Risk: Low | Reliability: ~70-80%

Add detailed properties to end_session.report in brun_e_sideband_tool.schemas.ts:

report: {
  type: 'object',
  additionalProperties: false,
  required: [
    'title', 'summary', 'emap_projection', 'analysis_raw',
    'insights', 'recommendations', 'alerts', 'generalNote',
    'confidence', 'meta',
  ],
  properties: {
    title: { type: 'string', description: 'Short session title' },
    summary: { type: 'string', description: 'Concise recap of the session' },
    emap_projection: {
      type: 'object',
      required: ['affective', 'effective', 'perspective'],
      properties: {
        affective: {
          type: 'object',
          required: ['score', 'label', 'insight'],
          properties: {
            score: { type: 'number', minimum: 0, maximum: 1, description: 'Score 0.0-1.0' },
            label: { type: 'string', description: 'High/Medium/Low' },
            insight: { type: 'string', description: 'Explanation for score' },
          },
        },
        effective: { /* same structure as affective */ },
        perspective: { /* same structure as affective */ },
      },
    },
    analysis_raw: { type: 'string' },
    insights: { type: 'array', items: { type: 'string' } },
    recommendations: { type: 'array', items: { type: 'string' } },
    alerts: { type: 'array', items: { type: 'string' } },
    generalNote: { type: 'string' },
    confidence: { type: 'number', minimum: 0, maximum: 1 },
    meta: {
      type: 'object',
      required: ['schema_version', 'generated_at'],
      properties: {
        schema_version: { type: 'string' },
        generated_at: { type: 'string', format: 'date-time' },
      },
    },
  },
}

Make report required in end_session.arguments.required

Enhance tool description:

"Closes the coaching session and generates the final assessment report. MUST include a complete report object with emap_projection scores (0.0-1.0 scale), title, summary, insights, and recommendations based on the conversation."

Enhance system instructions (session-runtime.ts) to include example schema

Pros: Simple, no new infrastructure, immediate improvement. Cons: Without strict, still best-effort. Model may still deviate. No fallback for user-initiated close.

Option B: Server-side report generation via Chat Completions (Robust, high reliability)

Effort: ~6-8h | Risk: Medium | Reliability: ~95%+

Add a post-session report generation step on the backend using Chat Completions API with response_format: json_schema (which DOES support strict: true).

Flow:

Session ends (any path)
       │
       ▼
CompleteBrunESessionHandler
       │
       ├── If sideband provided valid report → use it ✓
       │
       └── If report is null/invalid → trigger ChatCompletionsReportGenerator
              │
              ▼
       POST /v1/chat/completions
         model: gpt-4o
         response_format: { type: "json_schema", strict: true, schema: FinalReportSchema }
         messages: [system prompt + conversation summary from session data]
              │
              ▼
       Store generated report → return in /complete response

Implementation: 1. New port: ISessionReportGenerator 2. New adapter: ChatCompletionsReportGeneratorAdapter using Chat Completions with structured output 3. Modify CompleteBrunESessionHandler to trigger report generation when report is missing 4. The conversation context can come from: session metadata, sideband function call history, or OpenAI stored prompt context

Pros: Guarantees report shape via strict: true. Works for all close paths (user, model, expiry). Decouples report quality from Realtime API limitations. Cons: Additional API call cost. Slight latency increase. Needs conversation context pipeline.

Option C: Hybrid — Enriched schema + Chat Completions fallback (Recommended)

Effort: ~8-12h | Risk: Low-Medium | Reliability: ~98%+

Combines Option A and Option B:

Enrich tool schema + instructions (Option A) — makes the model produce valid reports ~70-80% of the time via Realtime
Server-side fallback (Option B) — when the Realtime model fails to produce a valid report, generate one via Chat Completions with strict structured output
Frontend parser resilience — make parseFinalReport more tolerant, log what the model actually sends for debugging

Flow:

end_session tool call arrives
       │
       ├── Has valid report with emap_projection? → complete with report ✓
       │
       └── Report missing/malformed?
              │
              ▼
       Generate via Chat Completions (strict schema)
              │
              ▼
       complete with generated report ✓

User-initiated close (HTTP only)
       │
       ▼
       Generate via Chat Completions (strict schema)
       complete with generated report ✓

Pros: Maximum reliability. Covers all edge cases. Graceful degradation. Cons: Highest effort. Chat Completions fallback needs conversation context.

Option D: Prompt-only fix via OpenAI Stored Prompt (Lowest effort, least reliable)

Effort: ~1h | Risk: Low | Reliability: ~50-60%

Only modify the stored prompt in OpenAI dashboard to include explicit report JSON template and examples. Don't change code.

Pros: Zero code changes. Instant deployment. Cons: Lowest reliability. No schema enforcement. Doesn't fix user-initiated close.

Recommendation

Start with Option A (immediate, ~2-4h) to unblock sessions and get ~70-80% reliability. Then implement Option B as a follow-up to reach ~95%+ and cover the user-initiated close path.

This gives you: - Day 1: Working reports for model-initiated closes (most sessions) - Week 1: Guaranteed reports for ALL close paths

Files to Modify

Option A (Quick Fix)

File	Change
`back/src/modules/brun-e/infrastructure/sideband/schemas/brun_e_sideband_tool.schemas.ts`	Add detailed `report` properties, make required
`back/src/modules/brun-e/infrastructure/adapters/brun_e_openai_tools.ts`	Update tool description for `end_session`
`front/lib/brun-e/session-runtime.ts`	Enhance injected system instructions with report schema example
`front/lib/brun-e/json-api-parse.ts`	Add logging when report parsing fails; consider graceful degradation
`back/src/modules/brun-e/infrastructure/sideband/schemas/brun_e_sideband_schema.contract.spec.ts`	Update tests for new required fields
`back/src/modules/brun-e/infrastructure/adapters/brun_e_openai_tools.spec.ts`	Verify tool mapping still correct

Option B (Robust Fallback) — additional files

File	Change
`back/src/modules/brun-e/application/ports/session_report_generator.port.ts`	New port interface
`back/src/modules/brun-e/infrastructure/adapters/chat_completions_report_generator.adapter.ts`	New adapter using Chat Completions
`back/src/modules/brun-e/application/commands/complete_session/complete_brun_e_session.handler.ts`	Add fallback report generation
`back/src/modules/brun-e/brun_e.module.ts`	Register new port/adapter
`back/src/modules/brun-e/infrastructure/http/brun_e_http.controller.ts`	Potentially pass conversation context

Conversation Context for Option B

For the Chat Completions fallback to generate a meaningful report, it needs conversation context. Options:

OpenAI Stored Prompt + session ref — If the stored prompt already contains coaching context, pass the session ref to Chat Completions
Sideband function call log — The idempotency keys table stores function call metadata per session; reconstruct context from get_user_context results
Input audio transcription — Enable input_audio_transcription in the Realtime session config and store transcripts
Outbox event payload — The BrunESessionCompletedEvent could carry conversation metadata

Recommended: Start with option 2 (sideband function call log) as it requires minimal infrastructure changes and provides the user profile + emap context that the report needs.

Questions for Decision

Which option to pursue first? (A alone, A+B hybrid, or C full?)
User-initiated close: Should it trigger the model to call end_session first (prompt the model to wrap up) or go straight to server-side generation?
Conversation context: Do you want to enable audio transcription storage for richer report generation?
Cost tolerance: The Chat Completions fallback adds ~$0.01-0.05 per session. Acceptable?