# LIA - Complete System Brief

## Project Overview

**Lia** is an AI-powered WhatsApp assistant for **Lincoln Audi** dealer, designed to manage end-of-term (EOT) PCP/HP finance customer engagement via automated WhatsApp campaigns orchestrated through **Make.com** and **Twilio**.

**Live API URL**: `https://bacecfc8-69bb-4900-a1cd-dad6c50868e6-00-13bm1gzdpj9n6.riker.replit.dev:8000`

> **IMPORTANT**: Always include `:8000` in the URL - the Dashboard Server runs on port 8000

---

## API Authentication

### For Make.com Endpoints (External Automation)
```
Header: x-api-key: lia_renewal_2025_super_secret_1
```

### For Dashboard Endpoints (Internal UI)
- Session-based authentication via `dp_session` cookie
- Password stored in `LOYALTY_DASHBOARD_PASSWORD` secret

---

## Core API Endpoints

### 1. POST /api/lia/outbound
**Purpose**: Generate outbound WhatsApp campaign openers for customers

**Security**: `x-api-key` header (Make.com)

**Request Body**:
```json
{
  "customer_id": "uuid-string",
  "payments_remaining": 12,
  "campaign": "pcp_stage_auto",
  "asset_type": "new",
  "channel": "whatsapp",
  "sandbox_mode": false,
  "from_number": "+447700000000"
}
```

**Response**:
```json
{
  "message_text": "Hi John, it's Lia from Lincoln Audi...",
  "conversation_id": "uuid-string",
  "to_number": "+447700900123",
  "stage": "12"
}
```

**Campaign-Safety Gate**: Only processes if `dp_campaigns` has a matching row where:
- `type` = request.campaign
- `status` = 'running'
- `date_from` <= today (or NULL)
- `date_to` >= today (or NULL)

---

### 2. POST /api/lia/inbound
**Purpose**: Process customer replies using Universal Reply Engine (URE)

**Security**: `x-api-key` header (Make.com)

**Request Body**:
```json
{
  "from_number": "+447700900123",
  "body": "Hi, can I pop in on Saturday?",
  "channel": "whatsapp",
  "conversation_id": "uuid-string",
  "payments_remaining": 12
}
```

**Response**:
```json
{
  "conversation_id": "uuid-string",
  "mode": "APPOINTMENT_CONFIRMED",
  "status": "active",
  "defer_until": null,
  "reply": "Thanks John – I've booked you in for saturday at Lincoln Audi...",
  "stage": "12"
}
```

---

### 3. POST /api/lia/nudge
**Purpose**: Send follow-up nudge messages (no inbound required)

**Security**: `x-api-key` header (Make.com)

**Request Body**:
```json
{
  "conversation_id": "uuid-string",
  "nudge_type": "NUDGE_1_12_NEW",
  "payments_remaining": 12
}
```

**Response**:
```json
{
  "conversation_id": "uuid-string",
  "nudge_type": "NUDGE_1_12_NEW",
  "template_key": "NUDGE_1_12_NEW",
  "reply": "Hi [customer_name], just checking in..."
}
```

**Available Nudge Types**:
- Generic: `nudge_1`, `nudge_2`, `nudge_3` (maps to `NUDGE_1_EOT`, etc.)
- Stage-specific: `NUDGE_1_12_NEW`, `NUDGE_2_12_USED`, `NUDGE_1_9_NEW`, etc.

---

### 4. POST /api/conversations/{id}/escalate
**Purpose**: Flag conversation for manager intervention (pause + set escalation reason)

**Security**: `x-api-key` header (Make.com)

**Request Body**:
```json
{
  "reason": "unrecognised_message"
}
```

**Valid Reasons**:
| Reason | Dashboard Badge |
|--------|-----------------|
| `customer_issue` | 🔴 Red |
| `unrecognised_message` | 🟣 Purple |
| `appointment_confirmation_needed` | 🟠 Orange |

---

### 5. GET /api/conversations/{id}/messages
**Purpose**: Retrieve conversation message history

**Security**: `x-api-key` header (Make.com)

**Response**:
```json
{
  "conversation_id": "uuid-string",
  "message_count": 5,
  "messages": [
    {
      "id": "uuid-string",
      "direction": "outbound",
      "channel": "whatsapp",
      "sender": "lia",
      "message_body": "Hi John...",
      "template_key": "EOT_12_NEW",
      "mode": "outbound_campaign",
      "created_at": "2025-11-25T10:30:00Z"
    }
  ]
}
```

---

### 6. GET /api/conversations/{id}/status
**Purpose**: Get conversation status including escalation info

**Security**: Session-based (dashboard)

**Response**:
```json
{
  "conversation_id": "uuid-string",
  "status": "paused",
  "defer_until": null,
  "defer_reason": null,
  "escalation_reason": "appointment_confirmation_needed",
  "escalation_at": "2025-11-25T20:59:53.614620+00:00"
}
```

---

## Stage Selection Logic

Based on `payments_remaining` value:

| Payments Remaining | Stage | Description |
|--------------------|-------|-------------|
| 16+ | 18 | 18-month EOT (early planning) |
| 10-15 | 12 | 12-month EOT (peak equity window) |
| 7-9 | 9 | 9-month EOT (unused loyalty) |
| 4-6 | 6 | 6-month EOT (factory order deadline) |
| 1-3 | 3 | 3-month EOT (end-of-contract check) |

```python
def select_outbound_stage(payments_remaining):
    if payments_remaining >= 16:
        return "18"
    elif payments_remaining >= 10:
        return "12"
    elif payments_remaining >= 7:
        return "9"
    elif payments_remaining >= 4:
        return "6"
    else:
        return "3"
```

---

## Universal Reply Engine (URE) - The Brain

### Location: `app/lia_ure.py`

The URE is a **campaign-agnostic** reply generator that processes all customer inbound messages and generates appropriate responses based on:

1. **Message Classification** (Tier system)
2. **Appointment Detection** (specific vs vague times)
3. **Conversation State** (soft_no_count, confusion_count, etc.)

### Tier Classification System

| Tier | Name | Detection Keywords | Behaviour |
|------|------|-------------------|-----------|
| T6 | OPT_OUT | "stop", "unsubscribe", "not interested" | Immediate exit, update preferences |
| T5 | MONEY | "too expensive", "can't afford", "budget" | Acknowledge concern, offer showroom visit (phone fallback after 2 mentions) |
| T4 | SCHEDULED | "next week", "try me again", "contact me in" | Defer conversation, schedule follow-up |
| T3 | APPOINTMENT | Day + time detected | Book or offer time slots |
| T2 | SOFT_OBJECTION | "not right now", "maybe later", "bit busy" | Two-no rule: 1st → gentle persist, 2nd → graceful exit |
| T1 | GENERAL | Default | Positive engagement, push for showroom visit |

### Appointment Detection

#### Specific Times (Immediate Confirmation)
Detected via regex patterns:
- Clock format: `10:30`, `9:00`, `14:15`
- AM/PM: `9am`, `10 pm`, `3pm`
- Day + at + number: `Saturday at 10`

**Response**: Immediate booking confirmation

#### Vague Times (Offer Slot Choices)
Detected when message contains:
- Day word (monday-sunday, tomorrow, next week)
- Time-of-day (morning, afternoon, evening)
- NO specific digits

**Response**: Offer time-matched slots:
- Morning → "9:00am or 11:00am"
- Afternoon → "1:00pm or 4:00pm"
- Evening → "5:30pm or 7:00pm"

### Slot Choice Reply Handling

After offering slots, customer can reply with just a number:
- "11" → matches "11:00am"
- "the 4 one" → matches "4:00pm"

System extracts day context from `pending_slot_context` and confirms full booking.

### Confusion Safety Net

| Confusion Count | Response |
|-----------------|----------|
| 0 (first unclear) | Ask for clarity: "Could you say that again in a slightly different way?" |
| 1 (second unclear) | Silent escalation: Empty reply + `needs_human_review=True` + pause conversation |
| 2+ | Stay silent (awaiting manager) |

### Two-No Rule

| Soft No Count | Response |
|---------------|----------|
| 1 (first objection) | Gentle persist: "Would a weekday or a Saturday generally work better?" |
| 2 (second objection) | Graceful exit: "I'll leave things with you for now..." + defer 60 days |

---

## Template System

### Location: `app/lia_rules.py`

### Campaign Opener Templates (`LIA_EOT_CAMPAIGN_OPENERS`)

| Template Key | Stage | Asset Type | Character Count |
|--------------|-------|------------|-----------------|
| `EOT_18_NEW` | 18 | New | ~544 |
| `EOT_18_USED` | 18 | Used | ~487 |
| `EOT_12_NEW` | 12 | New | ~535 |
| `EOT_12_USED` | 12 | Used | ~442 |
| `EOT_9_NEW` | 9 | New | ~559 |
| `EOT_9_USED` | 9 | Used | ~435 |
| `EOT_6_NEW` | 6 | New | ~600 |
| `EOT_6_USED` | 6 | Used | ~429 |
| `EOT_3_NEW` | 3 | New | ~550 |
| `EOT_3_USED` | 3 | Used | ~520 |

### First Reply Intro Templates (`LIA_EOT_INTROS`)

Used when customer replies to initial opener:
- `EOT_18_INTRO`, `EOT_18_INTRO_USED`
- `EOT_12_INTRO_NEW`, `EOT_12_INTRO_USED`
- `EOT_9_INTRO_NEW`, `EOT_9_INTRO_USED`
- `EOT_6_INTRO_NEW`, `EOT_6_INTRO_USED`
- `EOT_3_INTRO_NEW`, `EOT_3_INTRO_USED`

### Stage-Specific Nudge Templates

Each stage has 6 nudges (3 NEW + 3 USED):

**12-Month Nudges** (`LIA_EOT_12_NUDGES`):
- `NUDGE_1_12_NEW`, `NUDGE_2_12_NEW`, `NUDGE_3_12_NEW`
- `NUDGE_1_12_USED`, `NUDGE_2_12_USED`, `NUDGE_3_12_USED`

**9-Month Nudges** (`LIA_EOT_9_NUDGES`):
- `NUDGE_1_9_NEW`, `NUDGE_2_9_NEW`, `NUDGE_3_9_NEW`
- `NUDGE_1_9_USED`, `NUDGE_2_9_USED`, `NUDGE_3_9_USED`

**6-Month Nudges** (`LIA_EOT_6_NUDGES`):
- `NUDGE_1_6_NEW`, `NUDGE_2_6_NEW`, `NUDGE_3_6_NEW`
- `NUDGE_1_6_USED`, `NUDGE_2_6_USED`, `NUDGE_3_6_USED`

**3-Month Nudges** (`LIA_EOT_3_NUDGES`):
- `NUDGE_1_3_NEW`, `NUDGE_2_3_NEW`, `NUDGE_3_3_NEW`
- `NUDGE_1_3_USED`, `NUDGE_2_3_USED`, `NUDGE_3_3_USED`

### Generic Nudge Templates (`LIA_NUDGE_TEMPLATES`)

Fallback nudges not tied to specific stage:
- `NUDGE_1_EOT`: Curiosity trigger
- `NUDGE_2_EOT`: FOMO / timing hint
- `NUDGE_3_EOT`: Advisor call bridge

### URE Reply Templates (Tier-Based)

| Template | Use Case |
|----------|----------|
| `SMALL_TALK_REPLY` | Greetings, thanks, short messages |
| `DIRECT_QUESTION_REPLY` | Opening hours, visit duration |
| `T1_GENERAL_REPLY` | Default positive engagement |
| `T2_SOFT_NO_FIRST` | First soft objection |
| `T2_SOFT_NO_SECOND` | Second soft objection (graceful exit) |
| `T3_HARD_STOP_REPLY` | Opt-out confirmation |
| `T4_SCHEDULED_REPLY` | Defer acknowledgement |
| `T5_MONEY_REPLY` | Cost concerns (showroom invite) |
| `T5_MONEY_PHONE_FALLBACK` | Cost concerns (phone alternative) |
| `T6_SUPPORT_REPLY` | Service/MOT queries |
| `CLARITY_FIRST_REPLY` | Unclear message (first time) |

---

## Database Schema (Key Tables)

### dp_conversations
```sql
conversation_id UUID PRIMARY KEY
customer_id UUID
dealer_id UUID
channel TEXT ('whatsapp', 'sms')
campaign TEXT
template_key TEXT
stage TEXT ('18', '12', '9', '6', '3')
status TEXT ('active', 'paused', 'dnc')
from_number TEXT
to_number TEXT
soft_no_count INTEGER DEFAULT 0
appointment_status TEXT ('NONE', 'INVITED', 'BOOKED')
confusion_count INTEGER DEFAULT 0
needs_human_review BOOLEAN DEFAULT FALSE
escalation_reason TEXT
escalation_at TIMESTAMP
defer_until DATE
defer_reason TEXT
created_at TIMESTAMP
updated_at TIMESTAMP
```

### dp_messages
```sql
message_id UUID PRIMARY KEY
conversation_id UUID
direction TEXT ('inbound', 'outbound')
channel TEXT
sender TEXT
message_body TEXT
template_key TEXT
mode TEXT
ab_variant TEXT
created_at TIMESTAMP
```

### dp_campaigns
```sql
campaign_id UUID PRIMARY KEY
type TEXT
name TEXT
status TEXT ('draft', 'running', 'paused', 'complete')
date_from DATE
date_to DATE
channel TEXT
```

### dp_customer
```sql
customer_id UUID PRIMARY KEY
first_name TEXT
last_name TEXT
phone_mobile TEXT
email TEXT
```

---

## Key Python Files

| File | Purpose |
|------|---------|
| `app/lia_ure.py` | Universal Reply Engine (message classification, reply generation) |
| `app/lia_inbound.py` | Inbound message processing (conversation lookup, URE integration, logging) |
| `app/lia_outbound.py` | Outbound message generation (stage selection, template selection, campaign-safety) |
| `app/lia_rules.py` | All templates + tier definitions + classify functions |
| `app/db.py` | Database functions (asyncpg-based) |
| `app/main.py` | FastAPI endpoints |
| `app/config.py` | Environment variable loading |

---

## Brand Voice Guidelines

From `LIA_TONE_GUIDELINES`:

- **UK English only** - British spelling and phrasing
- **Warm, calm, professional tone** - Friendly yet professional
- **Showroom appointment first** - Always prioritise face-to-face
- **Phone appointment fallback** - Only if customer indicates difficulty attending
- **Short, clear paragraphs** - 1-3 sentence paragraphs, 2-4 paragraphs max
- **No figures over WhatsApp** - Never quote prices/payments in messages
- **End with clear question** - Except for opt-outs and 2nd soft no
- **Use "there" as fallback** - When customer name is missing (not "Unknown")

---

## Conversation State Flow

```
NONE → [outbound sent] → INVITED → [customer confirms time] → BOOKED
                              ↓
                    [vague time given]
                              ↓
              APPOINTMENT_TIME_CHOICES (pending_slot_options set)
                              ↓
                    [customer picks slot]
                              ↓
                           BOOKED
```

### Status Flow
```
active → [soft no x2 OR defer request] → paused (with defer_until)
active → [opt-out detected] → dnc
active → [confusion x2 OR escalation] → paused (with escalation_reason)
```

---

## Testing with FastAPI Docs

Interactive API testing available at:
```
https://bacecfc8-69bb-4900-a1cd-dad6c50868e6-00-13bm1gzdpj9n6.riker.replit.dev:8000/docs
```

Use "Authorize" button with `x-api-key: lia_renewal_2025_super_secret_1` for Make.com endpoints.

---

## Make.com Integration Summary

1. **Outbound Trigger**: Call `/api/lia/outbound` with customer data
2. **Get message_text and to_number**: Use for Twilio WhatsApp send
3. **Store conversation_id**: For tracking replies
4. **Inbound Webhook**: Call `/api/lia/inbound` when customer replies
5. **Use reply field**: Send Lia's response via Twilio
6. **Nudge Scheduling**: Call `/api/lia/nudge` for follow-ups
7. **Escalation Handling**: Call `/api/conversations/{id}/escalate` when stuck

---

## Current Build Status

**Completed**:
- ✅ Universal Reply Engine (URE) with tier-based responses
- ✅ 5-stage EOT campaign system (18/12/9/6/3 months)
- ✅ NEW vs USED template variants for all stages
- ✅ Stage-specific nudge sequences (24 nudges total)
- ✅ Appointment detection (specific + vague time handling)
- ✅ Slot choice reply handling
- ✅ Two-no rule (graceful conversation parking)
- ✅ Confusion safety net (escalation after 2 unclear messages)
- ✅ Campaign-safety gate for outbound
- ✅ Escalation endpoint with reason validation
- ✅ Dashboard escalation badges (red/purple/orange)
- ✅ Message history endpoint for Make.com

**Integration Points Ready**:
- Make.com → `/api/lia/outbound` (campaign launch)
- Make.com → `/api/lia/inbound` (customer replies)
- Make.com → `/api/lia/nudge` (follow-up sequence)
- Make.com → `/api/conversations/{id}/escalate` (manager alert)
- Make.com → `/api/conversations/{id}/messages` (history lookup)

---

## Secrets Required

| Secret | Purpose |
|--------|---------|
| `DATABASE_URL` | PostgreSQL connection string |
| `OPENAI_API_KEY` | For future AI enhancements |
| `SUPABASE_URL` | Supabase project URL |
| `SUPABASE_SERVICE_KEY` | Supabase service role key |
| `LOYALTY_DASHBOARD_PASSWORD` | Dashboard login password |
| `INGEST_FUNCTION_URL` | CSV ingestion edge function |
| `INGEST_FUNCTION_TOKEN` | Edge function auth token |

---

## File References for Deeper Review

- **URE Logic**: `app/lia_ure.py` (lines 1-650)
- **Templates**: `app/lia_rules.py` (lines 1-676)
- **Inbound Processing**: `app/lia_inbound.py` (lines 1-405)
- **Outbound Processing**: `app/lia_outbound.py` (lines 1-304)
- **API Endpoints**: `app/main.py` (lines 654-777)
- **Database Functions**: `app/db.py` (lines 1-399)
- **Make.com Guide**: `tests/make_com_12_month_guide.md`