Microservices & Architecture

System Overview

The system is composed of four independently deployable services (three FastAPI backends + one nginx frontend) orchestrated as 4 Deployments behind an Ingress in AKS. All services are stateless; persistent state lives in Databricks (MLflow + Delta Lake).

Service Responsibilities

1. authService (Port 8001)

Role: JWT token generation and validation, user registration, RBAC claim issuance.

Endpoints:

Key Implementation:

• Tokens: HS256 (symmetric, shared secret across all services)
• Token payload: {userId, role, exp, iat}
• Access tokens: 60-minute expiry; refresh tokens: 7 days
• Roles: user (demo access), analyst (stats endpoint access)
• Password: bcrypt hashing, no plaintext storage

Example JWT Payload:

{
  "userId": "demo-user-001",
  "role": "user",
  "exp": 1717287600,
  "iat": 1717284000
}

Environment Variables:

JWT_SECRET_KEY    → openssl rand -hex 32 (shared with other services)
JWT_ALGORITHM     → HS256
JWT_EXPIRY_MINUTES→ 60

2. inferenceService (Port 8002)

Role: Load ML models (pretrained ST-GCN and trained Random Forest), run inference, write predictions to Delta Lake, serve model metadata.

Endpoints:

Model Loading:

• At startup, pulls both aslStgcn and aslRandomForest from MLflow (Databricks hosted)
• ST-GCN: PyTorch model, 30 MB, inference on CPU
• RF: sklearn pickle, ~50 MB (100 trees, depth 15)

Landmark Preprocessing (app/preprocessing.py):

def validate_landmarks(payload):
    """
    Expects: { landmarks: [ [27, 2], [27, 2], ... ] }
    - Max 200 frames
    - Each frame exactly 54 floats (27 keypoints × 2 coords)
    - Body size ≤ 1 MB
    """

Inference Flow:

1. Receive landmarks (T, 27, 2)
2. Load model from memory (preloaded at startup)
3. For ST-GCN:
   • Forward pass → 2,731 logits
   • Logit-mask to 100 in-vocab signs (set non-vocab logits to -∞)
   • Softmax → probabilities (now summing to 1 over 100 signs only)
   • Return top-3: [(label, confidence), ...]
4. For Random Forest:
   • Extract per-clip features (mean/std/velocity/trajectory per keypoint)
   • predict_proba() → class probabilities
   • Return top-3
5. Write one row to asl.inferenceEvents: {timestamp, modelName, sign, confidence, latencyMs, eventId}
6. Return response

Event Schema (asl.inferenceEvents Delta table):

{
  "timestamp": "2026-06-01T22:30:00Z",
  "modelName": "aslStgcn",
  "sign": "HELLO",
  "confidence": 0.87,
  "latencyMs": 145,
  "eventId": "evt-001",
  "modelVersion": "1.2.3"
}

Environment Variables:

JWT_SECRET_KEY        → (shared)
DATABRICKS_HOST       → https://your-workspace.cloud.databricks.com
DATABRICKS_TOKEN      → personal access token
MLFLOW_TRACKING_URI   → https://your-workspace.cloud.databricks.com/api/2.0/mlflow

3. agentService (Port 8003)

Role: Two-agent pipeline (Gloss Assembler → Translator) for interpretation and correction. Runs on OpenAI gpt-4o-mini with structured outputs.

Endpoints:

Agent Workflow:

1. Input: ordered sequence of recognized signs (each with confidence + top-3 alternatives from inferenceService)

   [
     {"label": "HELLO", "confidence": 0.92, "alternatives": ["WAVE", "HI"]},
     {"label": "WORLD", "confidence": 0.41, "alternatives": ["EARTH", "LIFE"]},
     {"label": "I", "confidence": 0.88, "alternatives": ["ME", "YOU"]}
   ]
   

2. Gloss Assembler agent (Pydantic schema + gpt-4o-mini):
   • Drops signs with confidence < 0.40
   • Flags signs with 0.40 ≤ confidence < 0.65
   • Contextual correction: if top-1 is implausible given surrounding signs, swap with an alternative and log the reason

   Output schema:

   {
     "cleaned_gloss": ["HELLO", "I"],
     "flags": [{"index": 1, "label": "WORLD", "reason": "below threshold"}],
     "corrections": [
       {"original": "HELLO", "chosen": "WAVE", "reason": "context: greeting expected"}
     ]
   }
   

3. Translator agent (Pydantic schema + gpt-4o-mini):
   • Converts ASL-ordered gloss to grammatical English
   • Handles ASL grammar quirks: topic-comment, time markers up front, no articles/copulas

   • Inserts missing English function words (copulas, articles, prepositions)

   • Input: ["TOMORROW", "STORE", "I", "GO"]
   • Output: "I'm going to the store tomorrow."
4. Response includes:

   {
     "gloss": "HELLO I",
     "english": "I said hello.",
     "trace": {"gloss_assembler_output": {}, "translator_output": {}},
     "corrections": []
   }
   

Prompts (app/prompts.py):

Gloss Assembler System Prompt (abbreviated):

You are an ASL gloss correction expert. You receive a time-ordered sequence
of recognized signs. Each sign carries confidence and alternatives.

YOUR TASKS:
1. Drop signs with confidence < 0.40.
2. Flag (keep but mark) signs with 0.40 ≤ confidence < 0.65.
3. Contextual correction: if top-1 is implausible given surrounding signs,
   swap it with an alternative and explain why.

Return a JSON object with:
- cleaned_gloss: list of corrected labels
- flags: list of low-confidence items
- corrections: list of {original, chosen, reason}

Translator System Prompt (abbreviated):

You are an ASL-to-English translator. You receive an ASL-ordered gloss list.
ASL grammar is topic-comment; time markers appear at sentence start.
English is SVO; you must insert copulas and articles as needed.

Examples:
- ASL gloss: ["TOMORROW", "STORE", "I", "GO"]
- English: "I'm going to the store tomorrow."

- ASL gloss: ["RAIN", "OUTSIDE"]
- English: "It's raining outside."

Return JSON with:
- english: the translated sentence (one sentence only)
- explanation: brief notes on grammar transformations applied

Environment Variables:

JWT_SECRET_KEY   → (shared)
OPENAI_API_KEY   → sk-...
OPENAI_MODEL     → gpt-4o-mini

4. Frontend (Port 8080)

Role: Single-page application served by nginx — branded as SignStream. Runs MediaPipe Holistic in the browser, preprocesses landmarks, handles user interaction, displays predictions and corrections.

Pages:

• / — Main SPA: status indicators, record button, sign display, alternatives dropdown, feedback chips

JavaScript Modules (frontend/js/):

• mediapipe.js — MediaPipe Holistic setup, frame capture
• keypointPreprocessing.js — Subset to 27 keypoints, drop z coords, shoulder-normalize
• api.js — HTTP calls to auth, inference, agent services (with JWT refresh)
• ui.js — Render predictions, alternatives, feedback buttons, translation display

User Flow (see Frontend & UI for screenshots):

1. Page load — Check health of all three services (3 status dots)
2. Start signing — User clicks record; MediaPipe captures frames
3. Sign — Frames collected; sliced clips appear as buttons
4. Predict — Each clip → /predict → top-3 candidates
5. Display — Render prediction + top-3 alternatives in UI
6. Translate — Batch collected signs → /translate (agentService)
7. Final English — Display English caption + correction log
8. Feedback loop — User thumbs-up/down or swap alternative

End-to-End Request Flow

Communication Security

• All traffic: HTTPS (Ingress TLS)
• Authentication: Bearer token (JWT in Authorization header)
• Token validation: Middleware on each service checks signature + expiration
• Cross-origin: CORS configured to allow browser origin only
• Rate limiting: slowapi on /predict endpoint

Scaling Considerations

• Stateless services — Each service can be scaled horizontally (replicas in K8s)
• Database layer — Databricks MLflow and Delta are managed; no scaling concern on client side
• Inference latency — P95 around 165–180 ms; CPU-only means no GPU contention
• Concurrent users — Limited by AKS pod count and Databricks connection pool