← Home
Technical Report · ASL Pipeline

Technology Stack

This section details the tools and frameworks selected for each layer of the system, with rationale for each choice.

Layer-by-Layer Breakdown

Layer Technology Version/Notes Why Chosen
Version Control GitHub — Industry standard, native GitHub Pages support for documentation
CI/CD GitHub Actions ci.yml, cd.yml, securityScan.yml Integrated with GitHub, free tier sufficient for Docker builds and security scans
Container Runtime Docker 24.x Language-agnostic, fast local iteration, matches production (AKS)
Orchestration Azure Kubernetes Service (AKS) 4 Deployments behind ingress University Azure subscription, managed K8s reduces ops overhead
Container Registry Azure Container Registry (ACR) — Tight AKS integration, native authentication
Landmark Extraction (Browser) MediaPipe Holistic JS API v0.21+ Industry-standard for pose/hand/face detection, runs client-side (no server GPU needed)
OpenHands Subset 27-keypoint standard — Published paper (2023) defines exact indices, reduces payload size to 13 KB per clip
Data Processing PySpark 3.5.0 Distributed feature engineering, managed on Databricks
Data Storage Databricks Unity Catalog + Delta Lake — Free Edition constraint: no DBFS, so UC Volumes + Delta are mandatory; immutable, versioned tables
Primary Model Pretrained ASL Citizen ST-GCN Microsoft Research, MIT Published 2023, proven on full 2.7k-class ASL Citizen corpus, fast CPU inference
ST-GCN Inference PyTorch 2.0+ Matches checkpoint format, CPU-only sufficient for 30 MB model
Baseline Model sklearn Random Forest 100 trees, depth 15 End-to-end training satisfies ML-lifecycle rubric; Spark RF hit Free-Edition 256 MB serialization cap
Model Registry MLflow on Databricks Hosted Hosted MLflow avoids self-managed tracking server; versioning + stage promotion built-in
API Framework FastAPI 0.109+ Async, automatic OpenAPI docs, Pydantic validation out-of-box
Authentication PyJWT (HS256) 2.8+ Simple, stateless; tokens valid across stateless AKS pods
Rate Limiting slowapi 0.1.8+ Easy-to-configure middleware, prevents abuse on prediction endpoint
Frontend Vanilla HTML/JS + nginx — No build step, MediaPipe JS bundled directly, single HTML file served by nginx static container
Agent Orchestration OpenAI SDK + Pydantic gpt-4o-mini Structured outputs guarantee schema compliance; cheap inference model; no LangGraph overhead for two-agent workflow
LLM (Runtime) OpenAI gpt-4o-mini — Cheap, fast, structured output support, good NLP quality for glosses and English reconstruction
Security Scanning (CI) Trivy HIGH/CRITICAL CVEs Container image scanning on each build for all four service images
Development Assistants Claude Code, GitHub Copilot — See AI-Assisted Development for quantified impact

Key Design Rationale

Why CPU-Only Inference?

Landmark extraction happens in the browser via MediaPipe Holistic JS. The inference payload is tiny:

  • Per frame: 27 keypoints × 2 coordinates = 54 floats
  • Per sign (60 frames): 3,240 floats ≈ 13 KB JSON

ST-GCN at 30 MB runs single-clip inference on CPU at p95 around 180 ms. This eliminates GPU pod costs in AKS and simplifies deployment.

Why Pretrained ST-GCN + Custom Random Forest?

  • Pretrained ST-GCN — Production recognizer. Microsoft Research released weights on the full 2,731-class corpus; we logit-mask to 100 signs at inference time. Reproducing their training run is not the contribution; the contribution is the correction pipeline on top.
  • Custom Random Forest — Satisfies the ML-lifecycle rubric (training, tuning, deployment, monitoring). Trained end-to-end by the team on Databricks, demonstrating the full MLflow workflow.

Why Databricks Free Edition?

  • Free tier covers dev + demo workloads
  • No GPU necessary (inference is CPU, no model retraining)
  • Unity Catalog Volumes (no DBFS) store datasets
  • Delta Lake provides versioned, ACID tables
  • Hosted MLflow simplifies model registry
  • Serverless compute only; no custom cluster configs needed

Why No React / Build Pipeline?

The team’s time is better spent on the ML and agent logic. A single index.html with vanilla JS modules (mediapipe.js, api.js, ui.js) is sufficient for the demo and avoids npm/webpack overhead.

Why FastAPI?

  • Automatic OpenAPI docs at /openapi.json
  • Built-in Pydantic request/response validation
  • Async support for concurrent requests
  • Minimal boilerplate for microservices

Why Structured Outputs (gpt-4o-mini)?

OpenAI’s structured output API (chat.completions.parse with Pydantic response_format) guarantees responses conform to schema. This eliminates JSON-parsing errors; retries only fire on transient API failures, not malformed responses.

Deployment Architecture

GitHub
Source code & workflows
↓
git push / merge to main
GitHub Actions
ci.yml — ruff lint + pytest
cd.yml — build · push · deploy
securityScan.yml — Trivy (weekly)
↓
push images
Azure Container Registry
authService · inferenceService · agentService · frontend
↓
kubectl apply
Azure Kubernetes Service (AKS)
authService pod (Deployment)
inferenceService pod (Deployment)
agentService pod (Deployment)
nginx frontend pod (Deployment)
Ingress — HTTPS · JWT validation
↓
API calls
Databricks Free Edition
MLflow — model registry
Delta Lake — asl.* tables
SQL Dashboard — real-time
Spark Jobs — evaluation

Development Environment

Local development uses Docker Compose to simulate the AKS deployment:

services:
  auth-service:        # FastAPI, port 8001
  inference-service:   # FastAPI, port 8002
  agent-service:       # FastAPI, port 8003
  frontend:            # nginx, port 8080

All services are stateless and can be scaled horizontally. The only persistent storage is Databricks (MLflow registry, Delta tables).

Summary

The stack is chosen to minimize operational overhead while enabling full-stack feature development:

  • Client-side ML preprocessing reduces server load
  • Stateless FastAPI services scale easily in Kubernetes
  • Databricks Free Edition covers storage + model registry + dashboarding
  • GitHub Actions + ACR provide fast CI/CD feedback
  • Structured LLM outputs eliminate parsing brittleness