System Design Case Study

PR Intelligence Platform for async code review analysis and resilient delivery.

This architecture absorbs webhook bursts from GitHub, fans work out through Kafka-backed queues, runs analyzers and model-assisted review workers, and stores review findings with strong observability and recovery paths.

Open Repo Back to System Design

Problem

Review pull requests asynchronously without blocking GitHub delivery or losing findings.

Engineering teams want AI-assisted and rule-driven PR analysis, but webhook traffic is bursty, model calls can be slow, and reviews must still be queryable, rate-limited, and fault-tolerant.

Requirements

Latency, replayability, and operational visibility all matter.

10,000 PR events per hourDesign assumption for org-wide webhook traffic.

Sub-5 second acknowledgementDesign assumption so webhook senders are not blocked.

Review durabilityEvery analysis request and result must be stored.

Rate limitingProtect model-serving and analysis workers from abuse.

Failure recoveryJobs retry safely without losing PR state.

ObservabilityOperator insight into queues, failures, latency, and model errors.

Architecture Diagram

Webhook ingestion through analysis workers to review storage.

flowchart TB GitHub["GitHub Webhook"] --> Gateway["API Gateway"] Gateway --> Intake["Webhook Intake Service"] Intake --> Kafka["Kafka Review Queue"] subgraph Workers["Analysis Layer"] Static["Static Analysis Workers"] Policy["Policy Engine"] Model["AI / Model Service"] end Kafka --> Static Kafka --> Policy Kafka --> Model subgraph Storage["Storage and Delivery"] ReviewStore[("Review Storage")] RateLimit[("Rate Limit Store")] Notify["Notification Service"] end Static --> ReviewStore Policy --> ReviewStore Model --> ReviewStore Gateway --> RateLimit ReviewStore --> Notify subgraph Recovery["Recovery and Observability"] Retry["Retry Topics"] DLQ["Dead-Letter Queue"] Metrics["Metrics / Traces / Logs"] end Static -. transient failure .-> Retry Policy -. transient failure .-> Retry Model -. poison event .-> DLQ Intake --> Metrics Static --> Metrics Model --> Metrics

Ingress Flow: GitHub -> API Gateway -> Intake Service -> Kafka Analysis Flow: Kafka -> Static / Policy / AI Workers -> Review Storage Recovery Flow: Failures -> Retry Topics -> DLQ with observable review state

Cloud-Native Deployment Architecture

How the review platform runs across compute, messaging, persistence, and monitoring.

flowchart LR Webhook["GitHub Webhooks"] --> Ingress["Ingress / Load Balancer"] Ingress --> K8s subgraph K8s["Kubernetes Cluster"] ApiPods["API Pods"] IntakePods["Intake Pods"] WorkerPods["Analysis Worker Pods"] ModelPods["Model Adapter Pods"] end ApiPods --> IntakePods IntakePods --> WorkerPods WorkerPods --> ModelPods subgraph Platform["Platform Dependencies"] Kafka[("Apache Kafka")] Postgres[("PostgreSQL")] Redis[("Redis")] Prom["Prometheus"] Graf["Grafana"] end IntakePods --> Kafka WorkerPods --> Kafka WorkerPods --> Postgres ApiPods --> Postgres ApiPods --> Redis WorkerPods --> Prom ModelPods --> Prom Prom --> Graf

Scale Unit: API, intake, worker, and model adapter pods scale independently Durability: Kafka and PostgreSQL preserve review events and findings Monitoring: queue lag, worker latency, and model errors surface through Prometheus and Grafana

Key Decisions

Async orchestration is what makes review quality and reliability coexist.

Why Kafka?

Kafka buffers webhook bursts, decouples expensive analysis from intake, and supports replaying jobs after worker or model incidents.

Why persistent review storage?

Review findings need queryable history, diff correlation, status tracking, and future auditability across retries and model reruns.

How do you partition work?

Partition by repository and pull request identifier so events for the same PR stay ordered while still enabling cross-repo scale-out.

How are duplicates handled?

Webhook delivery IDs and PR event version checks prevent duplicate analysis jobs and duplicate comment delivery.

Where is caching useful?

Cache repository metadata, policy bundles, and recent rate-limit tokens so repeat reads do not overwhelm core services.

How is the system monitored?

Track queue lag, worker execution time, model error rates, API throttling, and review completion latency end to end.

Trade-Offs

Asynchronous analysis is resilient, but not free.

Queue-first design

Improves resilience and throughput, but means analysis is eventually consistent rather than instant.

AI / model integration

Improves review coverage and developer signal, but introduces rate limits, cost controls, and fallback requirements.

Persistent history

Improves traceability and debugging, but adds schema design and retention planning.

Worker specialization

Improves isolation between rule engines and model calls, but increases orchestration and deployment complexity.

Scaling And Failure Handling

Failure recovery is part of the product, not just operations.

Horizontal scaling

Scale intake services independently from analysis workers and model-serving adapters.

Retries

Transient GitHub API and model service failures retry with bounded backoff and status visibility.

Circuit breakers

If the model service degrades, fallback to deterministic rules so the review pipeline still produces signal.

Dead-letter queues

Malformed webhook payloads or poisoned jobs land in DLQ topics for targeted inspection and replay.

Idempotency

Use delivery IDs, PR state versions, and stable review IDs to prevent duplicate comments or duplicate persisted findings.

Disaster recovery

Retain review events and storage snapshots so historical analyses and system state can be rebuilt if needed.

Repository

Actual implementation link.

Repository: jeshwinwilliam/pr-intelligence-platform-jeshwin. This case study maps directly to the project’s webhook ingestion, asynchronous orchestration, persistence, and recovery patterns.