Online Code Sandbox

Interview-Ready Approach

1) Clarify Scope and SLOs

• •Problem statement: Design an online code execution sandbox (like CodeSandbox / Replit) that runs user code safely with real-time collaboration.
• •Design for a peak load target around 100 RPS (including burst headroom).
• •Registered users: ~200,000
• •Code executions/day: ~500,000
• •Max execution time: 10 seconds
• •Max memory per sandbox: 256 MB
• •Sandbox startup time: < 2 seconds

2) Capacity Planning Method

• •Convert traffic and growth constraints into request rate, storage growth, and concurrency budgets.
• •Keep at least 2-3x safety margin per tier (ingress, compute, storage, async workers).
• •Reserve explicit latency budgets per hop so p95 can be defended in review.

3) Architecture Decisions

• •Containerization: Run services/jobs in isolated containers with reproducible images and resource quotas.
• •Databases: Define a clear system-of-record and design read/write paths separately before adding optimizations.
• •WebSockets: Use persistent connection gateways and decouple fanout via pub/sub or queues.
• •API Design: Standardize API boundaries, idempotency keys, pagination, and error contracts first.
• •Storage: Use object storage for large blobs and keep metadata/authorization separate in the API tier.

4) Reliability and Failure Strategy

• •Use rolling deploys with readiness probes and fast rollback.
• •Use strong write constraints (transactions or conditional writes) and explicit backup/restore strategy.
• •Track connection churn, backpressure, and session resumption behavior.
• •Apply strict input validation and backward-compatible versioning.
• •Enforce lifecycle policies, retention tiers, and checksum validation.

5) Validation Plan

• •Run one peak-load test, one dependency-degradation test, and one failover test.
• •Verify idempotency for all retried writes and async consumers.
• •Track user-facing SLOs first: p95 latency, error rate, and successful throughput.

6) Trade-offs to Call Out in Interviews

• •Containerization: Containerization standardizes environments but increases orchestration complexity.
• •Databases: SQL gives stronger transactional guarantees; NoSQL often gives better write scaling and flexibility.
• •WebSockets: WebSockets reduce interaction latency but complicate scaling and state management.
• •API Design: Rich APIs improve developer speed but can create long-term compatibility burden.
• •Storage: Object storage is cheap and durable, but random low-latency reads are weaker than databases/caches.

Practical Notes

• •Use lightweight containers (gVisor or Firecracker microVMs) for sandboxing - they provide strong isolation with fast startup times.
• •Pre-warm a pool of idle sandboxes per language so users don't wait for container creation.
• •Stream stdout/stderr via WebSocket from the sandbox worker to the browser for real-time output.

Why This Solution Works

Request path: The solution keeps ingress, service logic, and stateful dependencies separated so each layer can scale independently.

Reference flow: Web Clients -> Load Balancer -> API Gateway -> API Service -> Primary NoSQL DB -> Realtime Bus -> Object Storage

Design strengths

• •The architecture keeps synchronous paths short and isolates stateful dependencies behind clear boundaries.

Interview defense

• •This design makes bottlenecks explicit (ingress, core compute, persistence, async workers).
• •It supports progressive scaling without re-architecting the core request path.
• •It keeps correctness-sensitive state changes in durable systems while offloading background work asynchronously.