IoT Platform 2 - Industrial Scale & Edge Computing

Interview-Ready Approach

1) Clarify Scope and SLOs

• •Problem statement: Scale to 50 M industrial sensors with edge processing, anomaly detection, and five-nines uptime.
• •Design for a peak load target around 20,000 RPS (including burst headroom).
• •Total sensors: ~50,000,000
• •Peak events/second: ~50,000,000
• •Facilities: ~10,000
• •Anomaly alert latency: < 10 seconds
• •Raw data retention: 90 days

2) Capacity Planning Method

• •Convert traffic and growth constraints into request rate, storage growth, and concurrency budgets.
• •Keep at least 2-3x safety margin per tier (ingress, compute, storage, async workers).
• •Reserve explicit latency budgets per hop so p95 can be defended in review.

3) Architecture Decisions

• •Databases: Define a clear system-of-record and design read/write paths separately before adding optimizations.
• •Message Queues: Move non-blocking and retry-heavy work to async consumers with explicit retry and DLQ policies.
• •Monitoring: Instrument golden signals (latency, traffic, errors, saturation) per tier and per tenant/domain.
• •Sharding: Choose shard keys around access patterns and growth hotspots, not just data size.
• •Analytics: Maintain separate OLTP and analytics paths; stream events into a warehouse/time-series layer.
• •Containerization: Run services/jobs in isolated containers with reproducible images and resource quotas.

4) Reliability and Failure Strategy

• •Use strong write constraints (transactions or conditional writes) and explicit backup/restore strategy.
• •Guarantee idempotent consumers and trace every message with correlation IDs.
• •Alert on user-impact SLOs, not only infrastructure metrics.
• •Support rebalancing and hotspot detection from day one.
• •Version event schemas and monitor drop/late-event rates.

5) Validation Plan

• •Run one peak-load test, one dependency-degradation test, and one failover test.
• •Verify idempotency for all retried writes and async consumers.
• •Track user-facing SLOs first: p95 latency, error rate, and successful throughput.

6) Trade-offs to Call Out in Interviews

• •Databases: SQL gives stronger transactional guarantees; NoSQL often gives better write scaling and flexibility.
• •Message Queues: Async pipelines absorb spikes well, but increase eventual-consistency complexity.
• •Monitoring: Deep observability speeds incident response but raises ingestion and tooling costs.
• •Sharding: Sharding improves horizontal scale but makes cross-shard queries and transactions harder.
• •Analytics: Analytics pipeline unlocks insights, but adds eventual consistency and governance overhead.

Practical Notes

• •Time-series databases (TimescaleDB, InfluxDB, or QuestDB) are purpose-built for this ingest rate.
• •Edge gateways can run lightweight containers (K3s) that buffer data locally during outages and sync when reconnected.
• •Downsample older data - raw 1-second data for 90 days, 1-minute aggregates for 5 years.