Drone Data Architect

Role Summary

The Drone Data Architect owns the data foundation of the platform. You will design how drone video and imagery (both live streams and bulk uploads) are ingested, validated, processed, indexed, stored and served to tenants; how flight telemetry is bound to every frame and image; and how existing MVP customer data is migrated safely into the multi-tenant model. Your designs set the standard for every team that touches media, metadata, geospatial data or inference results.

Key Responsibilities

Ingestion Architecture

• Bulk upload: Design resumable, chunked uploads (S3 multipart with presigned URLs, or tus protocol) with per-part checksums (SHA-256/CRC32C), retry and resume for low-bandwidth field connections, client-side manifest, and server-side completeness validation.

• Live ingest: Architect live feeds over RTSP, RTMP/SRT and WebRTC (WHIP) into a media gateway (MediaMTX, Kinesis Video Streams or equivalent), with reconnect handling, jitter and packet-loss tolerance, and fan-out to HLS/LL-HLS for viewing and to inference workers.

• Processing: Define FFmpeg/GStreamer pipelines: probe and validation (ffprobe), codec normalisation (H.264/H.265/AV1), proxy renditions and adaptive bitrate ladders, keyframe and interval frame extraction, thumbnails, sprite sheets, tiling of large orthomosaics and high-resolution stills.

• Real-time vs. batch: Specify which flows are real-time (latency targets, frame-sampling rates) and which are batch; define back-pressure, queue topology (Kafka topics or SQS queues), partitioning keys by tenant and flight, ordering guarantees and exactly-once effect via idempotent processing.

• Throughput sizing: Model ingest bandwidth, storage growth per tenant, concurrent streams, GPU hand-off rate and egress cost; publish sizing sheets and scaling thresholds.

Telemetry and Metadata Model

• Image metadata: Extract and normalise EXIF/XMP (including DJI drone-dji namespace: GPS, relative/absolute altitude, gimbal yaw/pitch/roll, flight yaw, camera intrinsics, RTK flags) with exiftool; handle vendor differences and missing fields.

• Video telemetry: Parse DJI SRT/subtitle telemetry, MAVLink tlog and dataflash logs, PX4 ULog and vendor flight-log exports into a canonical telemetry schema (time-series of position, attitude, speed, battery, sensor state).

• Time synchronisation: Define frame-to-telemetry alignment (clock offsets, interpolation, drift correction, timezone and GPS-time handling) so every frame carries a position and pose with a quality flag.

• Canonical model: Design the entity model: Organisation/Tenant, Site, Mission, Flight, Media Asset, Frame, Telemetry Sample, Detection, Track, Annotation, Report; define identifiers (ULID/UUIDv7), versioning and provenance (source file hash, software version, processing lineage).

Geospatial Data Design

• Spatial store: Decide and document the split between MongoDB (2dsphere indexes, GeoJSON for operational queries) and PostGIS (analytics, joins, raster/vector processing); define SRIDs (EPSG:4326 storage, projected CRS for measurement), GiST indexes, partitioning and query patterns.

• Formats: Support GeoJSON, KML/KMZ, GeoPackage, Shapefile import/export, and cloud-optimised outputs (COG, PMTiles, vector tiles via pg_tileserv or equivalent).

• Catalogue: Design a STAC-style catalogue so assets can be discovered by tenant, site, time, footprint and sensor; define footprint computation from pose, camera model and terrain (DEM) where available.

• Detection geo-referencing: Specify how pixel detections are projected to world coordinates (camera intrinsics, gimbal pose, altitude/DEM, ground sampling distance) and how accuracy is recorded.

Storage, Tiering and Retention

• Layout: Design object key layout and prefixes by tenant/site/flight/asset, immutable originals plus derived artefacts, and versioning rules; avoid hot-prefix throttling.

• Tiering: Define hot, warm and archive tiers (e.g. S3 Standard, Intelligent-Tiering, Glacier Instant Retrieval, Deep Archive; Azure Hot/Cool/Archive) with lifecycle rules, restore workflows and cost models by plan.

• Isolation: Implement tenant isolation using per-tenant prefixes, IAM/bucket policies, per-tenant KMS data keys (envelope encryption), signed URL scope and access logging.

• Retention and deletion: Define retention policies per plan, legal hold, tenant offboarding and verified deletion (including derived data and backups), in line with GDPR.

Results Data and API Delivery

• Inference outputs: Design schemas for detections, tracks and segmentation masks (COCO/GeoJSON/Parquet), model-version tagging, confidence thresholds and reprocessing without data loss.

• Query and delivery: Define APIs for time/space/class queries, tiled map delivery, signed media delivery (CDN with token auth), pagination and bulk export; set performance budgets (e.g. p95 query latency).

• Analytics: Define a columnar analytics layer (Parquet on object storage queried by Athena/Synapse or similar) for reporting and model evaluation datasets.

MVP Migration

• Assessment: Inventory MVP data stores, files and metadata quality; map MVP schemas to the canonical model with a documented transformation spec.

• Execution: Design idempotent, resumable migration jobs with manifests, checksum verification, dual-read or shadow validation, per-tenant cutover, rollback plan and reconciliation reports.

• Remediation: Backfill missing telemetry/EXIF where possible and flag data-quality exceptions for customer review.

Quality, Observability and Governance

• Data quality: Define validation rules, quarantine paths for bad media, and data-quality scorecards.

• Observability: Define pipeline metrics (ingest lag, bytes/s, failed-asset rate, time-to-first-result, storage per tenant) and alerts with the DevOps/SRE engineer.

• Governance: Maintain data dictionary, lineage, ADRs and schema-evolution policy; review all schema and storage changes.

Required Skills and Experience

• 8+ years in data or software architecture, including 3+ years with drone, aerial or video data.

• Deep experience designing large video/image pipelines on cloud object storage (S3, Azure Blob or GCS), including multipart upload, lifecycle and tiering.

• Hands-on FFmpeg and/or GStreamer; streaming protocols RTSP, RTMP/SRT, WebRTC, HLS/DASH.

• Drone metadata expertise: EXIF/XMP, DJI telemetry (SRT), MAVLink or PX4 logs; time-sync of telemetry with video.

• Geospatial data engineering: PostGIS, GeoJSON, KML, GDAL/OGR, coordinate reference systems, spatial indexing.

• MongoDB data modelling and indexing, including geospatial (2dsphere) and time-series patterns.

• Event-driven pipelines (Kafka, RabbitMQ or SQS), idempotency and back-pressure design.

• Multi-tenant data isolation patterns, envelope encryption and KMS.

• Data migration at scale with validation and rollback.

Preferred / Added Advantage

• STAC, COG, PMTiles, TiTiler or similar cloud-native geospatial tooling.

• Photogrammetry outputs (orthomosaic, point cloud, DEM) and tools such as ODM or Pix4D.

• Parquet/Iceberg, Athena or Spark for analytics.

• Kinesis Video Streams, AWS IVS or Azure Media Services.

• Experience sizing GPU inference hand-off and frame-sampling strategies.

• Cost optimisation at petabyte scale; FinOps tagging by tenant.

Key Deliverables

• Ingestion architecture document and sequence diagrams (live and bulk).

• Canonical data model, telemetry schema and data dictionary.

• Geospatial design and storage decision record (MongoDB vs. PostGIS).

• Storage layout, tiering, retention and tenant-isolation specification.

• Sizing and cost model.

• MVP migration plan, tooling spec and reconciliation report templates.

• Pipeline observability dashboard specification.

Education

Bachelor’s degree in Computer Science, Geoinformatics, Engineering or equivalent experience.

Pay: ₹1,500,000.00 - ₹2,500,000.00 per year

Work Location: In person