AWS Data Architect-PAN India

Job Position Title:

Location

Years of experience

Responsibilities:

-Design and build robust streaming and batch data pipelines with AWS Glue, Lambda, Kinesis Data Streams/Firehose, Amazon MSK (Kafka), and Step Functions.
-Architect end-to-end systems for real-time ingestion, processing, and analytics, leveraging Kinesis, MSK, Managed Flink, EMR, and Lambda for low-latency, high-availability operations. -Implement batch ETL pipelines using Glue, EMR (Spark/Hadoop), and S3, optimizing for cost and performance while handling large-scale datasets. -Integrate source systems (IoT, logs, transactional DBs, third-party feeds) and downstream destinations (Redshift, S3, DynamoDB, OpenSearch, Aurora, etc.). -Design and implement architectures supporting event-driven microservices, streaming analytics, materialized views, and near-real-time dashboards. -Ensure scalability, durability, ordering, and fault-tolerance in stream storage, using Kinesis/MSK and S3 for replay, backfill, and recovery scenarios. -Standardize patterns for Change Data Capture (CDC), late/duplicate data handling, data enrichment, stream normalization, and schema evolution. -Govern data security (encryption, access control, network isolation), ensure compliance, and embed monitoring/observability pipelines for SLA/SLO adherence.

Mandatory skill sets:

- 8+ years of experience in data engineering/architecture, including significant leadership in AWS cloud data platforms.
-Expert in designing streaming architectures with Kinesis, Kafka (MSK), Managed Flink, Lambda, and Step Functions. -Strong proficiency in batch ETL (AWS Glue, EMR, Spark/Hadoop), large-scale data lake/lakehouse implementation (S3, Delta Lake, Redshift). -Hands-on experience architecting hybrid streaming/batch workflows for analytics, reporting, ML, and near-real-time applications. -Deep understanding of distributed computing, parallelization, and fault-tolerance strategies for high-velocity, high-volume data. -Familiarity with real-time dashboards, API publication (REST, WebSockets), and materialized views for user-facing analytics. Proficient in network/security controls, IAM design, encryption (KMS), and data governance. Experience with DevOps/IaC (CloudFormation, Terraform). -Excellent communication and stakeholder leadership; ability to translate business requirements to technical solution patterns.

Preferred skill sets:

Redshift, S3, AWS Glue, Lambda, SQL, PySpark, SQL