Jobs

Interviews
Job Alerts
Tools

Upskill and Grow with AI

Mock Interview Practice interviews in realistic simulations

Coding Practice Improve your coding skills with challenges

Certification Earn certifications to validate your skills

AI Learning Get trained with AI expert sessions

Career Path AI insights for smarter career decisions

AI Job Match Score AI-Powered Job Match Against Your Resume and Optimize Your Resume

Career Tools and Resources

Resume Builder Build Professional Resume with Ease

ATS Friendliness Check Check Resume Friendliness for Applicant Tracking Systems

Auto Apply Apply to hundreds of jobs on any platform effortlessly

Co-Pilot (Chrome Extension) Your AI Assistant for Seamless Browsing Efficiency

Interview Questions Streamline interviews with ready-to-use questions

Salaries Discover market-driven salary insights across skillsets and geographies

Companies Explore leading companies actively hiring talent
For Employers

Home
>
Jobs in pune
>
Accion Labs
>
Data Engineer

Data Engineer

Accion Labs

6 - 10 years

15 - 30 Lacs

pune bengaluru mumbai (all areas)

Posted:2 weeks ago| Platform:

Apply

Skills Required

data engineering framework python pyspark aws sql

Work Mode

Hybrid

Job Type

Full Time

Job Description

Role Overview

You will own the design, development, and optimization of large-scale data pipelines and platforms. This role needs someone who can combine strong engineering fundamentals with hands-on expertise in Python, distributed data processing (Spark), cloud architecture (AWS), and modern container/orchestration stacks (Docker, Kubernetes). You will guide the data engineering strategy, enforce best practices, and lead complex initiatives end-to-end.

Key Responsibilities

Architect, build, and maintain scalable batch and streaming data pipelines using Spark (PySpark preferred).
Develop robust, modular, production-grade Python services for ingestion, transformation, and ML-adjacent workflows.
Optimize Spark jobs for performance, cost efficiency, and reliability in large datasets (TB-scale).
Lead the design of data lake, lakehouse, or warehouse architecture (S3 + Glue + Athena, EMR, Redshift, or similar).
Implement CI/CD pipelines for data services using tools like GitHub Actions / GitLab / Jenkins.
Manage containerized workloads using Docker and deploy/operate them on Kubernetes (EKS preferred).
Enforce data quality, lineage, governance, and schema management standards.
Collaborate with ML engineers and data scientists to productionize models, ensuring scalable feature pipelines and deployment workflows.
Perform code reviews, mentor team members, and uphold engineering best practices.
Drive architectural decisions, capacity planning, and platform scalability.

Required Skills & Experience

610+ years in data engineering, software engineering, or distributed systems.
Strong Python fundamentals: modular design, testing, async patterns, performance profiling.
Advanced Spark/PySpark experience (RDDs, DataFrames, structured streaming, tuning, partitioning strategies).
Strong AWS knowledge:

S3, EMR, Glue, Lambda, Step Functions
IAM policies & security
EKS experience is a strong plus

Hands-on Docker and Kubernetes (K8s operators, manifests, Helm is a plus).
Solid understanding of distributed systems concepts (fault tolerance, parallelization, memory management).
Experience designing data models, ETL frameworks, and metadata management.
Working knowledge of ML workflows (feature engineering pipelines, model deployment, feature stores).
Experience with monitoring (Prometheus, Grafana), logging (ELK), and alerting systems.
Strong problem-solving ability and the ability to cut through noise.

Nice to Have

Experience with Delta Lake / Iceberg / Hudi.
Terraform or CloudFormation for IaC.
CI/CD for ML (SageMaker pipelines, MLflow, Kubeflow, or similar).
Real-time streaming platforms (Kafka, Kinesis).
Security and compliance experience (PII handling, encryption-at-rest/in-transit).

Soft Skills (the practical ones)

Drives clarity, not chaos.
Communicates architecture decisions clearly and defends them with logic, not buzzwords.
Can mentor mid-level engineers without hand-holding.
Can operate independently and handle ambiguity.
Pushes back when something is poorly defined or technically unsound.

More Jobs at Accion Labs

Senior Associate (Business process)

Bengaluru

3 - 8 yrs

INR 14 - 20 Lacs

Data Engineer

Hyderabad

3 - 7 yrs

INR 10 - 20 Lacs

Java Developer

Pune, Bengaluru

6 - 10 yrs

INR 15 - 30 Lacs

Technical Architect (Python)

Hyderabad, Bengaluru, Mumbai (All Areas)

10 - 15 yrs

INR 25 - 40 Lacs

Full Stack_ Dot Net Developer(Angular)

Bengaluru, Mumbai (All Areas)

4 - 8 yrs

INR 20 - 30 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now

Accion Labs

IT Services and IT Consulting

Bridgeville Pennsylvania

Login to

Please Verify Your Phone or Email

Confirm Action

Data Engineer