Agilisium logo

Walk-in || Data Engineer

Agilisium

3 - 8 years

Pune

Posted:15 hours ago

Job Description

Key Responsibilities

  • Design, develop, and maintain scalable data pipelines using Python, PySpark, and Databricks.
  • Build and optimize ETL/ELT workflows for data extraction, transformation, and loading.
  • Develop complex SQL queries and perform query optimization to improve data processing performance.
  • Work with AWS services such as S3, Glue, EMR, Redshift, and Lambda for data integration and processing.
  • Implement and manage data pipeline orchestration using Apache Airflow or AWS Step Functions.
  • Develop and maintain data models, data warehouses, and data lake solutions.
  • Ensure data quality, accuracy, consistency, and reliability across data pipelines.
  • Identify and resolve performance bottlenecks in data processing workflows.
  • Implement version control, CI/CD pipelines, and deployment best practices using Git.
  • Collaborate with cross-functional teams to understand business requirements and deliver effective data engineering solutions.

Required Skills & Qualifications

  • 38 years of experience in Data Engineering or related roles.
  • Strong hands-on experience with Python for data engineering and data processing.
  • Proficiency in SQL, including complex queries, joins, and performance tuning.
  • Strong expertise in PySpark and Databricks.
  • Hands-on experience working with AWS cloud services, particularly S3, Glue, EMR, Redshift, and Lambda.
  • Experience with data pipeline orchestration tools such as Apache Airflow or AWS Step Functions.
  • Strong understanding of data modelling, data warehousing, and ETL/ELT concepts.
  • Experience working with large-scale datasets and distributed data processing.
  • Knowledge of Git, version control systems, and CI/CD practices.
  • Excellent analytical, problem-solving, and communication skills.

Preferred Candidate Profile

  • Candidates with practical experience in developing and managing end-to-end data engineering solutions.
  • Strong understanding of Databricks architecture, Delta Lake, and Lakehouse concepts.
  • Experience in building and optimizing batch and real-time data processing pipelines.
  • Exposure to cloud-based data lake and data warehouse architectures.
  • Ability to troubleshoot complex data pipeline issues and improve overall performance.
  • Experience working in Agile development environments.
  • AWS or Databricks certifications will be an added advantage.

Skills Required

Data Bricks Bricks SQL Data Python AWS Modeling Microsoft Azure

Learn the skills this job asks for

See your skill gap for Data Engineer →

Free courses, interview questions and practice for each skill.

Work Mode

Work from Office

Job Type

Full Time

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now
Agilisium logo
Agilisium

RecommendedJobs for You

CODERS BRAIN logo

Pune

Indium Software logo

Bengaluru

CODERS BRAIN logo

Pune