Data Engineer

Job Title: Data Engineer

Employment Type:- Full Time, Permanent Position

Location: Noida/Hyderabad

Qualifications: BE/B.Tech/MCA Degree in Computer Science, Engineering, or similar relevant field

Total Experience 3+ years

Working Model Work from Office

ABOUT THE ROLE

We are seeking a skilled Data Engineer/ Senior Data Engineer to design, build, and maintain robust data infrastructure and pipelines that enable our organization to leverage data for strategic decision making. The ideal candidate will have strong technical expertise in data engineering, cloud technologies, and data architecture, with a passion for building scalable and efficient data solutions.

PRIMARY RESPONSIBILITIES:

• Translate business requirements into technical specifications for data solutions

• Develop and maintain data models and schema designs that support analytical and operational needs

• Implement and manage data warehousing solutions on platforms like Amazon Redshift or Snowflake

• Design and implement middleware solutions to enable seamless data flows between applications and systems

• Design, develop, and maintain scalable ETL/ELT pipelines to ingest, process, and transform data from various sources

• Build automated data workflows using orchestration frameworks like Apache Airflow to ensure reliable and timely data delivery

• Process large-scale datasets using distributed computing frameworks, particularly Apache Spark

• Perform complex data wrangling and transformation tasks using Python libraries (Pandas, NumPy) and Spark Data Frames

• Work with geospatial (GIS) data, understanding spatial data types, coordinate systems, and GIS-specific processing requirements

• Optimize existing pipelines for performance, cost-efficiency, and maintainability

• Optimize cloud resource utilization to balance performance and cost

• Create and maintain documentation for data pipelines, data models, and technical processes

Knowledge & Skills Requirement

Technical Skills

Programming & Scripting:

• Advanced proficiency in Python for data engineering tasks, including experience with libraries such as Pandas, NumPy.

• Expert-level SQL skills with ability to write complex queries, optimize query performance, and design efficient database schemas

• Experience with version control systems (Git) and collaborative development practices

Data Integration & ETL/ELT:

• Hands-on experience in building and maintaining ETL/ELT pipelines

• Proficiency with Apache Spark for distributed data processing

• Experience with workflow orchestration tools, particularly Apache Airflow

• Strong understanding of API design and integration patterns (REST, GraphQL)

• Knowledge of message queuing systems (Kafka, RabbitMQ) is a plus

Data Storage & Management:

• Experience with SQL/NoSQL databases and understanding of when to use each type

• Hands-on experience with data warehousing platforms such as Amazon Redshift or Snowflake

• Knowledge of or experience working with geospatial (GIS) data.

• Understanding of data modeling techniques (dimensional modeling, normalization, denormalization)

Cloud & Distributed Computing:

• Experience with AWS cloud platform and managed data services (S3, Redshift, Glue, Lambda, RDS)

• Experience with distributed computing frameworks and big data technologies

• Understanding of cloud cost optimization and resource management

Data Quality & Governance:

• Strong expertise in data cleaning, validation, and quality assurance techniques

• Experience with data profiling tools and methodologies

• Understanding of data governance principles and best practices

• Ability to implement data lineage and metadata management solutions

Reporting & Visualization:

• Experience working with BI tools, particularly Power BI

• Ability to design data models that support efficient reporting and visualization

• Understanding of dimensional modeling for analytical use cases

Professional Skills

• Strong problem-solving abilities with analytical and critical thinking skills

• Excellent communication skills with ability to explain technical concepts to non-technical stakeholders

• Proven ability to work collaboratively in cross-functional teams

• Experience in client handling, including gathering requirements, managing expectations, and providing regular status updates

• Ability to build and maintain strong client relationships through responsive communication and proactive problem-solving work.

Preferred Qualifications

• Bachelor’s degree in computer science, Information Systems, Data Engineering, or related field

• Experience with real-time data streaming and event-driven architectures

• Certifications in AWS :- o AWS Certified Solutions Architect – Associate o AWS Certified Developer – Associate o AWS Certified Solutions Architect – Professional

• Experience with CI/CD practices for data pipelines

• Understanding of data security, encryption, and compliance requirements (GDPR, CCPA)