Interesting Job Opportunity: Techno-Comp - Data Engineer - ETL/Python

Description

Data Pipeline Development

• Design, develop, and maintain scalable and robust data pipelines for collecting, processing, and transforming large datasets.
• Implement ETL (Extract, Transform, Load) processes to ensure efficient movement of data across multiple systems.
  

Database Management

• Design, implement, and optimize relational and non-relational databases to support business needs.
• Ensure data quality, consistency, and accuracy by building validation checks, monitoring systems, and automating data reconciliation processes.
  

Cloud And Infrastructure Management

• Work with cloud platforms (AWS, Azure, GCP) to deploy and manage data storage, compute, and processing resources.
  

Experience

• 5.5 years IT experience and good expertise in SDLC/Agile
• 5.5 years experience in SQL, complex queries, and optimization
  

Mandatory Skills And Qualifications Required

• 5.5 years experience in Spark ecosystem, Python/Scala programming, MongoDB data loads, Snowflake and AWS platform (EMR, Glue, S3), Kafka
• Hands on experience in writing advanced SQL queries, familiarity with variety of databases.
• Experience in coding solutions using Python/Spark and performing performance tuning/optimization.
• Experience in building and optimizing Big-Data pipelines in Cloud.
• Experience in handling different file formats like JSON, ORC, Avro, Parquet, CSV.
• Hands on experience in data processing with NoSQL databases like MongoDB.
• Familiarity and understanding of jobs scheduling.
• Hands on experience on working with APIs to process data.
• Understanding of data streaming, such as Kafka services
• 2+ years experience in Healthcare IT projects
• Certification on Snowflake (Snow PRO certification), and AWS (Cloud Practitioner/Solution Architect)
• Hands-on experience on Kafka streaming pipelines implementation.