Job
Description
About The Role
Project Role :Data Engineer
Project Role Description :Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems.
Must have skills :Amazon Web Services (AWS)
Good to have skills :NA
Minimum 3 year(s) of experience is required
Educational Qualification :15 years full time education
Summary :As a Data Engineer, you will play a pivotal role in integrating, and maintaining the infrastructure needed to support data processing and analysis within the client ecosystem. You will collaborate with data scientists, analysts, and other stakeholders to ensure data availability, integrity, and usability for business intelligence and analytics purposes.
Roles & Responsibilities:
Integrate and unify various Database systems into the client ecosystem by implementing & maintaining scalable ETL pipelines to process large volumes of data. Ensuring data governance and security policies.Work in cross-functional teams (data scientists, analysts, and business partners) to provide them with solutions; communicate to non-technical stakeholders. Develop and optimize data architectures and data transformation technologies. to support analytical needs.Manage and optimize databases for performance, scalability, and availability.Integrate data from various sources, including APIs, external cloud platforms, and internal data systems.Identify and resolve bottlenecks in data processing, optimizing the performance of databases and data pipelines.Implement monitoring and logging solutions to proactively address data reliability and quality issues.Professional & Technical
Skills:AWS, Database management systems (e.g., SQL - a must, NoSQL, PostgreSQL, MongoDB), ,Python a must, Kafka, big data technologies (Hadoop, pySpark), data warehousing solutions (e.g., Snowflake, Redshift, BigQuery), DevOps tools (gitlab, Docker, Kubernetes).Strong analytical skills and a proactive approach to problem-solving.Ability to work independently and as part of a collaborative team. Able to coach junior team.
Additional Information:The candidate should have minimum 8 years of experience in Amazon Web Services (AWS).Must have skills :AWS Services , -
Good to have skills Python , Kafka , DevOps tools Qualification 15 years full time education