Data Engineer-Customer Analytics AIML

What awaits you/ Job Profile

• · Working closely with Stakeholders and Cross functional teams for the Data analysis.

What should you bring along

• Build data systems and pipelines
• Standardize the data assets which maximize data reusability
• Evaluate business needs and objectives
• Implementing data pipeline to ingest and cleanse raw data from various source systems including SaaS, Cloud based and On-prep databases
• Conduct complex data analysis and report on results
• Combine raw information from different sources
• Explore ways to enhance data quality and reliability
• Implementing functional requirements including incremental loads, data snapshots and identity resolution
• Optimizing data pipelines to improve performance and cost, while ensuring a high quality of data within the data lake:
• Monitoring services and jobs for cost and performance, ensuring continual operations of data pipelines, and fixing of defects.
• Constantly looking for opportunities to optimize data pipelines to improve performance.

Must have technical skill

• · Must have coding skills in Spark/Pyspark, Python and SQL
· Must have Knowledge of AWS tools, Glue, Athena ,step functions and S3

Good to have technical skills

• Good knowledge of CI/CD (Github / Github Actions, Terraform)

• Knowledge on Agile methodologies.