Data Engineer

The purpose of the job is to develop through iterative process data Integration and analytics solutions. Develop data products answering to Projects/Customer needs, from the data ingestion to ML productization.

You will Build data analytics solutions, developing Data applications (Data ingestion, Cleaning, data processing, etc.) to provide state-of-the-art advanced analytic and quantitative tools and modelling techniques to derive business insights, solve complex business problems and improve decisions.

Reporting to the Data Architect, the role will develop, support and advice on the day-to-day Data platform, Data processing and Data analytics requirements of Business areas and Deployment Projects key operational processes while continually improving the impact of these processes.

SCOPE OF WORK:

Design and develop data pipelines from different sources.

Produce efficient and elegant code that meets business requirements.

Focus on business applications, cost optimization, clients strategic needs and future growth goals.

Data ingestion, data processing, pipelines orchestration, data access, data quality, ML training modules, performance evaluation module: all deployed on AKS cluster.

Create and maintain high quality detailed technical documentation.

Design, implement and deploy customer facing projects on AKS.

Design and implement CI/CD automation pipelines (Gitlab, Helm).

Experience implementing modern cloud data architectures and engineering (our solutions are deployed on Azure and On-Premises)

Responsible for overall data lake stack and MLOps platform, with production deployments.

SKILLS AND EDUCATION:

Essential: Degree in Computer Science or Engineering is required for this role. You should have minimum 2-3 years of working experience in developing Data applications, developing software for different sub system. Experience in Data processing applications based on Python / Pyspark architectures. Strong experience in programming is required for this role (Agile methodologies). Experience in big data, Data Integration or Data analytics application development. Implementing and deploying Data processing and advanced analytics (Machine Learning/statistics) solutions based on MLOps for large enterprise or commercial customers in complex heterogeneous environments. Hands-on experience in the Big Data stack and related technologies:

Programming using Python.

Understanding of enterprise data concepts such as Master Data Management and Enterprise Data Warehouse.

Database application development, using SQL and NoSQL stores.

Understanding of SQL, report generation using visualization tools such as Tableau/Power BI.

Experience designing and implementing data pipelines for consumption by machine learning and BI solutions.

Python

Delta

GitLab(CI)/Helm(CD)

SQL Databases / NoSQL / Blob Storage

Azure

Kafka

AKS and Docker.

Visualization and BI tools (PowerBI)

Desired:

Pyspark

Real projects' hands-on experience with Azure PaaS solutions.

Good understanding of RBAC and IAM Access Control

Experience with Azure data stores (e.g., Azure Data Lake GEN2, Azure SQL Data Warehouse, Azure Blob Storage), messaging systems (Azure Event Hubs and Queues), and data processing engines (Azure Data Lake Analytics, Azure HDInsight).

Experience implementing cloud data architecture and data integration patterns like Azure Data Factory, Databricks, etc.

Experience developing/deploying Machine Learning pipelines. Azure ML model execution architecture

Others requirements:

- High level of English (daily working language). Working understanding of French is a plus.

- Availability to travel as required