Data Engineer

Role & responsibilities

1. Responsible to maintain the data required for managed application like Advanced Planning System, Dealer Management System, etc.,

2. Responsible for user access management on data upload and download

3. Working with functional team to manage the required data in a structured manner

4. Make use of Azure services like Data Factory, Data Lake, Machine Learning, SQL Warehouse, Data Bricks for data extraction.

5. Building real-time data pipelines from on-prem SAP DB to cloud along with required API.

6. Python model development and continuous improvement through containerization. Conduct basic exploratory analytics for insights.

7. Working with Azure SQL databases and NoSQL Databases (Azure Cosmos DB) for data management

Preferred candidate profile

1. A minimum of 2-3 years of experience in Data engineering and migration

2. Graduation/ Post graduation or equivalent. BE CS / B. Tech IT/ MCA preferred

3. SQL knowledge to create complex queries.

4. Experience in working with relational databases, Time-series Database as well as working familiarity with a variety of databases (structured & Unstructured).

5. Good experience in object-oriented programming in python and using python tools such as PySpark, NumPY, TensorFlow and Jupyter etc.

6. Experience in API creation.

7. Performance Monitoring & Troubleshooting Azure data pipeline performance, identify bottlenecks, and troubleshoot complex issues in production environments.

8. Hands on Experience in working on Kafka.

9. Knowledge on continuous integration/continuous deployment.

10. Experience of data migration and deployment from On-Prem to Cloud environment and vice-versa.

11. Ability to identify problems, mediate issues, develop solutions, and implement appropriate courses of action.