Azure Data Engineer

We are seeking a skilled Data Engineer with experience to join the AI R and D team at Infosys. The ideal candidate will have strong expertise in Python, SQL, and Azure Data Engineering, with a solid foundation in data structures, algorithms, and data modeling principles. This is a fully remote, day-shift role.

Responsibilities:
• Design and enhance information models (e. g., PO, Invoice, SES, RO, Vendor) based on functional design specifications.• Perform technical and functional validation of information model objects.• Troubleshoot and resolve coding issues, ensuring data integrity and smooth data flow.• Manage and support daily data loads from SAP and non-SAP systems into the information model.• Collaborate with cross-functional teams to build, optimize, and scale data pipelines in Azure.• Integrate and orchestrate data pipelines using Azure Data Factory and Databricks.• Ensure performance optimization using advanced SQL techniques, index optimization, and data partitioning.

Requirements:
• A minimum of 4 years of experience in data engineering and software engineering with a focus on building business data products.• Excellent command of Python and SQL for data processing and analytics.• Strong experience working within Azure Data Engineering platforms and tools.• Exposure to SAP systems is a plus but not mandatory.• Experience in product-based companies is required.

Python Expertise:
• Proficiency in Python, including experience with coding challenges such as decorators, generators, sliding windows, and dictionary comprehension.• Strong knowledge of PySpark, including coding with RDDs and transformation logic.• Expertise in Pandas and Numpy libraries for data manipulation (e. g., reading files, creating DataFrames, filtering, interpolation).

SQL Proficiency:
• Advanced SQL skills, with expertise in complex queries, joins window functions, partitioning, indexing, and aggregate functions.• Strong understanding of SQL optimizations and index fragmentation.

Azure Data Engineering:
• Hands-on experience with Azure Data Factory for pipeline orchestration.• Experience with Azure Databricks for large-scale data processing.• Proficiency with Azure Data Lake Storage for efficient data management.• Familiarity with Logic Apps, Azure Key Vault, and Azure Synapse.