Data Engineer

Data Engineer Bengaluru, Karnataka, India Job Description Key responsibilities include, but are not limited to: Design and implement end-to-end data engineering solutions on Azure, including Azure Data Factory (ADF), Azure Databricks, Synapse Analytics and Other Azure Services. Develop and maintain data pipelines that extract, transform, and load (ETL) data from various sources into Azure data repositories, which may include data lakes. Optimize and fine-tune data pipelines for performance, scalability, and cost-efficiency. The ability to automate tasks and deploy production standard code (with unit testing, continuous integration, versioning) Load transformed data into storage and reporting structures in destinations including data warehouse, high speed indexes, real-time reporting systems and analytics applications. Ensuring data quality and accuracy through testing and validation Implementing data security measures, including encryption, access controls, and auditing, to protect sensitive information. Providing guidance and support for data governance, including metadata management, data lineage, and data cataloging. Collaborate with cross-functional teams to understand data requirements and translate them into scalable data architectures. Troubleshoot and resolve data pipeline issues and ensure minimal downtime. Utilize SQL for data transformation and perform data modeling when necessary. Create and maintain comprehensive documentation of data processes and architectures. Required Skills ETL, Data Analysis, Business Analysis