Area Manager-Data Engineering-Cloud Data Engineering

As a Data Engineer, you will be responsible for designing, developing, and maintaining scalable and efficient data pipelines using PySpark and Databricks. Your key responsibilities will include: - Performing data extraction, transformation, and loading (ETL) from diverse structured and unstructured data sources. - Developing and maintaining data models, data warehouses, and data marts in Databricks. - Being proficient in Python, Apache Spark, and PySpark. - Integrating third party data from multiple sources with internal data. - Writing and optimizing complex SQL queries for high performance and scalability across large datasets. - Collaborating closely with data scientists, analysts, and business stakeholders to gather and understand data requirements. - Ensuring data quality, consistency, and integrity throughout the data lifecycle using validation and monitoring techniques. - Developing and maintaining modular, reusable, and well-documented code and technical documentation for data workflows and processes. - Implementing data governance, security, and compliance best practices. If there are any additional details about the company in the job description, please provide that information separately.,