Data Engineer

Primary & Secondary Skill Databricks – Pyspark, Python and Collibra ( Primary) Unity catalog ETL AWS JD (Detailed) Design of data solutions on Databricks including delta lake, data warehouse, data marts and other data solutions to support the analytics needs of the organization. Proficiency in using Collibra Data Governance Center, Data Catalog, and Collibra Connect for data management and governance. Apply best practices during design in data modeling (logical, physical) and ETL pipelines (streaming and batch) using cloud-based services especially Python & Pyspark Design, develop and manage the pipelining (collection, storage, access), data engineering (data quality, ETL, Data Modelling) and understanding (documentation, exploration) of the data. Interact with stakeholders regarding data landscape understanding, conducting discovery exercises, developing proof of concepts, and demonstrating it to stakeholders. Implement data quality frameworks and standards using Collibra to ensure the integrity and accuracy of data Excellent collaboration skills to work effectively with cross-functional teams Strong verbal and written communication skills Show more Show less