Aws Databricks

As a Data Engineer with 5+ years of hands-on experience in data engineering/ETL using Databricks on AWS/Azure cloud infrastructure and functions, your key responsibilities will include: - Performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement. - Implementing highly performant data ingestion pipelines from multiple sources using data warehousing concepts such as Dimensional (star-schema), SCD2, Data Vault, Denormalized, OBT. - Demonstrating a strong proficiency in Python and SQL for data manipulation and analysis. - Utilizing the features of the Databricks platform including Delta Lake, Databricks SQL, and MLflow. - Integrating the end-to-end Databricks pipeline to ensure the quality and consistency of data from source systems to target data repositories. - Collaborating within an Agile delivery/DevOps methodology to deliver proof of concept and production implementation in iterative sprints. - Having a basic working knowledge of API or Stream-based data extraction processes like Salesforce API and Bulk API. - Understanding and applying Data Management principles such as data quality, governance, security, privacy, life cycle management, and cataloguing. Qualifications required for this role include: - 5+ years of hands-on experience in data engineering/ETL using Databricks on AWS/Azure cloud infrastructure. - 3+ years of experience in Power BI and Data Warehousing. - Strong proficiency in Python and SQL. - Experience with CI/CD on Databricks using tools such as BitBucket, GitHub Actions, and Databricks CLI. - Nice to have: Databricks certifications and AWS Solution Architect certification. - Nice to have: experience with building data pipeline from various business applications like Salesforce, Marketo, NetSuite, Workday etc.,