Title AWS Databricks

You should have at least 5 years of hands-on experience in data engineering/ETL using Databricks on AWS/Azure cloud infrastructure and functions. Additionally, you must possess a minimum of 3 years of experience in Power BI and Data Warehousing, with the ability to perform root cause analysis on internal and external data and processes to address specific business questions and identify areas for improvement. It is preferred that you have experience with AWS services such as S3, Athena, Glue, and Lambda. A deep understanding of data warehousing concepts including Dimensional (star-schema), SCD2, Data Vault, Denormalized, OBT is essential for implementing highly performant data ingestion pipelines from multiple sources. Proficiency in Python and SQL is required for this role. You should also have a thorough understanding of Databricks platform features such as Delta Lake, Databricks SQL, and MLflow. Experience with CI/CD on Databricks using tools like BitBucket, GitHub Actions, and Databricks CLI is expected. Your responsibilities will include integrating the end-to-end Databricks pipeline to ensure data quality and consistency while moving data from source systems to target data repositories. Working within an Agile delivery/DevOps methodology to deliver proof of concept and production implementation in iterative sprints is also a key aspect of this role. Experience with Delta Lake, Unity Catalog, Delta Sharing, Delta Live Tables (DLT), and MLflow is highly beneficial. Basic knowledge of API or Stream-based data extraction processes like Salesforce API and Bulk API is also required. An understanding of Data Management principles including data quality, governance, security, privacy, life cycle management, and cataloging is essential. Databricks certifications and AWS Solution Architect certification would be a plus. Experience with building data pipelines from various business applications like Salesforce, Marketo, NetSuite, and Workday is also desirable.,