Hello World Tech Consulting

Job Summary We are seeking a highly skilled Data Engineer with Databricks certifications and extensive practical experience working with the Databricks Lakehouse platform and its supporting capabilities. The ideal candidate will possess a robust understanding of AWS services relevant to data engineering and demonstrate proficiency in data pipeline development, programming, data modeling, and governance. Key Responsibilities • Design and implement data pipelines using AWS and Databricks. • Manage and orchestrate data workflows, ensuring eEicient data ingestion processes. • Utilize Delta Lake for data storage and management, ensuring secure data sharing across platforms with Delta Sharing. • Implement data governance and access control using Unity Catalog. • Apply the Medallion architecture (Bronze, Silver, Gold layers) to projects. • Develop and manage ETL pipelines using AWS Glue. • Leverage Amazon S3 for data storage and management. • Utilize AWS Lambda and Step Functions for serverless computing in data processing. • Apply data warehousing solutions and best practices in data transformation. • Monitor and implement data quality checks and governance practices. • Collaborate with data scientists, analysts, and other stakeholders in an Agile environment. • Deliver large-scale data processing and analytics projects within cloud environments. Required Skills and Qualifications • Databricks certifications with practical experience in the Databricks Lakehouse platform. • Proficient in AWS services: Glue, S3, Lambda, Step Functions, and data warehousing solutions. • Strong programming skills in Python, Scala, or Java. • Proficient in SQL for querying and manipulating data. • Experience in data modeling for Lakehouse architecture. • Knowledge of ETL best practices and data transformation techniques. • Understanding of data governance frameworks and practices. • Proven track record of delivering data engineering projects, particularly in cloud environments. • Experience with large-scale data processing and analytics projects. • Ability to work in Agile teams and collaborate eEectively with various stakeholders. Preferred Qualifications • Advanced AWS certifications. • Experience with additional cloud platforms (e.g., Azure). • Familiarity with other data engineering tools and frameworks.