Job
Description
As an ideal candidate for this role, you should have strong hands-on experience with Microsoft Azure in enterprise-scale deployments. You must possess expert-level experience with Azure Databricks in production environments and advanced SQL skills with experience in modern data warehouses. Additionally, you should have proven experience in designing and supporting large-scale batch and streaming pipelines. Your deep hands-on experience with ETL/ELT architectures using Spark and Azure Data Factory is crucial for this position. Proficiency in Python for distributed data processing is also required. Your role will involve working with various Azure services such as Azure Data Lake, Blob Storage, Azure Synapse, Snowflake, and Cosmos DB. You should be familiar with Databricks MosaicAI and supporting ML/Generative AI data pipelines. Experience in preparing, optimizing, and governing data for AI model training, inference, and RAG-style workloads is highly beneficial for this position. Preferred qualifications include experience in streaming (Spark Structured Streaming, Kafka), CI/CD experience for Databricks (Azure DevOps, GitHub Actions), Azure Data Engineering certifications, and Databricks certifications. In summary, the key responsibilities for this role include: - Strong hands-on experience with Microsoft Azure in enterprise-scale deployments - Expert-level experience with Azure Databricks in production environments - Advanced SQL skills and experience with modern data warehouses - Proven experience designing and supporting large-scale batch and streaming pipelines - Deep hands-on experience with ETL/ELT architectures using Spark, Azure Data Factory - Strong proficiency in Python for distributed data processing - Experience working with Azure Data Lake, Blob Storage, Azure Synapse, Snowflake, Cosmos DB - Familiarity with Databricks MosaicAI and supporting ML/Generative AI data pipelines - Experience preparing, optimizing, and governing data for AI model training, inference, and RAG-style workloads If you meet these requirements and possess the preferred qualifications, you could be a great fit for this position. As an ideal candidate for this role, you should have strong hands-on experience with Microsoft Azure in enterprise-scale deployments. You must possess expert-level experience with Azure Databricks in production environments and advanced SQL skills with experience in modern data warehouses. Additionally, you should have proven experience in designing and supporting large-scale batch and streaming pipelines. Your deep hands-on experience with ETL/ELT architectures using Spark and Azure Data Factory is crucial for this position. Proficiency in Python for distributed data processing is also required. Your role will involve working with various Azure services such as Azure Data Lake, Blob Storage, Azure Synapse, Snowflake, and Cosmos DB. You should be familiar with Databricks MosaicAI and supporting ML/Generative AI data pipelines. Experience in preparing, optimizing, and governing data for AI model training, inference, and RAG-style workloads is highly beneficial for this position. Preferred qualifications include experience in streaming (Spark Structured Streaming, Kafka), CI/CD experience for Databricks (Azure DevOps, GitHub Actions), Azure Data Engineering certifications, and Databricks certifications. In summary, the key responsibilities for this role include: - Strong hands-on experience with Microsoft Azure in enterprise-scale deployments - Expert-level experience with Azure Databricks in production environments - Advanced SQL skills and experience with modern data warehouses - Proven experience designing and supporting large-scale batch and streaming pipelines - Deep hands-on experience with ETL/ELT architectures using Spark, Azure Data Factory - Strong proficiency in Python for distributed data processing - Experience working with Azure Data Lake, Blob Storage, Azure Synapse, Snowflake, Cosmos DB - Familiarity with Databricks MosaicAI and supporting ML/Generative AI data pipelines - Experience preparing, optimizing, and governing data for AI model training, inference, and RAG-style workloads If you meet these requirements and possess the preferred qualifications, you could be a great fit for this position.
