4863 Azure Databricks Jobs - Page 44

Hire Type: Fulltime Tech Stack: Proficiency in Python, SQL, Databricks, Snowflake Data Modeling, ETL process, Apache Spark and PySpark, data integration and workflow orchestration, real time data processing experience/frameworks, Cloud experience (Azure preferred) Job Description Minimum 5 years of experience in Data Engineering role that involves Analyzing, organizing raw data, and building data systems & pipelines in Cloud Platform [Azure] Experienced in migrating data from on-prem to cloud-based solution architectures [Azure] Extensive experience with Python, Spark and SQL Experience in developing ETL processes. Proficient with Azure Data Lake, Azure Data Factory, Azure SQL, Azure Databri...

Description: experience in AI architecture and machine learning solution design, with a strong focus on Generative AI applications. Experience designing and deploying AI models using Azure cloud services and frameworks like LangChain and OpenAI. Strong understanding of Natural Language Processing (NLP) and Generative AI principles. Lead the design and architecture of Generative AI solutions, ensuring they meet business needs, scalability, and security requirements. Design systems leveraging LangChain, OpenAI models, and Azure Cognitive Services to integrate AI into enterprise workflows. Build end-to-end AI solutions, including data processing, model development, deployment, and monitoring. E...

Seeking an Azure Data Engineer with expertise in data architecture and .NET development. Responsibilities include building and optimizing data pipelines, as well as integrating .NET applications with cloud data solutions. Experience with Azure Data Factory, SQL, and .NET Core required. Strong data engineering and development skills are essential for the role. Location: Flexible.

JR REQ--- Azure Data Engineer --4to8year---HYD,BANGALORE----Naveenraja A S --TCS C2H ---900000

We are seeking a highly skilled Data Engineer with deep experience in Python, PySpark, Scala, Azure Databricks, SQL Server, NoSQL (Cassandra, etc.) , and data streaming technologies . The ideal candidate will design and maintain scalable batch and real-time data pipelines , enabling efficient data ingestion, transformation, and analytics in a cloud environment. Key Responsibilities Design, develop, and maintain data pipelines (batch and real-time) using PySpark , Scala , and Azure Databricks . Build and optimize data ingestion frameworks integrating streaming and static data sources. Implement real-time data streaming solutions using tools like Apache Kafka , Azure Event Hubs , or Spark Stru...

Job Summary: Technical Experience Should have experience on working in Datawarehouse/ETL Testing Big data testing and BI testing in Snowflake Azure ADF ALS Snowpark and Synapse Experience in Writing complex Spark SQL queries/ Hive QL SQL performance tuning and validation of DWH Experience in data analysis including profiling auditing balancing and reconciliation. Experience in load testing of ETL pipeline and backend system with performance tuning indexes partitions query tuning) Experience in Snowflake Azure ADF Azure Databricks Experience in Test Management and defect management tools Design Test cases Test Data and perform ETL test execution and reporting Experience in Unix and Proficienc...

Advanced Data Engineer :Azure Databricks + Pyspark / Python + SQL + Data Modeling. ETLL Process, Apache Spark, Pyspark, Data Integration and workflow orchestration , real time data processing experience and frameworks, Azure cloud experience

Experienced Azure Data Engineer Tech Lead ( Tech Lead ) with 7 to 10 years of expertise Design, develop, and maintain robust data pipelines using Azure Databricks, Azure Data Factory, Pyspark / Python, Azure Data Lake Gen2, and Azure SQL Database. Integrate and manage data from various source systems, ensuring data quality and consistency. Mandatory Skills : Azure Databrick + Pyspark + Azure Data Facory + SQL +CICD + Strong Coding skills.

Azure Databricks: 1. Mounting Files 2. PAT Tokens 3. Cluster Auto scaling 4. Cluster Types 5. Nesting notebooks 6. Reading file using default and custom schema 7. Types Secrets and scopes in databricks 8. Delta Tables 9. Basic file loading scenario with its steps 10. Optimization 11. User defined functions 12. Transformations ADF: 1. Integration runtimes familiar with and where they are used 2. Components in creating pipelines 3. Scenario based question for creating a basic pipeline for loading data to Azure SQL table 4. Basic steps to improve the loading activity of a huge amount of file , ex: 1TB file size

Key Responsibilities Design, develop, and maintain ETL/ELT pipelines using Azure Data Factory , Databricks , and related Azure services. Integrate data from various sources (SQL, APIs, Blob Storage, Data Lake, etc.) into a centralized data warehouse or data lake. Build and manage data ingestion , transformation , and orchestration workflows . Optimize ADF pipelines for performance, scalability, and cost efficiency. Collaborate with business and analytics teams to understand data requirements and deliver end-to-end solutions. Implement and manage data quality , monitoring , and error-handling frameworks . Work closely with Azure Synapse , SQL Database , Power BI , and Azure Data Lake Storage ...

The Azure Data Bricks Engineer plays a critical role in establishing and maintaining an efficient data ecosystem within an organization. This position is integral to the development of data solutions leveraging the capabilities of Microsoft Azure Data Bricks. The engineer will work closely with data scientists and analytics teams to facilitate the transformation of raw data into actionable insights. With increasing reliance on big data technologies and cloud-based solutions, having an expert on board is vital for driving data-driven decision-making processes. The Azure Data Bricks Engineer will also be responsible for optimizing data workflows, ensuring data quality, and deploying scalable d...

The Azure Data Bricks Engineer plays a critical role in establishing and maintaining an efficient data ecosystem within an organization. This position is integral to the development of data solutions leveraging the capabilities of Microsoft Azure Data Bricks. The engineer will work closely with data scientists and analytics teams to facilitate the transformation of raw data into actionable insights. With increasing reliance on big data technologies and cloud-based solutions, having an expert on board is vital for driving data-driven decision-making processes. The Azure Data Bricks Engineer will also be responsible for optimizing data workflows, ensuring data quality, and deploying scalable d...

The Azure Data Bricks Engineer plays a critical role in establishing and maintaining an efficient data ecosystem within an organization. This position is integral to the development of data solutions leveraging the capabilities of Microsoft Azure Data Bricks. The engineer will work closely with data scientists and analytics teams to facilitate the transformation of raw data into actionable insights. With increasing reliance on big data technologies and cloud-based solutions, having an expert on board is vital for driving data-driven decision-making processes. The Azure Data Bricks Engineer will also be responsible for optimizing data workflows, ensuring data quality, and deploying scalable d...

The Azure Data Bricks Engineer plays a critical role in establishing and maintaining an efficient data ecosystem within an organization. This position is integral to the development of data solutions leveraging the capabilities of Microsoft Azure Data Bricks. The engineer will work closely with data scientists and analytics teams to facilitate the transformation of raw data into actionable insights. With increasing reliance on big data technologies and cloud-based solutions, having an expert on board is vital for driving data-driven decision-making processes. The Azure Data Bricks Engineer will also be responsible for optimizing data workflows, ensuring data quality, and deploying scalable d...

The Azure Data Bricks Engineer plays a critical role in establishing and maintaining an efficient data ecosystem within an organization. This position is integral to the development of data solutions leveraging the capabilities of Microsoft Azure Data Bricks. The engineer will work closely with data scientists and analytics teams to facilitate the transformation of raw data into actionable insights. With increasing reliance on big data technologies and cloud-based solutions, having an expert on board is vital for driving data-driven decision-making processes. The Azure Data Bricks Engineer will also be responsible for optimizing data workflows, ensuring data quality, and deploying scalable d...

The Azure Data Bricks Engineer plays a critical role in establishing and maintaining an efficient data ecosystem within an organization. This position is integral to the development of data solutions leveraging the capabilities of Microsoft Azure Data Bricks. The engineer will work closely with data scientists and analytics teams to facilitate the transformation of raw data into actionable insights. With increasing reliance on big data technologies and cloud-based solutions, having an expert on board is vital for driving data-driven decision-making processes. The Azure Data Bricks Engineer will also be responsible for optimizing data workflows, ensuring data quality, and deploying scalable d...

Responsibilities: * Design data solutions using Azure Databricks, Synapse & DW. * Collaborate with cross-functional teams on project delivery. * Optimize performance through data modeling and ETL processes.

About the Role We are seeking an experienced Big Data & Azure Engineer to join our data platform team. This role is critical to supporting and optimizing our enterprise-scale data infrastructure, managing migrations from on-premises Big Data systems to Azure cloud services, and ensuring seamless 24x7 production operations. The ideal candidate will combine strong technical expertise in both traditional Big Data technologies and modern cloud platforms with excellent problem-solving abilities. Required Skills and Qualifications Technical Expertise : Strong experience in Big Data technologies (Hive, Impala). Hands-on experience with Azure services : Network, VMs, Synapse, Databricks. Proficiency...

job description Technical Experience experience in Data Platform Engineering Extensive coding experience (3+ years) including Python/Pyspark (1 years +) Experience developing in Azure with key data technologies (e.g. ADLS, ADF, Azure Databricks etc.) Software development methodologies Experience working with at least one DevOps tool (GIT, Azure DevOps, Maven, Jenkins) Prior roles that demonstrate utilisation of high-quality Agile development best practices. In depth knowledge of at least one scheduling tool (e.g. Control-M, Oozie, ADF, etc.) Data lake, JIRA experience. Experience working on Agile projects Responsibilities Design, build, implement and maintain data pipeline components for the...

The Azure Data Bricks Engineer plays a critical role in establishing and maintaining an efficient data ecosystem within an organization. This position is integral to the development of data solutions leveraging the capabilities of Microsoft Azure Data Bricks. The engineer will work closely with data scientists and analytics teams to facilitate the transformation of raw data into actionable insights. With increasing reliance on big data technologies and cloud-based solutions, having an expert on board is vital for driving data-driven decision-making processes. The Azure Data Bricks Engineer will also be responsible for optimizing data workflows, ensuring data quality, and deploying scalable d...

Position : Azure Data Architect Experience: 12 to 15 Years Tech Stack: Azure Databricks (Data Fabric is a Plus) Role Overview: Seeking a highly skilled Azure Data Architect with deep expertise in Databricks to lead enterprise data modernization and migration projects. The ideal candidate will serve as a hands-on technical leaderdefining architecture, guiding delivery teams, and ensuring alignment between client strategy and technical implementation. Key Responsibilities: Lead data modernization and migration projects, from architecture design to deployment and optimization. Architect scalable, secure, and efficient data platforms leveraging Azure Databricks and Azure Data Services. Provide t...

Role Summary : As a Snowflake Subject Matter Expert (SME), you will be responsible for architecting, developing, and optimizing data warehouse and lakehouse solutions using the Snowflake platform. Your expertise will enable the data team to build a modern, scalable, and high-performance data platform that supports business analytics, data science, and reporting needs. You will work closely with data architects, engineers, and business stakeholders to ensure robust data ingestion, storage, and access frameworks that meet security, compliance, and performance standards. Key Responsibilities : - Design and implement Snowflake data warehouse and lakehouse architectures. - Develop and optimize Sn...

Are you ready to turn data into impact ? Vanderlande is looking for a Data & AI Engineer passionate about Machine Learning, GenAI, RAG, and LLM technologies . You will design and build scalable Azure-based data platforms , develop ML models , and work on cutting-edge AI solutions that power global logistics automation. Role & responsibilities Design, build, and maintain Azure-based data platform . Develop and manage data pipelines using Azure and Databricks. Build and deploy secure, scalable ML models (Python, PySpark, SQL). Process unstructured data (e.g., parsing, Retrieval-Augmented Generation). Implement and maintain CI/CD pipelines for data and ML workloads. Ensure platform performance ...

Roles and Responsibilities Design, develop, test, and deploy scalable data pipelines using Azure Databricks, Azure Data Factory, and Spark. Collaborate with cross-functional teams to gather requirements and deliver high-quality solutions. Troubleshoot complex technical issues related to big data processing and analytics. Stay up-to-date with industry trends and best practices in Scala programming language. Participate in code reviews and contribute to the improvement of existing systems. Desired Candidate Profile 6-11 years of experience as a Data Engineer or similar role. B.Tech/B.E. degree from any specialization (or equivalent). Strong expertise in Scala technologies such as Spark, Scala ...

As a Data Architect at Diageo, you will play a key role in transforming business capabilities by leveraging data and digital technology. Your responsibilities will include: - Proficiency in the overall IT landscape and technologies, with the ability to analyze how different technologies integrate. - Working experience in data modeling using Kimball methodologies, Schema design (Star schema, Snowflake schema), and Data Architectures (Vault, Mesh, etc.). - Experience with enterprise data management, Data-warehouse, Data modeling, Business Intelligence, and Data integration. - Strong capabilities to review data models for completeness, quality, and compliance with established architecture stand...