3649 Azure Databricks Jobs - Page 20

Job Summary: We are looking for a highly hands-on Back-End Engineer with a strong foundation in Python , system design , and practical experience in building and deploying Agentic Generative AI systems . The ideal candidate is passionate about solving complex problems using LLMs , understands the architecture of modern AI agent frameworks like LangChain/LangGraph , and can deliver scalable, cloud-native back-end services with a GenAI focus. Key Responsibilities: Design and implement robust, scalable back-end systems for GenAI agent-based platforms . Work closely with AI researchers and front-end teams to integrate LLMs and agentic workflows into production services. Develop and maintain serv...

As a Technical Lead-AI ML Expert at ZF, you will play a crucial role in shaping the future of mobility by leveraging your expertise in artificial intelligence and machine learning. Here is what you can expect in this role: **Role Overview:** You will lead the development process by owning the design, implementation, deployment, and maintenance of traditional ML and Generative AI solutions. Additionally, you will translate business requirements into data-driven and GenAI-driven use cases, scope features, estimates, and timelines. Your role will also involve mentoring and coaching junior/mid-level engineers on best practices in ML, MLOps, and GenAI. **Key Responsibilities:** - Lead end-to-end ...

As the Principal Information Architect at Fractal, you will serve as the Overall Portfolio Technical Architect. Your responsibilities will include leading and defining the architecture to align with account standards and best practices. You will assess the current technology landscape, lead project implementations, resolve issues, troubleshoot, and maintain project communication plans. Your role will also involve reviewing systems under development, coordinating coding activities, and ensuring adherence to company policies, standards, and technologies. Additionally, you will closely collaborate with the Scrum Master to align project plans, ensure compliance with application architecture stan...

Role Overview: You will serve as the Senior Analytics Engineer at Lennox, taking on the responsibility of designing, developing, and supporting the Azure Databricks platform and Power BI Dashboards & reports. Your key focus will be on independently gathering requirements, designing and building models and reports. You will also be required to interact with various levels of business audiences to clearly define the necessary data to support strategic and tactical decision making. In collaboration with the Data Engineering team, Business Users, and other stakeholders, you will design and deploy end-to-end Analytics solutions to meet business needs. Your involvement in the project will span acr...

**Role Overview:** You will be working as a Senior Azure Data Engineer in the IT services and enterprise data engineering sector, delivering cloud-native analytics, ETL/ELT pipelines, and modern data platform implementations on Microsoft Azure for large enterprise customers. This is an on-site, hands-on engineering role supporting strategic data modernization initiatives across business domains. **Key Responsibilities:** - Design, build and maintain scalable ETL/ELT pipelines using Azure Data Factory and Azure Databricks to ingest, cleanse, and transform large datasets. - Develop PySpark and Python-based data processing jobs; optimize Spark workloads for throughput, latency and cost on Datab...

Role Overview: As a Senior Data Analytics Engineer on the Data Analytics Data Platform (DADP) team, you will be responsible for analyzing, developing, and delivering business intelligence solutions using the Microsoft Analytics stack. Your expertise will be utilized to ensure client expectations and business decision-making tool needs are met. Your role will involve using analytical and problem-solving skills to create scalable and reliable data solutions through data analytics solution development. Key Responsibilities: - Lead and participate in the development of high-quality software solutions for client projects, utilizing modern programming languages and frameworks. - Contribute to syst...

As an IT Data Analyst at Syneos Health, your role will involve: - Writing and understanding complex SQL queries, including joins and CTE, to extract and manipulate data. - Managing and monitoring data pipelines in Azure Data Factory and Databricks. - Understanding the integration with Veeva CRM and Salesforce. - Ensuring data quality checks and business level checks in the data pipeline of ADF and Databricks. - Monitoring and troubleshooting data and pipeline related issues, performing root cause analysis, and implementing corrective actions. - Monitoring system performance and reliability, troubleshooting issues, and ensuring data delivery before committed SLA. - Having a strong understandi...

We looking for a candidate who has hands on experience on Azure Cloud and Microsoft Fabrics. Also candidates should have hands on experience with Python, SQL and Pyspark. Knowledge on Azure Data Factory and Azure Data Bricks is a plus. Required Candidate profile Azure Data Engineer

We are seeking a skilled Application & Platform Support Engineer to provide production support for applications built on Azure Databricks and associated technologies. The role involves ensuring the stability, performance, and resilience of critical data platforms used in Capital Markets. The ideal candidate will have strong system administration, CI/CD expertise, and experience with monitoring, troubleshooting, and automation. Key Responsibilities: Provide 24/7 production support for applications leveraging: Azure Databricks Collibra Kafka Cassandra Azure Kubernetes Service (AKS) Monitor platform health, performance, and security to proactively identify and resolve issues. Troubleshoot and r...

Please find below for reference. Key Qualifications: 10+ years of ETL/data engineering experience, with at least 3+ years on Azure Strong experience with Azure Data Factory, Synapse, ADLS Gen2, and Databricks (PySpark) Solid SQL skills and deep understanding of data modelling (dimensional, star/snowflake) Experience with CI/CD tools (e.g., Azure DevOps), source control, and infrastructure as code for data workflows Familiarity with Azure security features (Key Vault, Managed Identity, RBAC) Proven ability to troubleshoot and optimize large-scale data pipelines Strong documentation and communication skills Preferred (Nice to Have): Experience with real-time data ingestion (Kafka/Event Hubs) A...

As a Microsoft Fabric Professional at YASH Technologies, you will be responsible for: - Experience in Azure Fabric, Azure Data factory, Azure Databricks, Azure Synapse, Azure SQL, ETL - Creating Pipelines, datasets, dataflows, Integration runtimes, and monitoring Pipelines, and trigger runs - Extracting Transformation and Load Data from the source system and processing the data in Azure Databricks - Creating SQL scripts to perform complex queries - Creating Synapse pipelines to migrate data from Gen2 to Azure SQL - Implementing Data Migration pipeline to Azure cloud (Azure SQL) - Handling database migration from on-prem SQL server to Azure Dev Environment using Azure DMS and Data Migration A...

Role Overview: As an IT professional at Capgemini Invent, you will be responsible for creating data warehouses, data lakes, ETL/ELT, and data pipelines on cloud platforms. Your focus will be on delivering cutting-edge solutions tailored to the challenges faced by clients in the Life Sciences Domain. You will leverage your expertise in cloud storage, cloud database, and data warehousing solutions such as Snowflake, BigQuery, AWS Redshift, ADLS, and S3. Additionally, you will use cloud data integration services like Azure Databricks, Azure Data Factory, Azure Synapse Analytics, AWS Glue, AWS EMR, Dataflow, and Dataproc for handling structured, semi-structured, and unstructured data. Your role ...

About Position: We are seeking a highly analytical and detail-oriented Cloud Data Engineer. Role: Cloud Data Engineer Location: All Persistent Locations Experience: 5+ Years Job Type: Full Time Employment What You'll Do: Minimum of 5+ years of experience as a Cloud Data Engineer. Hands-on experience with Azure Cloud data tools (ADF, SHIR, Logic Apps, ADLS Gen2, Blob Storage) and Snowflake. Strong experience in ETL development using on-premises databases ETL technologiesExperience with Python or other scripting languages for data processing.Proficiency in DevOps and CI/CD practices using GitActions. Experience with Agile methodologies. Excellent problem-solving skills and ability to work inde...

Organization Background: At Providence, we are grounded in our goal to serve all as we engineer the future of healthcare. Providence Global Innovation Center, launched in Feb 2020, is the first development and innovation center of Providence outside the United States. Providence is a $26B non-profit organization and is one of the largest health systems in the United States. The ESI Enablement & Platform Technology - Power BI Platform Team provides analytical services to business units and IS Stakeholders. In this role you will engineer, develop, and support the complete lifecycle of the Enterprise PowerBI/ Fabric Services Platform, PowerBI Reports for ESI EPT Service Lines & Strategic Projec...

About Position: We are seeking a highly skilled and experienced Data Engineer with expert-level proficiency in Azure Databricks, Apache Spark, and Azure Data Services. The ideal candidate will have a strong background in building production-grade Delta Live Tables (DLT) pipelines, implementing Medallion Architecture, and managing infrastructure using Terraform. This role demands hands-on experience in real-time data ingestion, structured streaming, and CI/CD practices. Familiarity with the Healthcare domain and Snowflake is a plus. Role: Senior Data Engineer Location: All Persistent Locations Experience: 6+ Years Job Type: Full Time Employment What You'll Do: Design, develop, and maintain sc...

About Position: As a Data Engineer, you will design and manage robust data pipelines using Azure Databricks, Spark, and Kafka. You'll process large-scale datasets and implement CDC pipelines to power analytics platforms. Your role includes modeling data using the Medallion Architecture and ensuring data quality and governance. Collaboration with analytics teams for visualization and insights is key to success. Role: Data Engineer (Azure Databricks) Location: Navi Mumbai Experience: 5 to 11 Years Job Type: Full Time Employment What You'll Do: Design, build, and maintain scalable and efficient data pipelines using Azure Data Factory and Azure Databricks. Implement Change Data Capture (CDC) pip...

Role & responsibilities Understand Requirement, Analyze, Design, Develop, Test and Deploy End to End Azure based Data and Analytics solutions Develop, Test, Debug, Monitor data solutions, like data pipelines, data warehousing, and big data solutions, with Azure Data Fabric and Azure Synapse Analytics Follow Best Practices, Guideline, Review, Tune and optimize Microsoft Azure Synapse Analytics experience is essential. (Dedicated SQL Pool, Azure Data Factory, Azure Storage) End-to-end Data Warehouse experience: ingestion, ETL, big data pipelines, data architecture, message queuing, stream processing, BI/Reporting, and Data Security. Ability to build processes supporting data transformation, da...

What you'll do Develop, maintain, and enhance new data sources and tables, contributing to data engineering efforts to ensure comprehensive and efficient data architecture. Serves as the liaison between Data Engineer team and the Airport operation teams, developing new data sources and overseeing enhancements to existing database; being one of the main contact points for data requests, metadata, and statistical analysis Migrates all existing Hive Metastore tables to Unity Catalog, addressing access issues and ensuring smooth transition of jobs and tables. Collaborate with IT teams to validate package (gold level data) table outputs during the production deployment of developed notebooks Deve...

Educational Requirements Bachelor of Engineering Service Line Application Development and Maintenance Responsibilities A day in the life of an Infoscion As part of the Infosys consulting team, your primary role would be to get to the heart of customer issues, diagnose problem areas, design innovative solutions and facilitate deployment resulting in client delight. You will develop a proposal by owning parts of the proposal document and by giving inputs in solution design based on areas of expertise. You will plan the activities of configuration, configure the product as per the design, conduct conference room pilots and will assist in resolving any queries related to requirements and solutio...

Educational Requirements Bachelor of Engineering Service Line Data & Analytics Unit Responsibilities A day in the life of an Infoscion As part of the Infosys consulting team, your primary role would be to lead the engagement effort of providing high-quality and value-adding consulting solutions to customers at different stages- from problem definition to diagnosis to solution design, development and deployment. You will review the proposals prepared by consultants, provide guidance, and analyze the solutions defined for the client business problems to identify any potential risks and issues. You will identify change Management requirements and propose a structured approach to client for mana...

Role & responsibilities 8 to 10 years of experience is Data Modeling and Design Design and maintain conceptual, logical, and physical data models using tools Exp in Azure Cloud is mandatory Develop dimensional models for analytics and 3NF models for transactional systems. Create and manage source-to-target mappings for ETL/ELT processes. Write and optimize SQL scripts for transformation, and analysis. Collaborate with cross-functional teams to gather requirements and deliver data solutions. Support data integration from various sources including APIs, files, and databases .Contribute to data governance efforts including metadata management and data quality Ensure documentation of data archit...

Metyis is a Netherlands based fast-growing international company with a diverse team of over 700+ professionals with a mix of Analytics, Data Science and Technology Team. We are looking for an experienced Data Engineer to join our team with below requirements: Responsibilities: Independently lead & manage execution of data engineering projects Engineer complete technical solutions to solve concrete business challenges in the areas of digital marketing, eCommerce, Business Intelligence and self-service analytics Collect functional and non-functional requirements, consider technical environments, business constraints and enterprise organizations Support our clients in executing their Big Data ...

Metyis is a Netherlands based fast-growing international company with a diverse team of over 700+ professionals with a mix of Analytics, Data Science and Technology Team. We are looking for an experienced Data Engineer to join our team with below requirements: Responsibilities: Independently execute data engineering projects Diagnose problems, disaggregate issues, develop hypotheses and execute relevant solutions Undertake processing of structured and unstructured data Undertake work related to ELT activities Development of data processing codes for automation & building scalable solutions Develop and maintain data solutions for clients / projects Communicate and interact with clients at the...

Role & responsibilities • Experience in ADF • Experience in SQL Server • Ready to work in Shift (We have 24 X 7 Shift) • Maintaining and monitoring of Data Pipelines • Ability to understand the issues with pipelines and perform root cause analysis • Monitor data quality pillars like uniqueness, completeness, correctness, timeliness, consistency, availability • Good documenting skills for capturing issue logs, sequence of events, root cause analysis and short term, long term fix details • Updating wiki page for each pipeline enhancement or change. • Assisting partner teams on information related to data issues so as to arrive at issue resolution • Send regular updates on status of open issues...

Role & responsibilities Mandatory Skills: Microsoft Fabric, Azure datafactory, Azure Data Lake Storage, Azure data bricks with Py-Spark Good to have: Power BI, Azure IAAS services. Very strong understanding on ETL and ELT Very strong understanding on Lakehouse architecture. Very strong knowledge in Pyspark and Spark architecture. Good knowledge in Azure data lake architecture and access controls Good knowledge in Microsoft Fabric architecture Good knowledge in T-SQL Good knowledge in CI /CD process using Azure devops Power BI Preferred candidate profile