4854 Azure Databricks Jobs - Page 43

Azure Data Engineer Skills:Azure Data Factory, Data Bricks, Data Warehousing, Data Modelling, SQL, ETL, Python, Pyspark Location-Pune/ Bangalore Exp-5+ Years Pkg-15 LPA MAX Send CV: riya.imaginators@gmail.com

You will have good knowledge of Linux/Unix/Windows administration including Patch management. You will have good knowledge of Azure cloud infrastructure. You will have experience of containerization and orchestration (Docker Kubernetes). You will have experience with configuration management tools Ansible and IAC using Terraform. You will also have experience with monitoring in Azure Monitor datadog or Appdynamics. You will have experience with continuous integration and continuous delivery (CI/CD Azure DevOp's). You will have experience with scripting languages such as (Python Bash). You will also have experience with building ETL Pipelines for Azure Databricks Azure DataFactory Databricks ...

Description: Data Engineer Expertise in Azure data platform skills include Data Factory Databricks Synapse (Azure SQL DW) and Azure Data Lake Python/R/Scala . Expertise in traditional and NoSQL Database. Able to deliver as Technical lead Hands-on experience on Azure modern data platform that includes Data Factory Databricks Synapse (Azure SQL DW) and Azure Data Lake Meaningful experience of data analysis and transformation using Python/R/Scala on Azure Databricks or Apache Spark Well versed NoSQL data store concepts Good knowledge in Distributed Processing using Databricks (preferred) or Apache Spark Ability to debug using tools like Ganglia UI expertise in Optimizing Spark Jobs The ability ...

Job Description SQL DBA The SQL DBA will be responsible for managing day to day operations Technical Skills: SQL DBA experience with MS SQL and Azure Data Warehouse knowledge will be added advantage Well versed with SQL Administrative tasks Performance Tuning, Query Optimization, using Performance Monitor, SQL Profiler and other related troubleshooting tools Indexes, index management, integrity checks, configuration, patching SQL database backup, restore and recoveries Configuration of SQL Servers; Clustering, log shipping, mirroring, etc 6 Database designs tables, datatypes, stored procedures, functions, etc Hands on knowledge on Query performance tuning Critical incident support and intera...

Job Description: Data Engineer (Azure Databricks) About the Role We are looking for a passionate Data Engineer experienced in building scalable, high-performance data platforms on Azure Databricks . The ideal candidate will have strong hands-on experience in ETL/ELT pipeline development , data modeling , and performance optimization , with a focus on delivering clean, reliable, and business-ready data. Key Responsibilities Design, develop, and optimize end-to-end ETL/ELT pipelines using Azure Databricks, Synapse, and Data Factory (ADF) Implement Medallion / Delta Lake architectures ensuring quality and consistency of data Build and maintain data lake and warehouse solutions supporting analyt...

Description: 10+ years of relevant IT experience in the BI/DW domain with minimum 3 years of hands-on experience on Azure modern data platform that includes Data Factory Databricks Synapse (Azure SQL DW) and Azure Data Lake Meaningful experience of data analysis and transformation using Python/R/Scala on Azure Databricks or Apache Spark Well versed NoSQL data store concepts Good knowledge in Distributed Processing using Databricks (preferred) or Apache Spark Ability to debug using tools like Ganglia UI expertise in Optimizing Spark Jobs The ability to work across structured semi-structured and unstructured data extracting information and identifying linkages across disparate data sets Expert...

Provide support for Azure resources, focusing on PaaS and IaaS components, to ensure optimal performance and reliability. Troubleshoot and resolve environment and application-level issues in a timely and effective manner. Work with Azure Kubernetes Services (AKS), Container Registry, App Service, Function Apps, Load Balancer, Application Gateway, Traffic Manager, Azure Front Door, Azure Databricks, Storage Accounts, and Azure Data Factory. Manage and maintain virtual machines (VMs), including both Linux and Windows environments. Utilize automation tools such as PowerShell and Azure CLI to streamline operations and deployment processes. Experience with Terraform for infrastructure as code is ...

Description: Will be responsible for the design development and administration of data management solutions that align with the business needs of the organization. Design relational databases to support business enterprise applications and physical data modeling according to project requirements for data acquisition and security as well as customer-defined deliverables. Has at least 2 years of experience in Data Engineering using ADF to orchestrate and development of data pipelines speically for ingestion harmonisation and curation of the source data. This includes good hands on experience on writing Stored procedures complex views performance tuning of the SQL good underastanding and manage...

Architectural Design:Develop and design scalable reliable and secure Azure-based solutions that align with business requirements and technical constraints Solution Implementation:Lead the implementation of Azure solutions including virtual networks storage databases and compute resources ensuring best practices and high performance Consultation:Collaborate with stakeholders including business leaders IT teams and developers to gather requirements and provide technical guidance and solutions Integration:Design and oversee the integration of Azure services with existing systems and third-party applications Optimization:Continuously evaluate and optimize Azure solutions for performance cost and...

Description: 10+ years of relevant IT experience in the BI/DW domain with minimum 3 years of hands-on experience on Azure modern data platform that includes Data Factory Databricks Synapse (Azure SQL DW) and Azure Data Lake Meaningful experience of data analysis and transformation using Python/R/Scala on Azure Databricks or Apache Spark Well versed NoSQL data store concepts Good knowledge in Distributed Processing using Databricks (preferred) or Apache Spark Ability to debug using tools like Ganglia UI expertise in Optimizing Spark Jobs The ability to work across structured semi-structured and unstructured data extracting information and identifying linkages across disparate data sets Expert...

Description experience in AI architecture and machine learning solution design, with a strong focus on Generative AI applications. Experience designing and deploying AI models using Azure cloud services and frameworks like LangChain and OpenAI. Strong understanding of Natural Language Processing (NLP) and Generative AI principles. Lead the design and architecture of Generative AI solutions, ensuring they meet business needs, scalability, and security requirements. Design systems leveraging LangChain, OpenAI models, and Azure Cognitive Services to integrate AI into enterprise workflows. Build end-to-end AI solutions, including data processing, model development, deployment, and monitoring. En...

Azure data factory 6 to 9 yrs Bangalore

JPC REQ-- AZURE DATA ENGINEER ---4-6 YEARS--- BANGALORE --- CONGNIZANT C2H ---reshma - 30 LPA ---- Sharvin

• Design and implement scalable, secure, and high-performance data pipelines using Azure-native services and Azure Databricks. • Develop and maintain ETL/ELT processes to ingest, transform, and integrate data from diverse sources. • Build and optimize data workflows using Databricks notebooks, PySpark, and SparkSQL for both batch and streaming data. • Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver actionable insights. • Monitor, troubleshoot, and resolve issues in data workflows and pipelines. • Implement data governance, security, and privacy best practices. • Contribute to DevSecOps practices, including CI/CD automation and...

Hi Work Location : Chennai AND Bangalore Work location : Imm - 30 days Primary: Azure Databricks,ADF, Pyspark SQL Sharing JD for your reference : Overall, 6-12 yrs of IT experience preferably in cloud Min 4 years in Azure Databricks on development projects Should be 100% hands on in Pyspark coding Should have strong SQL expertise in writing advanced/complex SQL queries DWH experience is a must for this role Experience in programming using Python is an advantage Experience in data ingestion, preparation, integration, and operationalization techniques in optimally addressing the data requirements Should be able to understand system architecture which involves Data Lakes, Data Warehouses and Da...

Databriks + Pyspark About The Role 1 Skill:Databriks + Pyspark Exp 5 to 8 yr Location Any PSL , Bangalore, Delhi, Pune , Chennai Dont look for Hyderabad Location Azure Data bricks,PySpark,Oracle PL SQL,ETL,SQL,Data Integration Data Engineer - Data AnalystSummaryStrategically manages and maintains corporate global clinical data for all major business units, while ensuring data quality and integrity. Provides system support, testing, and training for applications accessing clinical data.Proven 2-5 years work experience as a Data Analyst in Clinical data Identifies, investigates, and resolves data quality issues, communicating with business units and IT and performing root cause analysis as nec...

Hire Type: Fulltime Tech Stack: Proficiency in Python, SQL, Databricks, Snowflake Data Modeling, ETL process, Apache Spark and PySpark, data integration and workflow orchestration, real time data processing experience/frameworks, Cloud experience (Azure preferred) Job Description Minimum 5 years of experience in Data Engineering role that involves Analyzing, organizing raw data, and building data systems & pipelines in Cloud Platform [Azure] Experienced in migrating data from on-prem to cloud-based solution architectures [Azure] Extensive experience with Python, Spark and SQL Experience in developing ETL processes. Proficient with Azure Data Lake, Azure Data Factory, Azure SQL, Azure Databri...

Description: experience in AI architecture and machine learning solution design, with a strong focus on Generative AI applications. Experience designing and deploying AI models using Azure cloud services and frameworks like LangChain and OpenAI. Strong understanding of Natural Language Processing (NLP) and Generative AI principles. Lead the design and architecture of Generative AI solutions, ensuring they meet business needs, scalability, and security requirements. Design systems leveraging LangChain, OpenAI models, and Azure Cognitive Services to integrate AI into enterprise workflows. Build end-to-end AI solutions, including data processing, model development, deployment, and monitoring. E...

Seeking an Azure Data Engineer with expertise in data architecture and .NET development. Responsibilities include building and optimizing data pipelines, as well as integrating .NET applications with cloud data solutions. Experience with Azure Data Factory, SQL, and .NET Core required. Strong data engineering and development skills are essential for the role. Location: Flexible.

JR REQ--- Azure Data Engineer --4to8year---HYD,BANGALORE----Naveenraja A S --TCS C2H ---900000

We are seeking a highly skilled Data Engineer with deep experience in Python, PySpark, Scala, Azure Databricks, SQL Server, NoSQL (Cassandra, etc.) , and data streaming technologies . The ideal candidate will design and maintain scalable batch and real-time data pipelines , enabling efficient data ingestion, transformation, and analytics in a cloud environment. Key Responsibilities Design, develop, and maintain data pipelines (batch and real-time) using PySpark , Scala , and Azure Databricks . Build and optimize data ingestion frameworks integrating streaming and static data sources. Implement real-time data streaming solutions using tools like Apache Kafka , Azure Event Hubs , or Spark Stru...

Job Summary: Technical Experience Should have experience on working in Datawarehouse/ETL Testing Big data testing and BI testing in Snowflake Azure ADF ALS Snowpark and Synapse Experience in Writing complex Spark SQL queries/ Hive QL SQL performance tuning and validation of DWH Experience in data analysis including profiling auditing balancing and reconciliation. Experience in load testing of ETL pipeline and backend system with performance tuning indexes partitions query tuning) Experience in Snowflake Azure ADF Azure Databricks Experience in Test Management and defect management tools Design Test cases Test Data and perform ETL test execution and reporting Experience in Unix and Proficienc...

Advanced Data Engineer :Azure Databricks + Pyspark / Python + SQL + Data Modeling. ETLL Process, Apache Spark, Pyspark, Data Integration and workflow orchestration , real time data processing experience and frameworks, Azure cloud experience

Experienced Azure Data Engineer Tech Lead ( Tech Lead ) with 7 to 10 years of expertise Design, develop, and maintain robust data pipelines using Azure Databricks, Azure Data Factory, Pyspark / Python, Azure Data Lake Gen2, and Azure SQL Database. Integrate and manage data from various source systems, ensuring data quality and consistency. Mandatory Skills : Azure Databrick + Pyspark + Azure Data Facory + SQL +CICD + Strong Coding skills.

Azure Databricks: 1. Mounting Files 2. PAT Tokens 3. Cluster Auto scaling 4. Cluster Types 5. Nesting notebooks 6. Reading file using default and custom schema 7. Types Secrets and scopes in databricks 8. Delta Tables 9. Basic file loading scenario with its steps 10. Optimization 11. User defined functions 12. Transformations ADF: 1. Integration runtimes familiar with and where they are used 2. Components in creating pipelines 3. Scenario based question for creating a basic pipeline for loading data to Azure SQL table 4. Basic steps to improve the loading activity of a huge amount of file , ex: 1TB file size