3649 Azure Databricks Jobs - Page 25

KiE Square is looking for a passionate and experienced Sr. Data Scientist to join our Analytics Consulting team. The ideal candidate shall possess a strong blend of technical expertise, analytical thinking, and business consulting skills. You will be responsible for developing and deploying scalable machine learning solutions, building end-to-end data pipelines, and translating analytical insights into actionable business strategies for our clients. Responsibilities Partner with clients to understand business challenges and identify data-driven opportunities. Design, build, and deploy end-to-end machine learning models and analytical solutions tailored to client objectives. Develop data inge...

experience on Azure data engineering - Azure databricks Analysis, Design, Development, Implementation, unstructured data, Enterprise Data warehousing, Data marts, Data Conversion and Data Integration. Experienced and knowledgeable at a mastery-level of industry best practices and vendors in order to actively analyze, design, and recommend third-party tools where applicable. Drives analysis of required functionality and implements customizations to platforms or develops custom solutions to fulfill business requirements with a mastery focus on technical excellence and high quality. Cloud migration methodologies and processes including tools like Azure Data Factory, Data Migration Service, SSIS...

Experience on Azure data engineering - Azure databricks Analysis, Design, Development, Implementation, unstructured data, Enterprise Data warehousing, Data marts, Data Conversion and Data Integration. Experienced and knowledgeable at a mastery-level of industry best practices and vendors in order to actively analyze, design, and recommend third-party tools where applicable. Drives analysis of required functionality and implements customizations to platforms or develops custom solutions to fulfill business requirements with a mastery focus on technical excellence and high quality. Cloud migration methodologies and processes including tools like Azure Data Factory, Data Migration Service, SSIS...

Join our Data Analytics team and be a key player in developing innovative, data-driven solutions for Aftersales. We are looking for team-oriented, responsible, and passionate professionals who thrive on innovation and want to help shape our Aftersales data strategy. Your focus will be on applying intelligent data analysis to generate valuable business insights, directly impacting strategic decisions and operational efficiency. If youre excited about data engineering, decentralized architectures, and building high-impact data products, wed love to hear from you! JD of the position Roles & Responsibilities Design & Build Scalable Data Solutions Improve data processing efficiency, ensuring scal...

Azure data bricks developer with 5 to 10 years of experience We are seeking a skilled Azure Databricks Developer to design, develop, and optimize big data pipelines using Databricks on Azure. The ideal candidate will have strong expertise in PySpark, Azure Data Lake, and data engineering best practices in a cloud environment. Key Responsibilities: Design and implement ETL/ELT pipelines using Azure Databricks and PySpark. Work with structured and unstructured data from diverse sources (e.g., ADLS Gen2, SQL DBs, APIs). Optimize Spark jobs for performance and cost-efficiency. Collaborate with data analysts, architects, and business stakeholders to understand data needs. Develop reusable code co...

Mandatory Skills: Data factory, Databricks, SQL DB, Python, ADLS, PySpark *Preferred Skills: PBI, Power APP stack, Data Modelling , AWS lambda, Glue, GMR, Airflow, Kinesis, Redshift Experience in designing and implementing Azure Data solutions using services such as: Azure Synapse Analytics Azure Databricks Azure Data Lake Storage Gen2 Azure SQL Database Azure Data Factory Azure DevOps Azure Stream Analytics Azure Blob storage Azure Cosmos DB ARM templates Familiar with Microsoft Power BI Familiar with Azure Purview An understanding of Master Data Management and Data Governance frameworks Familiar with Infrastructure as Code approaches and implementations Familiar with development approaches...

Responsibilities: Develop and maintain scalable data pipelines using Databricks (PySpark, Delta Lake). Integrate data from multiple APIs (Azure Monitor, Azure Cost Management, Log Analytics, Purview). Implement ETL/ELT processes for structured/unstructured data. Ensure data quality, reliability, and observability throughout the pipeline. Optimize pipeline performance (cost, CPU, memory usage). Collaborate with frontend developers to expose APIs for dashboards. Write documentation and maintain version control (Git/DevOps). Required Skills: Strong experience with Azure Data Services (Databricks, ADLS, Synapse optional). Knowledge of Delta Lake, PySpark, SQL . Experience with API integration (R...

Provide support for Azure resources, focusing on PaaS and IaaS components, to ensure optimal performance and reliability. Troubleshoot and resolve environment and application-level issues in a timely and effective manner. Work with Azure Kubernetes Services (AKS), Container Registry, App Service, Function Apps, Load Balancer, Application Gateway, Traffic Manager, Azure Front Door, Azure Databricks, Storage Accounts, and Azure Data Factory. Manage and maintain virtual machines (VMs), including both Linux and Windows environments. Utilize automation tools such as PowerShell and Azure CLI to streamline operations and deployment processes. Experience with Terraform for infrastructure as code is ...

Description: Will be responsible for the design development and administration of data management solutions that align with the business needs of the organization. Design relational databases to support business enterprise applications and physical data modeling according to project requirements for data acquisition and security as well as customer-defined deliverables. Has at least 2 years of experience in Data Engineering using ADF to orchestrate and development of data pipelines speically for ingestion harmonisation and curation of the source data. This includes good hands on experience on writing Stored procedures complex views performance tuning of the SQL good underastanding and manage...

Architectural Design:Develop and design scalable reliable and secure Azure-based solutions that align with business requirements and technical constraints Solution Implementation:Lead the implementation of Azure solutions including virtual networks storage databases and compute resources ensuring best practices and high performance Consultation:Collaborate with stakeholders including business leaders IT teams and developers to gather requirements and provide technical guidance and solutions Integration:Design and oversee the integration of Azure services with existing systems and third-party applications Optimization:Continuously evaluate and optimize Azure solutions for performance cost and...

Description: 10+ years of relevant IT experience in the BI/DW domain with minimum 3 years of hands-on experience on Azure modern data platform that includes Data Factory Databricks Synapse (Azure SQL DW) and Azure Data Lake Meaningful experience of data analysis and transformation using Python/R/Scala on Azure Databricks or Apache Spark Well versed NoSQL data store concepts Good knowledge in Distributed Processing using Databricks (preferred) or Apache Spark Ability to debug using tools like Ganglia UI expertise in Optimizing Spark Jobs The ability to work across structured semi-structured and unstructured data extracting information and identifying linkages across disparate data sets Expert...

Job Summary We are seeking a Data Engineer to work on a Generative AI initiative to join our team. The ideal candidate will have a deep understanding of data modeling, data schemas and has developed ETLs from various sources eensuring high data availability, fault tolerance and security and governance. Responsibilities: Collaborate with stakeholders to understand data requirements and design scalable and efficient data models, schemas, and architectures on the Azure platform. Develop and implement data integration and ETL (Extract, Transform, Load) processes to ingest, transform, and load data from various sources into Azure data storage systems. Build and maintain scalable and reliable data...

Description experience in AI architecture and machine learning solution design, with a strong focus on Generative AI applications. Experience designing and deploying AI models using Azure cloud services and frameworks like LangChain and OpenAI. Strong understanding of Natural Language Processing (NLP) and Generative AI principles. Lead the design and architecture of Generative AI solutions, ensuring they meet business needs, scalability, and security requirements. Design systems leveraging LangChain, OpenAI models, and Azure Cognitive Services to integrate AI into enterprise workflows. Build end-to-end AI solutions, including data processing, model development, deployment, and monitoring. En...

Azure data factory 6 to 9 yrs Bangalore

JPC REQ-- AZURE DATA ENGINEER ---4-6 YEARS--- BANGALORE --- CONGNIZANT C2H ---reshma - 30 LPA ---- Sharvin

Databriks + Pyspark About The Role 1 Skill:Databriks + Pyspark Exp 5 to 8 yr Location Any PSL , Bangalore, Delhi, Pune , Chennai Dont look for Hyderabad Location Azure Data bricks,PySpark,Oracle PL SQL,ETL,SQL,Data Integration Data Engineer - Data AnalystSummaryStrategically manages and maintains corporate global clinical data for all major business units, while ensuring data quality and integrity. Provides system support, testing, and training for applications accessing clinical data.Proven 2-5 years work experience as a Data Analyst in Clinical data Identifies, investigates, and resolves data quality issues, communicating with business units and IT and performing root cause analysis as nec...

Hire Type: Fulltime Tech Stack: Proficiency in Python, SQL, Databricks, Snowflake Data Modeling, ETL process, Apache Spark and PySpark, data integration and workflow orchestration, real time data processing experience/frameworks, Cloud experience (Azure preferred) Job Description Minimum 5 years of experience in Data Engineering role that involves Analyzing, organizing raw data, and building data systems & pipelines in Cloud Platform [Azure] Experienced in migrating data from on-prem to cloud-based solution architectures [Azure] Extensive experience with Python, Spark and SQL Experience in developing ETL processes. Proficient with Azure Data Lake, Azure Data Factory, Azure SQL, Azure Databri...

Description: experience in AI architecture and machine learning solution design, with a strong focus on Generative AI applications. Experience designing and deploying AI models using Azure cloud services and frameworks like LangChain and OpenAI. Strong understanding of Natural Language Processing (NLP) and Generative AI principles. Lead the design and architecture of Generative AI solutions, ensuring they meet business needs, scalability, and security requirements. Design systems leveraging LangChain, OpenAI models, and Azure Cognitive Services to integrate AI into enterprise workflows. Build end-to-end AI solutions, including data processing, model development, deployment, and monitoring. E...

Seeking an Azure Data Engineer with expertise in data architecture and .NET development. Responsibilities include building and optimizing data pipelines, as well as integrating .NET applications with cloud data solutions. Experience with Azure Data Factory, SQL, and .NET Core required. Strong data engineering and development skills are essential for the role. Location: Flexible.

JR REQ--- Azure Data Engineer --4to8year---HYD,BANGALORE----Naveenraja A S --TCS C2H ---900000

Job Summary: Technical Experience Should have experience on working in Datawarehouse/ETL Testing Big data testing and BI testing in Snowflake Azure ADF ALS Snowpark and Synapse Experience in Writing complex Spark SQL queries/ Hive QL SQL performance tuning and validation of DWH Experience in data analysis including profiling auditing balancing and reconciliation. Experience in load testing of ETL pipeline and backend system with performance tuning indexes partitions query tuning) Experience in Snowflake Azure ADF Azure Databricks Experience in Test Management and defect management tools Design Test cases Test Data and perform ETL test execution and reporting Experience in Unix and Proficienc...

Advanced Data Engineer :Azure Databricks + Pyspark / Python + SQL + Data Modeling. ETLL Process, Apache Spark, Pyspark, Data Integration and workflow orchestration , real time data processing experience and frameworks, Azure cloud experience

Experienced Azure Data Engineer Tech Lead ( Tech Lead ) with 7 to 10 years of expertise Design, develop, and maintain robust data pipelines using Azure Databricks, Azure Data Factory, Pyspark / Python, Azure Data Lake Gen2, and Azure SQL Database. Integrate and manage data from various source systems, ensuring data quality and consistency. Mandatory Skills : Azure Databrick + Pyspark + Azure Data Facory + SQL +CICD + Strong Coding skills.

Azure Databricks: 1. Mounting Files 2. PAT Tokens 3. Cluster Auto scaling 4. Cluster Types 5. Nesting notebooks 6. Reading file using default and custom schema 7. Types Secrets and scopes in databricks 8. Delta Tables 9. Basic file loading scenario with its steps 10. Optimization 11. User defined functions 12. Transformations ADF: 1. Integration runtimes familiar with and where they are used 2. Components in creating pipelines 3. Scenario based question for creating a basic pipeline for loading data to Azure SQL table 4. Basic steps to improve the loading activity of a huge amount of file , ex: 1TB file size

Bachelor's degree in Computer Science, Information Technology, or a related field. A master's degree is a plus. Proven experience as a Data Engineer or in a similar role, with a focus on ETL processes and database management. Proficiency in the Microsoft Azure data management suite (MSSQL, Azure Databricks , PowerBI , Data factories, Azure cloud monitoring, etc.) and Python scripting. Strong knowledge of SQL and experience with database management systems Strong development skills in python and pyspark . Experience with data warehousing solutions and data mart creation. Familiarity with big data technologies (e.g., Hadoop, Spark) is a plus. Strong problem-solving skills and attention to deta...