3649 Azure Databricks Jobs - Page 23

RoleAdvanced Data Engineer , LocationHyderabad Hire TypeFulltime Tech Stack: Proficiency in Python, SQL, Databricks, Snowflake Data Modeling, ETL process, Apache Spark and PySpark, data integration and workflow orchestration, real time data processing experience/frameworks, Cloud experience (Azure preferred) About The Role Minimum 5 years of experience in Data Engineering role that involves Analyzing, organizing raw data, and building data systems & pipelines in Cloud Platform [Azure] Experienced in migrating data from on-prem to cloud-based solution architectures [Azure] Extensive experience with Python, Spark and SQL Experience in developing ETL processes. Proficient with Azure Data Lake, ...

Description Responsibilities Perform testing in Databricks environment which includes Participating in developing Test frameworks. Participating in testing scripts development, code reviews Partner with the product owner, data engineers, data architects and analysts to create scalable and robust solutions. Prepare the status report of testing activities and share updates with the team during Scrum meeting. Experience using appropriate test management tool like Azure DevOps Competencies / Required Skills: ETL Testing experience of 7+ years Report Testing experience of 2+ years, preferably PowerBI Hands on experience of 3+ years working in Databricks/ADLS environment. Good at Python/Pyspark sc...

Skills: Cloudera, Big Data, Hadoop, SPARK, Kafka, Hive, CDH Clusters Design and implement Cloudera-based data platforms, including cluster sizing, configuration, and optimization. Install, configure, and administer Cloudera Manager and CDP clusters, managing all aspects of the cluster lifecycle. Monitor and troubleshoot platform performance, identifying and resolving issues promptly. Review the maintain the data ingestion and processing pipelines on the Cloudera platform. Collaborate with data engineers and data scientists to design and optimize data models, ensuring efficient data storage and retrieval. Implement and enforce security measures for the Cloudera platform, including authenticat...

Azure Data Factory: - Develop Azure Data Factory Objects - ADF pipeline, configuration, parameters, variables, Integration services runtime - Hands-on knowledge of ADF activities(such as Copy, SP, lkp etc) and DataFlows - ADF data Ingestion and Integration with other services Azure Databricks: - Experience in Big Data components such as Kafka, Spark SQL, Dataframes, HIVE DB etc implemented using Azure Data Bricks would be preferred. - Azure Databricks integration with other services - Read and write data in Azure Databricks - Best practices in Azure Databricks Synapse Analytics: - Import data into Azure Synapse Analytics with and without using PolyBase - Implement a Data Warehouse with Azure...

- Develop Azure Data Factory Objects - ADF pipeline, configuration, parameters, variables, Integration services runtime - Hands-on knowledge of ADF activities(such as Copy, SP, lkp etc) and DataFlows - ADF data Ingestion and Integration with other services Azure Databricks: - Experience in Big Data components such as Kafka, Spark SQL, Dataframes, HIVE DB etc implemented using Azure Data Bricks would be preferred. - Azure Databricks integration with other services - Read and write data in Azure Databricks - Best practices in Azure Databricks Synapse Analytics: - Import data into Azure Synapse Analytics with and without using PolyBase - Implement a Data Warehouse with Azure Synapse Analytics -...

Azure Data Factory: - Develop Azure Data Factory Objects - ADF pipeline, configuration, parameters, variables, Integration services runtime - Hands-on knowledge of ADF activities(such as Copy, SP, lkp etc) and DataFlows - ADF data Ingestion and Integration with other services Azure Databricks: - Experience in Big Data components such as Kafka, Spark SQL, Dataframes, HIVE DB etc implemented using Azure Data Bricks would be preferred. - Azure Databricks integration with other services - Read and write data in Azure Databricks - Best practices in Azure Databricks Synapse Analytics: - Import data into Azure Synapse Analytics with and without using PolyBase - Implement a Data Warehouse with Azure...

EPAM has presence across 40+ countries globally with 55,000 + professionals & numerous delivery centers, Key locations are North America, Eastern Europe, Central Europe, Western Europe, APAC, Mid East & Development Centers in India (Hyderabad, Pune & Bangalore). Exp: 5-14 years Location: Pune/Hyderabad/Bengaluru/Chennai/Coimbatore Work Mode: Hybrid (2-3 days office in a week) Job Description: 5-14 Years of in Big Data & Data related technology and Palantir experience Expert level understanding of distributed computing principles Expert level knowledge and experience in Apache Spark Hands on programming with Python Experience with building stream-processing systems, using technologies such as...

Data Engineer (Azure Databricks) - Immediate to 15days candidates preferably Experience : 2 to 4years Responsibilities: Develop and maintain scalable data pipelines using Databricks (PySpark, Delta Lake). Integrate data from multiple APIs (Azure Monitor, Azure Cost Management, Log Analytics, Purview). Implement ETL/ELT processes for structured and unstructured data. Ensure data quality, reliability, and observability across pipelines. Optimize performance (cost, CPU, and memory usage). Collaborate with frontend teams to expose APIs for dashboards. Maintain documentation and version control (Git/DevOps). Required Skills: Strong experience with Azure Data Services (Databricks, ADLS, Synapse op...

We are hiring a Senior Azure Data Engineer to build scalable data pipelines using Azure Databricks, PySpark, and Spark Pools. The role involves transforming raw data into insights, integrating cloud data platforms, and deploying machine learning models. Candidates should have 5+ years of data engineering experience and strong skills in Python, PySpark, Azure Data Lake, and Blob Storage. Experience with ML tools like MLlib or TensorFlow is preferred. The role demands strong problem-solving and communication skills, with added value for those familiar with CI/CD, Git, Docker, or Power BI.

We are hiring a Senior Azure Data Engineer to build scalable data pipelines using Azure Databricks, PySpark, and Spark Pools. The role involves transforming raw data into insights, integrating cloud data platforms, and deploying machine learning models. Candidates should have 5+ years of data engineering experience and strong skills in Python, PySpark, Azure Data Lake, and Blob Storage. Experience with ML tools like MLlib or TensorFlow is preferred. The role demands strong problem-solving and communication skills, with added value for those familiar with CI/CD, Git, Docker, or Power BI.

We are hiring a Senior Azure Data Engineer to build scalable data pipelines using Azure Databricks, PySpark, and Spark Pools. The role involves transforming raw data into insights, integrating cloud data platforms, and deploying machine learning models. Candidates should have 5+ years of data engineering experience and strong skills in Python, PySpark, Azure Data Lake, and Blob Storage. Experience with ML tools like MLlib or TensorFlow is preferred. The role demands strong problem-solving and communication skills, with added value for those familiar with CI/CD, Git, Docker, or Power BI.

* 7+ years experience in data eng, data platforms & analytics, 10+ years of consulting experience ? Completed Data Engineering Professional certification & required classes ? Minimum 6-8+ projects delivered with hands-on experience in development on databricks ? Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with deep expertise in at least one ? Deep experience with distributed computing with Spark with knowledge of Spark runtime internals ? Familiarity with CI/CD for production deployments ? Working knowledge of MLOps ? Current knowledge across the breadth of Databricks product and platform features ? Familiarity with optimizations for performance and scalability

4 Years Data Engineering Experience 5 years of consulting experience Completed Data Engineering Professional certification & required classes Minimum 2-3 projects delivered with hands-on experience in Databricks Completed Apache Spark Programming with Databricks, Data Engineering with Databricks, Optimizing Apache Spark on Databricks Experience in Spark and or Hadoop, Flink, Presto, other popular big data engines Familiarity with Databricks multi-hop pipeline architecture

2 Years Data Engineering Experience, 3 years of consulting experience Completed Data Engineering Associate certification & required classes Minimum 1 project delivered with hands-on experience in development on Databricks Completed Apache Spark Programming with Databricks, Data Engineering with Databricks, Optimizing Apache Spark on Databricks SQL delivery experience, and familiarity with Bigquery, Synapse or Redshift Proficient in Python, knowledge of additional databricks programming languages (Scala)

Immediate Joiners & Ready to Relocate to Madurai Roles and Responsibilities Design, develop, test, deploy, and maintain large-scale data processing pipelines using PySpark on Azure Databricks. Collaborate with cross-functional teams to gather requirements and deliver high-quality solutions that meet business needs. Troubleshoot complex issues related to big data processing, ETL processes, and SQL queries. Ensure scalability, performance, and reliability of the system by monitoring logs, debugging techniques, and implementing optimization strategies. Stay up-to-date with industry trends in big data technologies such as Hadoop ecosystem (HDFS), Spark (PySpark), AWS Glue.

About Position: We are seeking a highly skilled and experienced Data Engineer SQL / SSIS to join the Optum Advisory Team. The ideal candidate will bring deep expertise in cloud-based data engineering, healthcare data modeling, and consulting, with a strong background in Azure, Databricks, Azure Data Factory (ADF), SQL, and SSIS / SSRS. This role requires a self-starter who can work independently with minimal guidance and collaborate effectively with cross-functional teams to deliver high-impact data solutions in the healthcare domain Role: Cloud Data Platform Engineer Location: All Persistent Locations Experience: 6+ Years Job Type: Full Time Employment What You'll Do: Design, implement, and...

Python Gen AI

You will be responsible for performing multiple aspects involved in the development lifecycle such as design, cloud engineering, ingestion, data modeling, testing, CICD pipelines, performance tuning, deployments, consumption, BI, alerting, and prod support. Additionally, you will provide technical leadership, collaborate within a team environment, work independently, and be a part of a DevOps team that completely owns and supports their product. You will also lead the development of coding standards, best practices, and privacy and security guidelines while mentoring others on technical and domain skills to create multi-functional teams. - Implement batch and streaming data pipelines using c...

As a PySpark Developer at Capgemini Invent, you will play a crucial role in designing, developing, and implementing data solutions using PySpark for large-scale data processing and analytics. You will be working with Azure Databricks (ADB) and Azure Data Factory (ADF) to create efficient data pipelines and conduct financial risk assessments. Here is what you can expect in this role: - Design, develop, and deploy PySpark applications and workflows on Azure Databricks for data transformation, cleansing, and aggregation. - Implement data pipelines using Azure Data Factory (ADF) to orchestrate ETL/ELT processes across heterogeneous data sources. - Conduct regular financial risk assessments to id...

Are you a Big Data enthusiast with hands-on experience in Spark, PySpark, and Azure cloud technologies We are looking for talented and driven developers to join our team and make an impact! Mandatory Requirements: 8-12 years of relevant experience (strictly) Strong expertise in Apache Spark, PySpark, Azure Databricks, Azure Data Factory, and SQL Hands-on experience in building scalable data pipelines and optimizing Spark jobs Only candidates matching the JD and experience range will be considered

Purpose As a Senior Data Engineer at LogixHealth, you will work with a globally distributed team of engineers to design and build cutting edge solutions that directly improve the healthcare industry. Youll contribute to our fast-paced, collaborative environment and bring your expertise to continue delivering innovative technology solutions, while mentoring others. Duties and Responsibilities 1. Lead and contribute to the creation of a self-service data platform for reporting and analytics 2. Design and build data solutions using Databricks, SQL, Python, Spark, and Delta Lake in the Azure ecosystem (Blob Storage, Data Factory, Event Hubs) 3. Ensure best practices for ETL / ELT processes (data...

Develop and manage data lakes and data warehouses using Azure Data Lake Storage Gen2 and Azure SQL/Data Warehouse. Build and deploy machine learning models using Azure Machine Learning, Mlflow. Integrate AI capabilities using Azure Cognitive AI Services, Azure Open AI Implement real-time data streaming and analytics using Azure Stream Analytics and Event Hub Azure Open AI in Foundry Models Azure AI Document Intelligence Understanding of LLM pipelines, including data preprocessing for Gen AI models Experience deploying data pipelines for AI/ML workloads, ensuring scalability and efficiency

Web Application PaaS services PaaS based Monitoring PaaS Database setup NSG WAF Domain management

Web Application PaaS services PaaS based Monitoring PaaS Database setup NSG WAF Domain management

Web Application PaaS services PaaS based Monitoring PaaS Database setup NSG WAF Domain management