2404 Azure Databricks Jobs - Page 47

- Building and operationalizing large scale enterprise data solutions and applications using one or more of AZURE data and analytics services in combination with custom solutions - Azure Synapse/Azure SQL DWH, Azure Data Lake, Azure Blob Storage, Spark, HDInsights, Databricks, CosmosDB, EventHub/IOTHub. - Experience in migrating on-premise data warehouses to data platforms on AZURE cloud. - Designing and implementing data engineering, ingestion, and transformation functions - Azure Synapse or Azure SQL data warehouse - Spark on Azure is available in HD insights and data bricks - Good customer communication. - Good Analytical skill

Azure Data Factory: - Develop Azure Data Factory Objects - ADF pipeline, configuration, parameters, variables, Integration services runtime - Hands-on knowledge of ADF activities(such as Copy, SP, lkp etc) and DataFlows - ADF data Ingestion and Integration with other services Azure Databricks: - Experience in Big Data components such as Kafka, Spark SQL, Dataframes, HIVE DB etc implemented using Azure Data Bricks would be preferred. - Azure Databricks integration with other services - Read and write data in Azure Databricks - Best practices in Azure Databricks Synapse Analytics: - Import data into Azure Synapse Analytics with and without using PolyBase - Implement a Data Warehouse with Azure...

Azure Data Factory: - Develop Azure Data Factory Objects - ADF pipeline, configuration, parameters, variables, Integration services runtime - Hands-on knowledge of ADF activities(such as Copy, SP, lkp etc) and DataFlows - ADF data Ingestion and Integration with other services Azure Databricks: - Experience in Big Data components such as Kafka, Spark SQL, Dataframes, HIVE DB etc implemented using Azure Data Bricks would be preferred. - Azure Databricks integration with other services - Read and write data in Azure Databricks - Best practices in Azure Databricks Synapse Analytics: - Import data into Azure Synapse Analytics with and without using PolyBase - Implement a Data Warehouse with Azure...

- Building and operationalizing large scale enterprise data solutions and applications using one or more of AZURE data and analytics services in combination with custom solutions - Azure Synapse/Azure SQL DWH, Azure Data Lake, Azure Blob Storage, Spark, HDInsights, Databricks, CosmosDB, EventHub/IOTHub. - Experience in migrating on-premise data warehouses to data platforms on AZURE cloud. - Designing and implementing data engineering, ingestion, and transformation functions - Azure Synapse or Azure SQL data warehouse - Spark on Azure is available in HD insights and data bricks - Good customer communication. - Good Analytical skill

- Building and operationalizing large scale enterprise data solutions and applications using one or more of AZURE data and analytics services in combination with custom solutions - Azure Synapse/Azure SQL DWH, Azure Data Lake, Azure Blob Storage, Spark, HDInsights, Databricks, CosmosDB, EventHub/IOTHub. - Experience in migrating on-premise data warehouses to data platforms on AZURE cloud. - Designing and implementing data engineering, ingestion, and transformation functions - Azure Synapse or Azure SQL data warehouse - Spark on Azure is available in HD insights and data bricks - Good customer communication. - Good Analytical skill

Capgemini Invent is the digital innovation, consulting, and transformation brand of the Capgemini Group, a global business line that combines market-leading expertise in strategy, technology, data science, and creative design to help CxOs envision and build what's next for their businesses. In this role, you should have developed/worked on at least one Gen AI project and have experience in data pipeline implementation with cloud providers such as AWS, Azure, or GCP. You should also be familiar with cloud storage, cloud database, cloud data warehousing, and Data lake solutions like Snowflake, BigQuery, AWS Redshift, ADLS, and S3. Additionally, a good understanding of cloud compute services, l...

Role & responsibilities : At least 5 years of experience in data engineering with a strong background on Azure Databricks and Scala/Python. Experience in handling unstructured data processing and transformation with programming knowledge. Hands on experience in building data pipelines using Scala/Python Big data technologies such as Apache Spark, Structured Streaming, SQL, Databricks Delta Lake Strong analytical and problem solving skills with the ability to troubleshoot spark applications and resolve data pipeline issues. Familiarity with version control systems like Git, CICD pipelines using Jenkins

Azure Data Engineer Pyspark Python Azure Data Bricks Azure Data Factory SQL

Job Title: Senior Azure Health Data Services (AHDS) Engineer Location: Remote Duration: 5 Months (800+ hours) Project Overview We are seeking a highly skilled Senior Azure Health Data Services (AHDS) Engineer to support a client engagement focused on migrating and scaling healthcare data infrastructure on Azure. This contract will span multiple phases total 700+ hours and will involve complex FHIR migration, implementation of payer/provider APIs, and compliance with healthcare data standards. Phase 1 Responsibilities (Approx. 400 hours) Migrate existing FHIR services to Azure Health Data Services (AHDS) Apply security layers, manage platform performance tuning Plan and execute regression and...

Role & responsibilities: Write and understand complex SQL queries, including joins, CTE, to extract and manipulate the data. Manage and monitor the data pipelines in Azure Data Factory and Databricks. Manage to understand the integration with Veeva CRM and Salesforce Ensure data quality checks and business level checks in data pipeline of ADF and Databricks. Monitor and trouble shoot data and pipeline related issues, perform root cause analysis and implement corrective actions. Monitoring of system performance and reliability, troubleshooting issues and ensuring data delivery before committed SLA. Strong Understanding of Azure DevOps and Release process (Dev to Ops Handover) Preferred candid...

Azure Data Factory: - Develop Azure Data Factory Objects - ADF pipeline, configuration, parameters, variables, Integration services runtime - Hands-on knowledge of ADF activities(such as Copy, SP, lkp etc) and DataFlows - ADF data Ingestion and Integration with other services Azure Databricks: - Experience in Big Data components such as Kafka, Spark SQL, Dataframes, HIVE DB etc implemented using Azure Data Bricks would be preferred. - Azure Databricks integration with other services - Read and write data in Azure Databricks - Best practices in Azure Databricks Synapse Analytics: - Import data into Azure Synapse Analytics with and without using PolyBase - Implement a Data Warehouse with Azure...

- Building and operationalizing large scale enterprise data solutions and applications using one or more of AZURE data and analytics services in combination with custom solutions - Azure Synapse/Azure SQL DWH, Azure Data Lake, Azure Blob Storage, Spark, HDInsights, Databricks, CosmosDB, EventHub/IOTHub. - Experience in migrating on-premise data warehouses to data platforms on AZURE cloud. - Designing and implementing data engineering, ingestion, and transformation functions - Azure Synapse or Azure SQL data warehouse - Spark on Azure is available in HD insights and data bricks - Good customer communication. - Good Analytical skill

Skills: SQL data structures,,DataBricks,Azure data lake,pyspark,Snowflake,HADOOP,SPARK Experience: 8+ Years Areas of expertise: SQL Server, Azure Data factory, Pyspark, Microsoft Fabric, Data modeling, Azure Dev ops Develop metadata-driven pipelines in Azure Data Factory to promote reusability, scalability, and parameterization. Design and implement ETL solutions to extract, transform, and load data from SQL Server into Azure and Microsoft Fabric environments (Lake house/Warehouse) Build and manage incremental load, full load, and CDC-based pipelines using ADF or PySpark. Design & Implement Medallion architecture data platform Integrate ADF with Azure Dev Ops for version control, CI/CD autom...

Data Engineer with min 4+ years of experience strong with any cloud platform, ETL tools and scripting languages with atleast one basic cloud certification. Contact\Whatsapp: +919985831110 \ prashanth@livecjobs.com *JOB IN BANGALORE, PUNE, MUMBAI* Required Candidate profile Experience in any AWS, Azure, GCP SQL Any ETL tool Python or UNIX shell scripting Certification: Any basic Cloud (AWS, Azure, GCP) Certification

Overview Seeking an Associate Manager, Data Operations, to support our growing data organization. In this role, you will assist in maintaining data pipelines and corresponding platforms (on-prem and cloud) while working closely with global teams on DataOps initiatives. Support the day-to-day operations of data pipelines, ensuring data governance, reliability, and performance optimization on Microsoft Azure. Hands-on experience with Azure Data Factory (ADF), Azure Synapse Analytics, Azure Databricks, and real-time streaming architectures is preferred. Assist in ensuring the availability, scalability, automation, and governance of enterprise data pipelines supporting analytics, AI/ML, and busi...

Role & responsibilities We are seeking an experienced and visionary Technical Expert (Architect) with deep expertise in Microsoft technologies and a strong focus on Microsoft Analytics solutions. The ideal candidate will design, implement, and optimize end-to-end analytics architectures, enabling organizations to derive actionable insights from their data. This role requires a blend of technical prowess, strategic thinking, and leadership capabilities to guide teams and stakeholders toward innovative solutions. Key Responsibilities Architectural Design: Lead the design and development of scalable and secure data analytics architectures using Microsoft technologies (e.g., Power BI, Azure Syna...

Project Role : Application Lead Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills : Microsoft Azure Databricks Good to have skills : Microsoft Azure ArchitectureMinimum 7.5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Lead, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. Your typical day will involve collaborating with various teams to ensure that application development aligns with business objectives, overseeing project timelines, and facilitating communication a...

Project Role : Data Platform Engineer Project Role Description : Assists with the data platform blueprint and design, encompassing the relevant data platform components. Collaborates with the Integration Architects and Data Architects to ensure cohesive integration between systems and data models. Must have skills : Databricks Unified Data Analytics Platform Good to have skills : Microsoft Power Business Intelligence (BI), Microsoft Azure DatabricksMinimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Platform Engineer, you will assist with the data platform blueprint and design, encompassing the relevant data platform compone...

Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : PySpark, Microsoft Azure Databricks, Microsoft Azure Analytics Services, Microsoft Azure Data Services Good to have skills : NAMinimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical da...

Job Summary: We are looking for an experienced and visionary Lead Data Engineer to architect and drive the development of scalable, secure, and high-performance data solutions . This role requires deep technical expertise in Python, Apache Spark, Delta Lake, and orchestration tools like Databricks Workflows or Azure Data Factory . The ideal candidate will also bring a strong understanding of data governance, metadata management, and regulatory compliance in the insurance and financial services domains. Proficient in developing Python Applications, Spark-based workflows, leveraging Delta Lake, and orchestrating jobs using Databricks Workflows or Azure Data Factory. Good to understand retentio...

Hexaware Technologies Location - Chennai, Pune, Bengaluru, Mumbai, Coimbatore Role & responsibilities Test Lead Data ETL Testing with Cloud (Azure) ETL Tester with 3.6+ Years of Cloud Data testing experience • Experience in data migration and ETL transformation testing • Exposure to Azure Data lakes and data bricks is must • Develop technical specifications for data loading rules • Mentor and provide technical guidance for Software Engineers] • Adhere to and produce artifacts (technical specifications, architectural specifications, data models, installation and qualification guides, and release notes) for the software development process. • Provide application support for existing products •...

Data Engineering Lead Company Name: Blackstraw.ai Oce Location: Chennai (Work from Office) Job Type: Full-time Experience: 10 - 15 Years Candidates who can join immediately will be preferred. Job Description: As a lead data engineer you will oversee data architecture, ETL processes, and analytics pipelines, ensuring efficiency, scalability, and quality. Key Responsibilities: Working with clients to understand their data. Based on the understanding you will be building the data structures and pipelines. You will be working on the application from end to end collaborating with UI and other development teams. You will be working with various cloud providers such as Azure & AWS. You will be engi...

Role & responsibilities You Will- Owns the roadmap, timelines and delivery of data engineering and data science work streams by building end-to-end schedules, and managing cross team and cross functional project timelines in collaboration with engineering management, product management and business stakeholders. Lead multiple data-solution programs covering data pipelines, visualizations, data alerts, advanced analytics and machine learning methods, translating raw data into strategic insights and recommendations for leadership and business teams. Lead the technical delivery, implementation, and business adoption of new scalable and reliable data analytics, and business intelligence solution...

As an Associate Software Developer at IBM you will harness the power of data to unveil captivating stories and intricate patterns. You'll contribute to data gathering, storage, and both batch and real-time processing. Collaborating closely with diverse teams, you'll play an important role in deciding the most suitable data management systems and identifying the crucial data required for insightful analysis. As a Data Engineer, you'll tackle obstacles related to database integration and untangle complex, unstructured data sets. In this role, your responsibilities may include: Implementing and validating predictive models as well as creating and maintain statistical models with a focus on big ...

The ability to be a team player The ability and skill to train other people in procedural and technical topics Strong communication and collaboration skills Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Able to write complex SQL queries ; Having experience in Azure Databricks Preferred technical and professional experience Excellent communication and stakeholder management skills