3651 Azure Databricks Jobs - Page 41

Role & responsibilities Job Description: Job Summary: We are looking for a skilled Data Engineer to join our data team. The ideal candidate will have hands-on experience in building robust data pipelines using Airflow, managing data warehouses in Snowflake and Redshift, and working with cloud platforms such as AWS and Azure (Databricks, Glue). You will work closely with Data Scientists, Analysts, and Engineers to ensure data availability, quality, and reliability across the organization. Key Responsibilities: l Design, develop, and maintain scalable ETL/ELT pipelines using Apache Airflow, AWS Glue, and Databricks. l Integrate data from multiple sources into Snowflake and Redshift data wareho...

Responsibilities : - An Azure Data Engineer is responsible for designing, building, and maintaining the data infrastructure for an organization using Azure cloud services. This includes creating data pipelines, integrating data from various sources, and implementing data security and privacy measures. The Azure Data Engineer will also be responsible for monitoring and troubleshooting data flows and optimizing data storage and processing for performance and cost efficiency. Mandatory Skills : - Bachelors or higher degree in Computer Science or a related discipline; or equivalent (minimum 7 years work experience). At least 4+ years of consulting or client service delivery experience on Azure A...

In this role, you'll work in one of our IBM Consulting Client Innovation Centers (Delivery Centers), where we deliver deep technical and industry expertise to a wide range of public and private sector clients around the world. Our delivery centers offer our clients locally based skills and technical expertise to drive innovation and adoption of new technology. Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise The ability to be a team player The ability and skill to train other people in procedural and technical topics Strong communication and collaboration skills Able to write complex SQL queries Having experience in Azure ...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Microsoft Azure Databricks Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data qualit...

About The Role Project Role : Application Lead Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills : Microsoft Azure Databricks Good to have skills : NA Minimum 7.5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Lead, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. Your typical day will involve collaborating with various teams to ensure project milestones are met, facilitating discussions to address challenges, and guiding your team in implementing effective solutions....

Position: Azure/Data Bricks - Data Engineer Job Summary/ Job Opportunity: Xceedance has a talent pool of industry-savvy data scientists and insurance-expert specialists that strive to provide the best solutions to our clients. As a Data Engineer, you will assist re/insurers, agents, and brokers accelerate their data-to-insights lifecycles by using a robust proprietary data warehousing framework and leading-edge data enrichment and ingestion systems. This is an excellent opportunity for a highly efficient individual to secure the opportunity to work alongside a team of experts. If you possess a passion for what you do, we await your innovation at Xceedance. Key Objectives and Major Responsibi...

•Experience with ETL testing. •Ability to create data bricks notebooks to automate the manual tests •Ability to create and run Test pipelines and interpret the results. •Ability to test complex reports and write queries to check each metrics Required Candidate profile •Experience in Azure Databricks and SQL queries - ability to analyse data in a Data Warehouse environment. •Ability to test complex reports and write queries to check each metrics. Python & pyspark

Position: Azure/Data Bricks - Data Engineer Job Summary/ Job Opportunity: Xceedance has a talent pool of industry-savvy data scientists and insurance-expert specialists that strive to provide the best solutions to our clients. As a Data Engineer, you will assist re/insurers, agents, and brokers accelerate their data-to-insights lifecycles by using a robust proprietary data warehousing framework and leading-edge data enrichment and ingestion systems. This is an excellent opportunity for a highly efficient individual to secure the opportunity to work alongside a team of experts. If you possess a passion for what you do, we await your innovation at Xceedance. Key Objectives and Major Responsibi...

Design, develop, optimize scalable data pipelines using Azure Data Factory (ADF) Build maintain ETL/ELT workflows for structured and unstructured data Leverage Azure Databricks for big data processing, analytics, and machine learning integration Required Candidate profile Work with Azure Synapse Analytics, Cosmos DB, Azure Data Lake Storage to deliver high-performing data solutions Apply data warehousing concepts, dimensional modeling, best practices for performance Perks and benefits Perks and Benefits

About The Role Project Role : Application Lead Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills : Data Engineering Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Lead, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. Your typical day will involve collaborating with various teams to ensure that application development aligns with business objectives, overseeing project timelines, and facilitating communication among stakeholders to d...

About The Role Project Role : Application Lead Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills : Microsoft Azure Databricks Good to have skills : Python on Azure Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Lead, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. Your typical day will involve collaborating with various teams to ensure that application development aligns with business objectives, overseeing project timelines, and facilitating communication ...

Role Proficiency: This role requires proficiency in developing data pipelines including coding and testing for ingesting wrangling transforming and joining data from various sources. The ideal candidate should be adept in ETL tools like Informatica Glue Databricks and DataProc with strong coding skills in Python PySpark and SQL. This position demands independence and proficiency across various data domains. Expertise in data warehousing solutions such as Snowflake BigQuery Lakehouse and Delta Lake is essential including the ability to calculate processing costs and address performance issues. A solid understanding of DevOps and infrastructure needs is also required. Outcomes: Act creatively ...

Locations: Pune, Maharashtra, India Minimum Experience: 6 ctc 6+ upto 24 lac Skill Set: Databricks Engineer, Azure Databricks, SQL, Python Skill to Evaluate: Databricks Engineer, Azure Databricks, SQL,Pyspark, Python, Team Handling Experience: 8 to 14 Years Location: Pune, Maharashtra, India Job Description: Job Title: Senior Data Engineer Key Responsibilities: - Design, build, and maintain scalable data pipelines and ETL processes. - Collaborate with data scientists and analysts to understand data needs and provide support for their analytics requirements. - Develop and implement data models, database designs, and data management solutions. - Ensure data quality, integrity, and consistency ...

Role Overview: You will be leading a small multidisciplinary team in India specializing in Geospatial Analytics, Geographic Information Systems (GIS), and Data Engineering. As the Team Leader, you will be responsible for providing strategic direction, technical leadership, and ensuring project outcomes align with business objectives. Additionally, you will act as the primary interface between stakeholders in the UK and the team in India. Key Responsibilities: - Lead, mentor, and motivate a team of professionals in Geospatial Analytics, GIS, and Data Engineering. - Foster a collaborative work environment focusing on individual growth and team success. - Conduct performance reviews, provide fe...

As a Data Engineering Lead at EY, you will collaborate with the Data Architect to design and implement scalable data lake architecture and data pipelines. Your responsibilities will include designing and implementing scalable data lake architectures using Azure Data Lake services, developing and maintaining data pipelines to ingest data from various sources, optimizing data storage and retrieval processes for efficiency and performance, ensuring data security and compliance with industry standards, collaborating with data scientists and analysts to facilitate data accessibility, monitoring and troubleshooting data pipeline issues to ensure reliability, and documenting data lake designs, proc...

As an Azure Databricks Engineer at our fast-growing cloud data and analytics consultancy, you will play a crucial role in designing high-throughput lakehouse platforms, predictive analytics, and AI services on Microsoft Azure. Your responsibilities will include: - Designing, developing, and optimizing end-to-end data pipelines on Azure Databricks using PySpark/Scala and Delta Lake. - Building scalable ETL workflows to ingest structured and semi-structured data from Azure Data Lake, SQL, and API sources. - Implementing lakehouse architectures, partitioning, and performance tuning to ensure sub-second query response. - Collaborating with Data Scientists to prepare feature stores and accelerate...

As a Data Engineer Grade 5 at our company, your role involves working on solutions against delta lakes and relational databases. You will primarily focus on Azure Databricks, utilizing Python and SQL with PySpark for coding. Your responsibilities will include analysis, design, implementation, and testing, with a strong emphasis on automated testing. Additionally, you will be extending the functionality of the consistency check engine for sub-ledger data and other tenants, dealing with large datasets containing billions of rows and hundreds of columns, requiring deep dive into Spark performance optimization. Key Responsibilities: - Build solutions against delta lakes and relational databases ...

As a Cloud Data Engineer, your role involves designing, developing, and managing scalable, secure, and efficient data pipelines on Azure to process and transform large datasets from multiple sources. You will implement and optimize Azure data services such as Azure Synapse Analytics, Azure Data Lake Storage (ADLS), Azure SQL Database, Azure Data Factory (ADF), Azure Databricks, and Azure Functions. Your responsibilities also include architecting and maintaining ETL/ELT processes, implementing data models, collaborating with data architects, building data integration workflows, and utilizing Azure DevOps for source control and automation. Key Responsibilities: - Design, develop, and manage sc...

As a Microsoft Fabric Professional at YASH Technologies, you will be responsible for working with cutting-edge technologies to bring real positive changes in an increasingly virtual world. Your role will involve the following key responsibilities: - Experience required: 4-6 Years - Experience in Azure Fabric, Azure Data factory, Azure Databricks, Azure Synapse, Azure SQL, ETL - Create Pipelines, datasets, dataflows, Integration runtimes and monitoring Pipelines and trigger runs - Extract Transformation and Load Data from source system and processing the data in Azure Databricks - Create SQL scripts to perform complex queries - Create Synapse pipelines to migrate data from Gen2 to Azure SQL -...

Your mission: You will be joining a team that has laid a solid foundation for a robust DevOps practice to support an Azure-based data platform. The goal is to enhance deployments by making them faster, more reliable, and less dependent on manual work. As a hands-on DevOps Engineer, you will work independently and take ownership of various topics end-to-end. What you'll do: - Design and implement GitHub actions workflow for various solutions including Azure Databricks, Azure functions, App Services, REST API Solutions (APIOps), Power BI Solutions, and AI/ML Solutions (MLOps). - Define the Pull Request flow, encompassing Pull Request, Review, Merging, Build, Acceptance, and Deploy processes. -...

As a Data Engineer / Analyst at our company, your main responsibilities will include: - Designing, developing, and maintaining data solutions on Azure Databricks using Python, SQL, and PySpark. - Working extensively with Delta Lake and relational databases to build scalable data pipelines. - Performing analysis, design, implementation, and automated testing of data solutions. - Enhancing the consistency check engine for sub-ledger data and other tenants. - Optimizing Spark performance for very large datasets (10s of billions of rows, 200+ columns). - Identifying, troubleshooting, and resolving data issues proactively. - Collaborating with cross-functional teams to ensure data reliability, co...

As a Data Engineer, your role will involve collaborating with data scientists and business stakeholders to design, develop, and maintain efficient data pipelines feeding into the organization's data lake. It will be your responsibility to maintain the integrity and quality of the data lake, enabling accurate and actionable insights for data scientists and informed decision-making for business stakeholders. Leveraging your extensive knowledge of data engineering and cloud technologies, you will enhance the organization's data infrastructure to promote a culture of data-driven decision-making. Your expertise in data engineering will be crucial in defining and optimizing data pipelines using ad...

Data & Software Solutions Architect Experience: 9 - 15 Years Exp Salary: Highly Competitive Preferred Notice Period : 30 Days Shift : 10:00 AM to 7:00 PM IST Opportunity Type: Remote (Noida)- Remote 6 Months, Then Hybrid (Flexible) Placement Type: Permanent (*Note: This is a requirement for one of Uplers' Clients) Must have skills required : Azure, LLD, Architecture, J2EE OR Python or Java, AWS, SQL OR NoSQL Nuaav (One of Uplers' Clients) is Looking for: Technical Architect who is passionate about their work, eager to learn and grow, and who is committed to delivering exceptional results. If you are a team player, with a positive attitude and a desire to make a difference, then we want to he...

Technical Architect - Software & Data Engineering Experience: 9 - 15 Years Exp Salary: Highly Competitive Preferred Notice Period : 30 Days Shift : 10:00 AM to 7:00 PM IST Opportunity Type: Remote (Noida)- Remote 6 Months, Then Hybrid (Flexible) Placement Type: Permanent (*Note: This is a requirement for one of Uplers' Clients) Must have skills required : Azure, LLD, Architecture, J2EE OR Python or Java, AWS, SQL OR NoSQL Nuaav (One of Uplers' Clients) is Looking for: Technical Architect who is passionate about their work, eager to learn and grow, and who is committed to delivering exceptional results. If you are a team player, with a positive attitude and a desire to make a difference, then...

We are looking for a results-driven Data Analytics Professional to join our team. This role requires a strong analytical mindset, deep understanding of data models, and the ability to transform raw data into meaningful insights. The successful candidate will collaborate across departments to support business decisions and improve data-driven strategies. Key Responsibilities: Analyze structured and unstructured data to extract actionable business insights. Design and maintain reports, dashboards, and data visualizations using tools like Power BI or Tableau, or Looker . Hands on Experience in Azure Synapse and Databricks. Work with cross-functional teams to identify business needs and deliver ...