143 Spark Sql Jobs

Role Overview: As a Sr. Associate / Manager - Azure Databricks Developer in Kochi, you will be responsible for designing, developing, and implementing data integration solutions using Azure Databricks, PySpark, and Azure Data Factory. Your role will involve collaborating with cross-functional teams, maintaining high-quality code, troubleshooting technical issues, and staying updated with the latest Azure Cloud and data integration technologies. Key Responsibilities: - Design, develop, and implement data integration solutions using Azure Databricks, PySpark, and Azure Data Factory. - Collaborate with cross-functional teams to identify business requirements and develop data integration solutio...

As an experienced professional with over 7+ years of experience in Noida, your role will involve the following key responsibilities: - Reviewing and understanding business requirements to ensure timely completion of development tasks with minimal defects - Collaborating with a software development team to implement best practices and enhance the performance of Data applications for meeting client needs - Engaging with various teams and customers to translate their business challenges into innovative solutions - Conducting research on new Big Data technologies to align them with the business and technology strategy - Operating within a rapid and agile development process to accelerate speed t...

The ideal candidate will have experience in developing and implementing data integration solutions using Azure Databricks, PySpark, and Azure Data Factory. The role requires strong technical expertise in Azure Cloud, Spark SQL, and ADLS. Location Kochi Responsibilities Design, develop, and implement data integration solutions using Azure Databricks, PySpark, and Azure Data Factory. Collaborate with cross-functional teams to identify business requirements and develop data integration solutions. Develop and maintain high-quality code, ensuring scalability, performance, and reliability. Troubleshoot and resolve technical issues related to data integration and Azure Databricks. Stay up-to-date w...

Role Overview: You are an experienced Azure Databricks + SQL Developer / Big Data Engineer responsible for designing, developing, and maintaining scalable data solutions on Azure. Your primary focus will be on building efficient ETL/ELT pipelines, optimizing SQL queries, and leveraging Databricks and other Azure services for advanced data processing, analytics, and data platform engineering. Your strong background in traditional SQL development and modern big data technologies on Azure will be crucial for this role. Key Responsibilities: - Develop, maintain, and optimize ETL/ELT pipelines using Azure Databricks (PySpark/Spark SQL). - Write and optimize complex SQL queries, stored procedures,...

Job Title: Data Engineer Microsoft Fabric Experience: 10+ Years Location: Remote Timings: 2 PM 11 PM IST Data Enginee r with over 10 years of experience designing and building large-scale data platforms across cloud and on-prem ecosystems. Proven track record of developing high-performance ETL pipelines, optimizing data infrastructure, and enabling analytics teams through reliable, well- structured, and secure data systems. Deep expertise in Microsoft Fabric and GCP, with hands-on experience across data ingestion, transformation, modeling, and delivery. Comfortable working in fast-moving environments that require ownership, adaptability, and strong collaboration with analytics and product te...

As a Senior Automation QA at Barclays, you will play a crucial role in supporting the successful delivery of location strategy projects. Your responsibilities will include planning, budgeting, and ensuring quality and governance standards are met. You will be at the forefront of driving innovation and excellence in the digital landscape, leveraging cutting-edge technology to enhance our digital offerings and deliver exceptional customer experiences. Key Responsibilities: - Design, develop, and execute testing strategies to validate functionality, performance, and user experience. - Collaborate with cross-functional teams to identify and resolve defects, continuously improve testing processes...

As a Director Data Scientist at our company based in Bengaluru, you will play a crucial role in building end-to-end DS/AI/ML solutions. Your responsibilities will include: - Building and owning analyses, model training, deployment, business impact A/B evaluation, and subsequent iterations for best-in-class data science models. - Focusing primarily on dynamic pricing and discounting modules for various lines of business such as flights, hotels, domestic, international, B2C meta, and potentially other products of both brands MakeMyTrip and GoIbibo. - Collaborating with stakeholders at different project stages and leading project development. - Conducting iterative experiments to demonstrate in...

JD -7+ years of hands on experience in Python especially dealing with Pandas and Numpy Good hands-on experience in Spark PySpark and Spark SQL Hands on experience in Databricks Unity Catalog Delta Lake Lake house Platform Medallion Architecture Azure Data Factory ADLS Experience in dealing with Parquet and JSON file format Knowledge in Snowflake.

Required Skills: Must have: Proven experience as a Data Engineer with Assistant Manager designation or equivalent. Strong hands-on experience in Databricks (PySpark / Spark SQL). Proficiency in at least one cloud platform AWS, Azure, or GCP. Advanced SQL and Python programming skills. Experience in data lake / data warehouse solutions (Snowflake, Delta Lake, etc.). Good understanding of data modeling, ETL processes, and orchestration tools (Airflow, ADF, etc.). Domain experience in Pharma, Life Sciences

En Derevo, empoderamos a las empresas y las personas, liberando el valor de los datos en las organizaciones. Con ms de 15 aos de experiencia, diseamos soluciones de datos e IA de punta a punta, desde la integracin en arquitecturas modernas hasta la implementacin de modelos inteligentes en procesos clave del negocio. Buscamos tu talento como Data Engineer (MS Fabric)!! Es importante que vivas en Mxico o Colombia. Como Data Engineer en Derevo, tu misin ser clave para crear e implementar arquitecturas modernas de datos con alta calidad, impulsando soluciones analticas basadas en tecnologas de Big Data. Disears, mantendrs y optimizars sistemas de multiprocesamiento paralelo, aplicando las mejore...

Position Summary The Senior BI Data Analyst will collaborate with business stakeholders and team members to define BI solutions, data requirements, and project scope. They will recommend innovative BI solution approaches that meet or exceed business needs and are responsible for managing BI development projects from inception to completion. The role also includes providing ongoing support and enhancements, ensuring that solutions remain effective and aligned with organizational goals. Additionally, the analyst will contribute to the BI Center of Excellence by sharing knowledge, training users, and staying abreast of emerging BI trends to maintain a competitive edge. Key Responsibilities BI R...

As a senior Applications Development Technology Lead Analyst at our company, your role involves establishing and implementing new or revised application systems and programs in coordination with the Technology team. Your primary objective is to lead applications systems analysis and programming activities. Responsibilities: - Partner with multiple management teams to ensure appropriate integration of functions to meet goals and identify necessary system enhancements for deploying new products and process improvements - Resolve high impact problems/projects through in-depth evaluation of complex business processes and industry standards - Provide expertise in applications programming and ensu...

As a Senior Spark SQL/SQL Developer, you will be responsible for designing, developing, and optimizing complex database systems to meet business requirements and support critical applications. Your key responsibilities will include: - Designing, developing, and optimizing complex Spark SQL / SQL queries, stored procedures, views, and triggers for high-performance systems. - Leading the design and implementation of scalable database architectures to meet business needs. - Performing advanced query optimization and troubleshooting to ensure database performance, efficiency, and reliability. - Mentoring junior developers and providing guidance on best practices for SQL development, performance ...

As a Data Engineer & Warehousing professional, your primary responsibilities will include managing the full development cycle of Data Engineering & Warehousing concepts, from requirements gathering to maintenance. You will be identifying and resolving data issues to ensure data quality and consistency, and implementing automation opportunities. Developing expertise in data and data models within the data warehouse, and understanding supported business domains will also be part of your role. Effective communication with technical personnel, business managers, and senior leadership regarding data inquiries and requests is essential. You will conduct internal testing & user acceptance testing, ...

As a skilled PySpark Developer with hands-on experience in Reltio MDM, your role will involve designing and implementing scalable data processing solutions using PySpark and integrating with Reltio's cloud-native MDM platform. Key Responsibilities: - Develop and maintain data pipelines using PySpark in distributed computing environments such as AWS EMR and Databricks. - Integrate and synchronize data between enterprise systems and the Reltio MDM platform. - Design and implement data transformation, cleansing, and enrichment processes. - Collaborate with data architects, business analysts, and Reltio solution architects to ensure high-quality data modeling. - Work on API-based integration bet...

As a Data Engineer at UPS, your role involves participating in the support of batch and real-time data pipelines for Marketing and Finance business units. You will be responsible for integrating data from various sources, performing ETL data conversions, and facilitating data cleansing and enrichment. Your duties will include full systems life cycle management activities, such as analysis, technical requirements, design, coding, testing, and implementation of systems and applications software. Additionally, you will contribute to synthesizing disparate data sources to support reusable and reproducible data assets. **Key Responsibilities:** - Supervising and supporting data engineering projec...

As a highly skilled and experienced Hands-on Palantir Architect, your role will involve the end-to-end design, development, and implementation of complex data solutions on the Palantir platform. You will need to be proficient in both architectural design and hands-on development, including building robust data pipelines and creating powerful applications for critical insights. Key Responsibilities: - Lead the technical design and architecture of solutions on the Palantir platform, ensuring scalability, performance, and security. - Actively participate in the development of data pipelines using PySpark, Spark SQL, TypeScript, and Python to transform and integrate data from various sources. - ...

Role Overview: As a Senior Data Test Engineer at the company, you will be responsible for ensuring the quality and accuracy of data through comprehensive testing. Your expertise in SQL, Spark SQL, PostgreSQL, MySQL, MongoDB, and other related technologies will be utilized for data testing and quality assurance. Additionally, proficiency in API testing, GraphQL, data lakes, and cloud platforms such as AWS, Azure, or GCP will be required. Your scripting skills in JavaScript/TypeScript will play a crucial role in automating data validation tasks. Familiarity with JIRA and Bugzilla for defect tracking will also be essential in your responsibilities. Key Responsibilities: - Conduct data testing a...

As a Data Engineer at UPS, your role involves participating in the support of batch and real-time data pipelines for Marketing and Finance business units. You will be responsible for integrating data from various sources, performing ETL data conversions, and facilitating data cleansing and enrichment. Your duties will include full systems life cycle management activities, such as analysis, technical requirements, design, coding, testing, and implementation of systems and applications software. Additionally, you will contribute to synthesizing disparate data sources to support reusable and reproducible data assets. **Key Responsibilities:** - Supervising and supporting data engineering projec...

We are looking for a proactive and detail-oriented Data Support Engineer to join our growing team in Visakhapatnam . The ideal candidate should possess strong analytical and debugging skills, with hands-on experience in resolving production issues related to ADF, Snowflake, and Databricks (Spark SQL) . You will be responsible for monitoring, troubleshooting, and resolving daily production bugs/issues, while coordinating with cross-functional and US-based teams. Key Responsibilities: Identify, analyze, and resolve production issues across ADF, Snowflake, and Databricks environments. Perform SQL debugging and data validation to ensure smooth data flow across pipelines. Collaborate with interna...

Role Overview: As a seasoned Spark Developer with 5+ years of experience, you have a deep understanding of developing, testing, deploying, and debugging Spark Jobs using Scala on the Hadoop Platform. Your expertise includes proficiency in Spark Core and working with RDDs and Spark SQL, knowledge of Spark Optimization Techniques and Best practices, familiarity with Scala Functional Programming concepts like Try, Option, Future, and Collections, understanding of Scala OOPS principles including Classes, Traits, Objects (Singleton and Companion), and Case Classes, strong grasp of Scala Language Features such as Type System and Implicit/Givens, hands-on experience in Hadoop Environment (HDFS/Hive...

As a Sr. Data Engineer at NTT DATA in Bangalore, Karnataka (IN-KA), India, your primary responsibility is to build and implement PySpark-based data pipelines in Azure Synapse to transform and load data into ADLS in Delta format. You will design and implement dimensional (star/snowflake) and 3NF data models optimized for access using Power BI. Additionally, you will perform unit testing of data pipelines and transformations, design metadata-driven data pipelines using PySpark in Azure Synapse, and analyze and optimize Spark SQL queries. Collaborating with cross-functional teams to ensure data models align with business needs and maintaining version control and CI/CD pipelines in Git and Azure...

As a Senior Developer at TransUnion, you will play a crucial role in designing, developing, and implementing cutting-edge solutions using a variety of technologies such as Java, Big Data frameworks, AI/ML concepts, and cloud platforms. Your contributions will be vital to the architecture, development, and deployment of data-driven applications. Key Responsibilities: - Design, develop, and maintain high-performance applications utilizing Java and related technologies. - Work with Big Data technologies like Hadoop, HDFS, Spark, and Scala to process and analyze large datasets. - Develop data pipelines for ingestion, transformation, and storage. - Write, optimize, and debug SQL queries across pl...

Notice Period-Immediate Joiner or Max 10 days (Do not share long notice period profile.) Permanent Payroll - Anlage Infotech Client - Orion Innovation Location- Kochi, Chennai, Coimbatore, Pune - HYBRID (2/3 days in a week) Role & responsibilities Work Location: Kochi, Chennai, Coimbatore, Pune Hybrid Mode Mandate skills - Pyspark + Azure DataBricks + Azure Data Factory + MS Fabric Experience- 5+ years We are seeking a skilled and experienced Senior Data Engineer with a strong background in Azure Data Services, particularly Azure Databricks, Azure Data Factory, PySpark and MS Fabric. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines and...

Notice Period-Immediate Joiner or Max 10 days (Do not share long notice period profile.) Permanent Payroll - Anlage Infotech Client - Orion Innovation Location- Kochi - Chennai - Coimbatore - HYBRID (2/3 days in a week) Role & responsibilities Bachelors or Masters degree in Computer Science, Engineering, Information Systems, or a related field. 5+ years of hands-on experience in data engineering roles. Strong expertise in: Azure Databricks Azure Data Factory PySpark Experience working with Azure Data Lake Storage (ADLS) and Delta Lake. Strong SQL skills for data extraction and transformation. Experience with version control tools like Git and CI/CD practices. Familiarity with orchestration, ...