15947 Pyspark Jobs - Page 27

At American Express, our culture is built on a 175-year history of innovation, shared values and Leadership Behaviors, and an unwavering commitment to back our customers, communities, and colleagues. As part of Team Amex, you'll experience this powerful backing with comprehensive support for your holistic well-being and many opportunities to learn new skills, develop as a leader, and grow your career. Here, your voice and ideas matter, your work makes an impact, and together, you will help us define the future of American Express. How will you make an impact in this role? This Position is in GSG Advanced Analytics team as part of GSG MIS COE, is looking for full time candidates as Data Scien...

Skills : Experience with SQL, ETL, data modeling, and at least one programming language (e.g., Python, C++, C#, Scala, etc.) Data Modeling including expertise in Data Warehouse design and Dimensional Modeling , Data Science workflow, ERP System Notice Period: Immediate -45 days

Responsibilities: Design, develop, and maintain Azure data solutions using ADF, ADL, SQL Azure, SSIS, Spark, Python/Pyspark,Synapse, SQL Collaborate with cross-functional teams on data integration projects.

The Applications Development Programmer Analyst is an intermediate level position responsible for participation in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. The overall objective of this role is to contribute to applications systems analysis and programming activities. Responsibilities: Utilize knowledge of applications development procedures and concepts, and basic knowledge of other technical areas to identify and define necessary system enhancements Identify and analyze issues, make recommendations, and implement solutions Utilize knowledge of business processes, system processes, and industry standard...

A career in our Advisory Acceleration Centre is the natural extension of PwC’s leading class global delivery capabilities. We provide premium, cost effective, high quality services that support process quality and delivery capability in support for client engagements. Years of Experience: Candidates with 8+ years of hands on experience Required Skills Familiarity with the contact center architecture domain, conversational design & implementation and industry-specific challenges Understanding of conversational (chats, emails and calls) data and its preprocessing (including feature engineering if required) to train Conversational AI systems. Strong problem-solving and analytical skills to trou...

Cotiviti Hyderabad is looking for Technical Architect / Technical Lead for one of its product stream Java/SQL Server/Hadoop based application suite to Spring Boot, Appian based modern application suite. This will include high percentage of unit test coverage both on backend and frontend. JOB RESPONSIBILITIES Lead end-to-end solution architecture for enterprise platforms and data processing pipelines , leveraging technologies such as Appian , Java/Spring Boot , Python , and PySpark . Design scalable, secure, and maintainable microservices-based architectures integrating REST APIs , Kafka streams , and data pipelines for real-time and batch processing. Architect data engineering workflows on C...

Job Description Some careers shine brighter than others. If you’re looking for a career that will help you stand out, join HSBC and fulfil your potential. Whether you want a career that could take you to the top, or simply take you in an exciting new direction, HSBC offers opportunities, support and rewards that will take you further. HSBC is one of the largest banking and financial services organisations in the world, with operations in 64 countries and territories. We aim to be where the growth is, enabling businesses to thrive and economies to prosper, and, ultimately, helping people to fulfil their hopes and realise their ambitions. We are currently seeking an experienced professional to...

1.AWS + Python + Pyspark, 2.Python + AWS + Bigdata 3. Python & AWS + Snowflake 4. Python & AWS + ETL Testing

Ecolab is looking for a Data Engineer to be part of a dynamic team that's at the forefront of technological innovation. We're leveraging cutting-edge AI to create novel solutions that optimize operations for our clients, particularly within the restaurant industry. Our work is transforming how restaurants operate, making them more efficient and sustainable. As a key player in our new division, you'll have the unique opportunity to help shape its culture and direction. Your contributions will directly impact the success of our innovative projects and help define the future of our product offerings.Additionally, you will experience the best of both worlds with this team at Ecolab: the agility ...

Looking for strong experienced candidate for Azure Data Architect position in Bangalore location with 8+ Years of experience .Required skills for this position are Design and implement data solutions using Azure services such as Data Factory, Synapse Analytics, Databricks, and ADF. Develop complex data pipelines to extract, transform, and load large datasets from various sources into Azure storage solutions like Blobs, Files, SQL DW, Postgres etc. Collaborate with cross-functional teams to understand business requirements and design scalable data architectures that meet those needs. Ensure high availability, security, performance tuning of deployed systems.

Onsite Opportunity Sr. Data Engineer / Lead Data Engineer Client: MNC Experience:8+years Location: Ras Al-Khaimah, UAE Job Mode: Permanent Salary Range: 15K AED to 20K AED Note: Selected candidates must hold a valid passport. Selected candidates should be open to travel to Ras Al-Khaimah, UAE. Candidates must possess strong communication skills. Key skills: Python, Data Engineer, Databricks, SQL, Shell Scripting, Snowflake, PySpark, dataframes, Azure Data Factory, Apache Airflow, and Databricks workflows, FastAPI, Azure etc.. Job Description: We are looking for a skilled Full Stack Developer with hands-on experience in building scalable web applications using Python (FastAPI/Flask/Django). Y...

A career within Data and Analytics services will provide you with the opportunity to help organisations uncover enterprise insights and drive business results using smarter data analytics. We focus on a collection of organisational technology capabilities, including business intelligence, data management, and data assurance that help our clients drive innovation, growth, and change within their organisations in order to keep up with the changing nature of customers and technology. We make impactful decisions by mixing mind and machine to leverage data, understand and navigate risk, and help our clients gain a competitive edge. As part of our Analytics and Insights Consumption team, you’ll an...

Role Overview Cotiviti Hyderabad is hiring a Senior Full Stack Software Engineer with deep expertise in backend data engineering and a strong foundation in distributed systems . This role focuses on building reliable data pipelines, optimizing large-scale data processing, and developing backend services with optional contributions to frontend, cloud, or low-code platforms. Youll play a key role in driving high-performance application development, ensuring data accuracy, and fostering Agile and DevOps best practices across teams. Key Responsibilities Develop scalable, high performance backend applications and Distributed data pipelines using Python and Pyspark Design and optimize complex SQL ...

Job Title: Fabric Data Engineer – Azure Employment Type: Full-time About The Role We are seeking a highly skilled Fabric Data Engineer with strong expertise in Azure ecosystem to design, build, and maintain scalable data solutions. The ideal candidate will have hands-on experience with Microsoft Fabric, Databricks, Azure Data Factory, PySpark, SQL, and other Azure services to support advanced analytics and data-driven decision-making. Key Responsibilities Design, develop, and maintain scalable data pipelines using Microsoft Fabric and Azure data services. Implement data integration, transformation, and orchestration workflows with Azure Data Factory, Databricks, and PySpark. Work with stakeh...

Roles and Responsibilities : Design, develop, test, deploy and maintain large-scale data pipelines using AWS services such as S3, Lambda, Step Functions etc. Collaborate with cross-functional teams to gather requirements and design solutions that meet business needs. Develop scalable and efficient data processing workflows using PySpark on AWS cloud platform. Troubleshoot issues related to data pipeline failures and optimize performance for improved efficiency. Job Requirements : 4-12 years of experience in Data Engineering with expertise in AWS technologies (S3, Lambda). Strong understanding of Python programming language and its applications in data engineering. Experience working with big...

Role Summary We are seeking a highly skilled Data Architect to design and optimize cloud-native data architectures on platforms like Databricks and Snowflake . In this role, you will lead the development of scalable data pipelines, drive governance and security frameworks, and enable advanced analytics and AI/ML solutions for global enterprise clients. This is a leadership opportunity for professionals passionate about shaping modern data ecosystems and mentoring data engineering teams. Key Responsibilities Design and implement Lakehouse architectures using Databricks, Delta Lake, and Apache Spark. Build and optimize data pipelines, ETL/ELT processes, and integration strategies across platfo...

THE ROLE As a Senior Data Engineer, you will be part of a team who design, build and maintain scalable, secure, and high-performance data solutions on Microsoft Azure. You will support mission-critical operations across oil trading, refining, and retail fuel distribution by building robust data pipelines, integrating real-time telemetry, and ensuring compliance with regional and international regulations. This role sits within the Oil Assets IT team, reporting to the Data Engineering Team Lead, with occasional travel to affiliates in South Africa and Brazil. PROFILE Seasoned data engineer with 7+ years of experience, and a strong foundation in designing and managing cloud-native data pipelin...

Overview GCP, Big Query, Composer, Python, Pyspark, Dataproc, Airflow, SQL Responsibilities GCP, Big Query, Composer, Python, Pyspark, Dataproc, Airflow, SQL Requirements GCP, Big Query, Composer, Python, Pyspark, Dataproc, Airflow, SQL

Role Description Role Proficiency: Independently provides expertise on data analysis techniques using software tools; streamlining business processes and managing team Outcomes Managing and designing the reporting environment including data sources security and metadata. Providing technical expertise on data storage structures data mining and data cleansing. Supporting the data warehouse in identifying and revising reporting requirements. Supporting initiatives for data integrity and normalization. Assessing tests and implementing new or upgraded software and assisting with strategic decisions on new systems. Synthesize both quantitative and qualitative data into insights Generating reports ...

Data Engineer Data Lakes Exp: 7+ Years Location :Bangalore Notice:Immediate Key Experience & Expertise Extensive hands-on experience in Partitioning Delta Lake Tables to achieve optimized query performance and storage efficiency. Proficient in reading, interpreting, and reporting Delta Lake Transaction Logs to ensure data reliability and consistency. Strong expertise in transforming streaming data and building Delta Live Tables (DLT) pipelines for real-time analytics. Proven ability to design and implement Slowly Changing Dimensions (SCDs) using PySpark for scalable data warehousing solutions. Skilled in configuring and leveraging Auto Optimize to enhance Delta Lake performance. Practical kn...

Job Title: Data Engineer – ETL & Data Pipeline Development Location: Hyderabad / Bangalore Employment Type: Full-time | Long-term Engagement experience: 7+years About the Role We are seeking a highly skilled Data Engineer with strong expertise in building and managing data pipelines to join our growing team in Hyderabad/Bangalore. The ideal candidate will bring hands-on proficiency in ETL tools, Python, PySpark, SQL , and modern data warehousing/lakehouse technologies . This role demands independence, problem-solving capability, and the ability to deliver efficient, scalable, and cost-optimized data solutions in a dynamic environment. Key Responsibilities Design, develop, and maintain data p...

Greetings from TCS!! TCS is hiring for Azure Data Engineer Required Experience Range: 4 to 8 years Role: Azure Data Engineer(ADF, ADB) Locations: Mumbai, Pune, Bangalore, Bhubaneshwar, Kolkata Required Technical Skill Set: Azure Data Engineer, ADF, Azure Databricks Spark (PySpark or Scala), Python, PL/SQL. Role & responsibilities: Minimum 3+ years of troubleshooting and Supporting large databases and testing activities; Identifying reporting, and managing database security issues, user access/management; Designing database backup, archiving and storage, performance tuning, ETL importing large volume of data extracted from multiple systems, capacity planning Strong experience in Azure Data Fa...

Summary Job Description As a Data Engineer you will work on implementing complex data projects with a focus on collecting, parsing, managing, analysing, and visualising large sets of data to turn information into value using multiple platforms. You will work with business analysts and data scientists to understand customer business problems and needs, secure the data supply chain, implement analysis solutions, and visualise outcomes that support improved decision making for a customer. You will understand how to apply technologies to solve big data problems and to develop innovative big data solutions. Key Accountabilities Working with colleagues to understand and implement requirements Secu...

Eligibility Criteria: · Years of Experience : Minimum 7-9 years · Job Experience: Data Analysis/ Statistical Analysis, Data Profiling Expert in Database and Data warehouse tech (MS Fabric /PySpark/Azure Synapse/ SQL Server/SAP HANA) Experience with Visualization tools ( Power BI) Educational : Bachelor’s Degree: Math/Statistics/Operations Research/Computer Science Master’s Degree : Business Analytics (with a background in Computer Science) Primary Responsibilities: · Designs, Develops, and Tests Data Models to import data from source systems to meet Project requirements · Effectively analyzes the heterogeneous source data and writes SQL/PySpark scripts to integrate data from multiple data so...

9 - 12 Years 1 Opening Trivandrum Role description Role Proficiency: Leverage expertise in a technology area (e.g. Infromatica Transformation Terradata data warehouse Hadoop Analytics) Responsible for Architecture for a small/mid-size projects. Outcomes: Implement either data extract and transformation a data warehouse (ETL Data Extracts Data Load Logic Mapping Work Flows stored procedures data warehouse) data analysis solution data reporting solutions or cloud data tools in any one of the cloud providers(AWS/AZURE/GCP) Understand business workflows and related data flows. Develop design for data acquisitions and data transformation or data modelling; applying business intelligence on data o...