15947 Pyspark Jobs - Page 50

Experience in Big Data Technologies and Data Handling Experience in PySpark Knowledge & experience in SQL GCP/AWS/Azure Hadoop / HDFS/Other Databases Airflow/Other Automation tools Linux/ Bash scripting Kafka/Flume/Kinesis/Data Streaming Experience in Cloud, DevOps & CI/CD B.Tech Experience in Big Data Technologies and Data Handling Experience in PySpark Knowledge & experience in SQL GCP/AWS/Azure Hadoop / HDFS/Other Databases Airflow/Other Automation tools Linux/ Bash scripting Kafka/Flume/Kinesis/Data Streaming Experience in Cloud, DevOps & CI/CD

Responsible for managing multiple client engagements. Understand and analyse business requirements by working with various stakeholders and create the appropriate information architecture, taxonomy and solution approach Work independently to gather requirements, cleansing extraction and loading of data Translate business and analyst requirements into technical code Create interactive and insightful dashboards and reports using Power BI , connecting to various data sources and implementing DAX calculations. Design and build complete ETL/Azure Data Factory processes moving and transforming data for ODS, Staging, and Data Warehousing Design and development of solutions in Data Bricks, Scala, Sp...

Develop and maintain efficient ETL (Extract, Transform, Load) pipelines that ingest, process and transform large datasets from diverse sources. Build and manage data storage solutions using relational and non-relational databases, ensuring high availability, performance and scalability. Work with APIs, third-party data sources and internal databases to create unified datasets that provide actionable insights for business stakeholders. Partner with business intelligence analysts, data scientists and software engineers to design and implement data-driven solutions that meet business requirements. Design optimized data schemas and structures that support analytical workloads and improve query p...

Design and develop solutions using Bigdata tools and technologies like Bigquery, Hive, Spark etc. Extensive hands-on experience in object-oriented programming using Python, PySpark APIs etc. Experience in building data pipelines for huge volume of data. Experience in designing, implementing, and managing various ETL job execution flows. Experience in implementing and maintaining Data Ingestion process. Hands on experience in writing basic to advance level of optimized queries using HQL, SQL & Spark. Hands on experience in designing, implementing, and maintaining Data Transformation jobs using most efficient tools/technologies. Ensure the performance, quality, and responsiveness of solutions....

Design and develop solutions using Bigdata tools and technologies like Bigquery, Hive, Spark etc. Extensive hands-on experience in object-oriented programming using Python, PySpark APIs etc. Experience in building data pipelines for huge volume of data. Experience in designing, implementing, and managing various ETL job execution flows. Experience in implementing and maintaining Data Ingestion process. Hands on experience in writing basic to advance level of optimized queries using HQL, SQL & Spark. Hands on experience in designing, implementing, and maintaining Data Transformation jobs using most efficient tools/technologies. Ensure the performance, quality, and responsiveness of solutions....

At American Express, our culture is built on a 175-year history of innovation, shared values and Leadership Behaviors, and an unwavering commitment to back our customers, communities, and colleagues. As part of Team Amex, you'll experience this powerful backing with comprehensive support for your holistic well-being and many opportunities to learn new skills, develop as a leader, and grow your career. Here, your voice and ideas matter, your work makes an impact, and together, you will help us define the future of American Express. Sr Analyst, Business Insights, Global Strategy, Operations, & Performance, GMNS Global Merchant & Network Services (GMNS), the merchant and bank partner network of...

Decision Analytics EXL (NASDAQ:EXLS) is a leading operations management and analytics company that helps businesses enhance growth and profitability in the face of relentless competition and continuous disruption. Using our proprietary, award-winning Business EXLerator Framework™, which integrates analytics, automation, benchmarking, BPO, consulting, industry best practices and technology platforms, we look deeper to help companies improve global operations, enhance data-driven insights, increase customer satisfaction, and manage risk and compliance. EXL serves the insurance, healthcare, banking and financial services, utilities, travel, transportation and logistics industries. Headquartered...

EXL Decision Analytics EXL (NASDAQ: EXLS) is a leading operations management and analytics company that helps businesses enhance growth and profitability in the face of relentless competition and continuous disruption. Using our proprietary, award-winning Business EXLerator Framework™, which integrates analytics, automation, benchmarking, BPO, consulting, industry best practices and technology platforms, we look deeper to help companies improve global operations, enhance data-driven insights, increase customer satisfaction, and manage risk and compliance. EXL serves the insurance, healthcare, banking and financial services, utilities, travel, transportation and logistics industries. Headquar...

Job Description Role and Responsibilities: Emphasis is on end-to-end delivery of analysis Extremely comfortable working with data, including managing large number of data sources, analyzing data quality, and pro-actively working with client’s data/ IT teams to resolve issues Use variety of analytical tools (Python, SQL, PySpark etc.) to carry out analysis and drive conclusions Reformulate highly technical information into concise, understandable terms for presentations Candidate Profile: Required skills: Python, SQL, Tableau Good to have skills: Pyspark, SAS 4- 6 years of consulting, analytics delivery experience Experience in Banking and Financial Services domain is preferred. Very strong a...

Job Description Role and Responsibilities: Emphasis is on end-to-end delivery of analysis Extremely comfortable working with data, including managing large number of data sources, analysing data quality, and pro-actively working with client’s data/ IT teams to resolve issues Use variety of analytical tools (Python, SQL, PySpark etc.) to carry out analysis and drive conclusions Reformulate highly technical information into concise, understandable terms for presentations Specefic Requirements Good to have :- Any cloud platform knowledge(AWS), Airflow Candidate Profile: Required skills: Python, SQL,, Big Data, Hive,PySpark, Hadoop 2- 6 years of consulting, analytics delivery experience Experien...

Responsibilities: Support critical project initiative to migrate programming codes, processes, data, and reports from Big Data environment to Google Cloud Platform Big Data environment. Expert in SQL, Google BigQuery and Google Cloud Platform. Expertise in Python Notebook and programming languages such as PySpark/Python. Identify opportunities to optimize, streamline and automate SQL codes, processes, and reports. Perform detail testing and validation on programming codes, processes, and reports. Align with stakeholders on final outputs. Documentation of the programming codes, processes, and reports. Transition and training of the programming codes, processes and reports. Required Qualificat...

About the Role: We are seeking a results-driven and detail-oriented data analyst to support data-driven decision-making within banking risk operations. This role involves working closely with stakeholders to provide actionable insights, enhance strategies, and drive operational efficiencies using tools such as SQL, Python, and advanced analytics. Key Responsibilities: Analyze large volumes of data to identify trends, patterns, and performance drivers. Collaborate with different teams to support and influence decision-making processes. Perform root cause analysis and recommend improvements to optimize processes Design and track key KPIs Ensure data integrity and accuracy across reporting tool...

About the Role: We are seeking a results-driven and detail-oriented data analyst to support data-driven decision-making within banking risk operations. This role involves working closely with stakeholders to provide actionable insights, enhance strategies, and drive operational efficiencies using tools such as SQL, Python, and advanced analytics. Key Responsibilities: Analyze large volumes of data to identify trends, patterns, and performance drivers. Collaborate with different teams to support and influence decision-making processes. Perform root cause analysis and recommend improvements to optimize processes Design and track key KPIs Ensure data integrity and accuracy across reporting tool...

We are looking for PySpark developer with below skills: Experienced in design and maintain data pipelines using AWS services. Implemented data processing frameworks using Python and PySpark/Spark. Developed and maintained APIs for data access and manipulation. Monitor and manage data workflows. Experience with data engineering with cloud service knowledge. - proficient in AWS Glue, Lambda and other AWS services. Strong programming skill in Python, PySpark and SQL. Hands-on experience with CI/CD tools such as Jenkins and GitHub. Solid understanding of data storage solutions and management. Excellent in problem-solving skills and able to work on open ended problems Deliver with limited directi...

Position Purpose The Senior Developer will be a part of the ISPL Mumbai IHC ETL projects team. The developer position will primarily work on Apache Spark(python), Spark SQL, ETL tools, Unix, Autosys and DB Responsibilities Direct Responsibilities Expertise on PySpark, database migration, transformation, and integration solutions for any Data warehousing project. Must have excellent knowledge in Apache Spark and Python programming experience. Deep experience in developing data processing tasks using PySpark such as reading data from external sources, merging data, performing data enrichment, and loading into target data destinations. Experience in deployment and operationalizing the code, kno...

Role & responsibilities • Develop and maintain scalable data pipelines using Apache Spark on Databricks. • Build end-to-end ETL/ELT pipelines on AWS using services like S3, Glue, Lambda, EMR, and Step Functions. • Collaborate with data scientists, analysts, and business stakeholders to deliver high-quality data solutions. • Design and implement data models, schemas, and lakehouse architecture in Databricks. • Optimize and tune Spark jobs for performance and cost-efficiency. • Integrate data from multiple structured and unstructured data sources. • Monitor and manage data workflows, ensuring data quality, consistency, and security. • Follow best practices in CI/CD, code versioning (Git), and ...

Responsibilities Direct Responsibilities * Represent CEP IT at local and global CIB Forums on Technical Architecture subjects * Propose / Review/ Challenge Application Architecture and Design. * Guide/Review Technology Obsolescence Management plan for CEP teams. * Lead automation and guide teams to align with shift left and shift right strategy by encouraging a mindset for automation first and reduce recursive manual efforts * Goto person to find solutions to any technical challenges in the domain. * Hands on and lead by example. * Resolve performance bottlenecks. * Review Performance test reports and propose preventive measures to be undertaken. * Take ownership of POCs and technical feasib...

Senior Infrastructure Architect. Would being part of a digital transformation excite you. Are you passionate about infrastructure security. Join our digital transformation team. We operate at the heart of the digital transformation of our business. Our team is responsible for the cybersecurity, architecture and data protection for our global organization. We advise on the design and validation of all systems, infrastructure, technologies and data protection.. Partner the best. As a Senior Infrastructure Architect, you will be responsible for:. Participate in the domain technical and business discussions relative to future architect direction.. Assist in the analysis, design and development o...

Position Purpose DPR team member guarantees application production for both technical and functional support for a given Business Line or Business Line application chain, on the IT asset management for the infrastructure (IAAS , PAAS, CAAS and IT cloud ). He/she ensures that the service is maintained in operational condition. He/she ensures the smooth running of the processes (batch process[1]ing, transfer, etc.) and the proper functioning in the application sense of the term (relevant business data, functional result, correct execution of application workflows, etc.). In a context of accelerating change, he/she manages the various releases within his/her scope. He/she intervenes not only in...

Job Description Job Title: Software/Data Engineer AWS Role Overview We are seeking a skilled Data Engineer to develop and maintain a high-performance streaming data platform for a leading healthcare CRM product You will work across technologies like Python, SQL, NodeJS, Kafka, and cloud-native services, contributing to scalable, efficient, and robust data solutions, Key Responsibilities Design, develop, and maintain scalable data pipelines and streaming systems, Work with Python, SQL, Scala, NodeJS, and various data stores (relational, NoSQL, Kafka), Analyze healthcare and marketing datasets using SQL, Refactor and optimize existing codebases for performance improvements, Collaborate closely...

**Job Title:*Data Architect Azure Data Engineering **Company:*Happiest Minds **Location:*Bangalore **Job Type:*Full-Time **Minimum Years of Experience:*10 **Maximum Years of Experience:*15 --- **About Happiest Minds:** At Happiest Minds, we are committed to providing innovative digital transformation solutions that empower businesses and enhance user experiences We value creativity, collaboration, and a commitment to quality, and we are seeking like-minded individuals who are passionate about leveraging technology to drive positive change, --- **Job Description:** We are looking for an experienced **Data Architect*with a deep understanding of data management and analytics to join our dynamic...

Role & responsibilities We are looking for a Senior Data Engineer with strong expertise in SQL, Python, Azure Synapse, Azure Data Factory, Snowflake, and Databricks. The ideal candidate should have a solid understanding of SQL (DDL, DML, query optimization) and ETL pipelines while demonstrating a learning mindset to adapt to evolving technologies. Required Skills & Qualifications: SQL Expertise Strong hands-on experience with DDL, DML, query optimization, and performance tuning. Programming Languages – Proficiency in Python or Java for data processing and automation. Data Modelling – Good understanding of entity-relationship modelling, star schema, and Kimball methodology. Cloud Data Enginee...

Position Purpose Overall 10+ years in data management roles within banking, asset management, insurance, or financial services. We are seeking a highly skilled Senior Data Architect with deep technical expertise and strong domain knowledge in financial services. This role will be responsible for designing and implementing data platforms that support critical financial processes such as internal reporting, data analysis, non-financial risk management and financial reporting. You will mentor & work with a team of data engineers & developers to ensure data is accurate, accessible, secure, and aligned with both business and regulatory needs. The Data Architect should have key skills as mentioned...

Role Overview: SDE/DE We are seeking technically strong Senior Data Engineers / Leads who can take ownership of designing and developing cutting-edge data and AI platforms using big data and cloud technologies on Microsoft Azure. You will play a critical role in building scalable data pipelines, modern data architectures, and intelligent analytics solutions. Key Responsibilities Design and implement scalable, metadata-driven frameworks for data ingestion, quality, and transformation across both batch and streaming datasets. Develop and optimize end-to-end data pipelines to process structured and unstructured data, enabling the creation of analytical data products. Build robust exception hand...

Offshore Years exp 10-13 yrs Hyderabad- 5 days onsite About the Role : In this role, you will be responsible for developing and enhancing data-processing, orchestration, monitoring, and more by leveraging popular open-source software, AWS, and GitLab automation. Responsibilities : Develop and enhance data-processing, orchestration, monitoring, and more by leveraging popular open-source software, AWS, and GitLab automation. Collaborate with product and technology teams to design and validate the capabilities of the data platform. Identify, design, and implement process improvements: automating manual processes, optimizing for usability, re-designing for greater scalability. Provide technical ...