8417 Pyspark Jobs - Page 15

Diverse Lynx is looking for Pyspark Developer to join our dynamic team and embark on a rewarding career journey Designing and developing big data applications using the PySpark framework to meet the needs of the business Writing and optimizing Spark SQL statements to extract and manipulate large datasets Developing and deploying Spark algorithms to perform data processing and analytics tasks, such as machine learning and graph processing Debugging and troubleshooting Spark code to resolve any issues and improve the performance of the applications Collaborating with cross-functional teams, such as data engineers and data analysts, to ensure that the PySpark applications are integrated with other systems Creating and maintaining documentation to ensure that the big data architecture, design, and functionality are well understood by others Should be detail-oriented, have excellent problem-solving and communication skills

Project Role : Custom Software Engineer Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills : PySpark Good to have skills : NA Minimum 3 Year(s) Of Experience Is Required Educational Qualification : 15 years full time education Summary: As a Custom Software Engineer, you will develop custom software solutions to design, code, and enhance components across systems or applications. Your typical day will involve collaborating with cross-functional teams to understand business requirements, utilizing modern frameworks and agile practices to deliver scalable and high-performing solutions tailored to specific business needs. You will engage in problem-solving activities, ensuring that the software solutions meet the highest standards of quality and performance while adapting to evolving project requirements. Roles & Responsibilities: - Expected to be an SME. - Collaborate and manage the team to perform. - Responsible for team decisions. - Engage with multiple teams and contribute on key decisions. - Provide solutions to problems for their immediate team and across multiple teams. - Mentor junior team members to enhance their skills and knowledge. - Continuously evaluate and improve software development processes to increase efficiency. Professional & Technical Skills: - Must To Have Skills: Proficiency in PySpark. - Strong understanding of data processing frameworks and distributed computing. - Experience with modern software development methodologies, particularly Agile. - Familiarity with cloud platforms and services for deploying applications. - Ability to troubleshoot and optimize performance in software applications. Additional Information: - The candidate should have minimum 5 years of experience in PySpark. - This position is based at our Bengaluru office. - A 15 years full time education is required., 15 years full time education

What do we do? The TTS Analytics team provides analytical insights to the Product, Pricing, Client Experience and Sales functions within the global Treasury & Trade Services business. The team works on business problems focused on driving acquisitions, cross-sell, revenue growth & improvements in client experience. The team extracts relevant insights, identifies business opportunities, converts business problems into analytical frameworks, uses big data tools and machine learning algorithms to build predictive models & other solutions, and designs go-to-market strategies for a huge variety of business problems. Role Description The role will be Spec Analytics Analyst 2 (C10) in the TTS Analytics team The role will report to the AVP leading the team The role will involve working on multiple analyses through the year on business problems across the client life cycle – acquisition, engagement, client experience and retention – for the TTS business This will involve leveraging multiple analytical approaches, tools and techniques, working on multiple data sources (client profile & engagement data, transactions & revenue data, digital data, unstructured data like call transcripts etc.) to provide data driven insights to business and functional stakeholders Qualifications Experience: Bachelor’s Degree with 3+ years of experience in data analytics or Masters Degree with 2+ years of experience in data analytics Must have: Identifying and resolving business problems (around sales/marketing strategy optimization, pricing optimization, client experience, cross-sell and retention) preferably in the financial services industry Leveraging and developing analytical tools and methods to identify patterns, trends and outliers in data Applying Predictive Modeling techniques for a wide range of business problems Working with data from different sources, with different complexities, both structured and unstructured Utilizing text data to derive business value by leveraging different NLP techniques Good to have: Experience working with data from different sources and of different complexity Skills: Analytical Skills: Proficient in formulating analytical methodology, identifying trends and patterns with data Has the ability to work hands-on to retrieve and manipulate data from big data environments Tools and Platforms: Proficient in Python/R, PySpark and related tools Experience in Hive Proficient in MS Excel, PowerPoint Good to have: Experience with PySpark Experience with Tableau Soft Skills: Strong analytical and problem-solving skills Excellent communication and interpersonal skills Be organized, detail oriented, and adaptive to matrix work environment ------------------------------------------------------ Job Family Group: Decision Management ------------------------------------------------------ Job Family: Business Analysis ------------------------------------------------------ Time Type: Full time ------------------------------------------------------ Most Relevant Skills Please see the requirements listed above. ------------------------------------------------------ Other Relevant Skills For complementary skills, please see above and/or contact the recruiter. ------------------------------------------------------ Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law. If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi. View Citi’s EEO Policy Statement and the Know Your Rights poster.

Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : AWS Architecture Good to have skills : Python (Programming Language) Minimum 5 Year(s) Of Experience Is Required Educational Qualification : 15 years full time education Summary: As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and implementing ETL processes to effectively migrate and deploy data across various systems. You will collaborate with cross-functional teams to understand their data needs and provide innovative solutions that enhance data accessibility and usability. AWS Data Architect to lead the design and implementation of scalable, cloud-native data platforms. The ideal candidate will have deep expertise in AWS data services, along with hands-on proficiency in Python and PySpark for building robust data pipelines and processing frameworks. Roles & Responsibilities: - Expected to be an SME. - Collaborate and manage the team to perform. - Responsible for team decisions. - Engage with multiple teams and contribute on key decisions. - Provide solutions to problems for their immediate team and across multiple teams. - Mentor junior team members to enhance their skills and knowledge. - Continuously evaluate and improve data processes to ensure efficiency and effectiveness. - Design and implement enterprise-scale data lake and data warehouse solutions on AWS. - Lead the development of ELT/ETL pipelines using AWS Glue, EMR, Lambda, and Step Functions, with Python and PySpark. - Work closely with data engineers, analysts, and business stakeholders to define data architecture strategy. - Define and enforce data modeling, metadata, security, and governance best practices. - Create reusable architectural patterns and frameworks to streamline future development. - Provide architectural leadership for migrating legacy data systems to AWS. - Optimize performance, cost, and scalability of data processing workflows. Professional & Technical Skills: - Must To Have Skills: Proficiency in AWS Architecture. - Strong understanding of data modeling and database design principles. - Experience with ETL tools and data integration techniques. - Familiarity with data warehousing concepts and technologies. - Knowledge of programming languages such as Python or Java for data processing. - AWS Services: S3, Glue, Athena, Redshift, EMR, Lambda, IAM, Step Functions, CloudFormation or Terraform - Languages: Python ,PySpark .SQL - Big Data: Apache Spark, Hive, Delta Lake - Orchestration & DevOps: Airflow, Jenkins, Git, CI/CD pipelines - Security & Governance: AWS Lake Formation, Glue Catalog, encryption, RBAC - Visualization: Exposure to BI tools like QuickSight, Tableau, or Power BI is a plus Additional Information: - The candidate should have minimum 5 years of experience in AWS Architecture. - This position is based at our Pune office. - A 15 years full time education is required.

Join our digital revolution in NatWest Digital X In everything we do, we work to one aim. To make digital experiences which are effortless and secure. So we organise ourselves around three principles: engineer, protect, and operate. We engineer simple solutions, we protect our customers, and we operate smarter. Our people work differently depending on their jobs and needs. From hybrid working to flexible hours, we have plenty of options that help our people to thrive. This role is based in India and as such all normal working days must be carried out in India. Job Description Join us as a Software Engineer, PySpark This is an opportunity for a driven Software Engineer to take on an exciting new career challenge Day-to-day, you'll be engineering and maintaining innovative, customer centric, high performance, secure and robust solutions It’s a chance to hone your existing technical skills and advance your career while building a wide network of stakeholders We're offering this role at associate level What you'll do In your new role, you’ll be working within a feature team to engineer software, scripts and tools, as well as liaising with other engineers, architects and business analysts across the platform. You’ll also be: Producing complex and critical software rapidly and of high quality which adds value to the business Working in permanent teams who are responsible for the full life cycle, from initial development, through enhancement and maintenance to replacement or decommissioning Collaborating to optimise our software engineering capability Designing, producing, testing and implementing our working software solutions Working across the life cycle, from requirements analysis and design, through coding to testing, deployment and operations The skills you'll need To take on this role, you’ll need a background in software engineering, software design, and architecture, and an understanding of how your area of expertise supports our customers. You'll need at least six years of experience with PySpark, AWS and SQL. You’ll also need: Experience of working with development and testing tools, bug tracking tools and wikis Experience in multiple programming languages or low code toolsets Experience of DevOps and Agile methodology and associated toolsets A background in solving highly complex, analytical and numerical problems Experience of implementing programming best practice, especially around scalability, automation, virtualisation, optimisation, availability and performance

We are looking for a skilled AWS Data Engineer to design, develop, and maintain scalable data pipelines and cloud-based data infrastructure on Amazon Web Services (AWS). The ideal candidate will work closely with data scientists, analysts, and software engineers to ensure high availability and performance of data solutions across the organization. Responsibilities Build/support applications using speech-to-text AWS services like Transcribe, Comprehend, along with Bedrock. Experience working with BI tools like QuickSight. Design, build, and manage scalable data pipelines using AWS services (e. g., Glue, Lambda, Step Functions, S3 EMR, Kinesis, Snowflake). Optimize data storage and retrieval for large-scale datasets in data lakes or data warehouses. Monitor, debug, and optimize the performance of data jobs and workflows. Ensure data quality, consistency, and security across environments. Collaborate with analytics, engineering, and business teams to understand data needs. Automate infrastructure deployment using IaC tools like CloudFormation or Terraform. Apply best practices for cloud cost optimization, data governance, and DevOps. Stay current with AWS services and recommend improvements to data architecture. Understanding machine learning pipelines and MLOps (nice to have). Requirements Bachelor's degree in computer science or a related field. 5+ years of experience as a Data Engineer, with at least 3 years focused on AWS. Strong experience with AWS services, including Transcribe, Bedrock, and QuickSight. Familiarity with Glue, S3 Snowflake, Lambda, Step Function, Kinesis, Athena, EC2/EMR, Power BI, or Tableau. Proficient in Python, PySpark, or Scala for data engineering tasks. Hands-on experience with SQL and data modeling. Familiarity with CI/CD pipelines and version control (e. g., Git, CodePipeline). Experience with orchestration tools (e. g., Airflow, Step Functions). Knowledge of data security, privacy, and compliance standards (GDPR, HIPAA, etc. ). Good To Have Skills AWS certifications (e. g., AWS Certified Data Analytics - Specialty, AWS Certified Solutions Architect). Experience with containerization (Docker, ECS, EKS). Experience working in Agile/Scrum environments. This job was posted by Shailendra Singh from PearlShell Softech.

As a Senior Data Engineer, you will architect, build, and maintain our data infrastructure that powers critical business decisions. You will work closely with data scientists, analysts, and product teams to design and implement scalable solutions for data processing, storage, and retrieval. Your work will directly impact our ability to leverage data for business intelligence, machine learning initiatives, and customer insights. Responsibilities Design, build, and maintain our end-to-end data infrastructure on AWS and GCP cloud platforms. Develop and optimize ETL/ELT pipelines to process large volumes of data from multiple sources. Build and support data pipelines for reporting, analytics, and machine learning applications. Implement and manage streaming data solutions using Kafka and other technologies. Design and optimize database schemas and data models in ClickHouse and other databases. Develop and maintain data workflows using Apache Airflow and similar orchestration tools. Write efficient, maintainable, and scalable code using PySpark and other data processing frameworks. Collaborate with data scientists to implement ML infrastructure for model training and deployment. Ensure data quality, reliability, and security across all data platforms. Monitor data pipelines and implement proactive alerting systems. Troubleshoot and resolve data infrastructure issues. Document data flows, architectures, and processes. Stay current with industry trends and emerging technologies in data engineering. Requirements Bachelor's degree in Computer Science, Engineering, or related technical field (Master's preferred). 5+ years of experience in data engineering roles. Strong expertise in AWS and/or GCP cloud platforms and services. Proficiency in building data pipelines using modern ETL/ELT tools and frameworks. Experience with stream processing technologies such as Kafka. Hands-on experience with ClickHouse or similar analytical databases. Strong programming skills in Python and experience with PySpark. Experience with workflow orchestration tools like Apache Airflow. Solid understanding of data modeling, data warehousing concepts, and dimensional modeling. Knowledge of SQL and NoSQL databases. Strong problem-solving skills and attention to detail. Excellent communication skills and ability to work in cross-functional teams. Experience in D2C, e-commerce, or retail industries. Knowledge of data visualization tools (Tableau, Looker, Power BI). Experience with real-time analytics solutions. Familiarity with CI/CD practices for data pipelines. Experience with containerization technologies (Docker, Kubernetes). Understanding of data governance and compliance requirements. Experience with MLOps or ML engineering Technologies. Cloud Platforms: AWS (S3 Redshift, EMR, Lambda), GCP (BigQuery, Dataflow, Dataproc). Data Processing: Apache Spark, PySpark, Python, SQL. Streaming: Apache Kafka, Kinesis. Data Storage: ClickHouse, S, 3 BigQuery, PostgreSQL, MongoDB. Orchestration: Apache Airflow. Version Control: Git. Containerization: Docker, Kubernetes (optional). This job was posted by Sidharth Patra from Traya Health.

We are looking for a highly skilled and hands-on Senior Data Engineer to join our growing data engineering practice in Mumbai. This role requires deep technical expertise in building and managing enterprise-grade data pipelines, with a primary focus on Amazon Redshift, AWS Glue, and data orchestration using Airflow or Step Functions. You will be responsible for building scalable, high-performance data workflows that ingest and process multi-terabyte-scale data across complex, concurrent environments. The ideal candidate is someone who thrives in solving performance bottlenecks, has led or participated in data warehouse migrations (e. g., Snowflake to Redshift), and is confident in interfacing with business stakeholders to translate requirements into robust data solutions. Responsibilities Design, develop, and maintain high-throughput ETL/ELT pipelines using AWS Glue (PySpark), orchestrated via Apache Airflow or AWS Step Functions. Own and optimize large-scale Amazon Redshift clusters and manage high concurrency workloads for a very large user base: Lead and contribute to migration projects from Snowflake or traditional RDBMS to Redshift, ensuring minimal downtime and robust validation. Integrate and normalize data from heterogeneous sources, including REST APIs, AWS Aurora (MySQL/Postgres), streaming inputs, and flat files. Implement intelligent caching strategies, leverage EC2 and serverless compute (Lambda, Glue) for custom transformations and processing at scale. Write advanced SQL for analytics, data reconciliation, and validation, demonstrating strong SQL development and tuning experience. Implement comprehensive monitoring, alerting, and logging for all data pipelines to ensure reliability, availability, and cost optimization. Collaborate directly with product managers, analysts, and client-facing teams to gather requirements and deliver insights-ready datasets. Champion data governance, security, and lineage, ensuring data is auditable and well-documented across all environments. Requirements 2-4 years of core data engineering experience, especially focused on Amazon Redshift hands-on performance tuning and large-scale management capacity. Demonstrated experience handling multi-terabyte Redshift clusters, concurrent query loads, and managing complex workload segmentation and queue priorities. Strong experience with AWS Glue (PySpark) for large-scale ETL jobs. Solid understanding and implementation experience of workflow orchestration using Apache Airflow or AWS Step Functions. Strong proficiency in Python, advanced SQL, and data modeling concepts. Familiarity with CI/CD pipelines, Git, DevOps processes, and infrastructure-as-code concepts. Experience with Amazon Athena, Lake Formation, or S3-based data lakes. Hands-on participation in Snowflake, BigQuery, or Teradata migration projects. AWS Certifications such as: AWS Certified Data Analytics - Specialty. AWS Certified Solutions Architect - Associate/Professional. Exposure to real-time streaming architectures or Lambda architectures. Soft Skills & Expectations Excellent communication skills enable able to confidently engage with both technical and non-technical stakeholders, including clients. Strong problem-solving mindset and a keen attention to performance, scalability, and reliability. Demonstrated ability to work independently, lead tasks, and take ownership of large-scale systems. Comfortable working in a fast-paced, dynamic, and client-facing environment. This job was posted by Rituza Rani from Oneture Technologies.

Primary Responsibilities: Be a team player in an agile team within a release team / value stream Develop and automate business solutions by creating new and modifying existing software applications Be technically hands-on and excellent in Design, Coding and Testing End to End, product quality Participate and contribute to Sprint Ceremonies Promote and develop the culture of collaboration, accountability, and quality Provide technical support to the team and help the team in resolving technical issues Closely work with Tech Lead, Onshore partners, deployment, and infrastructure teams Basic, structured, standard approach to work Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so Required Qualifications: Graduate degree or equivalent experience 3+ years of experience working in Data warehousing and Data Mart Platforms 3+ years of working experience in warehousing ecosystemDesign & Development, scheduling jobs using Airflow, running, and monitoring refreshes 3+ years of working experience in Big Data Technologies around Spark Or PySpark and Databricks 3+ years of working experience in Agile team 2+ years of working experience in cloud and Dev Ops technologies preferably on AzureDocker/ Kubernetes/Terraform/Chef Working experience in CI/CD pipeline (test, build, deployment and monitoring automation) Knowledge of software configuration management and packaging Demonstrates excellent problem-solving skills Preferred Qualification: 3+ years of working experience in ELT/ETL Design & Development and solid experience in SQL on Teradata and Snowflake At UnitedHealth Group, our mission is to help people live healthier lives and make the health system work better for everyone. We believe everyoneof every race, gender, sexuality, age, location and incomedeserves the opportunity to live their healthiest life. Today, however, there are still far too many barriers to good health which are disproportionately experienced by people of color, historically marginalized groups and those with lower incomes. We are committed to mitigating our impact on the environment and enabling and delivering equitable care that addresses health disparities and improves health outcomes an enterprise priority reflected in our mission. #Nic

Skilled and motivated Data Engineer to join our dynamic technology team. The ideal candidate will have a strong background in data processing, cloud computing, and software development , with hands-on experience in Python, PySpark, Java , and Microsoft Azure . You will be responsible for designing, building, and maintaining scalable data pipelines and infrastructure to support advanced analytics and data science initiatives. Key Responsibilities: Design, develop, and maintain robust data pipelines using PySpark , Python , and Java . Implement and manage data workflows on Microsoft Azure and other public cloud platforms . Collaborate with data scientists, analysts, and IT operations to ensure seamless data integration and availability. Optimize data systems for performance, scalability, and reliability. Ensure data quality, governance, and security across all data platforms. Support DevOps practices for continuous integration and deployment of data solutions. Monitor and troubleshoot data infrastructure and resolve system issues. Document processes and maintain data architecture standards. Required Skills & Qualifications: Bachelors or Masters degree in Computer Science, Information Technology, or related field. 3+ years of experience in data engineering , software development , or IT operations . Proficiency in Python , PySpark , and Java . Experience with cloud computing platforms, especially Microsoft Azure . Strong understanding of data management , data processing , and data analysis . Familiarity with multi-paradigm programming and modern software development practices. Knowledge of DevOps tools and methodologies. Experience with system administration and cloud providers . Excellent problem-solving and communication skills. Preferred Qualifications: Certifications in Azure , Python Data Science , or related technologies. Experience with public cloud environments like AWS or GCP. Familiarity with big data tools and frameworks. Exposure to data science workflows and tools. Skilled and motivated Data Engineer to join our dynamic technology team. The ideal candidate will have a strong background in data processing, cloud computing, and software development , with hands-on experience in Python, PySpark, Java , and Microsoft Azure . You will be responsible for designing, building, and maintaining scalable data pipelines and infrastructure to support advanced analytics and data science initiatives. Key Responsibilities: Design, develop, and maintain robust data pipelines using PySpark , Python , and Java . Implement and manage data workflows on Microsoft Azure and other public cloud platforms . Collaborate with data scientists, analysts, and IT operations to ensure seamless data integration and availability. Optimize data systems for performance, scalability, and reliability. Ensure data quality, governance, and security across all data platforms. Support DevOps practices for continuous integration and deployment of data solutions. Monitor and troubleshoot data infrastructure and resolve system issues. Document processes and maintain data architecture standards. Required Skills & Qualifications: Bachelors or Masters degree in Computer Science, Information Technology, or related field. 3+ years of experience in data engineering , software development , or IT operations . Proficiency in Python , PySpark , and Java . Experience with cloud computing platforms, especially Microsoft Azure . Strong understanding of data management , data processing , and data analysis . Familiarity with multi-paradigm programming and modern software development practices. Knowledge of DevOps tools and methodologies. Experience with system administration and cloud providers . Excellent problem-solving and communication skills. Preferred Qualifications: Certifications in Azure , Python Data Science , or related technologies. Experience with public cloud environments like AWS or GCP. Familiarity with big data tools and frameworks. Exposure to data science workflows and tools.

Key Responsibilities: Live Instruction: Teach core DS/ML topics (supervised & unsupervised learning, deep learning, model evaluation, MLOps) through interactive sessions. Curriculum Collaboration: Work with content teams to design labs, code walkthroughs, and real-world case studies using Python, scikit-learn, TensorFlow/PyTorch, and cloud-based DS/ML services. Learner Support: Field technical questions, debug code, review notebooks, and provide actionable feedback on assignments and projects. Project Mentorship: Guide capstone worke.g., image/video models, NLP pipelines, recommendation systems, and deployment pipelines. Continuous Improvement: Analyze learner performance data to refine modules, introduce emerging topics (e.g., transformers, generative models), and enhance assessments. Requirements: 3+ years of industry or academic experience in DS/ML or AI. Minimum Masters degree in DS or ML &AI or CS specialization in AI & DS/ML Proficiency in Python and ML frameworks (scikit-learn, TensorFlow or PyTorch). Familiarity with MLOps tools (Docker, Kubernetes, MLflow) and cloud ML services (AWS SageMaker, GCP AI Platform, or Azure ML). Excellent presentation and mentoring skills in live and small-group settings. Prior teaching or edtech experience is a strong plus.

About Loyalytics Loyalytics is a fast-growing Analytics consulting and product organization based out of Bangalore. We work with large retail clients across the globe helping them monetize their data assets through our consulting assignments and product accelerators. We are a young dynamic team of 100+ analytics practitioners working on some of the most cutting-edge tools and technologies. Who We Are Technical team: A team full of data scientists, data engineers and business analysts who work with 1M+ data points every day. Market Size: Massive multi-billion $ global market opportunity. Leadership: Combined experience of 40+ years of experience in the industry. Customers: Word-of-mouth and referral driven marketing to acquire customers like big retail brands in GCC regions like Lulu, GMG, among others (Strong product-market fit). What makes us stand apart: 8 years old bootstrapped and 100+ people company that is still hiring. About The Job We are looking for a Data Analyst who will be working on designing various campaign experiments, reports and Analysis on company data and delivering them to top management and other interested parties. As a Data Analyst, you will be working with business users, collecting their requests, and then generating insights and finding and present then using suitable tools (SQL Power BI, PPT, excel etc.) Responsibilities Using data to drive analytical projects across all complexity levels and to generate key business insights & recommendations to enable decision making Draft daily Revenue, Traffic & other input metric plan/forecast for different customer cohorts for BAU and event days. Monitoring these metrics & doing deep-dive analyses for problem-solving sessions will also be a key part of the role Being the “data backbone” of the various campaigns run by the team – this will involve designing the experiments, creating various target groups and measuring the impact of the various initiatives related to pricing and customer relationship management. The recruit will need to liaison with cross-functional teams during the execution of these campaigns Independently leading critical projects with extensive cross – functional collaboration will be a key area of opportunity and learning Data visualization for easy consumption of insights for org and leadership, defining actionable next steps basis insights Skills Required Strong analytical problem-solving skills with appreciation for Math/Stats and Business. 1-3 years of experience as Data/Business Analyst would be strongly preferred Proficiency in using pyspark/python is mandatory, sequel is a plus. Efficient project and stakeholder management skills. Should have demonstrated the ability to work independently in a highly demanding environment with strict deadlines Strong attention to detail Good communication and presentation skills both written and verbal Tools/Languages: - Must-have SQL – Beginner/Intermediate Level Python – Expert/ intermediate level PySpark – Expert/ intermediate level Power BI – Expert/Intermediate Databricks – Beginner/Intermediate JIRA – Beginner/Intermediateate Work Mode: Work from Office Location:Whitefield , Bangalore

Mandatory Skills : GCP Storage,GCP BigQuery,GCP DataProc,GCP Cloud Composer,GCP DMS,Apache airflow,Java,Python,Scala,GCP Datastream,Google Analytics Hub,GCP Workflows,GCP Dataform,GCP Datafusion,GCP Pub/Sub,ANSI-SQL,GCP Dataflow,GCP Data Flow,GCP Cloud Pub/Sub,Big Data Hadoop Ecosystem Good to Have Skills : Apache airflow, Python, Scala

Programming: Python, SQL Databases: Relational (PostgreSQL, MySQL), NoSQL (MongoDB, Cassandra), Vector Databases (FAISS) APIs and Integration: Protocols (REST, GraphQL), Frameworks (Django, FastAPI) Big Data Frameworks: Hadoop, Spark Nice to have Data Processing: Apache Flink, Apache Kafka Large Language Models (LLMs): Integration of domain knowledge (RAG), Prompt Engineering, Embeddings

Greetings from LTIMindtree!! About the job Are you looking for a new career challenge? With LTIMindtree, are you ready to embark on a data-driven career? Working for global leading manufacturing client for providing an engaging product experience through best-in-class PIM implementation and building rich, relevant, and trusted product information across channels and digital touchpoints so their end customers can make an informed purchase decision – will surely be a fulfilling experience. Location: Pan India. Key Skill : Spark +Python Interested candidates kindly apply in below link and share updated cv to Hemalatha1@ltimindtree.com https://forms.office.com/r/zQucNTxa2U Job Description Key Skill: Hadoop-Spark SparkSQL – Python Mandatory Skills: Relevant Experience in ETL and Data Engineering Strong Knowledge in Spark, Python Strong experience in Hive/SQL, PL/SQL Good Understanding of ETL & DW Concepts, Unix Scripting Design, implement and maintain Dat Pipeline to meet business requirements. Convert the Business need into Technical complex PySpark Code. Ability to write complex SQL queries for reporting purpose. Monitor Pyspark code performance and troubleshoot issues Why join us? Work in industry leading implementations for Tier-1 clients Accelerated career growth and global exposure Collaborative, inclusive work environment rooted in innovation Exposure to best-in-class automation framework Innovation first culture: We embrace automation, AI insights and clean data Know someone who fits this perfectly? Tag them – let’s connect the right talent with right opportunity DM or email to know more Let’s build something great together

About the Role We are seeking a skilled and driven Data Engineer to join our growing data team. Required Skills Experience with data platforms and tools: Snowflake, DBT, Databricks, Azure (ADF & Fabric), and GCP BigQuery . Strong understanding of database optimization techniques: partitioning, indexes, query performance tuning , etc. Proficient in coding with Python, PySpark , and SQL for data extraction, transformation, and analysis. Strong SQL expertise is a must Experience in automating and testing data workflows , preferably using Azure ADF . Familiarity with analytical techniques including data modeling, data transformation , and algorithmic development . Role & responsibilities End-to-End Pipeline Development - Design, build, and test complete data pipelines for ingestion, integration, and curation. Workflow Automation & Data Quality - Automate data workflows and testing to boost efficiency and ensure data integrity. Pipeline Optimization - Continuously enhance pipeline performance to meet evolving business demands. Scalable Cloud Data Platform - Build and maintain a scalable, high-availability cloud-based data platform. Technical Debt Management - Identify and reduce technical debt for a clean and efficient codebase. Reusable Frameworks - Develop modular components and frameworks to accelerate data engineering tasks. Stakeholder Collaboration - Partner with analysts and cross-functional teams to deliver data-driven solutions. Documentation & Maintenance - Document workflows and systems for clarity, consistency, and long-term support.

Designation - Module Leader Role - ETL Developer Location - Chennai Notice Period - Immediate to 30 days Experience range b/w 5 - 7 years of Development experience in the Amazon Cloud Environment AWS (S3, AWS Glue, Amazon Redshift, Data Lake Experience in SSRS Expereince in SSIS Experience in ETL Experience in Power Bi Experience in Aws glue Create ETL jobs using Python/PySpark to fulfill the requirements. Ability to perform data manipulations, load, extract from several sources of data into another schema. Good experience with project management practices, proficiency with Agile and Waterfall methodologies and working with scrum teams and timely reporting. Experience with software development Life cycle and all the phases. 7 plus years Database development experience. Understanding of core AWS services, and basic AWS architecture best practices. AWS Technologies S3, AWS Glue, RDS, lambda, cloud watch, etc. Troubleshoot and resolve issues related to data quality, performance, and reliability. Document ETL processes and workflows for future reference and be able to demo completed demo. Optimize and maintain existing ETL processes to ensure high performance and efficiency. Strong analytical and collaboration skills and a team player. Excellent problem-solving and troubleshooting skills. Self-starter and be able to learn and adopt quickly. Strong verbal and written communication skills with an ability to understand frontend users requirements. Note: Work timings 1pm - 11pm Interested Candidates can also share their updated resume at megha.chattopadhyay@aspiresys.com

About the Role In this opportunity, as Senior Data Engineer, you will: Develop and maintain data solutions using resources such as dbt, Alteryx, and Python. Design and optimize data pipelines, ensuring efficient data flow and processing. Work extensively with databases, SQL, and various data formats including JSON, XML, and CSV. Tune and optimize queries to enhance performance and reliability. Develop high-quality code in SQL, dbt, and Python, adhering to best practices. Understand and implement data automation and API integrations. Leverage AI capabilities to enhance data engineering practices. Understand integration points related to upstream and downstream requirements. Proactively manage tasks and work towards completion against tight deadlines. Analyze existing processes and offer suggestions for improvement. About You Youre a fit for the role of Senior Data Engineer if your background includes: Strong interest and knowledge in data engineering principles and methods. 6+ years of experience developing data solutions or pipelines. 6+ years of hands-on experience with databases and SQL. 2+ years of experience programming in an additional language. 2+ years of experience in query tuning and optimization. Experience working with SQL, JSON, XML, and CSV content. Understanding of data automation and API integration. Familiarity with AI capabilities and their application in data engineering. Ability to adhere to best practices for developing programmatic solutions. Strong problem-solving skills and ability to work independently. #LI-SS6 Whats in it For You Hybrid Work Model Weve adopted a flexible hybrid working environment (2-3 days a week in the office depending on the role) for our office-based roles while delivering a seamless experience that is digitally and physically connected. Flexibility & Work-Life Balance: Flex My Way is a set of supportive workplace policies designed to help manage personal and professional responsibilities, whether caring for family, giving back to the community, or finding time to refresh and reset. This builds upon our flexible work arrangements, including work from anywhere for up to 8 weeks per year, empowering employees to achieve a better work-life balance. Career Development and Growth: By fostering a culture of continuous learning and skill development, we prepare our talent to tackle tomorrows challenges and deliver real-world solutions. Our Grow My Way programming and skills-first approach ensures you have the tools and knowledge to grow, lead, and thrive in an AI-enabled future. Industry Competitive Benefits We offer comprehensive benefit plans to include flexible vacation, two company-wide Mental Health Days off, access to the Headspace app, retirement savings, tuition reimbursement, employee incentive programs, and resources for mental, physical, and financial wellbeing. Culture: Globally recognized, award-winning reputation for inclusion and belonging, flexibility, work-life balance, and more. We live by our valuesObsess over our Customers, Compete to Win, Challenge (Y)our Thinking, Act Fast / Learn Fast, and Stronger Together. Social Impact Make an impact in your community with our Social Impact Institute. We offer employees two paid volunteer days off annually and opportunities to get involved with pro-bono consulting projects and Environmental, Social, and Governance (ESG) initiatives. Making a Real-World Impact: We are one of the few companies globally that helps its customers pursue justice, truth, and transparency. Together, with the professionals and institutions we serve, we help uphold the rule of law, turn the wheels of commerce, catch bad actors, report the facts, and provide trusted, unbiased information to people all over the world. Thomson Reuters informs the way forward by bringing together the trusted content and technology that people and organizations need to make the right decisions. We serve professionals across legal, tax, accounting, compliance, government, and media. Our products combine highly specialized software and insights to empower professionals with the data, intelligence, and solutions needed to make informed decisions, and to help institutions in their pursuit of justice, truth, and transparency. Reuters, part of Thomson Reuters, is a world leading provider of trusted journalism and news. We are powered by the talents of 26,000 employees across more than 70 countries, where everyone has a chance to contribute and grow professionally in flexible work environments. At a time when objectivity, accuracy, fairness, and transparency are under attack, we consider it our duty to pursue them. Sound excitingJoin us and help shape the industries that move society forward. As a global business, we rely on the unique backgrounds, perspectives, and experiences of all employees to deliver on our business goals. To ensure we can do that, we seek talented, qualified employees in all our operations around the world regardless of race, color, sex/gender, including pregnancy, gender identity and expression, national origin, religion, sexual orientation, disability, age, marital status, citizen status, veteran status, or any other protected classification under applicable law. Thomson Reuters is proud to be an Equal Employment Opportunity Employer providing a drug-free workplace. We also make reasonable accommodations for qualified individuals with disabilities and for sincerely held religious beliefs in accordance with applicable law. More information on requesting an accommodation here. Learn more on how to protect yourself from fraudulent job postings here. More information about Thomson Reuters can be found on thomsonreuters.com.

: Develop/enhance data warehousing functionality including the use and management of Snowflake data warehouse and the surrounding entitlements, pipelines and monitoring, in partnership with Data Analysts and Architects with guidance from lead Data Engineer. About the Role In this opportunity as Data Engineer, you will: Develop/enhance data warehousing functionality including the use and management of Snowflake data warehouse and the surrounding entitlements, pipelines and monitoring, in partnership with Data Analysts and Architects with guidance from lead Data Engineer Innovate with new approaches to meeting data management requirements Effectively communicate and liaise with other data management teams embedded across the organization and data consumers in data science and business analytics teams. Analyze existing data pipelines and assist in enhancing and re-engineering the pipelines as per business requirements. Bachelors degree or equivalent required, Computer Science or related technical degree preferred About You Youre a fit for the role if your background includes: Mandatory skills Data Warehousing, data models, data processing[ Good to have], SQL, Power BI / Tableau, Snowflake [good to have] , Python 3.5 + years of relevant experience in Implementation of data warehouse and data management of data technologies for large scale organizations Experience in building and maintaining optimized and highly available data pipelines that facilitate deeper analysis and reporting Worked on Analyzing data pipelines Knowledgeable about Data Warehousing, including data models and data processing Broad understanding of the technologies used to build and operate data and analytic systems Excellent critical thinking, communication, presentation, documentation, troubleshooting and collaborative problem-solving skills Beginner to intermediate Knowledge of AWS, Snowflake, Python Hands-on experience with programming and scripting languages Knowledge of and hands on experience with Data Vault 2.0 is a plus Also have experience in and comfort with some of the following skills/concepts: Good in writing SQL and performance tuning Data Integration tools lie DBT, Informatica, etc. Intermediate in programming language like Python/PySpark/Java/JavaScript AWS services and management, including Serverless, Container, Queueing and Monitoring services Consuming and building APIs. #LI-SM1 Whats in it For You Hybrid Work Model Weve adopted a flexible hybrid working environment (2-3 days a week in the office depending on the role) for our office-based roles while delivering a seamless experience that is digitally and physically connected. Flexibility & Work-Life Balance: Flex My Way is a set of supportive workplace policies designed to help manage personal and professional responsibilities, whether caring for family, giving back to the community, or finding time to refresh and reset. This builds upon our flexible work arrangements, including work from anywhere for up to 8 weeks per year, empowering employees to achieve a better work-life balance. Career Development and Growth: By fostering a culture of continuous learning and skill development, we prepare our talent to tackle tomorrows challenges and deliver real-world solutions. Our Grow My Way programming and skills-first approach ensures you have the tools and knowledge to grow, lead, and thrive in an AI-enabled future. Industry Competitive Benefits We offer comprehensive benefit plans to include flexible vacation, two company-wide Mental Health Days off, access to the Headspace app, retirement savings, tuition reimbursement, employee incentive programs, and resources for mental, physical, and financial wellbeing. Culture: Globally recognized, award-winning reputation for inclusion and belonging, flexibility, work-life balance, and more. We live by our valuesObsess over our Customers, Compete to Win, Challenge (Y)our Thinking, Act Fast / Learn Fast, and Stronger Together. Social Impact Make an impact in your community with our Social Impact Institute. We offer employees two paid volunteer days off annually and opportunities to get involved with pro-bono consulting projects and Environmental, Social, and Governance (ESG) initiatives. Making a Real-World Impact: We are one of the few companies globally that helps its customers pursue justice, truth, and transparency. Together, with the professionals and institutions we serve, we help uphold the rule of law, turn the wheels of commerce, catch bad actors, report the facts, and provide trusted, unbiased information to people all over the world. Thomson Reuters informs the way forward by bringing together the trusted content and technology that people and organizations need to make the right decisions. We serve professionals across legal, tax, accounting, compliance, government, and media. Our products combine highly specialized software and insights to empower professionals with the data, intelligence, and solutions needed to make informed decisions, and to help institutions in their pursuit of justice, truth, and transparency. Reuters, part of Thomson Reuters, is a world leading provider of trusted journalism and news. We are powered by the talents of 26,000 employees across more than 70 countries, where everyone has a chance to contribute and grow professionally in flexible work environments. At a time when objectivity, accuracy, fairness, and transparency are under attack, we consider it our duty to pursue them. Sound excitingJoin us and help shape the industries that move society forward. As a global business, we rely on the unique backgrounds, perspectives, and experiences of all employees to deliver on our business goals. To ensure we can do that, we seek talented, qualified employees in all our operations around the world regardless of race, color, sex/gender, including pregnancy, gender identity and expression, national origin, religion, sexual orientation, disability, age, marital status, citizen status, veteran status, or any other protected classification under applicable law. Thomson Reuters is proud to be an Equal Employment Opportunity Employer providing a drug-free workplace. We also make reasonable accommodations for qualified individuals with disabilities and for sincerely held religious beliefs in accordance with applicable law. More information on requesting an accommodation here. Learn more on how to protect yourself from fraudulent job postings here. More information about Thomson Reuters can be found on thomsonreuters.com.

We are looking for a self-motivated individual with appetite to learn new skills and be part of a fast-paced team that is delivering cutting edge solutions that drive new products and features that are critical for our customers. Our senior software engineers are responsible for designing, developing and ensuring the quality, reliability and availability of key systems that provide critical data and algorithms.Responsibilities of this role will include developing new and enhancing existing applications and you will work collaboratively with technical leads and architect to design, develop and test these critical applications. About the role Actively participate in the full life cycle of software delivery, including analysis, design, implementation and testing of new projects and features using Hadoop, Spark/Pyspark, Scala or Java, Hive, SQL, and other open-source tools and design patterns. Python knowledge is a bonus for this role. Working experience with HUDI , Snowflake or similar Must have technologies like Big Data, AWS services like EMR, S3, Lambdas, Elastic, step functions. Actively participate in the development and testing of features for assigned projects with little to no guidance. The position holds opportunities to work under technical experts and also to provide guidance and assistance to less experienced team members or new joiners in the path of the project. Appetite for learning will be key attribute for doing well in the role as the Org is very dynamic and have tremendous scope into various technical landscapes. We consider AI inclusion as a key to excel in this role, we want dynamic candidates who use AI tools as build partners and share experiences to ignite the Org. Proactively share knowledge and best practices on using new and emerging technologies across all of the development and testing groups Create, review and maintain technical documentation of software development and testing artifacts Work collaboratively with others in a team-based environment. Identify and participate in the resolution of issues with the appropriate technical and business resources Generate innovative approaches and solutions to technology challenges Effectively balance and prioritize multiple projects concurrently. About you Bachelors or Masters degree in computer science or a related field 7+ year experience in IT industry Product and Platform development preferred. Strong programming skill with Java or Scala. Must have technologies includes Big Data, AWS. Exposure to services like EMR, S3, Lambdas, Elastic, step functions. Knowledge of Python will be preferred. Experience with Agile methodology, continuous integration and/or Test-Driven Development. Self-motivated with a strong desire for continual learning Take personal responsibility to impact results and deliver on commitments. Effective verbal and written communication skills. Ability to work independently or as part of an agile development team. #LI-SP1 Whats in it For You Hybrid Work Model Weve adopted a flexible hybrid working environment (2-3 days a week in the office depending on the role) for our office-based roles while delivering a seamless experience that is digitally and physically connected. Flexibility & Work-Life Balance: Flex My Way is a set of supportive workplace policies designed to help manage personal and professional responsibilities, whether caring for family, giving back to the community, or finding time to refresh and reset. This builds upon our flexible work arrangements, including work from anywhere for up to 8 weeks per year, empowering employees to achieve a better work-life balance. Career Development and Growth: By fostering a culture of continuous learning and skill development, we prepare our talent to tackle tomorrows challenges and deliver real-world solutions. Our Grow My Way programming and skills-first approach ensures you have the tools and knowledge to grow, lead, and thrive in an AI-enabled future. Industry Competitive Benefits We offer comprehensive benefit plans to include flexible vacation, two company-wide Mental Health Days off, access to the Headspace app, retirement savings, tuition reimbursement, employee incentive programs, and resources for mental, physical, and financial wellbeing. Culture: Globally recognized, award-winning reputation for inclusion and belonging, flexibility, work-life balance, and more. We live by our valuesObsess over our Customers, Compete to Win, Challenge (Y)our Thinking, Act Fast / Learn Fast, and Stronger Together. Social Impact Make an impact in your community with our Social Impact Institute. We offer employees two paid volunteer days off annually and opportunities to get involved with pro-bono consulting projects and Environmental, Social, and Governance (ESG) initiatives. Making a Real-World Impact: We are one of the few companies globally that helps its customers pursue justice, truth, and transparency. Together, with the professionals and institutions we serve, we help uphold the rule of law, turn the wheels of commerce, catch bad actors, report the facts, and provide trusted, unbiased information to people all over the world. Thomson Reuters informs the way forward by bringing together the trusted content and technology that people and organizations need to make the right decisions. We serve professionals across legal, tax, accounting, compliance, government, and media. Our products combine highly specialized software and insights to empower professionals with the data, intelligence, and solutions needed to make informed decisions, and to help institutions in their pursuit of justice, truth, and transparency. Reuters, part of Thomson Reuters, is a world leading provider of trusted journalism and news. We are powered by the talents of 26,000 employees across more than 70 countries, where everyone has a chance to contribute and grow professionally in flexible work environments. At a time when objectivity, accuracy, fairness, and transparency are under attack, we consider it our duty to pursue them. Sound excitingJoin us and help shape the industries that move society forward. As a global business, we rely on the unique backgrounds, perspectives, and experiences of all employees to deliver on our business goals. To ensure we can do that, we seek talented, qualified employees in all our operations around the world regardless of race, color, sex/gender, including pregnancy, gender identity and expression, national origin, religion, sexual orientation, disability, age, marital status, citizen status, veteran status, or any other protected classification under applicable law. Thomson Reuters is proud to be an Equal Employment Opportunity Employer providing a drug-free workplace. We also make reasonable accommodations for qualified individuals with disabilities and for sincerely held religious beliefs in accordance with applicable law. More information on requesting an accommodation here. Learn more on how to protect yourself from fraudulent job postings here. More information about Thomson Reuters can be found on thomsonreuters.com.

This role involves the development and application of engineering practice and knowledge in the following technologiesStandards and protocols, application software and embedded software for wireless and satellite networks, fixed networks and enterprise networks; connected devices (IOT and device engineering), connected applications (5G/ edge, B2X apps); and Telco Cloud, Automation and Edge Compute platforms. This role also involves the integration of network systems and their operations, related to the above technologies. - Grade Specific Focus on Connectivity and Network Engineering. Develops competency in own area of expertise. Shares expertise and provides guidance and support to others. Interprets clients needs. Completes own role independently or with minimum supervision. Identifies problems and relevant issues in straight forward situations and generates solutions. Contributes in teamwork and interacts with customers.

Job Title:- Sr. Python Data Engineer Location:- Chennai & Bangalore (REMOTE) Job Type:- Permanent Employee Experience :- 8 to 12 Years Shift: 2 11 PM Responsibilities Design and develop data pipelines and ETL processes. Collaborate with data scientists and analysts to understand data needs. Maintain and optimize data warehousing solutions. Ensure data quality and integrity throughout the data lifecycle. Develop and implement data validation and cleansing routines. Work with large datasets from various sources. Automate repetitive data tasks and processes. Monitor data systems and troubleshoot issues as they arise. Qualifications Bachelor’s degree in Computer Science, Information Technology, or a related field. Proven experience as a Data Engineer or similar role (Minimum 6+ years’ experience as Data Engineer). Strong proficiency in Python and PySpark. Excellent problem-solving abilities. Strong communication skills to collaborate with team members and stakeholders. Individual Contributor Technical Skills Required Expert Python, PySpark and SQL/Snowflake Advanced Data warehousing, Data pipeline design – Advanced Level Data Quality, Data validation, Data cleansing – Advanced Level Intermediate/Basic Microsoft Fabric, ADF, Databricks, Master Data management/Data Governance Data Mesh, Data Lake/Lakehouse Architecture

We’re hiring Databricks Developers skilled in PySpark & SQL for cloud-based projects. Multiple positions are open based on experience level. Email: Anita.s@liveconnections.in *JOB AT HYDERABAD, MUMBAI, PUNE* Required Candidate profile Exciting walk-in drive on Aug 2 across Mumbai, Pune & Hyderabad. Shape the future with data 7–12 yrs total exp with 3–5 yrs in Databricks (Azure/AWS). Must know PySpark & SQL.

Ready to shape the future of work? At Genpact, we dont just adapt to changewe drive it. AI and digital innovation are redefining industries, and were leading the charge. Genpacts AI Gigafactory , our industry-first accelerator, is an example of how were scaling advanced technology solutions to help global enterprises work smarter, grow faster, and transform at scale. From large-scale models to agentic AI , our breakthrough solutions tackle companies most complex challenges. If you thrive in a fast-moving, tech-driven environment, love solving real-world problems, and want to be part of a team thats shaping the future, this is your moment. Genpact (NYSE: G) is an advanced technology services and solutions company that delivers lasting value for leading enterprises globally. Through our deep business knowledge, operational excellence, and cutting-edge solutions we help companies across industries get ahead and stay ahead. Powered by curiosity, courage, and innovation , our teams implement data, technology, and AI to create tomorrow, today. Get to know us at genpact.com and on LinkedIn , X , YouTube , and Facebook . Inviting applications for the role of Senior Principal Consultant, AWS DataLake! Responsibilities Having knowledge on DataLake on AWS services with exposure to creating External Tables and spark programming. The person shall be able to work on python programming. Writing effective and scalable Python codes for automations, data wrangling and ETL. ¢ Designing and implementing robust applications and work on Automations using python codes. ¢ Debugging applications to ensure low-latency and high-availability. ¢ Writing optimized custom SQL queries ¢ Experienced in team and client handling ¢ Having prowess in documentation related to systems, design, and delivery. ¢ Integrate user-facing elements into applications ¢ Having the knowledge of External Tables, Data Lake concepts. ¢ Able to do task allocation, collaborate on status exchanges and getting things to successful closure. ¢ Implement security and data protection solutions ¢ Must be capable of writing SQL queries for validating dashboard outputs ¢ Must be able to translate visual requirements into detailed technical specifications ¢ Well versed in handling Excel, CSV, text, json other unstructured file formats using python. ¢ Expertise in at least one popular Python framework (like Django, Flask or Pyramid) ¢ Good understanding and exposure on any Git, Bamboo, Confluence and Jira. ¢ Good in Dataframes and SQL ANSI using pandas. ¢ Team player, collaborative approach and excellent communication skills Qualifications we seek in you! Minimum Qualifications ¢BE/B Tech/ MCA ¢Excellent written and verbal communication skills ¢Good knowledge of Python, Pyspark Preferred Qualifications/ Skills ¢ Strong ETL knowledge on any ETL tool good to have. ¢ Good to have knowledge on AWS cloud and Snowflake. ¢ Having knowledge of PySpark is a plus. Why join Genpact? Be a transformation leader Work at the cutting edge of AI, automation, and digital innovation Make an impact Drive change for global enterprises and solve business challenges that matter Accelerate your career Get hands-on experience, mentorship, and continuous learning opportunities Work with the best Join 140,000+ bold thinkers and problem-solvers who push boundaries every day Thrive in a values-driven culture Our courage, curiosity, and incisiveness - built on a foundation of integrity and inclusion - allow your ideas to fuel progress Come join the tech shapers and growth makers at Genpact and take your career in the only direction that matters: Up. Lets build tomorrow together. Genpact is an Equal Opportunity Employer and considers applicants for all positions without regard to race, color, religion or belief, sex, age, national origin, citizenship status, marital status, military/veteran status, genetic information, sexual orientation, gender identity, physical or mental disability or any other characteristic protected by applicable laws. Genpact is committed to creating a dynamic work environment that values respect and integrity, customer focus, and innovation. Furthermore, please do note that Genpact does not charge fees to process job applications and applicants are not required to pay to participate in our hiring process in any other way. Examples of such scams include purchasing a 'starter kit,' paying to apply, or purchasing equipment or training.

Job Summary We are looking for a highly skilled Big Data & ETL Tester to join our data engineering and analytics team. The ideal candidate will have strong experience in PySpark, SQL, and Python, with a deep understanding of ETL pipelines, data validation, and cloud-based testing on AWS. Familiarity with data visualization tools like Apache Superset or Power BI is a strong plus You will work closely with our data engineering team to ensure data availability, consistency, and quality across complex data pipelines, and help transform business requirements into robust data testing frameworks. Key Responsibilities • Collaborate with big data engineers to validate data pipelines and ensure data integrity across ingestion, processing, and transformation stages. • Write complex PySpark and SQL queries to test and validate large-scale datasets. • Perform ETL testing, covering schema validation, data completeness, accuracy, transformation logic, and performance testing. • Conduct root cause analysis of data issues using structured debugging approaches. • Build automated test scripts in Python for regression, smoke, and end-to-end data testing. • Analyze large datasets to track KPIs and performance metrics supporting business operations and strategic decisions. • Work with data analysts and business teams to translate business needs into testable data validation frameworks. • Communicate testing results, insights, and data gaps via reports or dashboards (Superset/Power BI preferred). • Identify and document areas of improvement in data processes and advocate for automation opportunities. • Maintain detailed documentation of test plans, test cases, results, and associated dashboards. Required Skills and Qualifications 2+ years of experience in big data testing and ETL testing. • Strong hands-on skills in PySpark, SQL, and Python. • Solid experience working with cloud platforms, especially AWS (S3, EMR, Glue, Lambda, Athena, etc.). • Familiarity with data warehouse and lakehouse architectures. • Working knowledge of Apache Superset, Power BI, or similar visualization tools. • Ability to analyze large, complex datasets and provide actionable insights. • Strong understanding of data modeling concepts, data governance, and quality frameworks. • Experience with automation frameworks and CI/CD for data validation is a plus Preferred Qualifications • Experience with Airflow, dbt, or other data orchestration tools. • Familiarity with data cataloging tools (e.g., AWS Glue Data Catalog). • Prior experience in a product or SaaS-based company with high data volume environments. Why Join Us? • Opportunity to work with cutting-edge data stack in a fast-paced environment. • Collaborate with passionate data professionals driving real business impact. • Flexible work environment with a focus on learning and innovation