Job Job Title Pyspark Developer Responsibilities A day in the life of an Infoscion As part of the Infosys delivery team, your primary role would be to interface with the client for quality assurance, issue resolution and ensuring high customer satisfaction. You will understand requirements, create and review designs, validate the architecture and ensure high levels of service offerings to clients in the technology domain. You will participate in project estimation, provide inputs for solution delivery, conduct technical risk planning, perform code reviews and unit test plan reviews. You will lead and guide your teams towards developing optimized high quality code deliverables, continual knowledge management and adherence to the organizational guidelines and processes. You would be a key contributor to building efficient programs/ systems and if you think you fit right in to help our clients navigate their next in their digital transformation journey, this is the place for you!If you think you fit right in to help our clients navigate their next in their digital transformation journey, this is the place for you! Technical and Professional Requirements: Primary skills:Technology->Analytics - Packages->Python - Big Data,Technology->Big Data - Data Processing->Spark Preferred Skills: Technology->Analytics - Packages->Python - Big Data Technology->Big Data - Data Processing->Spark Additional Responsibilities: Knowledge of more than one technology Basics of Architecture and Design fundamentals Knowledge of Testing tools Knowledge of agile methodologies Understanding of Project life cycle activities on development and maintenance projects Understanding of one or more Estimation methodologies, Knowledge of Quality processes Basics of business domain to understand the business requirements Analytical abilities, Strong Technical Skills, Good communication skills Good understanding of the technology and domain Ability to demonstrate a sound understanding of software quality assurance principles, SOLID design principles and modelling methods Awareness of latest technologies and trends Excellent problem solving, analytical and debugging skills Educational Requirements Bachelor of Engineering Service Line Data & Analytics Unit * Location of posting is subject to business requirements

Job Job Title DNA_Danske_Bigdata Responsibilities SQL , Pyspark, CICD on AWS, Data engineering, Airflow on PremGood to have skill: API development(data ingestion using APIs (REST/gRPC)) , Airflow on AWSKafka, Scala Preferred Skills: Technology->Big Data->Big Data - ALL Educational Requirements Bachelor of Engineering Service Line Data & Analytics Unit * Location of posting is subject to business requirements

B2 Band 4yrs to 6yrs B3 Band 7yrs to 10yrs Notice Period Immediate to 30 days Location - Chennai/Pune/GNDC. JD : Hadoop Developer Responsibilities: Develop and maintain Hadoop applications: Write efficient and scalable code for data ingestion, processing, and analysis using Hadoop ecosystem tools (HDFS, Hive, HBase) and PySpark. Data pipeline development: Design and implement end-to-end data pipelines for batch and real-time processing. Data transformation: Utilize PySpark to transform and aggregate data from various sources. Performance optimization: Continuously monitor and optimize Hadoop jobs to ensure efficient resource utilization and timely processing. Collaboration: Work closely with business analysts and Data analysts to translate business requirements into technical solutions. Testing and deployment: Provide testing support during SIT/UAT phase and assist in deploying solutions to production environments. Required Skills and Experience: Hadoop ecosystem: Proficiency in Hadoop core components and related tools (Hive, HBase, Sqoop). PySpark expertise: Strong PySpark skills with experience in developing data processing pipelines. Python programming: Excellent Python programming skills with a focus on data manipulation and analysis libraries (Pandas, NumPy). SQL proficiency: Ability to write efficient SQL queries for data extraction and analysis within Hive or other SQL-like interfaces. Problem-solving: Strong analytical and problem-solving skills to tackle data-related challenges. Communication: Effective communication skills to collaborate with diverse stakeholders. Other: Retail Banking domain experience and Data Stage knowledge will be good to have. Hadoop Lead Developer Responsibilities: Technical leadership: Provide technical guidance and mentorship to a team of Hadoop developers. Architecture design: Design and implement scalable and reliable big data architectures using Hadoop and PySpark. Code review and optimization: Review code for quality and performance, and lead efforts to optimize Data pipelines. Project management: Plan and manage the development and deployment of data processing projects. Stakeholder collaboration: Collaborate with business stakeholders to understand requirements and translate them into technical solutions. Innovation: Proactively recommend improvements to the system. Required Skills and Experience: Proven leadership: Experience leading or mentoring technical teams. Hadoop/PySpark expertise: Advanced proficiency in Hadoop and PySpark, including experience with complex data processing pipelines. Architecture design: Experience designing and implementing scalable big data architectures. Performance optimization: Expertise in optimizing Hadoop/PySpark jobs for efficiency and performance. Communication: Excellent communication and interpersonal skills to effectively collaborate with stakeholders across departments. Other: Retail Banking domain experience and Data Stage knowledge will be good to have. Hadoop, Design, Technical Lead, Pyspark, Implementation

About Persistent We are a trusted Digital Engineering and Enterprise Modernization partner, combining deep technical expertise and industry experience to help our clients anticipate what’s next. Our offerings and proven solutions create a unique competitive advantage for our clients by giving them the power to see beyond and rise above. We work with many industry-leading organizations across the world including 12 of the 30 most innovative US companies, 80% of the largest banks in the US and India, and numerous innovators across the healthcare ecosystem. Our growth trajectory continues, as we reported $1,231M annual revenue (16% Y-o-Y). Along with our growth, we’ve onboarded over 4900 new employees in the past year, bringing our total employee count to over 23,500+ people located in 19 countries across the globe. Persistent Ltd. is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds. For more details please login to www.persistent.com About The Position We are looking for a Big Data Lead who will be responsible for the management of data sets that are too big for traditional database systems to handle. You will create, design, and implement data processing jobs in order to transform the data into a more usable format. You will also ensure that the data is secure and complies with industry standards to protect the company?s information. What You?ll Do Manage customer's priorities of projects and requests Assess customer needs utilizing a structured requirements process (gathering, analyzing, documenting, and managing changes) to prioritize immediate business needs and advising on options, risks and cost Design and implement software products (Big Data related) including data models and visualizations Demonstrate participation with the teams you work in Deliver good solutions against tight timescales Be pro-active, suggest new approaches and develop your capabilities Share what you are good at while learning from others to improve the team overall Show that you have a certain level of understanding for a number of technical skills, attitudes and behaviors Deliver great solutions Be focused on driving value back into the business Expertise You?ll Bring 6 years' experience in designing & developing enterprise application solution for distributed systems Understanding of Big Data Hadoop Ecosystem components (Sqoop, Hive, Pig, Flume) Additional experience working with Hadoop, HDFS, cluster management Hive, Pig and MapReduce, and Hadoop ecosystem framework HBase, Talend, NoSQL databases Apache Spark or other streaming Big Data processing, preferred Java or Big Data technologies, will be a plus Benefits Competitive salary and benefits package Culture focused on talent development with quarterly promotion cycles and company-sponsored higher education and certifications Opportunity to work with cutting-edge technologies Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards Annual health check-ups Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents Inclusive Environment •We offer hybrid work options and flexible working hours to accommodate various needs and preferences. •Our office is equipped with accessible facilities, including adjustable workstations, ergonomic chairs, and assistive technologies to support employees with physical disabilities Let's unleash your full potential. See Beyond, Rise Above

About Persistent We are a trusted Digital Engineering and Enterprise Modernization partner, combining deep technical expertise and industry experience to help our clients anticipate what’s next. Our offerings and proven solutions create a unique competitive advantage for our clients by giving them the power to see beyond and rise above. We work with many industry-leading organizations across the world including 12 of the 30 most innovative US companies, 80% of the largest banks in the US and India, and numerous innovators across the healthcare ecosystem. Our growth trajectory continues, as we reported $1,231M annual revenue (16% Y-o-Y). Along with our growth, we’ve onboarded over 4900 new employees in the past year, bringing our total employee count to over 23,500+ people located in 19 countries across the globe. Persistent Ltd. is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds. For more details please login to www.persistent.com About The Position We are looking for a Big Data Lead who will be responsible for the management of data sets that are too big for traditional database systems to handle. You will create, design, and implement data processing jobs in order to transform the data into a more usable format. You will also ensure that the data is secure and complies with industry standards to protect the company?s information. What You?ll Do Manage customer's priorities of projects and requests Assess customer needs utilizing a structured requirements process (gathering, analyzing, documenting, and managing changes) to prioritize immediate business needs and advising on options, risks and cost Design and implement software products (Big Data related) including data models and visualizations Demonstrate participation with the teams you work in Deliver good solutions against tight timescales Be pro-active, suggest new approaches and develop your capabilities Share what you are good at while learning from others to improve the team overall Show that you have a certain level of understanding for a number of technical skills, attitudes and behaviors Deliver great solutions Be focused on driving value back into the business Expertise You?ll Bring 6 years' experience in designing & developing enterprise application solution for distributed systems Understanding of Big Data Hadoop Ecosystem components (Sqoop, Hive, Pig, Flume) Additional experience working with Hadoop, HDFS, cluster management Hive, Pig and MapReduce, and Hadoop ecosystem framework HBase, Talend, NoSQL databases Apache Spark or other streaming Big Data processing, preferred Java or Big Data technologies, will be a plus Benefits Competitive salary and benefits package Culture focused on talent development with quarterly promotion cycles and company-sponsored higher education and certifications Opportunity to work with cutting-edge technologies Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards Annual health check-ups Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents Inclusive Environment •We offer hybrid work options and flexible working hours to accommodate various needs and preferences. •Our office is equipped with accessible facilities, including adjustable workstations, ergonomic chairs, and assistive technologies to support employees with physical disabilities. Let's unleash your full potential. See Beyond, Rise Above

About Persistent We are a trusted Digital Engineering and Enterprise Modernization partner, combining deep technical expertise and industry experience to help our clients anticipate what’s next. Our offerings and proven solutions create a unique competitive advantage for our clients by giving them the power to see beyond and rise above. We work with many industry-leading organizations across the world including 12 of the 30 most innovative US companies, 80% of the largest banks in the US and India, and numerous innovators across the healthcare ecosystem. Our growth trajectory continues, as we reported $1,231M annual revenue (16% Y-o-Y). Along with our growth, we’ve onboarded over 4900 new employees in the past year, bringing our total employee count to over 23,500+ people located in 19 countries across the globe. Persistent Ltd. is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds. For more details please login to www.persistent.com About The Position We are looking for a Big Data Lead who will be responsible for the management of data sets that are too big for traditional database systems to handle. You will create, design, and implement data processing jobs in order to transform the data into a more usable format. You will also ensure that the data is secure and complies with industry standards to protect the company?s information. What You?ll Do Manage customer's priorities of projects and requests Assess customer needs utilizing a structured requirements process (gathering, analyzing, documenting, and managing changes) to prioritize immediate business needs and advising on options, risks and cost Design and implement software products (Big Data related) including data models and visualizations Demonstrate participation with the teams you work in Deliver good solutions against tight timescales Be pro-active, suggest new approaches and develop your capabilities Share what you are good at while learning from others to improve the team overall Show that you have a certain level of understanding for a number of technical skills, attitudes and behaviors Deliver great solutions Be focused on driving value back into the business Expertise You?ll Bring 6 years' experience in designing & developing enterprise application solution for distributed systems Understanding of Big Data Hadoop Ecosystem components (Sqoop, Hive, Pig, Flume) Additional experience working with Hadoop, HDFS, cluster management Hive, Pig and MapReduce, and Hadoop ecosystem framework HBase, Talend, NoSQL databases Apache Spark or other streaming Big Data processing, preferred Java or Big Data technologies, will be a plus Benefits Competitive salary and benefits package Culture focused on talent development with quarterly promotion cycles and company-sponsored higher education and certifications Opportunity to work with cutting-edge technologies Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards Annual health check-ups Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents Inclusive Environment •We offer hybrid work options and flexible working hours to accommodate various needs and preferences. •Our office is equipped with accessible facilities, including adjustable workstations, ergonomic chairs, and assistive technologies to support employees with physical disabilities. Let's unleash your full potential. See Beyond, Rise Above

Description: Keyskills Must Have Platform/Framework Hadoop Spark Kafka Requirements: Skill Set Hadoop, Spark, ClickHouse, Kafka Job Responsibilities: Skill Set Hadoop, Spark, ClickHouse, Kafka What We Offer: Exciting Projects: We focus on industries like High-Tech, communication, media, healthcare, retail and telecom. Our customer list is full of fantastic global brands and leaders who love what we build for them. Collaborative Environment: You Can expand your skills by collaborating with a diverse team of highly talented people in an open, laidback environment — or even abroad in one of our global centers or client facilities! Work-Life Balance: GlobalLogic prioritizes work-life balance, which is why we offer flexible work schedules, opportunities to work from home, and paid time off and holidays. Professional Development: Our dedicated Learning & Development team regularly organizes Communication skills training(GL Vantage, Toast Master),Stress Management program, professional certifications, and technical and soft skill trainings. Excellent Benefits: We provide our employees with competitive salaries, family medical insurance, Group Term Life Insurance, Group Personal Accident Insurance , NPS(National Pension Scheme ), Periodic health awareness program, extended maternity leave, annual performance bonuses, and referral bonuses. Fun Perks: We want you to love where you work, which is why we host sports events, cultural activities, offer food on subsidies rates, Corporate parties. Our vibrant offices also include dedicated GL Zones, rooftop decks and GL Club where you can drink coffee or tea with your colleagues over a game of table and offer discounts for popular stores and restaurants!

Skill required: Talent & HR - SAP Talent & HR Designation: PPSM Specialist Qualifications: Any Graduation Years of Experience: 7 to 11 years Language - Ability: English(International) - Proficient What would you do? Improve workforce performance and productivity, boosts business agility, increases revenue and reduces costsTalent & HR processIn this role, you will be expected to leverage the part of the enterprise resource planning (ERP) that handles employee records and provides a framework to automate HR services like payroll, benefits, personnel activity and compliance. What are we looking for? in-depth knowledge of PMO activities excellent organizational skills, and strong attention to detail.This role involves collaboration with multiple stakeholders, supporting both project-specific processes, and managing critical tasks like onboarding, reporting, and session coordination. Microsoft Project Plan /ADO Maintenance Reporting Contractors Management (Ad-hoc) Proficient in tools such as ADO Microsoft Project Google Suite Beeline, and MS Office (Excel, PowerPoint). Roles and Responsibilities: In this role you are required to do analysis and solving of moderately complex problems May create new solutions, leveraging and, where needed, adapting existing methods and procedures The person would require understanding of the strategic direction set by senior management as it relates to team goals Primary upward interaction is with direct supervisor May interact with peers and/or management levels at a client and/or within Accenture Guidance would be provided when determining methods and procedures on new assignments Decisions made by you will often impact the team in which they reside Individual would manage small teams and/or work efforts (if in an individual contributor role) at a client or within Accenture Please note that this role may require you to work in rotational shifts Qualifications Any Graduation

Role: Big Data Engineer Experience: 5+ years Location: Bangalore Job Description Minimum Qualifications Bachelors Degree in Engineering, Computer Science, CIS, or related field (or equivalent work experience in a related field) 5 years of experience in Data, BI or Platform Engineering, Data Warehousing/ETL, or Software Engineering 5 years of experience working on project(s) involving the implementation of solutions applying development life cycles (SDLC) Data Engineering 5 years of experience in Hadoop or any Cloud Bigdata components (specific to the Data Engineering role) Expertise in Python, SQL,Hadoop (Sqoop, Hive, Pig, Map Reduce), Spark (Spark Streaming, MLib), Kafka or equivalent Cloud Bigdata components (specific to the Data Engineering role) Programming Languages : Scala /Python/Pyspark Big Data Frameworks: Spark, Hadoop/HDFS/Hive/Sqoop/Oozie, Kafka, HBase or any other NoSQL Store Good to Have: Streaming with Spark/Kafka, Cloud knowledge (AWS/Azure)Scheduling and creating workflows with Apache AirFlow Excellent communication skills Must have Very strong understanding of distributed computing concepts Very good knowledge on pyspark/python/scala and dataframe APIs Good understanding of Hive and usage with Spark Good knowledge on data integration tools like Sqoop Basic knowledge on shell scripting About the Client: The company is a Global Limited, multinational professional services partnership. It is one of the largest professional services networks in the world.

Required skills- - Relevant experience with Scala-Spark Big Data development. - Strong Database experience preferably with Hadoop, DB2, or Sybase. - Good understanding of Hadoop (HDFS) eco-system - Complete SDLC process and Agile Methodology (Scrum) - Strong oral and written communication skills - Experience working within a scrum team - Excellent interpersonal skills and professional approach - The ability to investigate and solve technical problems in the context of supporting production applications - Hands-on Data Mining and analytical work experience, big data or Scala on Spark - Unix OS, Scripting, Python - Good understanding of DevOps concepts including working experience of CI/CD tools like Jenkins

Capgemini Invent Capgemini Invent is the digital innovation, consulting and transformation brand of the Capgemini Group, a global business line that combines market leading expertise in strategy, technology, data science and creative design, to help CxOs envision and build whats next for their businesses. Your Role Analyse and organize raw data. Build data systems and pipelines. Evaluate business needs and objectives. Interpret trends and patterns. Conduct complex data analysis and report on results. Prepare data for prescriptive and predictive modelling. Build algorithms and prototypes. Combine raw information from different sources. Explore ways to enhance data quality and reliability. Identify opportunities for data acquisition. Develop analytical tools and programs. Collaborate with data scientists and architects on several projects. Participate in code peer reviews to ensure our applications comply with best practices. Your Profile Experience with any Big Data tools:Hadoop, Spark, Kafka, Sqoop, Flume, Hive etc. Experience with any relational SQL and NoSQL databases, including Postgres, Cassandra, Sql Server, Oracle, Snowflake. Experience with any data pipeline and workflow management tools:Azkaban, Luigi, Airflow, etc. Experience in any Cloud platforms:Azure, AWS or GCP. Experience with stream-processing systems:Storm, Spark-Streaming, etc. Experience with object-oriented/object function scripting languages:Python, Java, C++, Scala, etc Must have hands-on experience in DevOps and CI/CD deployments. Should know basic and advance SQL and can write complex queries. Strong experience into data warehousing and dimensional modelling. Should be a very good team player to work in a geographically dispersed team . What you will love about working here We recognize the significance of flexible work arrangements to provide support. Be it remote work, or flexible work hours, you will get an environment to maintain healthy work life balance. At the heart of our mission is your career growth. Our array of career growth programs and diverse professions are crafted to support you in exploring a world of opportunities. Equip yourself with valuable certifications in the latest technologies such as Generative AI. About Capgemini Capgemini is a global business and technology transformation partner, helping organizations to accelerate their dual transition to a digital and sustainable world, while creating tangible impact for enterprises and society. It is a responsible and diverse group of 340,000 team members in more than 50 countries. With its strong over 55-year heritage, Capgemini is trusted by its clients to unlock the value of technology to address the entire breadth of their business needs. It delivers end-to-end services and solutions leveraging strengths from strategy and design to engineering, all fueled by its market leading capabilities in AI, cloud and data, combined with its deep industry expertise and partner ecosystem. The Group reported 2023 global revenues of 22.5 billion.

Project Role :Data Platform Engineer Project Role Description :Assists with the data platform blueprint and design, encompassing the relevant data platform components. Collaborates with the Integration Architects and Data Architects to ensure cohesive integration between systems and data models. Must have skills :AWS Glue

Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Apache Spark Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years of full term education Project Role :Application Developer Project Role Description :Design, build and configure applications to meet business process and application requirements. Must have Skills :Apache SparkGood to Have Skills :No Industry SpecializationJob Requirements :Key Responsibilities :aShould be responsible for developing and maintaining applications with PySpark bPerformance Tuning with respect to executor sizing and other environmental parameters, code optimization, partitions tuning, etccWork with stakeholders including the product owner, data and design teams to assist with data-related technical issues and support their data infrastructure needsdInteract with business users to understand requirements and troubleshoot issues Technical Experience :aThe developer must have strong knowledge in Apache Spark and Python programmingbDeep experience in developing data processing tasks using pySpark such as reading data from external sources, merge data, perform data enrichment and load in to target data destinationscPossess validated knowledge in designing database models to store structured unstructured data efficiently and in creating effective data tools for analytics expertsdHands-on experience in enterprise data lakes, data analytic Professional Attributes :1Must have Good Communication skill Capable of Client facing2Ability to work creatively and analytically in a problem-solving 3Excellent communication skills; flexibility; responsiveness Educational Qualification:15 years of full term educationAdditional Info :1 The Candidate must have 4 years of experience. Qualifications 15 years of full term education

As an Software Developer at IBM you will harness the power of data to unveil captivating stories and intricate patterns. You'll contribute to data gathering, storage, and both batch and real-time processing. Collaborating closely with diverse teams, you'll play an important role in deciding the most suitable data management systems and identifying the crucial data required for insightful analysis. As a Data Engineer, you'll tackle obstacles related to database integration and untangle complex, unstructured data sets. In this role, your responsibilities may include: Implementing and validating predictive models as well as creating and maintain statistical models with a focus on big data, incorporating a variety of statistical and machine learning techniques Designing and implementing various enterprise seach applications such as Elasticsearch and Splunk for client requirements Work in an Agile, collaborative environment, partnering with other scientists, engineers, consultants and database administrators of all backgrounds and disciplines to bring analytical rigor and statistical methods to the challenges of predicting behaviours. Build teams or writing programs to cleanse and integrate data in an efficient and reusable manner, developing predictive or prescriptive models, and evaluating modelling results Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Strong technical abilities to understand, design, write and debug complex code.Bigdata, Pyspark,Scala,hadoop,Hive,Java,Python.Develops applications on Big Data technologies including API development, Knowledge in Relational Databases experience in troubleshooting, monitoring and performance tuning of Spark jobs.Presto,Impala, HDFS, Linux.. . Good to Have;-knowledge of Analytics libraries, open-source Natural Language Processing, statistical and big data computing libraries. Hands on Experience on cloud technology AWS/GCP Preferred technical and professional experience You thrive on teamwork and have excellent verbal and written communication skills. Ability to communicate with internal and external clients to understand and define business needs, providing analytical solutions Ability to communicate results to technical and non-technical audiences

As an Application Developer, you will lead IBM into the future by translating system requirements into the design and development of customized systems in an agile environment. The success of IBM is in your hands as you transform vital business needs into code and drive innovation. Your work will power IBM and its clients globally, collaborating and integrating code into enterprise systems. You will have access to the latest education, tools and technology, and a limitless career path with the worlds technology leader. Come to IBM and make a global impact Responsibilities: Responsible to manage end to end feature development and resolve challenges faced in implementing the same Learn new technologies and implement the same in feature development within the time frame provided Manage debugging, finding root cause analysis and fixing the issues reported on Content Management back end software system fixing the issues reported on Content Management back end software system Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Overall, more than 6 years of experience with more than 4+ years of Strong Hands on experience in Python and Spark Strong technical abilities to understand, design, write and debug to develop applications on Python and Pyspark. Good to Have;- Hands on Experience on cloud technology AWS/GCP/Azure strong problem-solving skill Preferred technical and professional experience Good to Have;- Hands on Experience on cloud technology AWS/GCP/Azure

Project Role : Application Lead Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills : PySpark Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : Engineering graduate preferably Computer Science graduate 15 years of full time education Summary :As an Application Lead, you will be responsible for leading the effort to design, build, and configure applications using PySpark. Your typical day will involve collaborating with cross-functional teams, developing and deploying PySpark applications, and acting as the primary point of contact for the project. Roles & Responsibilities: Lead the effort to design, build, and configure PySpark applications, collaborating with cross-functional teams to ensure project success. Develop and deploy PySpark applications, ensuring adherence to best practices and standards. Act as the primary point of contact for the project, communicating effectively with stakeholders and providing regular updates on project progress. Provide technical guidance and mentorship to junior team members, ensuring their continued growth and development. Stay updated with the latest advancements in PySpark and related technologies, integrating innovative approaches for sustained competitive advantage. Professional & Technical Skills: Must To Have Skills:Strong experience in PySpark. Good To Have Skills:Experience with Hadoop, Hive, and other Big Data technologies. Solid understanding of software development principles and best practices. Experience with Agile development methodologies. Strong problem-solving and analytical skills. Additional Information: The candidate should have a minimum of 5 years of experience in PySpark. The ideal candidate will possess a strong educational background in computer science or a related field, along with a proven track record of delivering impactful data-driven solutions. This position is based at our Bangalore, Hyderabad, Chennai and Pune Offices. Mandatory office (RTO) for 2- 3 days and have to work on 2 shifts (Shift A- 10:00am to 8:00pm IST and Shift B - 12:30pm to 10:30 pm IST) Qualifications Engineering graduate preferably Computer Science graduate 15 years of full time education

Project Role : Application Lead Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills : PySpark Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Lead, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. Your day will involve overseeing the application development process and ensuring seamless communication within the team and stakeholders. Roles & Responsibilities: Expected to be an SME Collaborate and manage the team to perform Responsible for team decisions Engage with multiple teams and contribute on key decisions Provide solutions to problems for their immediate team and across multiple teams Lead the application development process effectively Ensure timely delivery of projects Mentor and guide team members for their professional growth Professional & Technical Skills: Must To Have Skills:Proficiency in PySpark Strong understanding of big data processing Experience in designing and implementing scalable applications Knowledge of cloud platforms like AWS or Azure Hands-on experience in data processing and analysis Additional Information: The candidate should have a minimum of 5 years of experience in PySpark This position is based at our Bengaluru office A 15 years full-time education is required Qualifications 15 years full time education

Project Role : Data Platform Engineer Project Role Description : Assists with the data platform blueprint and design, encompassing the relevant data platform components. Collaborates with the Integration Architects and Data Architects to ensure cohesive integration between systems and data models. Must have skills : Databricks Unified Data Analytics Platform Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Platform Engineer, you will assist with the data platform blueprint and design, collaborating with Integration Architects and Data Architects to ensure cohesive integration between systems and data models. You will play a crucial role in shaping the data platform components. Roles & Responsibilities: Expected to be an SME, collaborate and manage the team to perform. Responsible for team decisions. Engage with multiple teams and contribute on key decisions. Provide solutions to problems for their immediate team and across multiple teams. Lead the implementation of data platform components. Ensure data platform scalability and performance. Conduct regular data platform audits. Stay updated on emerging data platform technologies. Professional & Technical Skills: Must To Have Skills:Proficiency in Databricks Unified Data Analytics Platform. Strong understanding of cloud-based data platforms. Experience with data integration and data modeling. Hands-on experience with data pipeline orchestration tools. Knowledge of data security and compliance standards. Additional Information: The candidate should have a minimum of 5 years of experience in Databricks Unified Data Analytics Platform. This position is based at our Bengaluru office. A 15 years full-time education is required. Qualifications 15 years full time education

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developing streaming pipelines Experience to work with Hadoop / AWS eco system components to implement scalable solutions to meet the ever-increasing data volumes, using big data/cloud technologies Apache Spark, Kafka, any Cloud computing etc Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Minimum 4+ years of experience in Big Data technologies with extensive data engineering experience in Spark / Python or Scala ; Minimum 3 years of experience on Cloud Data Platforms on AWS; Experience in AWS EMR / AWS Glue / DataBricks, AWS RedShift, DynamoDB Good to excellent SQL skills Exposure to streaming solutions and message brokers like Kafka technologies Preferred technical and professional experience Certification in AWS and Data Bricks or Cloudera Spark Certified developers

Client expectation apart from JD Longer AWS data engineering experience (glue, spark, ECR ECS docker), python, pyspark, hudi/iceberg/Terraform, Kafka. Java in early career would be a great addition but not a prio. (for the OOP part and java connectors).

As an Application Developer, you will lead IBM into the future by translating system requirements into the design and development of customized systems in an agile environment. The success of IBM is in your hands as you transform vital business needs into code and drive innovation. Your work will power IBM and its clients globally, collaborating and integrating code into enterprise systems. You will have access to the latest education, tools and technology, and a limitless career path with the worlds technology leader. Come to IBM and make a global impact Responsibilities: Responsible to manage end to end feature development and resolve challenges faced in implementing the same Learn new technologies and implement the same in feature development within the time frame provided Manage debugging, finding root cause analysis and fixing the issues reported on Content Management back end software system fixing the issues reported on Content Management back end software system Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Overall, more than 6 years of experience with more than 4+ years of Strong Hands on experience in Python and Spark Strong technical abilities to understand, design, write and debug to develop applications on Python and Pyspark. Good to Have;- Hands on Experience on cloud technology AWS/GCP/Azure strong problem-solving skill Preferred technical and professional experience Good to Have;- Hands on Experience on cloud technology AWS/GCP/Azure

JD Big data Developer Exp-4 to 8yrs Location- chennai,mumbai,pune, skills-Pyspark,scala , Hadoop ,spark Notice period-immediate joiner to 15 days

As a Senior General Manager in the IT Data department, the Data Engineer will be responsible for managing and organizing data. The candidate will play a pivotal role in the company's data infrastructure development and will be responsible for optimizing data systems and building them from the ground up. The Data Engineer will also be responsible for developing, testing, and maintaining architectures such as databases and large-scale processing systems.

Role & responsibilities The candidate must have overall 11+ years of experience in ETL and Data Warehouse of which 3-4 years on Hadoop platform and at least 2 year in Cloud Big Data Environment. Must have hands on experience on Hadoop services like HIVE/ Spark/ Scala /Sqoop Must have hands on in writing complex use case driven SQLs Should have about 3+ years of hands-on good knowledge of AWS Cloud and On-Prem related key services and concepts. Should have 3+ years of working experience with AWS Cloud tools like EMR, Redshift, Glue S3 Should have been involved in On-Prem to Cloud Migration process. Should have good knowledge with HIVE / Spark / Scala scripts Should have good knowledge on Unix Shell scripting Should be flexible to overlap US business hours Should be able to drive technical design on Cloud applications Should be able to guide & drive the team members for cloud implementations Should be well versed with the costing model and best practices of the services to be used for Data Processing Pipelines in Cloud Environment. AWS Certified applicants preferable

About Client Hiring for One of Our Multinational Corporations! Job Title: Data Engineer (Scala, Spark, Hadoop) Developer Location: Bangalore Job Type: Full Time WORK FROM OFFICE Job Summary: We are seeking a talented and motivated Data Engineer with strong expertise in Scala , Apache Spark , and Hadoop to join our growing team. As a Data Engineer, you will be responsible for building, optimizing, and maintaining scalable data pipelines, data processing systems, and data storage solutions. The ideal candidate will be passionate about working with big data technologies and developing innovative solutions for processing and analyzing large datasets. Key Responsibilities: Design, develop, and implement robust data processing pipelines using Scala , Apache Spark , and Hadoop frameworks. Develop ETL processes to extract, transform, and load large volumes of structured and unstructured data into data lakes and data warehouses. Work with large datasets to optimize performance, scalability, and data quality. Collaborate with cross-functional teams, including Data Scientists, Analysts, and DevOps, to deliver end-to-end data solutions. Ensure data processing workflows are automated, monitored, and optimized for efficiency and cost-effectiveness. Troubleshoot and resolve data issues, ensuring data integrity and quality. Work on the integration of various data sources into the Hadoop ecosystem and ensure effective data management. Develop and implement best practices for coding, testing, and deployment of data processing pipelines. Document and maintain clear and comprehensive technical documentation for data engineering processes and systems. Stay up-to-date with the latest industry trends, tools, and technologies in the data engineering and big data ecosystem. Required Skills and Qualifications: Proven experience as a Data Engineer, Data Developer, or similar role working with Scala , Apache Spark , and Hadoop . Strong knowledge of big data processing frameworks such as Apache Spark, Hadoop, HDFS, and MapReduce. Experience with distributed computing and parallel processing techniques. Solid experience with ETL processes and working with relational and NoSQL databases (e.g., MySQL, MongoDB, Cassandra, etc.). Proficiency in SQL for querying large datasets. Strong experience with data storage technologies such as HDFS , Hive , HBase , or Parquet . Familiarity with cloud platforms (e.g., AWS, Azure, Google Cloud) and their data-related services (e.g., S3, Redshift, BigQuery). Experience with workflow orchestration tools such as Airflow , Oozie , or Luigi . Knowledge of data warehousing , data lakes , and data integration patterns. Familiarity with version control tools such as Git and CI/CD pipelines. Strong problem-solving skills and the ability to debug complex issues. Excellent communication skills and the ability to collaborate with different teams. Preferred Skills: Experience with streaming data technologies like Kafka , Flink , or Kinesis . Familiarity with data visualization tools (e.g., Tableau, Power BI) and reporting. Knowledge of machine learning models and working with Data Science teams. Experience working in an Agile/Scrum environment. Degree in Computer Science, Engineering, Mathematics, or a related field. Why Join Us? Be a part of an innovative and dynamic team working on cutting-edge data engineering technologies. Opportunities for growth and career advancement in the data engineering domain. Competitive salary and benefits package. Flexible work arrangements and a supportive work environment Srishty Srivastava Black and White Business Solutions Pvt.Ltd. Bangalore,Karnataka,India. Direct Number:8067432456 srishty.srivastava@blackwhite.in |www.blackwhite.in