16 Bigdata Technologies Jobs

Join us as an ETL Developer at Barclays, where you will be responsible for supporting the successful delivery of Location Strategy projects to plan, budget, agreed quality, and governance standards. Spearheading the evolution of our digital landscape, you will drive innovation and excellence using cutting-edge technology to revolutionize our digital offerings and ensure unparalleled customer experiences. To be successful in this role, you should have experience with: - Good knowledge of Python - Extensive hands-on PySpark - Understanding of Data Warehousing concepts - Strong SQL knowledge - Proficiency in Bigdata technologies (HDFS) - Exposure to AWS working environment Additionally, highly valued skills may include: - Working knowledge of AWS - Familiarity with Bigdata As an ETL Developer at Barclays, you may be assessed on key critical skills relevant for success in this role, such as risk and controls, change and transformation, business acumen, strategic thinking, and digital and technology, in addition to job-specific technical skills. This role is based in Pune. Purpose of the role: To build and maintain systems that collect, store, process, and analyze data, including data pipelines, data warehouses, and data lakes, ensuring accuracy, accessibility, and security of all data. Accountabilities: - Building and maintaining data architecture pipelines for transfer and processing of durable, complete, and consistent data - Designing and implementing data warehouses and data lakes to manage data volumes, velocity, and security measures - Developing processing and analysis algorithms suitable for data complexity and volumes - Collaborating with data scientists to build and deploy machine learning models Analyst Expectations: - Impacting the work of related teams within the area - Partnering with other functions and business areas - Taking responsibility for end results of team's operational processing and activities - Escalating policy/procedure breaches appropriately - Embedding new policies/procedures for risk mitigation - Advising and influencing decision-making within expertise area - Managing risk and strengthening controls in work areas - Delivering work in line with relevant rules, regulations, and codes of conduct - Building an understanding of integration with function and organization's products, services, and processes - Demonstrating how areas contribute to achieving organization objectives - Resolving problems and selecting solutions based on technical experience - Guiding and persuading team members and communicating complex/sensitive information - Acting as a contact point for stakeholders outside the function and building a network of contacts All colleagues are expected to demonstrate the Barclays Values of Respect, Integrity, Service, Excellence, and Stewardship, as well as the Barclays Mindset to Empower, Challenge, and Drive.,

Greetings, We have an opening for one of our clients " Mphasis " for " Scala Developer " position. Role & responsibilities Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field. Minimum 5 years of professional experience in Data Engineering, with a strong focus on big data technologies. Proficiency in Scala for developing big data applications and transformations, especially with Apache Spark. Expert-level proficiency in SQL; ability to write complex queries, optimize performance, and understand database internals. Extensive hands-on experience with Apache Spark (Spark SQL, DataFrames, RDDs) for large-scale data processing and analytics. Solid understanding of distributed computing concepts and experience with the Hadoop ecosystem (HDFS, Hive). Experience with building and optimizing ETL/ELT processes and data warehousing concepts. Strong understanding of data modeling techniques (e.g., Star Schema, Snowflake Schema). Familiarity with version control systems (e.g., Git). Excellent problem-solving, analytical, and communication skills. Ability to work independently and collaboratively in an Agile team environment. Work mode : Hybrid ( 3 days ) Location : Pune / Chennai

Are you passionate about developing mission-critical, high-quality software solutions using cutting-edge technology in a dynamic environment Join Compliance Engineering, a global team of over 300 engineers and scientists working on the most complex, mission-critical problems. You will build and operate platforms and applications to prevent, detect, and mitigate regulatory and reputational risks, leveraging the latest technology and vast amounts of data. As part of a significant uplift and rebuild of the Compliance application portfolio, Compliance Engineering is seeking Systems Engineers. As a member of the team, you will partner with users, development teams, and colleagues globally to onboard new business initiatives, test Compliance Surveillance coverage, learn from experts, and mentor team members. You will work with technologies like Java, Python, PySpark, and Big Data tools to innovate, design, implement, test, and maintain software across products. The ideal candidate will have a Bachelor's or Master's degree in Computer Science or a related field, expertise in Java development, debugging, and problem-solving, as well as experience in project management. Strong communication skills are essential. Desired experience includes relational databases, Hadoop, big data technologies, knowledge of the financial industry (especially Capital Markets domain), compliance, or risk functions. Goldman Sachs, a leading global investment banking, securities, and investment management firm, commits to diversity, inclusion, and individual growth. They provide various opportunities for professional and personal development, fostering a culture of diversity and inclusion. Goldman Sachs is an equal employment/affirmative action employer. Accommodations for candidates with special needs or disabilities are available during the recruiting process. Learn more at GS.com/careers.,

Are you passionate about developing mission-critical, high-quality software solutions, using cutting-edge technology in a dynamic environment Compliance Engineering, a global team of over 300 engineers and scientists, is dedicated to working on the most complex, mission-critical problems. The team builds and operates a suite of platforms and applications that prevent, detect, and mitigate regulatory and reputational risk across the firm. Leveraging the latest technology and vast amounts of structured and unstructured data, we use modern frameworks to build responsive and intuitive front-end and Big Data applications. As the firm invests significantly in uplifting and rebuilding the Compliance application portfolio, Compliance Engineering seeks to fill several Systems Engineer roles. As a member of our team, you will partner globally with users, development teams, and engineering colleagues across multiple divisions to facilitate the onboarding of new business initiatives and test and validate Compliance Surveillance coverage. You will have the opportunity to learn from experts, train and mentor team members, leverage various technologies including Java, Python, PySpark, and other Big Data technologies, innovate and incubate new ideas, and be involved in the full software development life cycle. A successful candidate will possess a Bachelor's or Master's degree in Computer Science, Computer Engineering, or a similar field of study, expertise in Java development, debugging, and problem-solving, as well as experience in delivery or project management. The ability to clearly express ideas and arguments in meetings and on paper is essential. Experience in relational databases, Hadoop and Big Data technologies, knowledge of the financial industry (particularly in the Capital Markets domain), and compliance or risk functions is desired and can set you apart from other candidates. Goldman Sachs, a leading global investment banking, securities, and investment management firm founded in 1869 and headquartered in New York, is committed to fostering diversity and inclusion in the workplace and beyond. The firm provides numerous opportunities for professional and personal growth, from training and development to firmwide networks, benefits, wellness, personal finance offerings, and mindfulness programs. Learn more about the culture, benefits, and people at GS.com/careers. Goldman Sachs is an equal employment/affirmative action employer dedicated to finding reasonable accommodations for candidates with special needs or disabilities during the recruiting process.,

The Applications Development Group Manager is a senior management level position responsible for accomplishing results through the management of a team or department in an effort to establish and implement new or revised application systems and programs in coordination with the Technology Team. The overall objective of this role is to drive applications systems analysis and programming activities. Manage multiple teams of professionals to accomplish established goals and conduct personnel duties for team (e.g. performance evaluations, hiring and disciplinary actions). Provide strategic influence and exercise control over resources, budget management and planning while monitoring end results. Utilize in-depth knowledge of concepts and procedures within own area and basic knowledge of other areas to resolve issues. Ensure essential procedures are followed and contribute to defining standards. Integrate in-depth knowledge of applications development with overall technology function to achieve established goals. Provide evaluative judgement based on analysis of facts in complicated, unique, and dynamic situations including drawing from internal and external sources. Influence and negotiate with senior leaders across functions, as well as communicate with external parties as necessary. Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency, as well as effectively supervise the activity of others and create accountability with those who fail to maintain these standards. Qualifications: - 10+ years of relevant experience - Experience in applications development - Experience in management - Experience managing global technology teams - Working knowledge of industry practices and standards - Consistently demonstrates clear and concise written and verbal communication Education: - Bachelors degree/University degree or equivalent experience - Masters degree preferred Required Skills: - Programming skills including concurrent, parallel and distributed systems programming - Expert level knowledge of Java - Expert level experience with HTTP, ReSTful web services and API design - Messaging technologies (Kafka) - Experience with Bigdata technologies Developer Hadoop, Apache Spark, Python, PySpark - Experience with Reactive Streams Desirable Skills: - Messaging technologies - Familiarity with Hadoop SQL interfaces like Hive, Spark SQL, etc. - Experience with Kubernetes - Good understanding of the Linux OS - Experience with Gradle, Maven would be beneficial Note: For complementary skills, please see the job requirements above and/or contact the recruiter. If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi. View Citis EEO Policy Statement and the Know Your Rights poster.,

Job Description We are seeking a Sr. Data Engineer who has strong design, developments skills and upkeeps scalability, availability and excellence when building the next generation of our data pipelines and platform. You are an expert in various data processing technologies and data stores, appreciate the value of clear communication and collaboration, and devote to continual capacity planning and performance fine-tuning for emerging business growth. As the Senior Data Engineer, you will be mentoring Junior engineers in the team. Good to have: Experience in Web Services, API integration, Data exchanges with third parties is preferred. Experience in Snowflake is a big plus. Experience in NoSQL technologies (MongoDB, FoundationDB, Redis) is a plus. We would appreciate candidates who can demonstrate business-side functional understanding and effectively communicate the business context alongside their technical expertise. Job Requirements Must have 5+ years of experience in Data Engineering field, with a proven track record of exposure in Big Data technologies such as Hadoop, Amazon EMR, Hive, Spark. Expertise in SQL technologies and at least one major Data Warehouse technology (Snowflake, RedShift, BigQuery etc.). Must have experience in building data platform designing and building data model, integrate data from many sources, build ETL and data-flow pipelines, and support all parts of the data platform. Programming proficiency in Python and Scala, with experience writing modular, reusable, and testable code, including robust error handling and logging in data engineering applications. Hands-on experience with AWS cloud services, particularly in areas such as S3, Lambda, Glue, EC2, RDS, and IAM. Experience with orchestration tools such as Apache Airflow, for scheduling, monitoring, and managing data pipelines in a production environment. Familiarity with CI/CD practices, automated deployment pipelines, and version control systems (e.g., Git, GitHub/GitLab), ensuring reliable and repeatable data engineering workflows. Data Analysis skill – can make arguments with data and proper visualization. Energetic, enthusiastic, detail-oriented, and passionate about producing high-quality analytics deliverable. Must have experience in developing application with high performance and low latency. Ability to take ownership of initiatives and drive them independently from conception to delivery, including post-deployment monitoring and support. Strong communication and interpersonal skills with the ability to build relationships with stakeholders, understand business requirements, and translate them into technical solutions. Comfortable working cross-functionally in a multi-team environment, collaborating with data analysts, product managers, and engineering teams to deliver end-to-end data solutions.

We are seeking a highly skilled ETL/Data Engineer with expertise in Informatica DEI BDM to design and implement robust data pipelines handling medium to large-scale datasets. The role involves building efficient ETL frameworks that support batch .

Description - External Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by diversity and inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health equity on a global scale. Join us to start Caring. Connecting. Growing together. Primary Responsibilities: Analyzes and investigates Provides explanations and interpretations within area of expertise Participate in scrum process and deliver stories/features according to the schedule Collaborate with team, architects and product stakeholders to understand the scope and design of a deliverable Participate in product support activities as needed by the team. Understand product architecture, features being built and come up with product improvement ideas and POCs Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so Qualifications - External Required Qualifications: Undergraduate degree or equivalent experience Proven experience using Bigdata tech stack Sound knowledge on Java and Spring framework with good exposure to Spring Batch, Spring Data, Spring Web services, Python Proficient with Bigdata ecosystem (Sqoop, Spark, Hadoop, Hive, HBase) Proficient with Unix/Linux eco systems and shell scripting skills Proven Java, Kafka, Spark, Big Data, Azure ,analytical and problem solving skills Proven solid analytical and communication skills Role & responsibilities Preferred candidate profile

Qualifications we are looking for Master/Bachelor degree in Computer Science, Electrical Engineering, Information Systems or other technical discipline; advanced degree preferred. Minimum of 7+ years of software development experience (with a concentration in data centric initiatives), with demonstrated expertise in leveraging standard development best practice methodologies. Minimum 4+ years of experience in Hadoop using Core Java Programming, Spark, Scala, Hive and Go lang Expertise in Object Oriented Programming Language Java Experience using CI/CD Process, version control and bug tracking tools. Experience in handling very large data volume in Real Time and batch mode. Experience with automation of job execution and validation Strong knowledge of Database concepts Strong team player. Strong communication skills with proven ability to present complex ideas and document in a clear and concise way. Quick learner; self-starter, detailed and in-depth.

Job description Hiring for Bigdata Developer with experience range 5 to 15 years. Mandatory Skills: Bigdata, Scala, Spark, Hive, Kafka Education: BE/B.Tech/MCA/M.Tech/MSc./MSts

5-12 Years of in Big Data & Data related technology experience Expert level understanding of distributed computing principles Expert level knowledge and experience in Apache Spark Hands on programming with Python Proficiency with Hadoop v2, Map Reduce, HDFS, Sqoop Experience with building stream-processing systems, using technologies such as Apache Storm or Spark-Streaming Experience with messaging systems, such as Kafka or RabbitMQ Good understanding of Big Data querying tools, such as Hive, and Impala Experience with integration of data from multiple data sources such as RDBMS (SQL Server, Oracle), ERP, Files Good understanding of SQL queries, joins, stored procedures, relational schemas Experience with NoSQL databases, such as HBase, Cassandra, MongoDB Knowledge of ETL techniques and frameworks Performance tuning of Spark Jobs Experience with native Cloud data services AWS or AZURE Databricks Ability to lead a team efficiently Experience with designing and implementing Big data solutions Practitioner of AGILE methodology. Role & responsibilities Preferred candidate profile

2-5 Yrs in Exp Big Data testing, focusing on both automated and manual testing. Strong understanding of Hadoop. Proficient in SQL, Unix, Linux with hands-on experience. also Exp in ETL .

Open & Direct Walk-in Drive event | Hexaware technologies - Azure Data Engineer/Architect in Chennai, Tamilnadu on 10th May [Saturday] 2025 - Azure Databricks/ Data factory/ SQL & Pyspark Dear Candidate, I hope this email finds you well. We are thrilled to announce an exciting opportunity for talented professionals like yourself to join our team as an Azure Data Engineer. We are hosting an Open Walk-in Drive in Chennai, Tamilnadu on 10th May [Saturday] 2025, and we believe your skills in Databricks, Data Factory, SQL, and Pyspark align perfectly with what we are seeking. Details of the Walk-in Drive: Date: 10th May [Saturday] 2025 Experience: 5 years to 12 years Time: 9.00 AM to 5 PM Venue: HEXAWARE TECHNOLOGIES H-5, Sipcot It Park, Post, Navalur, Siruseri, Tamil Nadu 603103 Point of Contact: Azhagu Kumaran Mohan/+91-9789518386 Key Skills and Experience: As an Azure Data Engineer, we are looking for candidates who possess expertise in the following: Databricks Data Factory SQL Pyspark/Spark Roles and Responsibilities: As a part of our dynamic team, you will be responsible for: Designing, implementing, and maintaining data pipelines Collaborating with cross-functional teams to understand data requirements. Optimizing and troubleshooting data processes Leveraging Azure data services to build scalable solutions. What to Bring: Updated resume Photo ID, Passport size photo How to Register: To express your interest and confirm your participation, please reply to this email with your updated resume attached. Walk-ins are also welcome on the day of the event. This is an excellent opportunity to showcase your skills, network with industry professionals, and explore the exciting possibilities that await you at Hexaware Technologies. If you have any questions or require further information, please feel free to reach out to me at AzhaguK@hexaware.com - +91-9789518386 We look forward to meeting you and exploring the potential of having you as a valuable member of our team. ********* less than 4 years of total experience will not be Screen selected to attend the interview***********

Warm Greetings from SP Staffing!! Role: Pyspark Developer Experience Required :4 to 6 yrs Work Location :Pune/Hyderbad/Chennai/Bangalore Required Skills, Pyspark Python Interested candidates can send resumes to nandhini.spstaffing@gmail.com

Warm Greetings from SP Staffing!! Role: Big Data Developer Experience Required :4 to 8 yrs Work Location : Bangalore/Chennai/Pune/Delhi/Hyderabad/Kochi Required Skills, Spark and Scala Interested candidates can send resumes to nandhini.s@spstaffing.in

Software developer with hands-on experience on BigData technologies (Spark, Scala and hadoop) 1)Strong understanding and hands-on experience of Scala programming 2)Hands-on experience on Spark and Hadoop 3)Good working knowledge of GIT, Jenkins and Airflow 4)Good to have working knowledge of AWS / Azure 5) Good to have basic knowledge of Python 6)Strong interest for good coding practices (clean code, TDD, ....) 7)Experience of working in Agile methodology 8)Strong problem-solving skill 9)Strong verbal and written communication skills Desirable Skills: Should have been part of teams that did multiple Cloud migration.a. 5 plus years of relevant experience in cloud Infrastructure principles , methodologies and cloud architecture b. Public Cloud certifications (Azure) c. Work with cloud application migration leaders to define and create application migration plans d. Work with Infra engineer, Dev Ops, DB engineer and other teams in executing migration projects to ensure minimal risk and business disruption e. Provide cloud migration expertise covering infrastructure, application architectures, cloud capabilities, security, etc. f. In collaboration with application teams, review and identify candidate applications for cloud migration based on pre-defined application rationalization output of categorized applications g. Lead the discovery and assessment of on-prem application workloads to plan detailed migration h. Build detailed application migration plans in collaboration with partners and other stakeholders i. Define appropriate migration strategy for each application (Rehost, Re-Platform, Rearchitect, etc.) while considering cost and complexity of the various options j. Help identify potential risks that may impact migration projects and identify appropriate mitigating actions k. Ensure migrated applications comply with required security requirements l. Measure relevant cloud application migration KPIs to ensure overall program health and migration efficiency. m. Experience with configuration, migration, and deployment experience related to Azure or AWS IaaS/PaaS Technologies n. Experience working in a multi-cloud environment with demonstrated experience working on cloud deployment projects (including cloud native development and migration) o. Experience managing container orchestration tools such as Kubernetes, Docker or equivalent p. Experience of Virtualization technologies such as VMWare, OpenStack q. Experience of networking concepts DNS, Load Balancing, firewall, CDN etc. r. Must have an automation mindset to study/analyze the landscape and build/propose tools.