12278 Big Data Jobs - Page 5

56 years of total experience in data engineering or big data development. 2–3 years hands-on experience with Apache Spark. Strong programming skills in PySpark, Python, and Scala. 2+ years of experience in Scala backend development. Proficient in Scala, both object oriented and functional programming concepts. Deep understanding and application of advanced functional programming concepts like category theory, monads, applicatives, and type classes. Hands-On knowledge with Scala Typelevel libraries like Cats, Shapeless, and others used for building applications with strong typing and efficient concurrency. Solid understanding of data lakes, lakehouses, and Delta Lake concepts. Experience in S...

Role & responsibilities: . 1) Strong experience in ETL Testing. 2) Experience with Big Data Platforms (Hadoop/Hive/Spark). 3) Understanding business requirements, source to target mappings, ETL flows and batch jobs. 4) Experience in Test Strategy and Test Plan creation. 5) Design, Develop & Execute test cases, scenarious and test data for Data pipelines. 6) Familiar with DW Concepts. 7) Understanding of ADO Pipelines and processes. 8) Python for Automation.

Must have 5+ years of experience in below mentioned skills. Must Have: Big Data Concepts , Python(Core Python- Able to write code), SQL, Shell Scripting, AWS S3 Good to Have: Event-driven/AWA SQS, Microservices, API Development, Kafka, Kubernetes, Argo, Amazon Redshift, Amazon Aurora .

Role & responsibilities Job Summary We are seeking a talented Gen AI & Data Science Engineer to join our dynamic team in Bangalore. The ideal candidate will have 5 to 8 years of hands-on experience in developing AI and data science models, with expertise in Generative AI, machine learning, and data engineering. You will be responsible for designing and implementing AI solutions to address complex business challenges and drive data-driven decision-making. Mandatory Skills Proficiency in Generative AI frameworks (like GPT, BERT, DALL-E). Strong programming skills in Python and Pyspark. Experience with ML libraries (TensorFlow, PyTorch, Scikit-learn). Proficiency in data analysis and visualizat...

Job description Responsibilities: Data Pipeline Development: Design, build, and maintain robust and scalable data pipelines using Databricks and Apache Spark, primarily with Scala, to ingest, process, and transform large datasets from various sources. ETL/ELT Implementation: Develop and optimize Extract, Transform, Load ETL or Extract, Load, Transform ELT processes within the Databricks environment to ensure data quality, consistency, and availability for analytical purposes. Experience on Snowflake API Development: Design, develop, and deploy secure and efficient RESTful APIs using AWS services like Lambda and API Gateway to expose data or trigger data processing workflows. Cloud Infrastruc...

Job description The ideal candidate for this role should possess a Post Graduate Degree in Computer Science or a related field (or equivalent industry experience) with a strong background in Mathematics and Statistics, complemented by a minimum of 5+ years of development and design experience as a Data Engineer. Required expertise includes experience with Big Data platforms and distributed computing technologies such as Hadoop, Map/Reduce, Spark, HBase, and Hive. The role demands proficiency in data pipeline software engineering best practices in Python, encompassing linting, unit/integration tests, git flow/pull request process, object-oriented development, data validation, algorithms, data...

Job description 10+ Years experienced Software Engineer working for our leading client, you serve as member of an agile team to design and deliver trusted market-leading technology products in a secure, stable, and scalable way. Depending on the team that you join, you could be developing features that give our customers and clients more control over how they bank with the client, strategizing on how big data can make client''s trading systems quicker, creating the next innovation in payments, or supporting the integration . Strong proficiency in Java 11+, FullStack including JMS MQ, Java, Springboot, Oracle, SQL, Unix Scripting, Git, Jira, Dev Ops, CI /CD, Cucumber/TestNG, Design and develo...

We are looking for a highly skilled and experienced Data Engineering Tech Lead to join our team at ERNST YOUNG LLP. The ideal candidate will have a strong background in data engineering and technical leadership, with 6-10 years of experience. Roles and Responsibility Design and implement scalable data pipelines and architectures. Lead the development of data engineering solutions using various technologies. Collaborate with cross-functional teams to identify business requirements and develop technical solutions. Develop and maintain large-scale data systems and applications. Ensure high-quality data processing and analysis. Mentor junior engineers and provide technical guidance. Job Requirem...

Data Engineer NameData Engineer RoleData EngineerIndustrySoftware/ ITLocationUdaipur (Rajasthan)Job TypeFull TimeExperience3- 6years Skills: Data Engineer,ETL/ELT processes,data ingestion, cloud platforms,big data tools,CI/CD pipelinesSalaryBest in the industryEducationBTech (CS/IT/EC), BCA, MCA Description: 3+ years of experience in a data engineering or related role, managing complex data workflows and large-scale datasets. Expertise in SQL, with the ability to write, optimize, and troubleshoot complex queries for relational databases like MySQL, PostgreSQL, Oracle, or Snowflake. Proficiency in Python, including libraries like Pandas, PySpark, and others used for data manipulation and proc...

Joining: Immediate to 15 days Key Skills: Strong hands-on experience in PySpark Expertise in Big Data technologies Solid understanding of SQL, Data Pipelines, and ETL Interested candidates can share their resume with the following details to navaneetha@suzva.com Current CTC Expected CTC Location

We are hiring a Cloud Data Engineer to design, build, and maintain data solutions in cloud environments, supporting large-scale analytics, AI/ML, and business intelligence workloads. Key Responsibilities: Build data pipelines and workflows for ingestion, transformation, and storage. Implement and manage data lake and warehouse solutions. Work with structured and unstructured data using distributed processing tools. Optimize data performance, reliability, and governance in the cloud. Support data scientists, analysts, and stakeholders with accessible datasets. Required Skills & Qualifications: Experience with cloud data tools (AWS Redshift, Azure Synapse, GCP BigQuery). Strong knowledge of ET...

We are looking for a skilled Senior Data Engineer with 5 to 10 years of experience to join our team in Gurugram. The ideal candidate will have expertise in building Data Lakes, Data Warehouses using Big data & cloud stack, and data modeling. Roles and Responsibility Design and develop scalable data pipelines and architectures. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data systems and databases. Ensure data quality and integrity through data validation and testing procedures. Implement Agile development methodologies to ensure timely delivery of projects. Optimize data storage and retrieval processes for improved...

Job Purpose Creates the data structures from various sources including the data infrastructure or defines new ways of capturing data to be used for analysis and runs advanced analytics models to generate insights, in accordance with TechnipFMC standards, procedures, processes, schedules, and budget, with a continuous concern for quality, targets, and customer satisfaction. Proposes and drives tactical actions. Job Description Coaches the Data Analyst Team. Participates in the selection of data analytics technology with the Technical Teams. Develops organisational policies, standards, and guidelines around data analysis. Plans and co-ordinates the backlog with the Data Stakeholders. Plans and...

The Role and Responsibilities We have open positions ranging from Data Scientist to Director, providing talented and motivated professionals with excellent career and growth opportunities. We seek individuals with relevant prior experience in quantitatively intense areas to join our team. Youll be working with varied and diverse teams to deliver unique and unprecedented solutions across all industries. In the data scientist track, you will be primarily responsible for managing and delivering analytics projects and helping teams design analytics solutions and models that consistently drive scalable high-quality solutions. Your responsibilities will include Working alongside Oliver Wyman consu...

Your future duties and responsibilities: Lead data bricks developer with 7-9 years of experience We are seeking a skilled Azure Databricks Developer to design, develop, and optimize big data pipelines using Databricks on Azure The ideal candidate will have strong expertise in PySpark, Azure Data Lake, and data engineering best practices in a cloud environment Key Responsibilities : Design and implement ETL/ELT pipelines using Azure Databricks and PySpark Work with structured and unstructured data from diverse sources (e.g, ADLS Gen2, SQL DBs, APIs) Optimize Spark jobs for performance and cost-efficiency Collaborate with data analysts, architects, and business stakeholders to understand data ...

Role & responsibilities strong development experience in Snowflake, Cloud (AWS, GCP), SCALA, Python, Spark, Big Data and SQL. • Work closely with stakeholders, including product managers and designers, to align technical solutions with business goals. • Maintain code quality through reviews and make architectural decisions that impact scalability and performance. • Performs Root cause Analysis for any critical defects and address technical challenges, optimize workflows, and resolve issues efficiently. • Expert in Agile, Waterfall Program/Project Implementation. • Manages strategic and tactical relationships with program stakeholders. • Successfully executing projects within strict deadlines...

Job Summary: CGI is looking for a Lead Big Data Developer who will design, build, and optimize large-scale data pipelines and processing systems. The ideal candidate is a hands-on technologist with deep expertise in Python, PySpark, and SQL, capable of leading data initiatives, mentoring team members, and ensuring delivery of high-performance data solutions across the organization. Your future duties and responsibilities: Lead the design and development of scalable, efficient, and reliable data pipelines using PySpark, Python, and SQL Collaborate with data architects, analysts, and business stakeholders to understand data requirements and translate them into technical solutions Optimize data...

Position Description: Technologies Used: Odoo Platform v15+ Based Development Experience with Odoo development and customization Odoo User base (Logged-in users) - 1000 Users Odoo on Kubernetes (Microservices Based Architecture) with DevOps understanding Knowledge of Odoo modules, architecture, and APIs Ability to integrate Odoo with other systems and data sources Capable of creating custom modules Scale Odoo deployments for a large number of users and transactions Programming Languages: Proficiency in Python is essential Experience with other programming languages (e.g, Java, Scala) is a plus Data Analysis and Reporting: Ability to analyse and interpret complex data sets. Your future duties...

Job description We are seeking a highly skilled and motivated Big Data Engineer to join our data engineering team. The ideal candidate will have hands-on experience with Hadoop ecosystem, Apache Spark, and programming expertise in Python (PySpark), Scala, and Java. You will be responsible for designing, developing, and optimizing scalable data pipelines and big data solutions to support analytics and business intelligence initiatives. Qualification We are seeking a highly skilled and motivated Big Data Engineer to join our data engineering team The ideal candidate will have hands-on experience with Hadoop ecosystem, Apache Spark, and programming expertise in Python (PySpark), Scala, and Java...

Job description Position Summary The ETL Tester is responsible for validating and verifying data extraction transformation and loading processes to ensure data accuracy and integrity This role involves designing and executing test cases identifying data quality issues and collaborating with data engineers to resolve discrepancies The ETL Tester will also develop automated testing frameworks to enhance testing efficiency and coverage A strong understanding of ETL processes and data warehousing concepts is essential for success in this role Minimum Qualification At least 6 years of IT experience with minimum 4 years of experience in below skills Work experience in ETL testing Proficiency in SQ...

Job description Design and execute test plans for ETL processes, ensuring data accuracy, completeness, and integrity. Develop automated test scripts using Python or R for data validation and reconciliation. Perform source-to-target data verification, transformation logic testing, and regression testing. Collaborate with data engineers and analysts to understand business requirements and data flows. Identify data anomalies and work with development teams to resolve issues. Maintain test documentation, including test cases, test results, and defect logs. Participate in performance testing and optimization of data pipelines. Required Skills & Qualifications: Strong experience in ETL testing acr...

Role Overview: As a highly skilled Senior Vice President (SVP) in the Applications Development Group, your role is crucial in leading and overseeing the design, development, and implementation of scalable and robust technology solutions. You will provide hands-on technical expertise in Java, Python, and Database management to guide teams and ensure high-quality deliverables. Your deep understanding of modern technologies will drive innovation and technical excellence across the organization. Collaboration with cross-functional teams to define technical strategies aligned with business objectives is key, as well as mentoring and guiding technical teams to foster a culture of innovation and co...

Job Description: You will play a crucial role in evangelizing and demonstrating the value and impact of analytics for informed business decision making. Your responsibilities will include developing and deploying analytical solutions, as well as providing data-driven insights to business stakeholders to understand and solve various lending business nuances. Key Responsibilities: - Utilize your 2-5 years of experience in the analytics domain, specifically in analytics consulting and/or BFSI industry - Hold a Bachelors or Masters degree in statistics, economics, engineering, math, or relevant quantitative discipline with a strong academic performance - Demonstrate strong expertise in machine l...

Job Description: As the Head of GPS Data Platform at HSBC, you will have the opportunity to work closely with key Engineering leadership to develop and implement the roadmap for data platforms. Your responsibilities will include ensuring the upkeep and availability of systems to meet operational requirements with a strong focus on Risk and Control governance. You will also be accountable for finances, application design, development, testing, production implementation, and support/maintenance. Your role will involve providing thought leadership on DevOps and Agile development methodologies, innovation, and transformation, as well as managing cross-skilling and up-skilling initiatives within ...

As a Data Engineer, you will work with multiple teams to deliver solutions on the Azure Cloud using core cloud data warehouse tools (Azure Data Factory, Azure Databricks, Azure SQL DW and other Big Data related technologies). You will be responsible for developing the next generation of application data platforms and/or enhancing recent implementations. Your role will involve analyzing data, developing strategies for populating data lakes, and coding complex solutions using U-SQL, Scala or Python, and T-SQL. Responsibilities: - Work as part of a team to develop Cloud Data and Analytics solutions - Participate in development of cloud data warehouses, data as a service, and business intelligen...