12395 Big Data Jobs - Page 32

Role: Data Engineer Scala No. of Positions: 3 Experience: 4 - 8 Years Employment Type: Permanent / contractual Work Mode: Onsite (5 Days Work from Office) Work Location: Hyderabad / Bangalore Work Timings: 11:00 AM 8:00 PM IST Role Overview: We are looking for an experienced Data Engineer (Scala) to join our team in a client-facing, individual contributor role . The ideal candidate should have strong hands-on experience in Scala and PySpark and be comfortable working in an onsite, fast-paced environment. Key Responsibilities: Design, develop, and optimize scalable data processing solutions using Scala and PySpark Build and maintain robust data pipelines for large-scale data processing Work c...

Overview: The Azure Data Engineer / Data Architect will be responsible for designing, developing, and managing scalable data solutions on Microsoft Azure. You will ensure that data is accessible, optimized for performance, and compliant with organizational and security standards. You will have hands-on expertise with Azure services, strong ETL development skills, and the ability to architect modern data platforms that support business intelligence, analytics, and advanced data science initiatives. You will operate in a Scrum/Agile environment, collaborating closely with cross-functional teams to deliver value iteratively. Core Responsibilities: •Design, build, and maintain scalable and relia...

Desired Competencies (Technical/Behavioral Competency) Must-Have** (Ideally should not be more than 3-5) Strong proficiency in Python programming. Hands-on experience with PySpark and Apache Spark . Knowledge of Big Data technologies (Hadoop, Hive, Kafka, etc.). Experience with SQL and relational/non-relational databases. Familiarity with distributed computing and parallel processing . Understanding of data engineering best practices. Experience with REST APIs , JSON/XML , and data serialization. Exposure to cloud computing environments. Good-to-Have 5+ years of experience in Python and PySpark development. Experience with data warehousing and data lakes . Knowledge of machine learning libra...

As a Java Spark Developer, your role will involve working with various technologies and tools to develop efficient solutions for processing and analyzing large volumes of data. Here are the key responsibilities and qualifications required for this position: **Key Responsibilities:** - Experience in Java programming languages - Proficiency in Big Data technologies including Spark Java and Kafka - Understanding of organizational strategy architecture patterns, Microservices, and Event Driven architecture - Ability to coach the team and align project execution with organizational guidelines - Hands-on experience in working with large volumes of data for ingestion, processing, and storage - Prof...

We are seeking a highly skilled and experienced Senior Data Engineer with strong expertise in Streaming Data and Big Data Technologies , AWS & Scala Mandatory . The ideal candidate will have hands-on experience in architecting and implementing scalable data solutions using modern cloud-native tools and frameworks. Key Responsibilities: Architect and implement scalable data pipelines for real-time, batch, structured, and unstructured data . Design and develop solutions using AWS services such as EMR, Kinesis, Glue, Athena, S3, CloudFormation, API Gateway . Work extensively with streaming platforms like Kafka, Flink, Spark Streaming . Develop and optimize data ingestion workflows using Apache ...

As a Big Data Engineer located in Hyderabad, you will be responsible for building scalable data solutions using your strong Python expertise. The role involves working at the customer location with a hybrid model of 3 days in the office. **Key Responsibilities:** - Utilize Python for data processing - Work with Big Data technologies such as Hive, Spark or Snowflake, and Kafka - Proficiency in SQL and SQLServer **Qualifications Required:** - Mandatory Skills: - Python (for data processing) - Big Data: Hive, Spark or Snowflake, Kafka - SQL / SQLServer - Good to Have: - AWS - API development - C# - Ideal Candidate Profile: - 3-5 years of experience as a Software Engineer - Full stack developmen...

As an Analytics Consultant at EY, you will be part of a globally connected powerhouse of diverse teams that aim to build a better working world. Your role will involve leveraging your expertise in Analytics and HR consulting to lead global HR analytics projects and address people-related challenges. To qualify for this position, you should meet the following requirements: - 1-5 years of experience in Analytics and HR consulting domain - Minimum 2 years of experience in leading global HR analytics projects, including predictive analytics, modelling, descriptive analysis, logistical and regression analysis - Masters degree in Analytics or HR, full-time MBA/EMBA - Experience in master data mana...

Job description Hiring for Hadoop Developer Mandatory Skills: Hadoop, bigdata, spark Responsibilities A day in the life of an Infoscion As part of the Infosys consulting team, your primary role would be to actively aid the consulting team in different phases of the project including problem definition, effort estimation, diagnosis, solution generation and design and deployment You will explore the alternatives to the recommended solutions based on research that includes literature surveys, information available in public domains, vendor evaluation information, etc. and build POCs You will create requirement specifications from the business needs, define the to-be-processes and detailed funct...

Big Data Orchestration: Airflow, Spark on Kubernetes, Yarn, Oozie Big Data Processing: Hadoop, Kafka, Spark & Spark Structured Streaming. Experience on SOLID & DRY principles with Good Software Architecture & Design implementation experience ¢ Advanced Scala experience (e.g. Functional Programming, using Case classes, Complex Data Structures & Algorithms) ¢ Proficient in developing automated frameworks for unit & integration testing. ¢ Proficient with Kubernetes, Docker, Helm and related container technologies. ¢ Proficient in deploying and managing Spark workloads on Kubernetes clusters. ¢

We are hiring an experienced Big Data Hadoop Developer to join our growing engineering team. The ideal candidate will have strong hands-on experience in the Hadoop ecosystem and be responsible for designing, developing, and maintaining scalable big data systems and ETL pipelines. As a Big Data Hadoop Developer, you will work closely with data architects, analysts, and business stakeholders to build reliable and high-performance data solutions. You will be involved in processing large datasets using Hadoop, Spark, and Hive while ensuring data quality, availability, and performance. Key Responsibilities: Design, develop, and maintain scalable big data systems and data pipelines. Implement data...

Position Overview: We are seeking a highly skilled and tactically focused Technical Product Owner to play a critical role in supporting our data engineering initiatives, particularly in the ETL, big data, and integrations space. This role is critical to optimizing the value of the development teams work, managing the backlog, and ensuring seamless execution and delivery of enterprise cross-functional initiatives. In this role, you will collaborate with cross-functional teams, manage the product backlog, and ensure smooth execution of data engineering initiatives. ShyftLabs is a growing data product company that was founded in early 2020 and works primarily with Fortune 500 companies. We deli...

Senior Data Platform Engineer - Data platform You will be part of: Myntra Data Platform - Analytics Myntra is a one stop shop for all your fashion and lifestyle needs. Being India''s largest online store for fashion and lifestyle products, Myntra.com aims at providing a hassle free and enjoyable shopping experience to shoppers across the country with the widest range of brands and products on offer. The brand is making a conscious effort to bring the power of fashion to shoppers with an array of the latest and trendiest products available in the country. Myntra''s cloud based big data platform is highly scalable and processes over 7 billion events per day. We are on a journey to modernize ou...

About the Opportunity Job TypeApplication 20 June 2025 Title Data Scientist, Risk Data Analytics Department Data Value Location Gurgaon Reports To Associate Director, Risk Data Analytics Level 5 Were proud to have been helping our clients build better financial futures for over 50 years. How have we achieved thisBy working together - and supporting each other - all over the world. So, join our Data Value team and feel like youre part of something bigger. About Global Risk The Global Risk team in Fidelity covers the management oversight of Fidelitys risk profile, including key risk frameworks, policies and procedures and oversight and challenge processes. The team partner with the businesses ...

As a Data Engineer at IBM, you'll play a vital role in the development, design of application, provide regular support/guidance to project teams on complex coding, issue resolution and execution. Your primary responsibilities include: Lead the design and construction of new solutions using the latest technologies, always looking to add business value and meet user requirements. Strive for continuous improvements by testing the build solution and working under an agile framework. Discover and implement the latest technologies trends to maximize and build creative solutions Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Ex...

Skilled Multiple GCP services - GCS, BigQuery, Cloud SQL, Dataflow, Pub/Sub, Cloud Run, Workflow, Composer, Error reporting, Log explorer etc. Must have Python and SQL work experience & Proactive, collaborative and ability to respond to critical situation Ability to analyse data for functional business requirements & front face customer Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise 5 to 7 years of relevant experience working as technical analyst with Big Query on GCP platform. Skilled in multiple GCP services - GCS, Cloud SQL, Dataflow, Pub/Sub, Cloud Run, Workflow, Composer, Error reporting, Log explorer You love colla...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

As Data Engineer at IBM you will harness the power of data to unveil captivating stories and intricate patterns. You’ll contribute to data gathering, storage, and both batch and real-time processing. Collaborating closely with diverse teams, you’ll play an important role in deciding the most suitable data management systems and identifying the crucial data required for insightful analysis. As a Data Engineer, you’ll tackle obstacles related to database integration and untangle complex, unstructured data sets. In this role, your responsibilities may include: * Implementing and validating predictive models as well as creating and maintain statistical models with a focus on big data, incorporat...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

As a Big Data Engineer, you will develop, maintain, evaluate, and test big data solutions. You will be involved in data engineering activities like creating pipelines/workflows for Source to Target and implementing solutions that tackle the clients needs. Your primary responsibilities include: Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements. Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization. Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need too. Required education Bachel...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

The ability to be a team player The ability and skill to train other people in procedural and technical topics Strong communication and collaboration skills Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Able to write complex SQL queries ; Having experience in Azure Databricks Preferred technical and professional experience Excellent communication and stakeholder management skills

As an Associate Software Developer at IBM you will harness the power of data to unveil captivating stories and intricate patterns. You'll contribute to data gathering, storage, and both batch and real-time processing. Collaborating closely with diverse teams, you'll play an important role in deciding the most suitable data management systems and identifying the crucial data required for insightful analysis. As a Data Engineer, you'll tackle obstacles related to database integration and untangle complex, unstructured data sets. In this role, your responsibilities may include: Implementing and validating predictive models as well as creating and maintain statistical models with a focus on big ...

In this role, you'll work in one of our IBM Consulting Client Innovation Centers (Delivery Centers), where we deliver deep technical and industry expertise to a wide range of public and private sector clients around the world. Our delivery centers offer our clients locally based skills and technical expertise to drive innovation and adoption of new technology. Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise The ability to be a team player The ability and skill to train other people in procedural and technical topics Strong communication and collaboration skills Able to write complex SQL queries Having experience in Azure ...

Location : Hyderabad, Narsingi Work Mode - Work from Office (5 days a week) Joining : Within a week Role & responsibilities Design, develop, and maintain optimized SQL queries for large-scale data systems. Analyze slow-running queries and implement performance tuning strategies. Rewrite and refactor complex SQL queries for improved execution efficiency. Collaborate with data engineers to design scalable database schemas and models. Develop and maintain ETL processes, data pipelines, and data warehousing solutions. Work with Big Data platforms (e.g., Hadoop, Spark, Hive) to manage and query massive datasets. Ensure data integrity, accuracy, and consistency across systems. Partner with analyti...