12368 Big Data Jobs - Page 25

immediate joiners only Senior Data Engineer (Airflow & PostgreSQL Specialist) Role Overview We are looking for a heavy-hitting Senior Data Engineer to lead the architecture and optimization of our data ecosystem. With a core focus on Apache Airflow and PostgreSQL , you will be responsible for building industrial-grade data pipelines that are scalable, self-healing, and highly performant. This is a role for an engineer who thrives on solving complex orchestration challenges and fine-tuning relational databases for high-volume analytical workloads. Key Responsibilities 1. Advanced Airflow Orchestration (Core) Architect and manage complex Airflow DAGs , moving beyond basic scheduling to impleme...

As a Big Data Engineer specializing in the Hadoop Ecosystem, your primary role will involve designing, deploying, and maintaining Hadoop ecosystem clusters (HDFS, YARN) to ensure stability, scalability, and security for production workloads. You will be responsible for developing and optimizing batch ETL pipelines using Spark, Hive, Sqoop, and related tools to ingest, transform, and store large datasets. Additionally, you will build and operate streaming data pipelines using Kafka to guarantee low-latency delivery and fault tolerance. Your duties will also include performance tuning, capacity planning, and cluster troubleshooting, such as resource contention, job optimization, and storage fo...

Experience: 2+ years of experience in combined Data Engineering and Data Analyst roles. Technical Skills: Proficiency in SQL, Python and Big data technologies (PySpark, Hive, Hadoop) Strong understanding of data pipeline Familiarity with data visualization toolsmunication Skills: Ability to communicate complex technical concepts. Strong collaborative and team-oriented mindset

About The Role Project Role : Application Lead Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills : Business Requirements Analysis Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Lead, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. Your typical day will involve collaborating with various stakeholders to gather and analyze business requirements, ensuring that the applications meet the needs of the organization. You will also oversee t...

Your role and responsibilities As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform * Responsibilities * Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS * Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on ...

What you'll do Parse and manipulate raw data leveraging tools including R, Python, Tableau, with a strong preference for Python Ingest, understand, and fully synthesize large amounts of data from multiple sources to build a full comprehension of the story Analyze large data sets, while finding the truth in data, and develop efficient processes for data analysis and simple, elegant visualization Develop and automate daily, monthly, quarterly reporting for multiple business areas within Viasat Identifies data gaps, researches methods to fill these gaps and provide recommendations Gather and analyze facts and devise solutions to administrative problems Monitor big data with Business Intelligenc...

Key Responsibilities Collect, clean, and preprocess structured and unstructured data Perform exploratory data analysis (EDA) to identify trends and patterns Build, validate, and deploy statistical and machine learning models Develop predictive and prescriptive analytics solutions Communicate insights through dashboards, reports, and presentations Collaborate with product, engineering, and business teams Monitor model performance and retrain models as needed Ensure data quality, integrity, and governance best practices Master s or PhD in a quantitative field Experience with big data tools (Spark, Hadoop) Knowledge of cloud platforms (AWS, GCP, Azure) Experience deploying models into productio...

Role & responsibilities Project & Program Delivery • Own end-to-end delivery of large-scale enterprise data and AI programs across multiple workstreams. • Ensure on-time, on-budget, and high-quality delivery using Agile / Hybrid methodologies. • Drive sprint planning, PI planning, backlog grooming, release management, and delivery governance. Technical Leadership • Lead delivery of solutions using: Big Data & Analytics: Hadoop, Spark, Kafka, Hive, Presto Cloud Platforms: AWS, Azure. GCP (data & AI services) Modern Data Platforms: Databricks, Snowflake AI & Advanced Analytics: GenAI, LLMs, Agentic AI frameworks, MLOps • Collaborate with architects to ensure scalable, secure, and performant da...

About The Role Project Role : Custom Software Engineer Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills : Java Full Stack Development Good to have skills : NA Minimum 15 year(s) of experience is required Educational Qualification : 15 years full time education Position Title:Full Stack Enterprise Architect Overview:We are seeking a seasoned Full Stack Enterprise Architect with deep expertise in designing and implementing application-heavy, multi-cloud enterprise soluti...

About The Role Project Role : Custom Software Engineer Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills : Java Full Stack Development Good to have skills : NA Minimum 12 year(s) of experience is required Educational Qualification : 15 years full time education Position Title:Full Stack Enterprise Architect Overview:We are seeking a seasoned Full Stack Enterprise Architect with deep expertise in designing and implementing application-heavy, multi-cloud enterprise soluti...

About The Role Project Role : Custom Software Engineer Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills : Java Full Stack Development Good to have skills : NA Minimum 15 year(s) of experience is required Educational Qualification : 15 years full time education Position Title:Full Stack Enterprise Architect Overview:We are seeking a seasoned Full Stack Enterprise Architect with deep expertise in designing and implementing application-heavy, multi-cloud enterprise soluti...

About The Role Project Role : Custom Software Engineer Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills : Data Engineering Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will design, build, and configure applications to meet business process and application requirements. A typical day involves collaborating with various teams to understand their need...

Job Title Technical Project Delivery Manager Enterprise Data Solutions Location Bangalore Experience 8–10+ years (with strong technical delivery background) Role Summary We are seeking a Technical Project Delivery Manager to lead the end-to-end delivery of enterprise-scale data, analytics, and AI solutions . This role requires a strong blend of technical depth, delivery leadership, stakeholder management, and agile execution across modern data platforms including Big Data, Cloud, Analytics, and GenAI/Agentic AI ecosystems. The ideal candidate will manage complex, multi-stream programs, drive architectural alignment, ensure on-time and high-quality delivery, and act as a bridge between busine...

Oversee the delivery of data engagements across a portfolio of client accounts, understanding their specific needs, goals & challenges Provide mentorship & guidance for the Architects, Project Managers, & technical teams for data engagements Required Candidate profile 8+ years of experience and should be hands on in Data Architecture

About The Role Project Role : Custom Software Engineer Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills : Python (Programming Language) Good to have skills : PySpark Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer on the Skywise platform, you will design, build, and maintain scalable data pipelines that ingest, process, and transform large volumes of aviation and operational data. You will...

About The Role Project Role : Custom Software Engineer Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills : Java Full Stack Development Good to have skills : NA Minimum 12 year(s) of experience is required Educational Qualification : 15 years full time education Position Title:Full Stack Enterprise Architect Overview:We are seeking a seasoned Full Stack Enterprise Architect with deep expertise in designing and implementing application-heavy, multi-cloud enterprise soluti...

About The Role Project Role : Custom Software Engineer Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills : Data Engineering Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Lead, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. Your typical day will involve collaborating with various teams to ensure project...

About The Role Project Role : Custom Software Engineer Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills : Data Engineering Good to have skills : Machine Learning Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Software Development Lead, you will be responsible for developing and configuring software systems, either end-to-end or for specific stages of the product lifecycle. Your typical day will involve...

Job Summary We are seeking a Product Solution Architect with strong expertise in designing and delivering AWS-based big data solutions . The role focuses on building scalable, secure, and high-performance data platforms while providing technical leadership to data engineering teams. Key Responsibilities Design and architect end-to-end big data solutions on AWS aligned with best practices. Collaborate with product, engineering, and business teams to translate requirements into technical solutions. Lead implementation from design to production, ensuring performance, scalability, and cost optimization. Ensure data security, compliance, quality, and governance across data pipelines. Mentor data ...

Notice Period-Immediate-10 days The Data Engineer will lead end-to-end Big Data architecture design and build scalable data platforms that support high-volume, high-velocity data. The role emphasizes performance tuning of large-scale systems to deliver low-latency, cost-efficient data processing and analytics. The engineer will be hands-on with SQL and Python to design, implement, and optimize robust data pipelines. They will also strengthen data governance and security practices and explore AI-driven techniques to enhance data quality and processing efficiency. Required Qualifications Big Data architecture design Performance tuning for large-scale systems Advanced SQL (query design, optimiz...

JOB DISCRIPTION Experience: 4+Years Location: Hyderabad/pune Must have skill - Pyspark + GCP - The must-have skills are deep ETL tool experience, PySpark with GCP pipeline expertise, advanced SQL, Python. Description: At least 3-5 years of experience in Pyspark with GCP (Airflow, Dataproc, Big query) capable of configuring data pipelines. Strong Experience in writing complex SQL queries to perform data analysis on Databases SQL server, Oracle, HIVE etc. Possess the following technical skills SQL, Python, Pyspark, Hive, ETL, Unix, Control-M (or similar scheduling tools) Ability to work independently on specialized assignments within the context of project deliverables Take ownership of provid...

As a Solution Architect at our company, your role will involve developing effective optimized IT solutions that meet customer needs and constraints. You will be responsible for ensuring the robustness of designs across various parameters such as performance, security, usability, and scalability. It is essential to have a strong spirit of innovation, be a self-starter, and have the ability to come up with solutions to complex problems. Additionally, you will proactively develop reusable frameworks and assets to enhance the competitive advantage of the solution. Your responsibilities will include maintaining a big picture of the customer enterprise systems landscape, analyzing current processe...

Strong experience in programming languages such as Python, Java, or Scala for data manipulation and engineering tasks. Expertise in SQL and NoSQL databases. Hands-on experience with big data technologies like Hadoop, Spark, Kafka, and Hive to handle large-scale data processing and real-time data streams. In-depth knowledge of data warehousing solutions such as Amazon Redshift, Google BigQuery, and Snowflake for building and managing data warehouses. Proficiency in designing, developing, and maintaining ETL (Extract, Transform, Load) processes using tools like Apache NiFi, Talend, or Informatica. Familiarity with cloud platforms like AWS, Azure, or Google Cloud for deploying, managing, and sc...

We are looking for a skilled Senior Data Engineer to join our team, responsible for designing and implementing large-scale data systems. The ideal candidate will have a strong background in data engineering and excellent problem-solving skills. Roles and Responsibility Design and implement scalable data pipelines using various technologies. Collaborate with cross-functional teams to identify business requirements and develop solutions. Develop and maintain large-scale data systems, ensuring high performance and reliability. Troubleshoot complex data-related issues and provide timely resolutions. Implement data quality checks and ensure data integrity. Stay updated with industry trends and em...

We are looking for a highly skilled and experienced Senior Architect to lead our Data Engineering team, responsible for designing and implementing large-scale data systems. The ideal candidate will have a strong background in data engineering and architecture, with excellent leadership and communication skills. Roles and Responsibility Design and implement scalable and efficient data pipelines and architectures. Lead the development of data engineering solutions, including data warehousing and analytics. Collaborate with cross-functional teams to identify business requirements and develop technical solutions. Develop and maintain technical documentation for data engineering projects. Mentor ...