9690 Big Data Jobs - Page 8

Data Support Engineer Location: Bangalore Experience 6+yrs Rate 30 LPA AMVikas POC:Swati Patil Key Responsibilities: Database Development & Support: Write and optimize SQL queries, stored procedures, and views for data retrieval and transformation. Develop and maintain data pipelines to support business intelligence and analytics requirements. Support SQL Server and Amazon Redshift environments for data storage, transformation, and analytics. Ensure data integrity, security, and quality across all database solutions. Operational Support: Monitor ETL logs and troubleshoot data pipeline issues to minimize downtime. Perform data validation and reconciliation to ensure data accuracy. Maintain Ex...

JR REQ--- Data Engineer --6to8year---PUNE, THANE----Bhalchandra Suryawanshi ---TCS C2H ---900000

Experienced data modelers, SQL, ETL, with some development background to provide defining new data schemas, data ingestion for Adobe Experience Platform customers. Interface directly with enterprise customers and collaborate with internal teams. 10+ years of strong experience with data transformation & ETL on large data sets/5+ years of Data Modelling experience (i.e., Relational, Dimensional, Columnar, Big Data) 5+ years of complex SQL or NoSQL experience Experience in advanced Data Warehouse concepts.

Role; C2H Location: PAN India 1. Mandatory Skills - Big Data, Data Bricks, SQL, Python, Databricks Sr Data Engineer JD as below-Must have 8+ years of total experience. Hands-on experience in SQL, Python, Pyspark, AWS Databricks. Good knowledge of Big Data, Data Warehousing concepts. Good knowledge of GIT, CI/CD Customer-focused, react well to changes, work with teams and able to multi-task. Must be a proven performer and team player that enjoy challenging assignments in a high-energy, fast growing and start-up workplace. Must be a self-starter who can work well with minimal guidance and in fluid environment. Act as a technical mentor and guide/support junior team members technically. 2. Mand...

Summary Within this role, you would be a hands-on experience in data engineering functions including schema design, data movement, data transformation, encryption, and monitoring: all the activities needed to build, sustain, and govern big data pipelines. Responsibilities Own development of large-scale data platform including operational data store, real time metrics store and attribution platform, data warehouses and data marts for advertising planning, operation, reporting and optimization Wider team collaboration and system documentation Maintain next-gen cloud based big data infrastructure for batch and streaming data applications, and continuously improve performance, scalability and av...

JD GCP Data Engineer Responsibilities: Design, develop, and maintain data pipelines using GCP services such as Dataflow, BigQuery, and Pub/Sub. Implement and manage data warehouse solutions using BigQuery, ensuring data is stored securely and efficiently. Use Pub/Sub for real time data ingestion and streaming analytics. Provision and manage GCP infrastructure using Terraform, ensuring best practices in IaC are followed. Optimize data storage and retrieval processes to enhance performance and reduce costs. Monitor and troubleshoot data pipeline issues, ensuring high availability and reliability of data services. Ensure data quality and integrity through robust testing and validation processes...

Azure Databricks: 1. Mounting Files 2. PAT Tokens 3. Cluster Auto scaling 4. Cluster Types 5. Nesting notebooks 6. Reading file using default and custom schema 7. Types Secrets and scopes in databricks 8. Delta Tables 9. Basic file loading scenario with its steps 10. Optimization 11. User defined functions 12. Transformations ADF: 1. Integration runtimes familiar with and where they are used 2. Components in creating pipelines 3. Scenario based question for creating a basic pipeline for loading data to Azure SQL table 4. Basic steps to improve the loading activity of a huge amount of file , ex: 1TB file size

Roles and Responsibilities Troubleshoot issues related to EMR clusters, Spark jobs, and other big data technologies. Develop complex SQL queries to extract insights from massive datasets stored in Amazon Redshift. Collaborate with cross-functional teams to gather requirements and design solutions that meet business needs. Ensure high availability and performance of the system by monitoring logs, metrics, and alerts. Desired Candidate Profile 5-8 years of experience in developing scalable big data solutions on AWS platform. Strong expertise in working with Spark/Hive/Pyspark/Python programming languages. Proficiency in writing complex SQL queries for querying large datasets stored in Amazon R...

The Data Analyst is responsible for analyzing, managing, and visualizing data across the organization’s Identity and Access Management (IAM) ecosystem. The role focuses on generating actionable insights from identity data to enhance user security, streamline access management, and ensure compliance with organizational and regulatory standards. Roles and Responsibilities Lead the design and implementation of solutions within the team, ensuring compliance with design standards and alignment with business objectives. Analyze existing business models, data flows, and system architectures to develop functional specifications and test plans. Create high-level and detailed process models, sequence ...

About the Role We are seeking an experienced Big Data & Azure Engineer to join our data platform team. This role is critical to supporting and optimizing our enterprise-scale data infrastructure, managing migrations from on-premises Big Data systems to Azure cloud services, and ensuring seamless 24x7 production operations. The ideal candidate will combine strong technical expertise in both traditional Big Data technologies and modern cloud platforms with excellent problem-solving abilities. Required Skills and Qualifications Technical Expertise : Strong experience in Big Data technologies (Hive, Impala). Hands-on experience with Azure services : Network, VMs, Synapse, Databricks. Proficiency...

Notice Period: Immediate to 30 Days Mandatory Skills Data Science & Machine Learning Python & SQL Marketing Analytics / Retail / CPG Domain Experience AWS (Cloud & Big Data tools) Power BI Responsibilities Lead and deliver Data Science / ML models for Marketing Analytics, Retail, and CPG domains. Work on large datasets using Python, SQL, PySpark, and AWS cloud platforms. Apply supervised & unsupervised learning techniques (classification, regression, clustering, ensemble models, neural networks). Build and deploy predictive & prescriptive models for business optimization. Deliver insights via Power BI dashboards and storytelling. Collaborate with stakeholders to define business problems and ...

Job Title: Bigdata + Pyspark Location: Hyderabad Experience Level: 5-8 Years Notice Period: Immediate to 30 Days I nterview Mode : L1 Virtual / L2 F2F (Mandatory) Work Mode : Hybrid Job Summary We are seeking a Specialist with 5 to 8 years of experience in the Big Data Hadoop Ecosystem to design and implement scalable data solutions. Detailed Job Description Work extensively with the Big Data Hadoop Ecosystem to process and analyze large datasets. Utilize Hadoop, Spark, and SparkSQL technologies to build efficient and high-performance data pipelines. Leverage Python for data manipulation, automation, and integration tasks. Collaborate with cross-functional teams to gather data requirements a...

Python, Scala, or Java for data processing. Experience with big data tools such as Apache Spark, Hadoop, or Kafka. (AWS, Azure, or Google Cloud). Snowflake, Redshift, BigQuery, Databricks) Cloudera, HDFS, Hive, Kafka, Spark, ETL, Hadoop, HBase

Role & responsibilities Job Title: Developer Skills Required: Digital: BigData and Hadoop Ecosystems, Digital: PySpark Experience Range in Required Skills: 4 to 6 Years Job Description: Big Data (PySpark, HIVE) 1. Need to work as a developer in Bigdata, Hadoop or Data Warehousing Tools and Cloud Computing. 2. Work on Hadoop, Hive SQLs, Spark, and Bigdata Eco System Tools. 3. Experience in working with teams in a complex organization involving multiple reporting lines.? 4. The candidate should have strong functional and technical knowledge to deliver what is required and he/she should be well acquainted with Banking terminologies. 5. The candidate should have strong DevOps and Agile Developme...

Are you ready to turn data into impact ? Vanderlande is looking for a Data & AI Engineer passionate about Machine Learning, GenAI, RAG, and LLM technologies . You will design and build scalable Azure-based data platforms , develop ML models , and work on cutting-edge AI solutions that power global logistics automation. Role & responsibilities Design, build, and maintain Azure-based data platform . Develop and manage data pipelines using Azure and Databricks. Build and deploy secure, scalable ML models (Python, PySpark, SQL). Process unstructured data (e.g., parsing, Retrieval-Augmented Generation). Implement and maintain CI/CD pipelines for data and ML workloads. Ensure platform performance ...

About The Opportunity : We are seeking a highly skilled Lead Data Engineer with strong expertise in Python, Pandas, PySpark, AWS, and SQL to design, build, and manage scalable data solutions. The ideal candidate will lead a team of data engineers, develop robust ETL pipelines, and collaborate with analytics, data science, and business teams to ensure high data quality and performance across cloud-based environments. Role & Responsibilities : - Lead the design and development of data pipelines for ingestion, transformation, and integration from multiple sources into the enterprise data platform. - Implement data quality frameworks, validation checks, and monitoring solutions using Python and ...

4 to 10 years of hands-on data engineering experience. •Strong expertise in Apache Spark (batch + streaming) and Hive. •Proficiency in Python, Scala, or Java. •Knowledge of orchestration tools (Airflow / Control-M) and SQL transformation frameworks (DBT preferred). •Experience working with Kafka, Solace, and object stores (S3, MinIO). •Exposure to Docker/Kubernetes for deployment. •Hands on experience of data Lakehouse formats (Iceberg, Delta Lake, Hudi). Interested can share your resume to uthirakumar.k@tekskills.in

You are seeking a Spark, Big Data - ETL Tech Lead for Commercial Cards Global Data Repository development team at Citigroup. You will be interacting with various teams within Citigroup and need to have exceptional communication skills across technology and business areas. As a technical lead, you will design and implement large-scale data processing pipelines using Apache Spark on the BigData Hadoop Platform. Your responsibilities will include developing and optimizing Spark applications, providing technical leadership for global software solutions, integrating data from various sources, and managing development scope, budgets, and timelines. You will also mentor and guide junior developers ...

Role Overview: You will be working as a Data Analyst with a strong background in banking, regulatory reporting, and financial services. Your main responsibilities will include analyzing complex business requirements, ensuring data quality, and providing solutions that align with organizational goals. You will work closely with stakeholders to understand their needs and translate them into functional system design documents. Key Responsibilities: - Analyze and translate complex business requirements into functional system design documents. - Perform technology and system analysis to ensure business needs are met within SVB standards. - Conduct data mapping, data quality checks, and data profi...

As an Associate Data Architect at Quantiphi, you will play a crucial role in designing and delivering big data pipelines for structured and unstructured data across multiple geographies, specifically focusing on supporting healthcare organizations in achieving their business objectives through the utilization of data ingestion technologies, cloud services, and DevOps practices. Your collaboration with Architects specializing in Cloud engineering, Software engineering, and ML engineering will involve creating innovative platforms, solutions, and applications that align with the latest trends in the healthcare industry, including digital diagnosis, software as a medical product, and AI marketp...

Greetings From Tata Consultancy Services!!! Role: Big Data Developer (Hive, Pyspark) Years of Experience: 4 to 10 yrs Location: Chennai, Hyderabad, Bengaluru Notice Period: 0- 90 days Date of Interview: 14th Nov, 2025 Mode of Interview: Virtual Drive(Cisco or Webex) Time Of Interview: 5.00PM To 7.00PM Job Description: Roles & Responsibilities: Hands-on experience of Hadoop, Python, PySpark, Hive, Big Data Eco System Tools. Should be able to develop, tweak queries and work on performance enhancement. Solid understanding of object-oriented programming and HDFS concepts The candidate will be responsible for delivering code, setting up environment, connectivity, deploying the code in production ...

We are looking for a dynamic professional who brings a unique blend of software development expertise and business analysis acumen. The ideal candidate will have hands-on experience in coding and system design, coupled with a strong understanding of business processes and stakeholder management. Key Responsibilities: Collaborate with stakeholders to gather and analyze business requirements. Translate business needs into technical specifications and solutions. Participate in system design, development, and testing phases. Work closely with development teams to ensure business requirements are met. Create and maintain documentation including BRDs, FRDs, and user stories. Conduct gap analysis a...

We are seeking a highly skilled and experienced Senior Data Engineer with strong expertise in Streaming Data and Big Data Technologies , AWS & Scala Mandatory . The ideal candidate will have hands-on experience in architecting and implementing scalable data solutions using modern cloud-native tools and frameworks. Key Responsibilities: Architect and implement scalable data pipelines for real-time, batch, structured, and unstructured data . Design and develop solutions using AWS services such as EMR, Kinesis, Glue, Athena, S3, CloudFormation, API Gateway . Work extensively with streaming platforms like Kafka, Flink, Spark Streaming . Develop and optimize data ingestion workflows using Apache ...

As a Data Scientist at + +, you will be responsible for delivering end-to-end vis-a-vis projects in the Analytics space. Your role will involve identifying valuable data sources, supervising data preprocessing, analyzing large datasets, building predictive models, and collaborating with different teams to improve business outcomes. Your passion for discovering solutions hidden in large datasets will be crucial for this position. **Key Responsibilities:** - Identify valuable data sources and collection processes - Supervise preprocessing of structured and unstructured data - Analyze large amounts of information to discover trends and patterns for the insurance industry - Build predictive mode...

Roles and Responsibilities Design, develop, test, deploy, and maintain Collibra solutions for clients. Collaborate with cross-functional teams to identify business requirements and implement data governance frameworks. Ensure seamless integration of Collibra DAM with other systems through APIs and web services. Provide technical guidance on best practices for data management and compliance. Desired Candidate Profile 3-8 years of experience in Collibra development or related field (data governance, big data). Strong understanding of Collibra platform architecture and its components (e.g., Workbench, Access Management). Location: Pan India