63 Hadoop Spark Jobs

About the Role: We are looking for a skilled and passionate Spark Developer to design, build, and optimize large-scale data processing systems. The ideal candidate will have strong expertise in Apache Spark , Big Data ecosystems , and distributed computing frameworks . Key Responsibilities: Design, develop, and maintain data processing pipelines using Apache Spark (Core, SQL, Streaming, Structured Streaming) . Implement ETL workflows for data ingestion, transformation, and integration across multiple data sources. Optimize Spark applications for performance, scalability, and reliability in distributed environments. Work with large structured and unstructured datasets to support analytical an...

Qualification Experience applying AI/ML techniques, data mining,statistical modeling, and/or pattern recognition on real-world problems Proficiency in programming languages: Python,Java,R,C++,Scala Big data expertise with tools as Spark,Hadoop,Hive

As a SF Data Cloud Architect, you will play a critical role within Salesforce's Professional Services team. Your main responsibilities will include assisting in pre-sales activities and leading the design and implementation of enterprise-grade Data Management solutions. You will be responsible for architecting scalable solutions using Data Cloud, ensuring data readiness for enterprise AI, applying data governance guardrails, and supporting enterprise analytics and automation. This role covers markets in ANZ, ASEAN, and India. Key Responsibilities: - Serve as a Salesforce Data Cloud Trusted Advisor, supporting and leading project delivery and customer engagements during the pre-sales cycle. Y...

As an SF Data Cloud Architect, you play a critical role within Salesforce's Professional Services team. Your responsibilities involve assisting in pre-sales and leading the design and implementation of enterprise-grade Data Management solutions. Your focus will be on architecting scalable solutions across enterprise landscapes using Data Cloud, ensuring data readiness for enterprise AI, applying data governance guardrails, supporting enterprise analytics and automation. This role covers ANZ, ASEAN, and India markets. **Key Responsibilities:** - **Salesforce Data Cloud Trusted Advisor:** Be a trusted Subject Matter Expert (SME) for Data Cloud to support and lead project delivery and customer ...

Purpose Design and develop scalable ETL pipelines, APIs, and automations for big data processing in an on-premise Hadoop/Spark/Airflow environment, focusing on real-time log enrichment and exposure via APIs for a visualization app. Take ownership of proposing and implementing solutions for data flow, performance, and integration challenges. Key Responsibilities Architect and implement ETL/ELT pipelines for streaming (Kafka/Spark) and batch log processing, including data enrichment (e.g., IP geolocation, user mapping) in high-volume scenarios. Build and optimize high-performance APIs (e.g., using FastAPI) to deliver processed logs and automate integrations for internal apps, ensuring low-late...

As a Big Data Engineer, you will develop, maintain, evaluate, and test big data solutions. You will be involved in data engineering activities like creating pipelines/workflows for Source to Target and implementing solutions that tackle the clients needs. Your primary responsibilities include: Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements. Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization. Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need too Required education Bachelo...

Responsibilities: Perform complex data analysis on large datasets to extract actionable insights and support decision-making processes Design implement and maintain data lake house architectures that integrate the benefits of data lakes and data warehouses Develop and manage data processing systems on cloud platforms such as AWS Google Cloud or Microsoft Azure Design and optimize ETL (Extract Transform Load) processes to ensure efficient data ingestion transformation and storage Develop and deploy machine learning models to solve business problems and enhance data-driven decision-making Work closely with cross-functional teams including data engineers data analysts and IT professionals to en...

As a Big Data Engineer, you will develop, maintain, evaluate, and test big data solutions. You will be involved in data engineering activities like creating pipelines/workflows for Source to Target and implementing solutions that tackle the clients needs. Your primary responsibilities include: Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements. Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization. Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need too. Required education Bachel...

As a Big Data Engineer, you will develop, maintain, evaluate, and test big data solutions. You will be involved in data engineering activities like creating pipelines/workflows for Source to Target and implementing solutions that tackle the clients needs. Your primary responsibilities include: Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements. Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization. Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need too Required education Bachelo...

As a Big Data Engineer, you will develop, maintain, evaluate, and test big data solutions. You will be involved in data engineering activities like creating pipelines/workflows for Source to Target and implementing solutions that tackle the clients needs. Your primary responsibilities include: Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements. Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization. Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need too Required education Bachelo...

As a Salesforce Data Cloud Architect, you play a critical role within Salesforce's Professional Services team. Your main responsibilities include assisting in pre-sales and leading the design and implementation of enterprise-grade Data Management solutions using Data Cloud. Your role involves ensuring that data is ready for enterprise AI, applying data governance guardrails, supporting enterprise analytics, and automation. This position covers the ANZ, ASEAN, and India markets. Key Responsibilities: - Serve as a Salesforce Data Cloud Trusted Advisor, supporting and leading project delivery and customer engagements during the pre-sales cycle. Provide insights on how Data Cloud contributes to ...

As a Big Data Engineer, you will develop, maintain, evaluate, and test big data solutions. You will be involved in data engineering activities like creating pipelines/workflows for Source to Target and implementing solutions that tackle the clients needs. Your primary responsibilities include: Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements. Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization. Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need too Required education Bachelo...

As a Big Data Engineer, you will develop, maintain, evaluate, and test big data solutions. You will be involved in data engineering activities like creating pipelines/workflows for Source to Target and implementing solutions that tackle the clients needs. Your primary responsibilities include: Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements. Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization. Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need too. Required education Bachel...

Job Description: Role: Big data architech Required Skills: Data Architecture, Hadoop, Spark Must-Have** • > Experience of Hadoop Big Data Eco System Tools & Technologies (HIVE, Spark/PySpark) • > Experience in Various Data Architecture patterns and techniques • > Experience with E2E Architecture layers Data Ingestion, Data Storage, Data Visualization, Governance, Analytics • > Experience with multiple Data & Analytics technologies Good-to-Have • > Experience with Cloud Data platforms like Azure, AWS • > Preferably to have good DWH/ Data Lake knowledge • > BFSI domain knowledge Responsibility of / Expectations from the Role > Perform Data Inventory and Mapping > Define and Develop Data Archit...

Develop and manage applications involved with Search Systems Drive/coordinate cross-functional projects with broad range of Loan, Insurance,investment products owners, engineers, designers Analyze website metrics and customer behaviour data Required Candidate profile Search / Information Retrieval /NLP/ Machine Learning statistical tools -Python, R, SAS, or open-source libraries ML/DL frameworks-TensorFlow, PyTorch, scikit-learn Big Data frameworks (Hadoop, Spark)

As a Big Data Engineer, you will develop, maintain, evaluate, and test big data solutions. You will be involved in data engineering activities like creating pipelines/workflows for Source to Target and implementing solutions that tackle the clients needs. Your primary responsibilities include: Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements. Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization. Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need too Required education Bachelo...

As a Big Data Engineer, you will develop, maintain, evaluate, and test big data solutions. You will be involved in data engineering activities like creating pipelines/workflows for Source to Target and implementing solutions that tackle the clients needs. Your primary responsibilities include: Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements. Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization. Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need too. Required education Bachel...

As a Big Data Engineer, you will develop, maintain, evaluate, and test big data solutions. You will be involved in data engineering activities like creating pipelines/workflows for Source to Target and implementing solutions that tackle the clients needs. Your primary responsibilities include: Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements. Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization. Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need too Required education Bachelo...

As a Big Data Engineer, you will develop, maintain, evaluate, and test big data solutions. You will be involved in data engineering activities like creating pipelines/workflows for Source to Target and implementing solutions that tackle the clients needs. Your primary responsibilities include: Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements. Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization. Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need too Required education Bachelo...

As a Big Data Engineer, you will develop, maintain, evaluate, and test big data solutions. You will be involved in data engineering activities like creating pipelines/workflows for Source to Target and implementing solutions that tackle the clients needs. Your primary responsibilities include: Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements. Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization. Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need too Required education Bachelo...

As a Big Data Engineer, you will develop, maintain, evaluate, and test big data solutions. You will be involved in data engineering activities like creating pipelines/workflows for Source to Target and implementing solutions that tackle the clients needs. Your primary responsibilities include: Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements. Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization. Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need too. Required education Bachel...

As a Big Data Engineer, you will develop, maintain, evaluate, and test big data solutions. You will be involved in data engineering activities like creating pipelines/workflows for Source to Target and implementing solutions that tackle the clients needs. Your primary responsibilities include: Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements. Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization. Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need too. Required education Bachel...

As a Big Data Engineer, you will develop, maintain, evaluate, and test big data solutions. You will be involved in data engineering activities like creating pipelines/workflows for Source to Target and implementing solutions that tackle the clients needs. Your primary responsibilities include: Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements. Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization. Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need too. Required education Bachel...

Looking for senior pyspark developer with 6+ years of hands on experienceBuild and manage large scale data solutions using tools like Pyspark, Hadoop, Hive, Python & SQLCreate workflows to process data using IBM TWSAble to use pyspark to create different reports and handle large datasetsUse HQL/SQL/Hive for ad-hoc query data and generate reports, and store data in HDFS Able to deploy code using Bitbucket, Pycharm and Teamcity.Can manage folks, able to communicate with several teams and can explain problem/solutions to business team in non-tech manner -Primary Skill Pyspark-Hadoop-Spark - One to Three Years,Developer / Software Engineer

As a Big Data Engineer, you will develop, maintain, evaluate, and test big data solutions. You will be involved in data engineering activities like creating pipelines/workflows for Source to Target and implementing solutions that tackle the clients needs. Your primary responsibilities include: Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements. Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization. Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need too Required education Bachelo...