69 Spark Programming Jobs

About the Role: We are looking for a skilled and passionate Spark Developer to design, build, and optimize large-scale data processing systems. The ideal candidate will have strong expertise in Apache Spark , Big Data ecosystems , and distributed computing frameworks . Key Responsibilities: Design, develop, and maintain data processing pipelines using Apache Spark (Core, SQL, Streaming, Structured Streaming) . Implement ETL workflows for data ingestion, transformation, and integration across multiple data sources. Optimize Spark applications for performance, scalability, and reliability in distributed environments. Work with large structured and unstructured datasets to support analytical an...

Contract Duration : 3 months Skills : - PySpark or Scala with Spark, Spark Architecture, Hadoop, SQL - Streaming Technologies like Kafka etc.- Proficiency in Advanced SQL (Window functions)- Airflow, S3, and Stream Sets or similar ETL tools.- Basic Knowledge on AWS IAM, AWS EMR and Snowflake. Responsibilities : - Data Pipeline Development: Design, implement, and maintain scalable and efficient data pipelines to collect, process, and store large volumes of structured and unstructured data. - Data Modeling: Develop and maintain data models, schemas, and metadata to support the organization's data initiatives. Ensure data integrity and optimize data storage and retrieval processes. - Data Integ...

BANGALORE, Infosys Limited Spark Developer Work Experience 5 - 11 Years Educational Requirements Bachelor of Engineering,BTech,BCA,MTech,MCA Technical and Professional Requirements: Primary skills:Technology->Big Data - Data Processing->Spark Responsibilities A day in the life of an Infoscion • As part of the Infosys delivery team, your primary role would be to interface with the client for quality assurance, issue resolution and ensuring high customer satisfaction. • You will understand requirements, create and review designs, validate the architecture and ensure high levels of service offerings to clients in the technology domain. • You will participate in project estimation, provide input...

At EY, youll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And were counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all. EY-Consulting - Data and Analytics Manager AWS Databricks EY's Consulting Services is a unique, industry-focused business unit that provides a broad range of integrated services that leverage deep industry experience with strong functional and technical capabilities and product knowledge. EYs financial services practice provides integrated Cons...

4 Years Data Engineering Experience 5 years of consulting experience Completed Data Engineering Professional certification & required classes Minimum 2-3 projects delivered with hands-on experience in Databricks Completed Apache Spark Programming with Databricks, Data Engineering with Databricks, Optimizing Apache Spark on Databricks Experience in Spark and or Hadoop, Flink, Presto, other popular big data engines Familiarity with Databricks multi-hop pipeline architecture

2 Years Data Engineering Experience, 3 years of consulting experience Completed Data Engineering Associate certification & required classes Minimum 1 project delivered with hands-on experience in development on Databricks Completed Apache Spark Programming with Databricks, Data Engineering with Databricks, Optimizing Apache Spark on Databricks SQL delivery experience, and familiarity with Bigquery, Synapse or Redshift Proficient in Python, knowledge of additional databricks programming languages (Scala)

We are seeking a highly skilled and experienced Senior Data Engineer with strong expertise in Streaming Data and Big Data Technologies , AWS & Scala Mandatory . The ideal candidate will have hands-on experience in architecting and implementing scalable data solutions using modern cloud-native tools and frameworks. Key Responsibilities: Architect and implement scalable data pipelines for real-time, batch, structured, and unstructured data . Design and develop solutions using AWS services such as EMR, Kinesis, Glue, Athena, S3, CloudFormation, API Gateway . Work extensively with streaming platforms like Kafka, Flink, Spark Streaming . Develop and optimize data ingestion workflows using Apache ...

We are seeking a highly skilled and experienced Senior Data Engineer with strong expertise in Streaming Data and Big Data Technologies , AWS & Scala Mandatory . The ideal candidate will have hands-on experience in architecting and implementing scalable data solutions using modern cloud-native tools and frameworks. Key Responsibilities: Architect and implement scalable data pipelines for real-time, batch, structured, and unstructured data . Design and develop solutions using AWS services such as EMR, Kinesis, Glue, Athena, S3, CloudFormation, API Gateway . Work extensively with streaming platforms like Kafka, Flink, Spark Streaming . Develop and optimize data ingestion workflows using Apache ...

We are seeking a highly skilled and experienced Senior Data Engineer with strong expertise in Streaming Data and Big Data Technologies , AWS & Scala Mandatory . The ideal candidate will have hands-on experience in architecting and implementing scalable data solutions using modern cloud-native tools and frameworks. Key Responsibilities: Architect and implement scalable data pipelines for real-time, batch, structured, and unstructured data . Design and develop solutions using AWS services such as EMR, Kinesis, Glue, Athena, S3, CloudFormation, API Gateway . Work extensively with streaming platforms like Kafka, Flink, Spark Streaming . Develop and optimize data ingestion workflows using Apache ...

As a Data Engineer at IBM, you'll play a vital role in the development, design of application, provide regular support/guidance to project teams on complex coding, issue resolution and execution. Your primary responsibilities include: Lead the design and construction of new solutions using the latest technologies, always looking to add business value and meet user requirements. Strive for continuous improvements by testing the build solution and working under an agile framework. Discover and implement the latest technologies trends to maximize and build creative solutions Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Ex...

We are seeking a highly skilled and experienced Senior Data Engineer with strong expertise in Streaming Data and Big Data Technologies , AWS & Scala Mandatory . The ideal candidate will have hands-on experience in architecting and implementing scalable data solutions using modern cloud-native tools and frameworks. Key Responsibilities: Architect and implement scalable data pipelines for real-time, batch, structured, and unstructured data . Design and develop solutions using AWS services such as EMR, Kinesis, Glue, Athena, S3, CloudFormation, API Gateway . Work extensively with streaming platforms like Kafka, Flink, Spark Streaming . Develop and optimize data ingestion workflows using Apache ...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : PySpark Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and implementing...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : PySpark Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and implementing...

Key Responsibilities Design, develop, and maintain robust ETL/ELT pipelines using SQL, dbt, and orchestration tools. Architect and implement data warehouse and lakehouse solutions leveraging Snowflake, Databricks, and cloud-native services. Work on data modeling, schema design, and query optimization for performance and scalability. Migrate and integrate legacy/on-prem data platforms to cloud environments (Azure or AWS). Implement data governance, quality, and security practices to ensure reliability and compliance. Collaborate with data scientists, BI developers, and business stakeholders to deliver analytics-ready datasets. Optimize pipelines and storage for cost efficiency and cloud-nativ...

Knowledge of setting up operating & maintaining wire EDM machine to produce parts & components. Set up & operating of EDM equipment to ensure proper making precision parts. Hand of experience in operating wire EDM machine, programming knowledge Required Candidate profile Shall be responsible for executing the job as per drawing, GD & T knowledge, Knowledge of 5S, safety & autonomous maintenance. Hands on experience in wire-cut, EDM & Spark Machine.

Minimum 6+ years of experience in the fundamental technology work of architecting / designing / building / managing / Optimizing the core frameworks for our cloud-based system platforms and end-to-end solutions. Responsible for optimization of cloud infrastructure, including Databricks , ADLS, Blob, VMs etc., Conduct Proof-of-Concept activities in support of advanced use cases. Must have architectural focus for ensuring availability, reliability and scalability are necessary Responsible for documenting the technical solutions Administer and manage Azure Databricks workspaces, clusters, and jobs with a focus on maximizing efficiency and performance. Implement and manage Unity Catalog to provi...

Streaming data Technical skills requirements :- Experience- 5+ Years Solid hands-on and Solution Architecting experience in Big-Data Technologies (AWS preferred) - Hands on experience in: AWS Dynamo DB, EKS, Kafka, Kinesis, Glue, EMR - Hands-on experience of programming language like Scala with Spark. - Good command and working experience on Hadoop Map Reduce, HDFS, Hive, HBase, and/or No-SQL Databases - Hands on working experience on any of the data engineering analytics platform (Hortonworks Cloudera MapR AWS), AWS preferred - Hands-on experience on Data Ingestion Apache Nifi, Apache Airflow, Sqoop, and Oozie - Hands on working experience of data processing at scale with event driven syste...

Minimum 6+ years of experience in the fundamental technology work of architecting / designing / building / managing / Optimizing the core frameworks for our cloud-based system platforms and end-to-end solutions. Responsible for optimization of cloud infrastructure, including Databricks , ADLS, Blob, VMs etc., Conduct Proof-of-Concept activities in support of advanced use cases. Must have architectural focus for ensuring availability, reliability and scalability are necessary Responsible for documenting the technical solutions Administer and manage Azure Databricks workspaces, clusters, and jobs with a focus on maximizing efficiency and performance. Implement and manage Unity Catalog to provi...

About The Team/Role As a Senior Staff SWE of Data Lake House Engineering, this technical leader will be responsible for the design and implementation of the Data Lake house platform (supporting both Data Engineering and Data Lake house applications), Data Engineering pipeline productionalization (E2E data pipelines, model development, deployment, monitoring, refresh, etc), Data Lake house, and related technology, systems, and tools. This technologist will drive the technology development and architecture in these areas to ensure the platforms, systems, tools, models, and services generated from this team are technically sound with strong security, quality, reliability, usability, scalability...

As an Applications Development Senior Programmer Analyst, your role involves participating in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. You will contribute to applications systems analysis and programming activities. Key Responsibilities: - Conduct feasibility studies, time and cost estimates, IT planning, risk technology, applications development, and establish new or revised applications systems and programs to meet specific business needs - Monitor and control all phases of the development process including analysis, design, construction, testing, and implementation, providing user and operational supp...

Role Overview: As an Applications Development Senior Programmer Analyst, you will be an intermediate level individual contributor responsible for participating in the establishment and implementation of new or revised application systems and programs. Your main objective will be to contribute to applications systems analysis and programming activities, focusing on hands-on application development in Java technologies. Your expertise in applications development will be utilized to analyze complex problems, evaluate business processes, and adhere to industry standards. Key Responsibilities: - Participate in hands-on application development and application management activities in Java and rela...

Streaming data Technical skills requirements :- Experience- 5+ Years Solid hands-on and Solution Architecting experience in Big-Data Technologies (AWS preferred) - Hands on experience in: AWS Dynamo DB, EKS, Kafka, Kinesis, Glue, EMR - Hands-on experience of programming language like Scala with Spark. - Good command and working experience on Hadoop Map Reduce, HDFS, Hive, HBase, and/or No-SQL Databases - Hands on working experience on any of the data engineering analytics platform (Hortonworks Cloudera MapR AWS), AWS preferred - Hands-on experience on Data Ingestion Apache Nifi, Apache Airflow, Sqoop, and Oozie - Hands on working experience of data processing at scale with event driven syste...

Job Title: Principal Data Engineer, Logistics Employment Type: Full Time Experience: 12+ Years About the Role: We are looking for a Principal Data Engineer to lead the design and delivery of scalable data solutions using Azure Data Factory and Azure Databricks. This is a consulting-focused role that requires strong technical expertise, stakeholder engagement, and architectural thinking. You will work closely with business, functional, and technical teams to define data strategies, design robust pipelines, and ensure smooth delivery in an Agile environment. Responsibilities Collaborate with business and technology stakeholders to gather and understand data needs Translate functional requireme...

Ready to shape the future of work At Genpact, we don&rsquot just adapt to change&mdashwe drive it. AI and digital innovation are redefining industries, and we&rsquore leading the charge. Genpact&rsquos , our industry-first accelerator, is an example of how we&rsquore scaling advanced technology solutions to help global enterprises work smarter, grow faster, and transform at scale. From large-scale models to , our breakthrough solutions tackle companies most complex challenges. If you thrive in a fast-moving, tech-driven environment, love solving real-world problems, and want to be part of a team that&rsquos shaping the future, this is your moment. Genpact (NYSE: G) is an advanced technology ...

Responsibilities * Design, develop, test, and deploy Spark Streaming applications using Scala/Spark Core. * Collaborate with cross-functional teams to gather requirements and deliver high-quality solutions. * Troubleshoot issues in Spark Streaming applications and optimize performance for improved efficiency. * Stay up-to-date with the latest developments in Spark Streaming and contribute to its evolution. Job Requirements * 3-10 years of experience in developing Spark Streaming applications using Scala/Spark Core. * Strong understanding of Spark programming concepts, including RDDs, Dataframes, Datasets, etc. * Experience with big data processing technologies such as Hadoop/Hive/Pig is a pl...