281 Spark Streaming Jobs

Skills-Java, Spark, SQL Experience: 6 to 12 years Location: AIA-Hyderabad Responsibilities and JD in brief along with additional criteria to be considered (if any): At least 6+ years of experience and strong knowledge in Java and related frameworks. Expertise in building robust, scalable and maintainable applications with Java and Spark . Able to write clean, maintainable and efficient Java code following best practices. Having deep understanding of Java core and Java EE concepts, JVM internals such as class loading and memory management. At least 5+ years of experience in designing and developing large scale, distributed data processing pipelines using Apache Spark and related technologies....

We are seeking a highly skilled and experienced Senior Data Engineer with strong expertise in Streaming Data and Big Data Technologies , AWS & Scala Mandatory . The ideal candidate will have hands-on experience in architecting and implementing scalable data solutions using modern cloud-native tools and frameworks. Key Responsibilities: Architect and implement scalable data pipelines for real-time, batch, structured, and unstructured data . Design and develop solutions using AWS services such as EMR, Kinesis, Glue, Athena, S3, CloudFormation, API Gateway . Work extensively with streaming platforms like Kafka, Flink, Spark Streaming . Develop and optimize data ingestion workflows using Apache ...

Job Description: You should have a minimum of 8 to 10 years of experience in Java, REST API, and Spring boot. Additionally, hands-on experience in Angular JS, React JS, or View JS is required. A Bachelors degree or higher in computer science, data science, or a related field is a must. You should also possess hands-on experience with data cleaning, visualization, and reporting, along with experience working in an agile environment. Excellent analytical and problem-solving skills, as well as excellent communication and interpersonal skills, are essential. Key Responsibilities: - Minimum 8 to 10 years of experience in Java, REST API, Spring boot - Hands-on experience in Angular JS, React JS, o...

Infosys Hirings for PYSPARK DEVELOPER EXPERIENCE: 5-10 YEARS LOCATION: PAN INDIA SKILLS REQUIRED: PYSPARK, PYTHON, SPARK EDUCATION : BTECH/BE/MTECH/ME/BSC/MSC/BCA/MCA

Role Overview: You are a highly skilled Azure PySpark Solution Architect responsible for designing and implementing scalable data solutions on Microsoft Azure. Your expertise in Azure services, PySpark, and enterprise-grade solution architecture will drive efficient data processing and analytics workflows. Key Responsibilities: - Design and implement end-to-end data solutions using Azure Data Services and PySpark. - Develop high-performance ETL pipelines leveraging Azure Databricks, Azure Data Factory, and Synapse Analytics. - Architect scalable, secure, and cost-efficient cloud solutions aligning with business goals. - Collaborate with data engineers, data scientists, and stakeholders to de...

Position: Senior Kafka Developer Experience - 6+ Years Location: PAN India Remote Job Description: Senior Kafka Developer with strong Java Spring Boot expertise to design and implement real-time, scalable data pipelines and microservices. - Develop and maintain Kafka producers and consumers using Spring Kafka. - Design and implement microservices architecture using Spring Boot. - Optimize Kafka configurations for performance, reliability, and scalability. - Integrate Kafka Streams or KSQL for real-time data processing. - Collaborate with DevOps to deploy and monitor services on cloud platforms (GCP). - Ensure robust logging, error handling, and message delivery guarantees. - Participate in c...

As the Data Team Lead, you will play a crucial role in driving the strategy, governance, and operational excellence of the enterprise data platform. Your responsibilities will include: - Leading and managing vendor Data Engineering and Architecture teams, allocating tasks and planning resources effectively. - Acting as the primary owner of the Data Platform strategy and roadmap. - Overseeing Data Platform reporting activities such as system go-lives, KPI tracking, usage insights, and FinOps monitoring. - Serving as the subject matter expert for the Data Platform ecosystem, including Snowflake, AWS, Glue, etc. - Defining and evolving the long-term platform strategy in alignment with business ...

Must Have: Proficiency in at least one of the following programming languages: Java/Scala/Python Good understanding of SQL Experience of development and deployment of at least one end-to-end data storage/processing pipeline Strong Experience in Spark development with batch and streaming Intermediate level expertise in HDFS and Hive Experience with Pyspark and Data Engineering ETL implementation and migration to spark Experience of working with Hadoop cluster Python, PySpark, Data Bricks developer with knowledge of cloud Experience with Kafka and Spark streaming (Dstream and Structured Streaming) Experience with using Jupyter notebooks or any other developer tool Experience with Airflow or ot...

Role Overview: As a Lead Spark Scala Engineer under ICG TTS Operations Technology (OpsTech) Group, you will be responsible for developing and optimizing production-grade Spark applications in Scala. Your role will involve providing technical leadership, conducting code reviews, enforcing best practices, and optimizing performance. Your expertise in Spark, Scala, RDBMS, CI/CD tools, and other relevant technologies will be crucial in contributing to applications, systems analysis, and programming activities. Key Responsibilities: - Develop, test, and deploy production-grade Spark applications in Scala, ensuring optimal performance, scalability, and resource utilization. - Provide technical lea...

Role Overview: As a Solutions Architect at Koantek, you will collaborate with customers to design scalable data architectures utilizing Databricks technology and services. You will build secure, highly scalable big data solutions to achieve tangible, data-driven outcomes while focusing on simplicity and operational effectiveness. Leveraging your technical expertise and business acumen, you will engage in complex technology discussions, demonstrating the value of the Databricks platform throughout the sales process. Working with Account Executives, you will interact with customers" technical leaders to become a trusted advisor who delivers concrete outcomes. This role involves collaborating w...

Role Overview: You will be a Data Analytics focused Senior Software Engineer at PubMatic, responsible for developing advanced AI agents to enhance data analytics capabilities. Your role will involve building and optimizing AI agents using technologies such as Hadoop, Spark, Scala, Kafka, and cloud-based solutions. You will play a crucial role in improving data-driven insights and analytical workflows for the platform. Key Responsibilities: - Build, design, and implement a highly scalable and fault-tolerant big data platform for processing terabytes of data and providing in-depth analytics. - Develop backend services using Java, REST APIs, JDBC, and AWS. - Create and maintain Big Data pipelin...

Role Overview: You will be working as a Lead Spark Scala Engineer under ICG TTS Operations Technology (OpsTech) Group to assist in the implementation of the next generation Digital Automation Platform and Imaging Workflow Technologies. Your role will involve managing development teams in the distributed systems Eco-System and collaborating effectively as a team player. Your primary objective will be to contribute to applications, systems analysis, and programming activities. Key Responsibilities: - Develop, test, and deploy production-grade Spark applications in Scala, ensuring optimal performance, scalability, and resource utilization. - Provide technical leadership and guidance to a team o...

Role Overview: At EY, you will have the opportunity to build a career tailored to your unique strengths, supported by a global network, inclusive culture, and cutting-edge technology, enabling you to reach your full potential. Your voice and perspective are valued as essential contributions to EY's continuous improvement. By joining EY, you will not only create an exceptional experience for yourself but also contribute to building a better working world for all. Key Responsibilities: - Develop standardized practices for delivering new products and capabilities using Big Data & cloud technologies, encompassing data acquisition, transformation, analysis, modeling, governance, and data manageme...

Role & responsibilities Position: Senior Developer (Azure Data Engineer) Experience: 6-8 Years No. Of Position: 2 were always looking to strengthen our business by adding the best available people to our team. Were currently seeking a Senior Azure Data Engineer to work closely with our Data Analytics team in driving the adoption, optimization, and iteration, leveraging multiple integrations and technologies. The ideal candidate will have at least 3-10 years of experience with technical implementation of customer projects. Using their expertise, problem-solving ability, and creative flair, the consultant will design, build, deliver, and maintain. The consultant will also help develop business...

Role Overview: At PwC, our managed services team specializes in providing outsourced solutions to clients across various functions. As a Data Engineer Offshore, you will be responsible for designing, implementing, and maintaining scalable ETL pipelines and systems using technologies such as Teradata, DataStage, AWS, Databricks, SQL, Spark, Kafka, and more. Your role will involve collaborating with cross-functional teams to deliver data solutions that meet business requirements and ensure data quality, integrity, and security. Key Responsibilities: - Design, develop, and maintain scalable ETL pipelines using DataStage and other ETL tools. - Utilize AWS cloud services for data storage, process...

Role Overview: You should have a minimum of 8 to 10 years of experience in Java, REST API, and Spring Boot. Additionally, hands-on experience in either AngularJS, ReactJS, or VueJS is required. A background in computer science, data science, or a related field at a Bachelors degree level or higher is essential. You should possess hands-on experience with data cleaning, visualization, and reporting. Working in an agile environment should be familiar to you. Your strong analytical and problem-solving skills, along with excellent communication and interpersonal abilities, will be crucial for this role. Key Responsibilities: - Minimum 8 to 10 years of experience in Java, REST API, and Spring Boo...

ValueLabs is Hiring: Data Engineer | Cloudera Data Platform (CDP) Location: Hyderabad Experience: 5+ Years Employment Type: Full-Time About ValueLabs ValueLabs is a global technology company committed to delivering innovative solutions and exceptional client experiences. We are expanding our data engineering team in Hyderabad and are looking for a skilled Data Engineer with strong expertise and the Cloudera Data Platform (CDP) . Role Overview As a Data Engineer , you will design and maintain scalable ETL pipelines, ensuring high data quality, performance, and reliability across our data ecosystem. Key Responsibilities Build and optimize ETL pipelines using PySpark on CDP Ingest data from RDB...

As a skilled PySpark Developer with 4-5 or 2-3 years of experience, you will be responsible for developing and maintaining data processing pipelines using PySpark, Apache Spark's Python API. You will collaborate closely with data engineers, data scientists, and stakeholders to design and implement scalable and efficient data processing solutions. Key Responsibilities: - Designing, developing, and maintaining PySpark data processing pipelines to handle large volumes of structured and unstructured data - Collaborating with data engineers and data scientists to understand data requirements and create efficient data models and transformations - Optimizing and tuning PySpark jobs for improved per...

Role Overview: As a Data Engineer Offshore at PwC, you will be part of the managed services team focusing on various outsourced solutions to support clients across different functions. Your role involves helping organizations streamline operations, reduce costs, and enhance efficiency by managing key processes and functions on their behalf. You will utilize your skills in project management, technology, and process optimization to deliver high-quality services to clients. Your primary focus will be on transitioning and running services, managing delivery teams, programmes, commercials, performance, and delivery risk. Your work will revolve around continuous improvement and optimizing managed...

As a highly skilled Azure PySpark Solution Architect, your role involves designing and implementing scalable data solutions on Microsoft Azure. Your expertise in Azure services, PySpark, and solution architecture will ensure efficient data processing and analytics workflows for enterprise applications. **Key Responsibilities:** - Design and implement end-to-end data solutions using Azure Data Services and PySpark - Develop high-performance ETL pipelines leveraging Azure Databricks, Azure Data Factory, and Synapse Analytics - Architect scalable, secure, and cost-efficient cloud solutions aligning with business objectives - Collaborate with data engineers, data scientists, and business stakeho...

As an Applications Development Programmer Analyst at our company, your main objective will be to contribute to applications systems analysis and programming activities. You will participate in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. Your key responsibilities in this role will include: - Utilizing your knowledge of applications development procedures and concepts to identify and define necessary system enhancements. - Identifying and analyzing issues, making recommendations, and implementing solutions based on your understanding of business processes, system processes, and industry standards to solve com...

Min 3+ yrs in Data engineer(GenAI platform) ETL/ELT workflows using AWS,Azure Databricks,Airflow,Azure Data Factory Exp in Azure Databricks,Snowflake,Airflow,Python,SQL,Spark,Spark Streaming,AWS EKS, CI/CD(Jenkins) Elasticsearch,SOLR,OpenSearch,Vespa

Responsibilities: * Design, develop & maintain big data pipelines using Spark, Hive & Hadoop. * Optimize performance through data modeling & query optimization. * Collaborate with cross-functional teams on project delivery. Food allowance Annual bonus Health insurance Office cab/shuttle Provident fund

About the Role Were seeking a highly skilled Data Engineer with a strong development background and a passion for transforming data into valuable insights. The ideal candidate will play a key role in designing, building, and maintaining scalable data pipelines and analytics solutions that support critical business decisions. What youll be doing Design, build, and optimize robust data pipelines for large-scale data processing Write complex SQL queries for data extraction, transformation, and reporting Collaborate with analytics and reporting teams to deliver data-driven insights Develop scalable solutions using programming languages such as Java, Python, and Node. js Integrate APIs and third-...

Role Overview: As a Manager - Azure Data Architect at EY-GDS Data and Analytics (D&A) team, you will have the opportunity to work with diverse clients in industries such as Banking, Insurance, Manufacturing, Healthcare, Retail, Supply Chain, and Finance. Your role will involve designing and proposing Azure-based data and AI solutions, developing data platforms for real-time analytics, and leading the development of data engineering and governance frameworks. Key Responsibilities: - Architecture & Solutioning: - Design and propose Azure-based data and AI solutions aligning with customer objectives. - Architect and implement data platforms supporting real-time, batch, and hybrid analytics. - R...