905 Apache Spark Jobs - Page 12

What youll do: We seek Software Engineers with experience building and scaling services in on-premises and cloud environments. As a Principal Software Engineer in the Epsilon Attribution/Forecasting Product Development team, you will design, implement, and optimize data processing solutions using Scala, Spark, and Hadoop. Collaborate with cross-functional teams to deploy big data solutions on our on-premises and cloud infrastructure along with building, scheduling and maintaining workflows. Perform data integration and transformation, troubleshoot issues, Document processes, communicate technical concepts clearly, and continuously enhance our attribution engine/forecasting engine. Strong wri...

What youll do: We seek Software Engineers with experience building and scaling services in on-premises and cloud environments. As a Principal Software Engineer in the Epsilon Attribution/Forecasting Product Development team, you will design, implement, and optimize data processing solutions using Scala, Spark, and Hadoop. Collaborate with cross-functional teams to deploy big data solutions on our on-premises and cloud infrastructure along with building, scheduling and maintaining workflows. Perform data integration and transformation, troubleshoot issues, Document processes, communicate technical concepts clearly, and continuously enhance our attribution engine/forecasting engine. Strong wri...

As a Senior Data Engineer, you will be responsible for designing, building, and maintaining scalable data processing systems in cloud-based Big Data environments. Your passion for data architecture and experience in creating reliable pipelines using modern technologies will be crucial for this role. **Key Responsibilities:** - Understand business requirements and translate them into scalable and efficient data solutions. - Design, develop, and maintain robust data pipelines for batch and stream processing. - Build data lakes, warehouses, and data mesh solutions for enterprise-level consumption. - Transform complex data from various sources into actionable insights and production-ready data p...

As a Python Developer specializing in Machine Learning and Data Analytics, you will play a crucial role in designing, developing, and deploying Python applications that utilize data analytics, statistical modeling, and machine learning techniques. Your responsibilities will include: - Designing, developing, and deploying Python applications for data analytics and machine learning. - Independently working on machine learning model development, evaluation, and optimization. - Developing ETL pipelines and processing large-scale datasets for analysis. - Implementing scalable and efficient algorithms for predictive analytics and automation. - Optimizing code for performance, scalability, and main...

As a seasoned Developer passionate about writing clean and efficient code, building scalable systems, and driving engineering excellence in a fast-paced, Agile environment, you will have the following responsibilities: - Perform detailed impact analysis for code changes with an understanding of dependencies across the application components - Design and develop scalable, high-performance code using Java and Bigdata / Java Spark - Write high-quality, maintainable code that is modular, testable, and adheres to SOLID principles and industry-standard design patterns - Write robust unit tests using JUnit, focusing on code coverage, business logic, readability, and reusability - Conduct code revie...

We are seeking a seasoned Back-End Engineer with a strong foundation in Java , along with hands-on experience in data engineering within cloud environments. The ideal candidate will have a deep understanding of building scalable backend systems and working with modern data platforms such as Snowflake/ OpenSearch/AWS Glue . This role combines core backend development with data pipeline engineering to support both operational and analytical needs Job location: Mumbai/Kolkata/Chennai/Noida

Job Title: Data Engineer Location: PAN India Exp range: 6-8 years CONTRACT 2 HIRE Shift: 2pm - 11pm Skill Required: Digital : Apache Spark~Digital : Python for Data Science~Digital : PySpark~MySQL Role required for Data Engineer with Data science " Job Description: Must have: Candidate must have expertise/experience with below tasks - Candidate must have experience on Linux, git, CICD, Release management, production deployment and support. - Strong Knowledge on Apache Spark is MUST - Strong Knowledge of PySpark is MUST - Strong Knowledge on SQL is MUST - Good Knowledge of Data Science workload - Good Knowledge on Kubernetes/Docker - Good Knowledge of Python is MUST - Good Knowledge of Java l...

As a Data Engineering Consultant with GCP experience at LTIMINDTREE, your role will involve the following responsibilities: - Good experience in SQL Queries - Understanding of databases data flows and data architecture - Experience on Python and GCP platform - Willingness to Collaborate with Cross-Functional teams to drive validation and Project Execution - Review and evaluate policies and procedures - To develop, implement, monitor, and manage the compliance needs as per the AEMP70 policy Qualifications Required: - Mandatory Skills: Apache Spark, Java, Python, Scala, SparkSQL, Databricks (Note: No additional details about the company were provided in the job description),

When you join BairesDev, you become part of a team that has been at the forefront of technology projects for over 15 years. We are dedicated to delivering cutting-edge solutions to industry giants like Google and the most innovative startups in Silicon Valley. Our diverse team of over 4,000 professionals, who represent the world's Top 1% of tech talent, works on roles that have a significant global impact. When you apply for the position of Apache Spark Developer at BairesDev, you are embarking on a journey that goes beyond the ordinary. We are committed to matching your passions and skills with our job opportunities, guiding you towards exceptional career growth and success. Role Overview: ...

Role Overview: You will be responsible for developing generative AI use cases and deploying machine learning models. Additionally, you will be building and maintaining private cloud infrastructure for AI/ML solutions, automating workflows, and improving system performance. Collaboration with teams to analyze and deliver AI application requirements will also be a key part of your role. Ensuring the security, scalability, and efficiency of cloud infrastructure will be essential. Key Responsibilities: - Develop generative AI use cases and deploy machine learning models. - Build and maintain private cloud infrastructure for AI/ML solutions. - Automate workflows and improve system performance. - ...

As a Technical Lead AI, you will be responsible for driving the design, development, and deployment of AI-driven solutions across multiple product lines. You should have a proven track record of supporting AI-embedded products, leading Gen AI initiatives, and establishing best practices in data pipelines, ML model lifecycle, LLMs, and MLOps. Your role will involve collaborating with cross-functional teams to architect scalable AI systems, mentor engineers, and ensure successful integration of AI capabilities into business applications. - Lead the end-to-end design and implementation of AI/ML and GenAI solutions, including Retrieval-Augmented Generation (RAG) pipelines. - Partner with product...

As an experienced professional with 3+ years of experience in building and operating production-grade applications and services across the stack, you will be responsible for the following key responsibilities: - Demonstrating strong programming skills in Python and/or Scala and SQL to write modular, testable, and well-documented code for batch and streaming workloads. - Hands-on experience with modern data engineering stacks including: - Distributed processing using Apache Spark (Databricks preferred), PySpark/Scala. - Orchestration with Azure Data Factory or Apache Airflow; implementing event-driven patterns with Azure Functions/Logic Apps. - Working with storage & formats such as Delta Lak...

As a Data Engineer, your role involves designing and implementing scalable data architectures to efficiently store, process, and manage structured and unstructured data. You will be responsible for developing robust data pipelines, integrating disparate data sources, and ensuring high data quality and security across multiple cloud platforms. Key Responsibilities: - Design and implement scalable, cloud-agnostic Data Lake, Data LakeHouse, Data Mesh, and Data Fabric architectures. - Develop and maintain robust data pipelines using services like AWS Glue, Azure Data Factory, GCP Dataflow, Apache Spark, or Apache Airflow. - Optimize Extract, Transform, Load (ETL) processes to integrate disparate...

Role Overview: You will be a Senior Data Engineer responsible for building scalable, reliable, and cutting-edge data solutions on cloud platforms. Your expertise in Google Cloud Platform (GCP), BigQuery, and modern data engineering practices will be crucial in developing end-to-end data pipelines for actionable business intelligence, advanced analytics, and machine learning initiatives. Additionally, you will have the opportunity to work with diverse data ecosystems and influence the future of the company's data infrastructure. Key Responsibilities: - Architect, develop, and maintain robust ETL/ELT data pipelines on GCP, primarily utilizing BigQuery. - Design and implement data ingestion fra...

Role Overview: You will be working as a Solutions Architect at Koantek, collaborating with customers to design scalable data architectures using Databricks technology and services. Your role will involve building secure, highly scalable big data solutions, focusing on achieving tangible, data-driven outcomes while ensuring simplicity and operational effectiveness. With your technical expertise and business acumen, you will engage in complex technology discussions, demonstrating the value of the Databricks platform throughout the sales process. Collaborating with Account Executives, you will interact with customers" technical leaders to become a trusted advisor who delivers concrete results. ...

As a Lead Data Scientist at Meesho, you will be responsible for driving high-impact initiatives across multiple charters. Your role will involve leading a team of data scientists and collaborating with product, engineering, and business teams to transform raw data into scalable solutions. Your deep expertise in machine learning and data systems, combined with strategic problem-solving skills, will shape the future of the platform. Key Responsibilities: - Own and deliver solutions across multiple charters by formulating well-scoped problem statements - Mentor a team of data scientists (DS2s and DS3s) with project planning and execution - Design and optimize key user journeys by identifying us...

In this role, you will: (Principal Responsibilities) As a key member of the technical team alongside Engineers, Data Analysts and Business analysts, you will be expected to define and contribute at a high-level to many aspects of our collaborative Agile development process: Software design, Scala and Python - Spark development experience, automated testing of new and existing components in an Agile, DevOps and dynamic environment. Minimum 1 year experience in Scala Promoting development standards, code reviews, mentoring, knowledge sharing Production support & troubleshooting. Implement the tools and processes, handling performance, scale, availability, accuracy and monitoring Liaison with B...

In this role, you will: (Principal Responsibilities) As a key member of the technical team alongside Engineers, Data Analysts and Business analysts, you will be expected to define and contribute at a high-level to many aspects of our collaborative Agile development process: Software design, Scala and Python - Spark development experience, automated testing of new and existing components in an Agile, DevOps and dynamic environment. Minimum 1 year experience in Scala Promoting development standards, code reviews, mentoring, knowledge sharing Production support & troubleshooting. Implement the tools and processes, handling performance, scale, availability, accuracy and monitoring Liaison with B...

In this role, you will: (Principal Responsibilities) As a key member of the technical team alongside Engineers, Data Analysts and Business analysts, you will be expected to define and contribute at a high-level to many aspects of our collaborative Agile development process: Software design, Scala and Python - Spark development experience, automated testing of new and existing components in an Agile, DevOps and dynamic environment. Minimum 1 year experience in Scala Promoting development standards, code reviews, mentoring, knowledge sharing Production support & troubleshooting. Implement the tools and processes, handling performance, scale, availability, accuracy and monitoring Liaison with B...

In this role, you will: (Principal Responsibilities) As a key member of the technical team alongside Engineers, Data Analysts and Business analysts, you will be expected to define and contribute at a high-level to many aspects of our collaborative Agile development process: Software design, Scala and Python - Spark development experience, automated testing of new and existing components in an Agile, DevOps and dynamic environment. Minimum 1 year experience in Scala Promoting development standards, code reviews, mentoring, knowledge sharing Production support & troubleshooting. Implement the tools and processes, handling performance, scale, availability, accuracy and monitoring Liaison with B...

In this role, you will: (Principal Responsibilities) As a key member of the technical team alongside Engineers, Data Analysts and Business analysts, you will be expected to define and contribute at a high-level to many aspects of our collaborative Agile development process: Software design, Scala and Python - Spark development experience, automated testing of new and existing components in an Agile, DevOps and dynamic environment. Minimum 1 year experience in Scala Promoting development standards, code reviews, mentoring, knowledge sharing Production support & troubleshooting. Implement the tools and processes, handling performance, scale, availability, accuracy and monitoring Liaison with B...

In this role, you will: (Principal Responsibilities) As a key member of the technical team alongside Engineers, Data Analysts and Business analysts, you will be expected to define and contribute at a high-level to many aspects of our collaborative Agile development process: Software design, Scala and Python - Spark development experience, automated testing of new and existing components in an Agile, DevOps and dynamic environment. Minimum 1 year experience in Scala Promoting development standards, code reviews, mentoring, knowledge sharing Production support & troubleshooting. Implement the tools and processes, handling performance, scale, availability, accuracy and monitoring Liaison with B...

In this role, you will: (Principal Responsibilities) As a key member of the technical team alongside Engineers, Data Analysts and Business analysts, you will be expected to define and contribute at a high-level to many aspects of our collaborative Agile development process: Software design, Scala and Python - Spark development experience, automated testing of new and existing components in an Agile, DevOps and dynamic environment. Minimum 1 year experience in Scala Promoting development standards, code reviews, mentoring, knowledge sharing Production support & troubleshooting. Implement the tools and processes, handling performance, scale, availability, accuracy and monitoring Liaison with B...

In this role, you will: (Principal Responsibilities) As a key member of the technical team alongside Engineers, Data Analysts and Business analysts, you will be expected to define and contribute at a high-level to many aspects of our collaborative Agile development process: Software design, Scala and Python - Spark development experience, automated testing of new and existing components in an Agile, DevOps and dynamic environment. Minimum 1 year experience in Scala Promoting development standards, code reviews, mentoring, knowledge sharing Production support & troubleshooting. Implement the tools and processes, handling performance, scale, availability, accuracy and monitoring Liaison with B...

In this role, you will: (Principal Responsibilities) As a key member of the technical team alongside Engineers, Data Analysts and Business analysts, you will be expected to define and contribute at a high-level to many aspects of our collaborative Agile development process: Software design, Scala and Python - Spark development experience, automated testing of new and existing components in an Agile, DevOps and dynamic environment. Minimum 1 year experience in Scala Promoting development standards, code reviews, mentoring, knowledge sharing Production support & troubleshooting. Implement the tools and processes, handling performance, scale, availability, accuracy and monitoring Liaison with B...