908 Apache Spark Jobs - Page 11

Job Title: Spark Data Engineer SME (Part-Time) Remote (India) | Work From Home | Flexible Timings About the Opportunity We are looking for experienced Spark Data Engineers (5+ years) to create content and mentor future data professionals. This role is ideal for professionals with strong expertise in Apache Spark and big data engineering who want to share knowledge and guide learners. What Were Looking For 5+ years of hands-on experience in data engineering Strong expertise in Apache Spark (PySpark/Scala/Java) Experience with ETL workflows, big data platforms, and cloud deployments Solid SQL and data modeling skills Interest in mentoring and simplifying complex concepts Why Join Us Part-time ...

Position Summary We are looking for a highly skilled and experienced Data Engineering Manager to lead our data engineering team. The ideal candidate will possess a strong technical background, strong project management abilities, and excellent client handling/stakeholder management skills. This role requires a strategic thinker who can drive the design, development and implementation of data solutions that meet our clients needs while ensuring the highest standards of quality and efficiency. Job Responsibilities Technology Leadership Lead guide the team independently or with little support to design, implement deliver complex cloud-based data engineering / data warehousing project assignment...

We are seeking an experienced Java Technical Lead with 8-12 years of experience for an immediate joiner position in Noida. The role involves leading a technical team in the design, development, and implementation of scalable Java-based enterprise solutions using Agile methodologies. Responsibilities include defining architecture, capturing requirements, leveraging Spring Boot and Spring Cloud frameworks, performing code reviews, implementing PoCs, troubleshooting bottlenecks, and ensuring system scalability and performance. The candidate should possess strong expertise in distributed architecture, multithreading, RDBMS, and NoSQL databases, with added knowledge in Apache Spark, Data Science,...

In data analysis at PwC, you will focus on leveraging data to drive insights and make informed business decisions. Utilizing advanced analytics techniques, you will extract insights from large datasets and support clients in solving complex business problems through data-driven decision-making. Your role will involve skills in data manipulation, visualization, and statistical modeling. - **Architecture Design:** - Design and implement scalable, secure, and high-performance architectures for Generative AI applications. - Integrate Generative AI models into existing platforms, ensuring compatibility and performance optimization. - **Model Development And Deployment:** - Fine-tune pre-trained g...

As a [Job Title], your role is crucial in ensuring the success of the organization by [insert brief role overview here]. Key Responsibilities: - Develop user requirements for systems in production to ensure maximum usability - Utilize skills in SQL, Apache Spark, and Python for data pipeline development - Implement data modeling and ETL processes for efficient data processing - Work with Apache Hadoop and data warehousing technologies - Collaborate on projects related to cloud platforms and big data technologies - Communicate effectively with team members and stakeholders Qualifications: - Proficiency in SQL, Apache Spark, and Python - Experience with data modeling and ETL processes - Famili...

As the Head of Engineering at a dynamic AI-driven company in Mumbai, you will lead a team of skilled web developers to deliver high-quality end-to-end solutions using an agile approach. Your role involves understanding the flagship products Trinka and RAx, collaborating with various technical roles, and ensuring the product's flexibility and adherence to best engineering practices. **Key Responsibilities:** - Lead a team of talented web developers in delivering high-quality solutions - Understand the products Trinka and RAx thoroughly - Work closely with the Director Engineer, Technical Project Manager, Solution Architect, Principal Engineers, and Senior DevOps - Review solution architecture...

Role Overview: As an Applications Development Technology Lead Analyst at our company, you will be a senior member responsible for establishing and implementing new or revised application systems and programs in coordination with the Technology team. Your main objective in this role will be to lead applications systems analysis and programming activities. You should have at least two years of experience in building and leading highly complex, technical data engineering teams, with an overall of 10+ years of hands-on Data Engineering experience. Key Responsibilities: - Partner with multiple management teams to ensure appropriate integration of functions and identify necessary system enhancemen...

Role Overview: At Derevo, we empower companies and individuals by unlocking the value of data within organizations. With over 15 years of experience, we design end-to-end data and AI solutions - from integration into modern architectures to implementing intelligent models in key business processes. We are currently seeking a talented Senior Data Engineer specialized in Databricks to join our team. Key Responsibilities: - Design and implement modern data architectures with high quality, driving analytical solutions based on Big Data technologies. - Maintain and optimize parallel processing systems, applying best practices for storage and management in data warehouses, data lakes, and lakehous...

Role Overview: As an Applications Development Technology Lead Analyst at our company, you will be a senior member of the team responsible for implementing new or revised application systems and programs in coordination with the Technology team. Your main objective will be to lead applications systems analysis and programming activities. Key Responsibilities: - Partner with multiple management teams to ensure appropriate integration of functions and identify necessary system enhancements - Resolve high impact problems/projects through in-depth evaluation of complex business processes - Provide expertise in applications programming and ensure application design aligns with the overall architec...

As a Mid-Level Data Engineer at Thinkproject, you will be at the forefront of building, optimizing, and maintaining reliable data pipelines and workflows. Here is a glimpse of what your role entails and the qualifications required to excel in this position: Role Overview: You will play a crucial role in designing scalable data processing systems, handling both structured and unstructured data sources, and supporting data needs across the organization in a cloud-centric environment. Key Responsibilities: - Design, implement, and optimize data pipelines (batch and streaming) for efficient data ingestion, transformation, and storage. - Ensure high data integrity through monitoring, alerting, te...

As a Senior Technical Architect with expertise in Databricks, Apache Spark, and modern data engineering architectures, you will play a crucial role in leading the architecture, design, and implementation of scalable data platforms and AI-powered applications for global clients. Your responsibilities will include: - Leading the architecture, design, and deployment of scalable data solutions using Databricks and the medallion architecture. - Guiding technical teams in building batch and streaming data pipelines using Spark, Delta Lake, and MLflow. - Collaborating with clients and internal stakeholders to understand business needs and translate them into robust data and AI architectures. - Desi...

As a Data Engineer at Omnissa, you will play a crucial role in delivering as an individual contributor. You should have a minimum of 2 to 8 years of experience in Data Engineering on AWS. Your responsibilities will include: - Strong Proficiency in SQL, Python, Apache Spark, AWS Data Stack (S3, EMR, Glue Catalog, Athena, Lambda, Airflow), Starburst / Trino. - Solid Understanding of Lakehouse Architecture - Apache Iceberg / Delta Lake. - Experience with CI/CD practices using GitHub Actions. - Ability to collaborate in cross-functional teams and communicate technical concepts clearly. - Mindset to design solutions for scale, proactively involve in additional areas and domains. - Experience with...

As a Senior Software Developer (Java) at our company in Coimbatore, you will play a crucial role in leading the development of secure and scalable SaaS applications. You will contribute to architectural decisions, drive innovation in enterprise-grade systems, and ensure the implementation of best practices. Your deep expertise in Java, Spring, and microservices will be valuable in ensuring the success of our projects. **Key Responsibilities:** - Design and develop robust, scalable Java-based SaaS applications - Implement secure authentication and authorization using Spring Security - Lead the integration and implementation of JBoss BPM Suite (jBPM) for business process automation - Apply mic...

You have a job opportunity for a Data role based in Jaipur/Indore. **Role Overview:** As a Data professional in this role, you will be responsible for developing dashboard stories, creating dashboards, and managing data engineering pipelines. Your primary focus will be on analyzing log data, identifying patterns, trends, and anomalies to derive actionable insights for product improvements and feature optimization. **Key Responsibilities:** - Utilize your 5+ years of experience to develop interactive dashboards and visualize key metrics using tools like Tableau, Power BI, or ThoughtSpot AI. - Collaborate with cross-functional teams to gather business requirements and translate them into techn...

As a Cloud Data Engineer, your role involves designing, developing, and managing scalable, secure, and efficient data pipelines on Azure to process and transform large datasets from multiple sources. You will implement and optimize Azure data services such as Azure Synapse Analytics, Azure Data Lake Storage (ADLS), Azure SQL Database, Azure Data Factory (ADF), Azure Databricks, and Azure Functions. Your responsibilities also include architecting and maintaining ETL/ELT processes, implementing data models, collaborating with data architects, building data integration workflows, and utilizing Azure DevOps for source control and automation. Key Responsibilities: - Design, develop, and manage sc...

As a Data Engineer at Valorem Reply, you will be responsible for designing and contributing to the architecture of database systems, including both normalized transactional systems and dimensional reporting systems. Your expertise with SQL Server and Microsoft technologies, as well as experience in implementing Big Data solutions using Databricks, will be essential for success in this role. You will play a key role in providing data solutions to support business applications and reporting systems, working closely with customers to deliver effective solutions. Proficiency in Power BI and Azure Data Services is a must to excel in this position. Key Responsibilities: - Design and implement arch...

As a Senior Data Engineer at Milestone Technologies, you will play a crucial role in leading data ingestion, transformation, and optimization initiatives using the modern Apache and Azure data stack. Your responsibilities will include: - Building and managing high-volume data pipelines using Spark/Databricks. - Implementing ELT frameworks using Azure Data Factory/Synapse Pipelines. - Optimizing large-scale datasets in Delta/Iceberg formats. - Implementing robust data quality, monitoring, and governance layers. - Collaborating with Data Scientists, Analysts, and Business stakeholders. Your technical stack will involve: - Big Data: Apache Spark, Kafka, Hive, Airflow, Hudi/Iceberg. - Cloud: Azu...

As a Data Engineer with 3-5 years of experience, your role will involve implementing scalable and sustainable data engineering solutions using tools like Databricks, Snowflake, Teradata, Apache Spark, and Python. You will be responsible for creating, maintaining, and optimizing data pipelines as workloads transition from development to production for specific use cases. Your ownership will extend to end-to-end development tasks, including coding, testing, debugging, and deployment. Your key responsibilities will include: - Driving automation through the use of modern tools and techniques to automate repeatable data preparation and integration tasks, thereby enhancing productivity. - Mapping ...

As an AI/ML Engineer at Automation Anywhere, you will be part of the Cloud Operations (CloudOps) team, responsible for designing, developing, and deploying AI/ML models and automation solutions to optimize cloud infrastructure management and operational efficiency. **Primary Responsibilities:** - Design, develop, and deploy machine learning models and AI algorithms for automating cloud infrastructure tasks like monitoring, fault detection, and predictive maintenance. - Collaborate with Cloud Engineers, ML Engineers, Data Scientists, and DevOps teams to integrate AI/ML solutions into cloud orchestration and management platforms. - Build scalable data pipelines and workflows using cloud-native...

You will be responsible for leading the data engineering team and driving data architecture initiatives. Your primary focus will be on designing and developing end-to-end data solutions, data pipelines, and ETL processes using various data technologies such as Snowflake, Databricks, Apache Spark, Hadoop, Dbt, Fivetran, and Azure Data Factory. Your strong expertise in Python and SQL will be crucial for success in this role. Key Responsibilities: - Designing and developing end-to-end data solutions, data pipelines, and ETL processes - Ensuring data governance, data quality, and data security principles are adhered to - Leading projects and teams in an Agile environment - Engaging in pre-sales ...

**Job Description:** As a Full Stack AI Developer at GlobalLogic, you will be responsible for developing and implementing AI solutions using a variety of programming languages and data processing libraries. Your role will involve working on distributed computing platforms, API development, and cloud platforms to create cutting-edge solutions for our clients. **Key Responsibilities:** - Must Have: Proficiency in programming languages such as Python, Java, or Scala - Must Have: Experience with data processing libraries like Pandas, NumPy, and Scikit-learn - Must Have: Proficient in distributed computing platforms like Apache Spark (PySpark, Scala), Torch, etc. - Must Have: Proficiency in API d...

As a Snowflake Engineer - Digital Solution Consultant Sr. Analyst at NTT DATA, you will be part of a dynamic team in Hyderabad, Telangana (IN-TG), India (IN). Your role will involve the following responsibilities: - Experience with other cloud data warehousing solutions. - Knowledge of big data technologies such as Apache Spark and Hadoop. - Proficiency in CI/CD pipelines and DevOps practices. - Familiarity with data visualization tools. NTT DATA is a $30 billion global innovator of business and technology services, serving 75% of the Fortune Global 100. Committed to helping clients innovate, optimize, and transform for long-term success, we have diverse experts in over 50 countries. Our ser...

As a Data Architect at EY, you'll have the opportunity to showcase your deep knowledge of data management in cloud and big data analytics, along with proven experience in building solutions, handling large teams, and demonstrating thought leadership. Your responsibilities will include: - Building solutions and capabilities in cloud and big data analytics - Hands-on knowledge of all major Big Data analytics platforms and cloud data engineering stack - Designing and implementing robust data solutions using cloud-based technologies - Expertise in various data-related tools and languages - Familiarity with Cloud computing platforms such as Azure, Databricks, and Snowflake - Knowledge of Apache S...

As an experienced data engineer, your role will involve enabling data frameworks to automate domain-specific data and analytics workflows. You will be responsible for incorporating GenAI into data engineering workflows, applications, or internal data and analytics tools. Additionally, optimizing ETL workflows to enhance data processing efficiency and reliability will be a key part of your responsibilities. Ensuring data integrity and performance in big data workflows and building frameworks to automate test suites and streamline data operations will also be crucial tasks. You will be developing and deploying AI-powered support tools for developers, such as code suggestion bots, to help ident...

As a Senior Software Engineer Java at SAP, you will be part of the App2App Integration team within SAP Business Data Cloud. Your role is crucial in accelerating the integration of SAP's application ecosystem with its unified data fabric, ensuring low-latency, secure, and scalable data exchange. Your responsibilities will include: - Designing and developing App2App integration components and services using Java, RESTful APIs, and messaging frameworks like Apache Kafka. - Building and maintaining scalable data processing and ETL pipelines to support real-time and batch data flows. - Integrating data engineering workflows with tools such as Databricks, Spark, or other cloud-based processing pla...