13 Apache Spark Jobs

Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : PySpark Good to have skills : Python (Programming Language), Apache Spark, Google BigQueryMinimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipeline...

Role Overview: As a BI Architect specializing in TIBCO Spotfire, your role involves designing and developing enterprise BI architectures within AWS. You will collaborate with data engineers and architects to ensure optimal data modeling and query performance. Your responsibilities also include designing scalable and secure Spotfire environments, developing reusable frameworks for dashboards, and optimizing BI query performance for enhanced dashboard responsiveness. Key Responsibilities: - Design and develop enterprise BI architectures for TIBCO Spotfire hosted in AWS - Partner with data engineers to optimize data modeling and caching in Spotfire - Design scalable and secure Spotfire environm...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark and Hive, Hbase or other NoSQL databases on Azure Cloud Data Platform or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developing strea...

As an expert contributor in the S&P Global Rating Commercial application and Data distribution Engineering Team, you will play a crucial role in leading and delivering strategy projects using the latest cloud-based, Databricks, & Containerization technologies. Your role will involve designing technology strategies, providing technical expertise, collaborating with different teams, and ensuring compliance with enterprise standards. You will have the opportunity to work with diverse technology stacks and be part of a team that values leadership, innovation, and timely delivery. **Key Responsibilities:** - Lead and design technology strategy on Data collection and enrichment projects - Provide ...

As a Staff Software Development Engineer (SDE) at LiveRamp, you will play a crucial role in designing and implementing scalable backend systems that power the data collaboration clean room platform. Your responsibilities will include: - Designing, building, and maintaining scalable backend systems to support high-performance data ingestion, partitioning, and constructing data pipelines. - Planning and delivering key development milestones to improve system reliability, throughput, and maintainability. - Writing clean, efficient, and well-tested code in Java, Go, Scala, and Spark for deployment in production environments. - Collaborating cross-functionally with product, infrastructure, and in...

As an experienced and talented Data Engineer specializing in Databricks, you will join our growing data engineering team on a full-time employment basis. Your role will involve designing and maintaining scalable data pipelines using Databricks and Apache Spark, optimizing ETL/ELT processes for structured and unstructured data, and implementing Lakehouse architecture for efficient data storage, processing, and analytics. You will also be responsible for managing Databricks Workflows, Jobs API, and AWS/Azure/GCP data services, ensuring data governance, security policies, and access controls. Key Responsibilities: - Design and maintain scalable data pipelines with Databricks and Apache Spark - ...

As a Bigdata Engineer with over 5 years of experience based in Hyderabad and currently working on a contract basis, you are equipped with the skills required to excel in this role. Your primary expertise lies in Spark, Scala, and Python, with a strong command over Apache Spark and Scala programming language. Your in-depth knowledge of Spark's core concepts, RDDs, DataFrames, and Spark SQL enables you to develop optimized and scalable Spark applications efficiently. Key Responsibilities: - Demonstrating proficiency in Apache Spark and Scala programming language - Utilizing your knowledge of Spark's core concepts, RDDs, DataFrames, and Spark SQL to develop optimized applications - Handling lar...

The Applications Development Senior Programmer Analyst position is an intermediate level role that involves collaborating with the Technology team to establish and implement new or updated application systems and programs. Your main objective will be to contribute to applications systems analysis and programming activities. Responsibilities: - Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, and model development. - Monitor and control all phases of the development process. - Provide user and operational support. - Analyze complex problems and recommend security measures post-implementation. - Consult with users/cl...

As a seasoned Senior Developer & Tech Lead, you will play a crucial role in guiding a team of skilled developers in an Agile setting. Your main responsibilities will include: - Conducting detailed impact analyses for code modifications, ensuring a thorough understanding of dependencies across application components. - Designing and implementing scalable, high-performance code using Java and Bigdata / Apache Spark. - Crafting high-quality, maintainable code that is modular, testable, and aligns with SOLID principles and industry-standard design patterns. - Developing robust unit tests using JUnit, with a focus on code coverage, business logic, readability, and reusability. - Leading and parti...

As a Senior Data Engineer, you will be responsible for leading the design, development, and implementation of scalable data pipelines and ELT processes using tools such as Databricks, DLT, dbt, and Airflow. Your main responsibilities will include collaborating with stakeholders to understand data requirements, delivering high-quality data solutions, and optimizing existing data pipelines to ensure data quality, reliability, and performance. Additionally, you will develop and enforce data engineering best practices, mentor junior data engineers, and monitor and troubleshoot data pipeline issues to ensure minimal disruption to business operations. Qualifications: - Bachelor's degree in Managem...

Role Overview: You will be an Advanced Data Engineer with a focus on utilizing a tech stack that includes proficiency in Python, SQL, Databricks, Snowflake Data Modeling, ETL process, Apache Spark and PySpark, data integration and workflow orchestration, and real-time data processing experience/frameworks. Cloud experience, particularly with Azure, is preferred for this role. Key Responsibilities: - Analyze and organize raw data to build data systems and pipelines in the Azure Cloud Platform - Migrate data from on-premises to cloud-based solution architectures using Azure - Develop ETL processes using Python, Spark, and SQL - Utilize Azure Data Lake, Azure Data Factory, Azure SQL, Azure Data...

Project Role : Data Platform Engineer Project Role Description : Assists with the data platform blueprint and design, encompassing the relevant data platform components. Collaborates with the Integration Architects and Data Architects to ensure cohesive integration between systems and data models. Must have skills : Apache Spark Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Platform Engineer, you will assist with the data platform blueprint and design, encompassing the relevant data platform components. Your typical day will involve collaborating with Integration Architects and Data Architects ...

Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Apache Spark Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and implementing ETL proce...