876 Apache Spark Jobs - Page 6

Should have experience in building scalable enterprise data integration solutions preferred on the Microsoft SQL Server Azure Data Warehouse Database platforms

Data Pipelines & ETL Processes: Snowflake Data Modeling: Cloud & On-Prem Solutions: Design and manage cloud-based data infrastructure (AWS, Azure, GCP), Real-Time Data Processing:

Data Pipelines & ETL Processes: Snowflake Data Modeling: Cloud & On-Prem Solutions: Design and manage cloud-based data infrastructure (AWS, Azure, GCP), Real-Time Data Processing:

Primarily looking for a Data Engineer with expertise in processing data pipelines using Databricks PySpark SQL on Cloud distributions like AWS Must have AWS Databricks Good to have PySpark Snowflake Talend

Primarily looking for a Data Engineer with expertise in processing data pipelines using Databricks PySpark SQL on Cloud distributions like AWS Must have AWS Databricks Good to have PySpark Snowflake Talend

Primarily looking for a Data Engineer with expertise in processing data pipelines using Databricks PySpark SQL on Cloud distributions like AWS Must have AWS Databricks Good to have PySpark Snowflake Talend

Primarily looking for a Data Engineer with expertise in processing data pipelines using Databricks PySpark SQL on Cloud distributions like AWS Must have AWS Databricks Good to have PySpark Snowflake Talend

Advanced Data Engineer :Azure Databricks + Pyspark / Python + SQL + Data Modeling. ETLL Process, Apache Spark, Pyspark, Data Integration and workflow orchestration , real time data processing experience and frameworks, Azure cloud experience

As a skilled Data Engineer with at least 2 years of experience and proficiency in Python, along with a strong background in Natural Language Processing (NLP) and Machine Learning (ML), you will play a crucial role in designing, developing, and maintaining data pipelines and systems to support NLP and ML initiatives at our company. Working closely with data scientists and software engineers, you will deploy cutting-edge solutions that leverage NLP and ML techniques effectively. **Responsibilities:** - Develop and maintain data pipelines to ingest, process, and analyze large volumes of text data. - Implement NLP algorithms and techniques for tasks such as text classification, named entity reco...

As a GCP Data Engineer, you will be responsible for designing and implementing solutions on Google Cloud Platform (GCP) utilizing various GCP components. Key Responsibilities: - Implementing and architecting solutions on GCP using components such as BigQuery, SQL, Cloud Composer/Python, Cloud Functions, Dataproc with PySpark, Python Injection, Dataflow with PUB/SUB. - Experience with Apache Beam, Google Dataflow, and Apache Spark in creating end-to-end data pipelines. - Proficiency in Python, Hadoop, Spark, SQL, BigQuery, BigTable, Cloud Storage, Datastore, Spanner, Cloud SQL, and Machine Learning. - Programming expertise in Java, Python, and other relevant technologies. - Certified in Googl...

As a dynamic and organized self-starter interested in joining Sciative's Tech Team, you will have the opportunity to collaborate with data scientists, software engineers, and business stakeholders. Your primary responsibilities will include: - Collaborating with team members to understand data requirements and design efficient data models. - Developing, implementing, and maintaining robust and scalable data pipelines, ETL processes, and data integration solutions. - Optimizing data processing and storage systems to handle large volumes of structured and unstructured data efficiently. - Performing data cleaning, normalization, and enrichment tasks to prepare datasets for analysis and modeling...

Role Overview: As a Data Analyst at PwC, you will play a crucial role in leveraging data to drive insights and support clients in making informed business decisions in the field of data and analytics. Your responsibilities will include utilizing advanced analytical techniques to extract insights from large datasets, facilitating data-driven decision-making for clients, performing data manipulation, visualization, and statistical modeling, and supporting clients in solving complex business problems. Key Responsibilities: - Designing and implementing scalable, secure, and high-performance architectures for Generative AI applications - Integrating Generative AI models into existing platforms - ...

As a PySpark Data Engineer at our company, you will play a crucial role in developing robust data processing and transformation solutions within our data platform. Your responsibilities will include designing, implementing, and maintaining PySpark-based applications to handle complex data processing tasks, ensure data quality, and integrate with diverse data sources. The ideal candidate for this role would possess strong PySpark development skills, experience with big data technologies, and the ability to thrive in a fast-paced, data-driven environment. Key Responsibilities: - Design, develop, and test PySpark-based applications for processing, transforming, and analyzing large-scale dataset...

As a Senior Data Analyst at Elsevier, you will be part of the Content and Data Analytics team under DataOps in Global Operations. Your primary focus will be on providing data analysis services using Databricks for product owners and data scientists of Elsevier's Research Data Platform, contributing to the development of leading data analytics products for scientific research, including Scopus and SciVal. - **Support data scientists within the Domains of the Research Data Platform by engaging in various analytical activities such as analyzing large datasets, performing data preparation, and reviewing data science algorithms. - Create advanced-level insights and recommendations, as well as lea...

As an Expert Software Engineer Java at SAP, you will play a crucial role in the App2App Integration team within SAP Business Data Cloud. Your responsibilities will involve leading strategic initiatives to enhance seamless integration patterns across SAP applications and the BDC data fabric. Your deep technical expertise in Java, ETL, distributed data processing, Kafka, cloud-native development, and DevOps will be pivotal in guiding architectural direction and providing hands-on leadership in building real-time, event-driven, and secure communication across a distributed enterprise landscape. **Responsibilities:** - Lead and design App2App integration components and services using Java, RESTf...

As a Production Specialist at Wipro Limited, your role is to support process delivery by ensuring daily performance of the team, resolving technical escalations, and developing technical capability within the team. - Oversee and support process by reviewing daily transactions on performance parameters - Review performance dashboard and the scores for the team - Support the team in improving performance parameters by providing technical support and process guidance - Record, track, and document all queries received, problem-solving steps taken, and total successful and unsuccessful resolutions - Ensure standard processes and procedures are followed to resolve all client queries - Resolve clie...

Hiring: Cloud DevOps Engineer Location: Remote (India-based) Were looking for a seasoned Cloud DevOps Engineer with 10+ years of hands-on experience to join a dynamic team driving innovation in cloud-native data platforms. Key Responsibilities: Architect & optimize Databricks Pipelines in Azure Data Lakehouse Manage Azure infrastructure: Databricks, AKS/ACI, Storage, Networking Implement monitoring with Azure Monitor, Log Analytics, Prometheus/Grafana Enable microservices architecture for scalable data solutions Drive CI/CD automation using Terraform, ARM templates, GitHub Actions/Azure DevOps Required Skills: Strong experience with Azure Databricks, Delta Lake, Apache Spark Deep knowledge o...

Role & responsibilities Develop and maintain data pipelines using AWS Glue , Lambda , EMR , and Snowflake . Implement real-time ingestion using Snowpipe and Streams for CDC (Change Data Capture). Write efficient PySpark or Scala Spark jobs for large-scale data processing. Automate workflows and orchestrate jobs using AWS Step Functions , Airflow , or similar tools. Optimize Snowflake queries and warehouse performance. Collaborate with Data Scientists, Analysts, and DevOps teams to deliver reliable data solutions. Monitor and troubleshoot data pipeline failures and latency issues. Preferred candidate profile Strong experience with Snowflake architecture, SQL, and performance tuning. Hands-on ...

As a Senior Data Engineer at our company, you will play a crucial role in designing, developing, and optimizing scalable data pipelines and applications using Apache Spark and Databricks. Your responsibilities will include: - Designing, developing, and implementing scalable data pipelines and applications with a focus on adhering to industry best practices. - Performing in-depth performance tuning and optimization of Spark applications within the Databricks environment. - Troubleshooting complex issues related to data ingestion, transformation, and pipeline execution. - Collaborating with cross-functional teams to deliver end-to-end data solutions. - Continuously evaluating and adopting new ...

Role Overview: As a Data Architect/Developer at HCL Technologies, your main responsibility will be designing and implementing data models for cloud-based data warehouses such as Microsoft Fabric, Azure Synapse, Azure SQL Database, and Snowflake. You will need to demonstrate expertise in OLAP modeling and optimization for analytical reporting. Moreover, hands-on experience in Power BI semantic model design and performance tuning will be crucial for this role. You will also be involved in performance tuning, indexing, and query optimization. Experience with big data technologies like Apache Spark and Databricks will be considered a plus. Key Responsibilities: - Design and implement data models...

As a Software Engineer at Fanatics, you will be responsible for designing and implementing components of the consent management framework, building privacy compliance reporting and data retention management features, and contributing to the data certification framework with configurable data quality validation rules. Additionally, you will help develop data lineage tracking capabilities and maintain APIs for integrating data quality, privacy, and governance tools. Your role will involve implementing data classification and privacy-preserving transformation pipelines, working cross-functionally with data engineering, analytics, and business teams, as well as participating in code reviews and ...

As a Staff Software Engineer in the Data Quality & Audience team at Warner Bros. Discovery, you will play a crucial role in maintaining data quality and facilitating data-driven decisions. Your technical expertise will be vital in developing systems that adhere to quality standards and in creating datasets using top-notch software engineering and data management practices. Your responsibilities will include: - Applying comprehensive knowledge of technology options within the Data Engineering ecosystem - Constructing systems and datasets while adhering to best practices and operational excellence - Grasping data warehousing concepts, ETL processes, and tools such as Apache Airflow and Talend ...

As an AI/ML Product Designer at our company, your role involves designing and building AI/ML products at scale to enhance customer Understanding & Sentiment analysis, recommend customer requirements, suggest optimal inputs, and improve the efficiency of processes. You will collaborate with product owners and business stakeholders to achieve these goals. Key Responsibilities: - Lead a team of junior and experienced data scientists - Lead and participate in end-to-end ML project deployments, including feasibility analysis, design, development, validation, and application of state-of-the-art data science solutions - Push the boundaries of data science applications by utilizing data mining, visu...

As a Big Data Developer at Viraaj HR Solutions, your role involves designing and implementing scalable Big Data solutions using Hadoop technologies. You will be responsible for developing and maintaining ETL processes to manage and process large data sets. Collaborating with data architects and analysts to gather requirements and deliver solutions will be a key part of your responsibilities. Additionally, you will optimize existing Hadoop applications for maximizing efficiency and performance. Key Responsibilities: - Design and implement scalable Big Data solutions using Hadoop technologies. - Develop and maintain ETL processes to manage and process large data sets. - Collaborate with data a...

As a Software Engineer at Google, you will be developing next-generation technologies that impact how billions of users connect, explore, and interact with information and each other. Your role will involve handling information at massive scales and contributing to various areas such as information retrieval, distributed computing, system design, networking, security, artificial intelligence, and more. You will have the opportunity to work on critical projects and collaborate with diverse teams, showcasing your leadership skills and adaptability as you tackle new challenges across the full technology stack. Key Responsibilities: - Enhance Apache Spark for improved performance, reliability, s...