530 Apache Spark Jobs - Page 10

As a Data Scientist at GlobalLogic, you will be responsible for working as a Full-stack AI Engineer. You must have proficiency in programming languages like Python, Java/Scala, and experience with data processing libraries such as Pandas, NumPy, and Scikit-learn. Additionally, you should be proficient in distributed computing platforms like Apache Spark (PySpark, Scala), and Torch. It is essential to have expertise in API development using Fast API, Spring Boot, and a good understanding of O&M - logging, monitoring, fault management, security, etc. Furthermore, it would be beneficial to have hands-on experience with deployment and orchestration tools like Docker, Kubernetes, Helm. Experience...

You will be joining one of the Big Four companies in India at either Bangalore or Mumbai. As a Spark/Scala Developer specializing in Big Data, you will play a key role in designing and implementing scalable data solutions while ensuring optimal performance. Your responsibilities will include translating business requirements into technical deliverables and contributing to the overall success of the team. To excel in this role, you should have 3 to 5 years of experience as a Big Data Engineer or in a similar position. Additionally, a minimum of 2 years of experience in Scala programming and SQL is required. You will be expected to design, modify, and implement solutions for handling data in H...

The Content and Data Analytics team at Elsevier is an integral part of Global Operations, delivering data analysis services using Databricks to product owners and data scientists. As a Senior Data Analyst, you will work independently to provide advanced insights and recommendations, leading analytics efforts with high complexity. Your responsibilities will include supporting data scientists within the Research Data Platform, performing various analytical activities such as diving into large datasets, data preparation, and evaluating data science algorithms. A keen eye for detail, strong analytical skills, and expertise in data analysis systems are essential, along with curiosity and dedicati...

We are looking for a seasoned Senior Data Architect with extensive knowledge in Databricks and Microsoft Fabric to join our team. In this role, you will be responsible for leading the design and implementation of scalable data solutions for BFSI and HLS clients. As a Senior Data Architect specializing in Databricks and Microsoft Fabric, you will play a crucial role in architecting and implementing secure, high-performance data solutions on the Databricks and Azure Fabric platforms. Your responsibilities will include leading discovery workshops, designing end-to-end data pipelines, optimizing workloads for performance and cost efficiency, and ensuring compliance with data governance, security...

Are you a seasoned engineer with a passion for data science and AI We are seeking a talented individual to lead our AM Analytics platform, a cutting-edge data science and AI platform that empowers our business users to develop innovative data models using the latest technologies. Your role will involve collaborating with business stakeholders across UBS Asset Management to gather and refine requirements, ensuring their successful delivery through technology solutions. You will act as a trusted advisor by identifying opportunities and guiding stakeholders on what's possible through technology. Additionally, you will lead the development of custom capabilities and workflows within the AM Analy...

You should have 5-12 years of experience in Big Data & Data related technologies. Your expertise should include a deep understanding of distributed computing principles and strong knowledge of Apache Spark. Proficiency in Python programming is required, along with experience using technologies such as Hadoop v2, Map Reduce, HDFS, Sqoop, Apache Storm, and Spark-Streaming for building stream-processing systems. You should have a good understanding of Big Data querying tools like Hive and Impala, as well as experience in integrating data from various sources such as RDBMS, ERP, and Files. Knowledge of SQL queries, joins, stored procedures, and relational schemas is essential. Experience with No...

As an AI Ops Expert, you will be responsible for taking full ownership of deliverables with defined quality standards, timelines, and budget constraints. Your primary role will involve designing, implementing, and managing AIops solutions to automate and optimize AI/ML workflows. Collaborating with data scientists, engineers, and stakeholders is essential to ensure the seamless integration of AI/ML models into production environments. Your duties will also include monitoring and maintaining the health and performance of AI/ML systems, developing and maintaining CI/CD pipelines specifically tailored for AI/ML models, and implementing best practices for model versioning, testing, and deploymen...

As a Pyspark Developer at Viraaj HR Solutions, you will be responsible for developing and maintaining scalable Pyspark applications for data processing. Your role will involve collaborating with data engineers to design and implement ETL pipelines for large datasets. Additionally, you will perform data analysis and build data models using Pyspark to derive insights. It will be your responsibility to ensure data quality and integrity by implementing data cleansing routines and leveraging SQL to query databases effectively. You will also create comprehensive data reports and visualizations for stakeholders, optimize existing data processing jobs for performance and efficiency, and implement ne...

As a Data Engineer, you will be responsible for collaborating with data scientists, software engineers, and business stakeholders to comprehend data requirements and develop efficient data models. Your key tasks will include designing and implementing robust data pipelines, ETL processes, and data integration solutions to ensure scalability and reliability. You will play a crucial role in the extraction, transformation, and loading of data from multiple sources while maintaining data quality, integrity, and consistency. It will be essential to optimize data processing and storage systems to effectively handle large volumes of structured and unstructured data. Additionally, you will be expect...

You are a highly skilled and experienced Senior Data Scientist with a strong background in Artificial Intelligence (AI) and Machine Learning (ML). You will be joining our team as an innovative, analytical, and collaborative team player with a proven track record in end-to-end AI/ML project delivery. This includes expertise in data processing, modeling, and model deployment. With a minimum of 5-7 years of experience in data science, your focus will be on AI/ML applications. Your technical skills will include proficiency in a wide range of ML algorithms such as regression, classification, clustering, decision trees, neural networks, and deep learning architectures (e.g., CNNs, RNNs, GANs). Str...

You are sought after by NTT DATA, a global innovator in business and technology services, to join as a Snowflake Engineer - Digital Solution Consultant Sr. Analyst in Hyderabad, Telangana (IN-TG), India (IN). The ideal candidate is expected to possess experience with cloud data warehousing solutions, knowledge of big data technologies like Apache Spark and Hadoop, familiarity with CI/CD pipelines, DevOps practices, as well as data visualization tools. NTT DATA, a trusted global innovator with a revenue of $30 billion, caters to 75% of the Fortune Global 100 companies. Committed to aiding clients in innovation, optimization, and transformation for long-term success, NTT DATA boasts a diverse ...

You are seeking a Lead - Python Developer / Tech Lead to take charge of backend development and oversee a team handling enterprise-grade, data-driven applications. In this role, you will have the opportunity to work with cutting-edge technologies such as FastAPI, Apache Spark, and Lakehouse architectures. Your responsibilities will include leading the team, making technical decisions, and ensuring timely project delivery in a dynamic work environment. Your primary duties will involve mentoring and guiding a group of Python developers, managing task assignments, maintaining code quality, and overseeing technical delivery. You will be responsible for designing and implementing scalable RESTful...

As a Research Scientist at Adobe, you will have the opportunity to engage in cutting-edge research within the Media and Data Science Research Laboratory. Your role will involve designing, implementing, and optimizing machine learning algorithms to address real-world problems related to understanding user behavior and enhancing marketing performance. You will also be responsible for developing scalable data generation techniques, analyzing complex data from various sources, and running experiments to study data quality. Collaboration will be a key aspect of your role, as you will work closely with product, design, and engineering teams to prototype and transition research concepts into produc...

As a Data Modeler, your primary responsibility will be to design and develop conceptual, logical, and physical data models supporting enterprise data initiatives. You will work with modern storage formats like Parquet and ORC, and build and optimize data models within Databricks Unity Catalog. Collaborating with data engineers, architects, analysts, and stakeholders, you will ensure alignment with ingestion pipelines and business goals. Translating business and reporting requirements into robust data architecture, you will follow best practices in data warehousing and Lakehouse design. Your role will involve maintaining metadata artifacts, enforcing data governance, quality, and security pro...

As a Big Data Architect specializing in Databricks at Codvo, a global empathy-led technology services company, your role is critical in designing sophisticated data solutions that drive business value for enterprise clients and power internal AI products. Your expertise will be instrumental in architecting scalable, high-performance data lakehouse platforms and end-to-end data pipelines, making you the go-to expert for modern data architecture in a cloud-first world. Your key responsibilities will include designing and documenting robust, end-to-end big data solutions on cloud platforms (AWS, Azure, GCP) with a focus on the Databricks Lakehouse Platform. You will provide technical guidance a...

As a Technical Lead / Data Architect, you will play a crucial role in our organization by leveraging your expertise in modern data architectures, cloud platforms, and analytics technologies. In this leadership position, you will be responsible for designing robust data solutions, guiding engineering teams, and ensuring successful project execution in collaboration with the project manager. Your key responsibilities will include architecting and designing end-to-end data solutions across multi-cloud environments such as AWS, Azure, and GCP. You will lead and mentor a team of data engineers, BI developers, and analysts to deliver on complex project deliverables. Additionally, you will define a...

You should have 9+ years of experience and be located in Chennai. You must possess in-depth knowledge of Python and have good experience in creating APIs using FastAPI. It is essential to have exposure to data libraries like Pandas, DataFrame, NumPy etc., as well as knowledge in Apache open-source components. Experience with Apache Spark, Lakehouse architecture, and Open table formats is required. You should also have knowledge in automated unit testing, preferably using PyTest, and exposure in distributed computing. Experience working in a Linux environment is necessary, and working knowledge in Kubernetes would be an added advantage. Basic exposure to ML and MLOps would also be advantageou...

As a Data Engineer at Lifesight, you will play a crucial role in the Data and Business Intelligence organization by focusing on deep data engineering projects. Joining the data platform team in Bengaluru, you will have the opportunity to contribute to defining the technical strategy and data engineering team culture in India. Your responsibilities will include designing and constructing data platforms and services, as well as managing data infrastructure in cloud environments to support strategic business decisions across Lifesight products. You will be expected to build highly scalable distributed data processing systems, data solutions, and data pipelines that optimize data quality and are...

As a Senior Data Engineer at our Pune location, you will play a critical role in designing, developing, and maintaining scalable data pipelines and architectures using Data bricks on Azure/AWS cloud platforms. With 6 to 9 years of experience in the field, you will collaborate with stakeholders to integrate large datasets, optimize performance, implement ETL/ELT processes, ensure data governance, and work closely with cross-functional teams to deliver accurate solutions. Your responsibilities will include building, maintaining, and optimizing data workflows, integrating datasets from various sources, tuning pipelines for performance and scalability, implementing ETL/ELT processes using Spark ...

As a Data Quality Engineer, your primary responsibility will be to analyze business and technical requirements to design, develop, and execute comprehensive test plans for ETL pipelines and data transformations. You will perform data validation, reconciliation, and integrity checks across various data sources and target systems. Additionally, you will be expected to build and automate data quality checks using SQL and/or Python scripting. It will be your duty to identify, document, and track data quality issues, anomalies, and defects. Collaboration is key in this role, as you will work closely with data engineers, developers, QA, and business stakeholders to understand data requirements and...

We are seeking a Senior Data Engineer who is proficient in Azure Databricks, PySpark, and distributed computing to create and enhance scalable ETL pipelines specifically for manufacturing analytics. Your responsibilities will include working with industrial data to support real-time and batch data processing needs. Your role will involve constructing scalable real-time and batch processing workflows utilizing Azure Databricks, PySpark, and Apache Spark. You will be responsible for data pre-processing tasks such as cleaning, transformation, deduplication, normalization, encoding, and scaling to guarantee high-quality input for downstream analytics. Designing and managing cloud-based data arch...

The Applications Development Technology Lead Analyst role at our organization involves working closely with the Technology team to establish and implement new or updated application systems and programs. Your primary responsibility will be to lead applications systems analysis and programming activities. As the Applications Development Technology Lead Analyst, you will collaborate with various management teams to ensure seamless integration of functions to achieve organizational goals. You will also be responsible for identifying necessary system enhancements for deploying new products and process improvements. Additionally, you will play a key role in resolving high-impact problems and proj...

As an ideal candidate for this role, you should possess in-depth knowledge of Python and have good experience in creating APIs using FastAPI. You should also have exposure to data libraries like Pandas, DataFrame, NumPy, as well as knowledge in Apache open-source components and Apache Spark. Familiarity with Lakehouse architecture and Open table formats is also desirable. Additionally, you should be well-versed in automated unit testing, preferably using PyTest, and have exposure to distributed computing. Experience working in a Linux environment is a must, while working knowledge in Kubernetes would be considered an added advantage. Basic exposure to ML and MLOps would also be advantageous ...

As a Custom Software Engineer, you will be responsible for developing custom software solutions to design, code, and enhance components across systems or applications. Your role will involve using modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. On a typical day, you will collaborate with cross-functional teams to understand business requirements and work towards aligning the software solutions with project goals. You are expected to be subject matter expert (SME) within the team, make team decisions, and engage in problem-solving activities to contribute to the success of the organization. Additionally, you will be men...

The Content and Data Analytics team is an integral part of Global Operations at Elsevier, within the DataOps division. The team primarily provides data analysis services using Databricks, catering to product owners and data scientists of Elsevier's Research Data Platform. Your work in this team will directly contribute to the development of cutting-edge data analytics products for the scientific research sector, including renowned products like Scopus and SciVal. As a Data Analyst II, you are expected to possess a foundational understanding of best practices and project execution, with supervision from senior team members. Your responsibilities will include generating basic insights and reco...