905 Apache Spark Jobs - Page 14

As an SDE-3 Backend at Crimson Enago, you will play a key role in leading a team of web developers and driving end-to-end project development and delivery. Your responsibilities will involve setting engineering examples, overseeing team recruitment and training, and dedicating 100% of your time to coding. Collaboration with various stakeholders such as the Engineering Manager, Principal Engineer, SDE-3 leads, and Technical Project Manager will be essential to ensure successful project outcomes. Key Responsibilities: - Lead a team of web developers and contribute significantly to project development and delivery - Set engineering examples and mentor team members - Recruit, train, and manage t...

As an Applications Development Programmer Analyst at our company, your role will involve participating in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. Your main objective will be to contribute to applications systems analysis and programming activities. **Key Responsibilities:** - Utilize your knowledge of applications development procedures and concepts to identify and define necessary system enhancements. - Identify and analyze issues, make recommendations, and implement solutions based on your understanding of business processes, system processes, and industry standards to solve complex issues. - Analyze ...

Role & responsibilities Preferred candidate profile Job Title: Data Engineer Azure Databricks & PySpark Location: Bangalore/ Kolkata Department: Data & Analytics Employment Type: Full-Time-ITC Infotech Hybrid 3+ Years immediate joiners up to 30 LPA Job Summary: We are seeking a skilled and motivated Data Engineer with hands-on experience in Azure Databricks , PySpark , and cloud-based data solutions. The ideal candidate will be responsible for designing, building, and maintaining scalable data pipelines and analytics solutions that support business decision-making and digital transformation initiatives. Key Responsibilities: Design and develop robust, scalable data pipelines using PySpark on...

Position Purpose The role is split between architectural / project-based tasks and third-line support of infrastructure hosting CIB SSC Data services. The team is responsible for the CIB SSC Data platform across EMEA, AMER and APAC with an application support team (APS) based in Chennai and Mumbai, India acting as a single, global team. Responsibilities Direct Responsibilities To provide architectural expertise thereby driving change and improvement in technology and process. To contribute to the development of the Data technology roadmap. To liaise with CIB architecture teams in undertaking architectural reviews. To stay abreast of emerging technologies and technology trends. To work with P...

Role & responsibilities • Develop and maintain scalable data pipelines using Apache Spark on Databricks. • Build end-to-end ETL/ELT pipelines on AWS using services like S3, Glue, Lambda, EMR, and Step Functions. • Collaborate with data scientists, analysts, and business stakeholders to deliver high-quality data solutions. • Design and implement data models, schemas, and lakehouse architecture in Databricks. • Optimize and tune Spark jobs for performance and cost-efficiency. • Integrate data from multiple structured and unstructured data sources. • Monitor and manage data workflows, ensuring data quality, consistency, and security. • Follow best practices in CI/CD, code versioning (Git), and ...

Position Purpose The Regulatory Reporting team is ramping up to about 20 members to address the ambitious project of standardizing the activity for APAC region. This long-term project involves the reporting of Balance Sheet, P&L, Capital and a more recent and intense focus on liquidity, especially Basel III LCR requirement. Our team captures reporting requirements from the users located all across APAC, comes up with efficient propositions, implements solution and follows up to guarantee user comfort on the long run. To operate efficiently, the team is constituted of complementing profiles: business analysts understand the logic from the users and develop the reports, data managers are the r...

Responsibilities Direct Responsibilities * Represent CEP IT at local and global CIB Forums on Technical Architecture subjects * Propose / Review/ Challenge Application Architecture and Design. * Guide/Review Technology Obsolescence Management plan for CEP teams. * Lead automation and guide teams to align with shift left and shift right strategy by encouraging a mindset for automation first and reduce recursive manual efforts * Goto person to find solutions to any technical challenges in the domain. * Hands on and lead by example. * Resolve performance bottlenecks. * Review Performance test reports and propose preventive measures to be undertaken. * Take ownership of POCs and technical feasib...

HCL Hiring for Python role!! Experience-4 to 7 Years Location-Chennai/Bangalore Notice Period-Immediate to 60 Days Design, develop, and implement robust and efficient ETL/ELT data pipelines using Python to extract, transform, and load data from diverse sources (e.g., databases, APIs, flat files, streaming data) into target systems. • Utilize Python libraries and frameworks commonly used in data engineering (e.g., Pandas, NumPy, Dask, Apache Spark with PySpark). • Implement data quality checks, data validation, and data reconciliation processes within pipelines to ensure data accuracy and integrity.

Role Overview: As an AI Ops Expert, you will be responsible for the delivery of projects with defined quality standards within set timelines and budget constraints. Your role will involve managing the AI model lifecycle, versioning, and monitoring in production environments. You will be tasked with building resilient MLOps pipelines and ensuring adherence to governance standards. Additionally, you will design, implement, and oversee AIops solutions to automate and optimize AI/ML workflows. Collaboration with data scientists, engineers, and stakeholders will be essential to ensure seamless integration of AI/ML models into production systems. Monitoring and maintaining the health and performan...

Role Overview: As an Applications Development Group Manager at Citigroup, you will be responsible for managing a team to establish and implement new or revised application systems and programs in coordination with the Technology Team. Your primary objective will be to drive applications systems analysis and programming activities. Key Responsibilities: - Manage multiple teams of professionals to accomplish established goals and conduct personnel duties for the team, such as performance evaluations, hiring, and disciplinary actions - Provide strategic influence, exercise control over resources, budget management, and planning, while monitoring end results - Utilize in-depth knowledge of conce...

Role Overview: You are a highly experienced Big Data Engineer with a strong background in Python, Java, or Scala. Your main responsibility is to design, build, and maintain scalable data pipelines and data lake architectures. You excel in managing and delivering on complex data engineering projects, providing you with an excellent opportunity to contribute to scalable data solutions in a collaborative environment. Key Responsibilities: - Develop robust data engineering solutions using technologies like Apache Spark, Hadoop, Hive, HBase, Kafka, Airflow, and Oozie. - Collaborate with cross-functional teams to ensure high data quality and governance. - Utilize cloud platforms such as AWS, Azure...

As a PySpark Developer at our company, you will be responsible for developing and maintaining data processing pipelines using PySpark, Apache Spark's Python API. You will collaborate with data engineers, data scientists, and other stakeholders to design and implement scalable and efficient data processing solutions. Key Responsibilities: - Designing, developing, and maintaining PySpark data processing pipelines for processing structured and unstructured data - Collaborating with data engineers and data scientists to understand data requirements and design efficient data models and transformations - Optimizing and tuning PySpark jobs for performance, scalability, and reliability - Implementin...

Role Overview: As an Applications Development Senior Programmer Analyst, you will play an intermediate role in participating in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. Your primary objective will be to contribute to applications systems analysis and programming activities. You will conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establishing and implementing new or revised applications systems and programs to meet specific business needs or user areas. Your responsibilities will also include monitoring ...

Role Overview: You will play a crucial role as a Data Platform Engineer Lead at Barclays, focusing on building and maintaining systems for data collection, storage, processing, and analysis. This includes managing data pipelines, data warehouses, and data lakes to ensure the accuracy, accessibility, and security of all data within the organization. Key Responsibilities: - Utilize hands-on coding experience in Java or Python and a strong understanding of AWS development to implement various services like Lambda, Glue, Step Functions, and IAM roles. - Proficiently build efficient data pipelines using Apache Spark and other AWS services to streamline data processing and analysis. - Demonstrate ...

Role Overview: At PwC, our focus is on leveraging data to drive insights and make informed business decisions in the field of data and analytics. As a data analyst, you will assist clients in optimizing their operations and achieving strategic goals by extracting insights from large datasets and facilitating data-driven decision-making. Your role will involve utilizing advanced analytics techniques such as data manipulation, visualization, and statistical modeling to help clients solve complex business problems effectively. Key Responsibilities: - **Architecture Design:** - Designing and implementing scalable, secure, and high-performance architectures for Generative AI applications. - Integ...

As a Software and Data Science Engineer, you will play a crucial role in the team by utilizing your strong engineering background and analytical skills. Your primary responsibility will involve developing and maintaining the platform, using large-scale data to extract valuable insights that drive business growth. Collaboration with various technical and non-technical stakeholders will be essential for success in this role. Key Responsibilities: - Utilize your analytical mindset and problem-solving skills to address complex technical challenges using data structures, storage systems, cloud infrastructure, and front-end frameworks. - Work on front-end and back-end components, contributing to t...

As an experienced data engineer specializing in dashboard story development and data engineering pipelines, you will play a crucial role in analyzing log data to extract actionable insights for product enhancements and feature optimization. Collaborating with cross-functional teams, you will gather business requirements, translate them into technical specifications, and design interactive dashboards using tools like Tableau, Power BI, or ThoughtSpot AI. - Manage large volumes of application log data using Google Big Query. - Ensure data integrity, consistency, and accessibility for analytical purposes. - Identify patterns, trends, and anomalies in log data to visualize key metrics and insigh...

Role Overview: Sciative, a dynamic pricing company powered by artificial intelligence and big data, is seeking a dynamic and organized self-starter to join their Tech Team. As a Data Engineer at Sciative, you will collaborate with data scientists, software engineers, and business stakeholders to design efficient data models, develop robust data pipelines, and ensure data quality and integrity. You will play a crucial role in optimizing data processing systems and contributing to the continuous improvement of data engineering practices within the organization. Key Responsibilities: - Collaborate with data scientists, software engineers, and business stakeholders to understand data requirement...

As a Data Science Manager in the Research and Development (R&D) team at our organization, you will play a crucial role in driving innovation through advanced machine learning and AI algorithms. Your primary responsibility will involve conducting applied research, development, and validation of cutting-edge algorithms to address complex real-world problems on a large scale. - Collaborate closely with the product team to gain insights into business challenges and product objectives - Devise creative algorithmic solutions - Create prototypes and demonstrations to validate new ideas - Transform research findings into practical innovations by collaborating with AI Engineers and software engineers...

As a Cloud Platform Administrator in Bengaluru, you will be responsible for various key tasks related to user and access management, cluster and resource management, job management, monitoring and troubleshooting, platform maintenance, collaboration, and cost optimization. Your primary responsibilities will include: - Creating users, setting roles, and managing access controls for the cloud platform. - Provisioning clusters, monitoring resource usage, and optimizing performance for efficient operations. - Scheduling, managing, and optimizing jobs and workflows. Furthermore, you will play a vital role in: - Monitoring system performance, troubleshooting issues, managing updates, backups, inte...

Position Overview: We are looking for an experienced and driven Data Engineer to join our team. The ideal candidate will have a strong foundation in big data technologies, particularly Spark, and a basic understanding of Scala to design and implement efficient data pipelines. As a Data Engineer at CloudKeeper, you will be responsible for building and maintaining robust data infrastructure, integrating large datasets, and ensuring seamless data flow for analytical and operational purposes. Key Responsibilities: Design, develop, and maintain scalable data pipelines and ETL processes to collect, process, and store data from various sources. Work with Apache Spark to process large datasets in a ...

Role: AWS Data Engineer Skills: AWS Cloud, Data Bricks, AWS services, Pyspark, RedShift, SQL , Data Warehouse, Apache Spark, ETL Location: Hyderabad, Chennai, Bangalore, Pune, Mumbai, Delhi, Kolkata, Indore, Noida, Gurugram Roles and Responsibilities : Strong hands-on experience in Python programming and PySpark Experience using AWS services (RedShift, Glue, EMR, S3 & Lambda) Experience working with Apache Spark and Hadoop ecosystem. Experience in writing and optimizing SQL for data manipulation Good Exposure to scheduling tools. Airflow is preferable. Data Warehouse Experience with AWS Redshift or Hive Experience in implementing security measures for data protection. Expertise to build/test...

Job Description: As an AI + Backend Developer at AskGalore, you will be responsible for developing and maintaining backend systems and APIs for AI-powered applications. Your role will involve designing, implementing, and optimizing machine learning models and algorithms for real-world applications. Collaborating with cross-functional teams to integrate AI models into scalable, production-ready systems will be a key aspect of your responsibilities. You will also be building and managing robust, secure, and efficient server-side architectures using Python frameworks such as Django, Flask, and FastAPI. Additionally, you will optimize application performance, troubleshoot backend issues, work wi...

As an Applications Development Senior Programmer Analyst, your main objective will be to contribute to applications systems analysis and programming activities. You will be responsible for participating in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. Your responsibilities will include: - Conducting tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establishing and implementing new or revised applications systems and programs to meet specific business needs or user areas. - Monitoring and controlling all phases of the ...

Role Overview: Join a dynamic and innovative educational technology organization at Curriculum Associates and play a pivotal role in developing impactful software solutions. As a Senior Software Engineer, you will leverage your robust experience in Scala, Snowflake, database systems, and Big Data technologies. This position emphasizes both individual technical contributions and collaborative efforts within an Agile environment to deliver scalable and efficient solutions that address complex business needs. Key Responsibilities: - Lead technical initiatives and contribute as a senior team member to achieve project goals and deadlines. - Collaborate with team members to design, implement, and ...