9690 Big Data Jobs - Page 22

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark and Hive, Hbase or other NoSQL databases on Azure Cloud Data Platform or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developing strea...

Your Role Gen AI ,Computer vision, Cloud exp , Azure Need to understand how the products are developed and even more important, as big data touches the privacy of consumers, they need to have a set of ethical Need to have a natural desire to go beneath the surface of a problem. Confident and self-secure as they more often than not will have to deal with situations where there is a lot of unknown. Good knowledge & experience in Artificial Neural Network ( ANN ), AI conversational Chatbot & Needs to be able to write the required Algorithm and coding programming. Preferably in different programming languages such as Python, Tensor Flow, Keras, AWS (ML and DL), Numpy, and Pytorch. Works in the a...

Are you ready to make an impact at DTCC? The Impact you will have in this role: The Kafka Administrator will play a critical role in ensuring the stability, security, and efficiency of our Confluent Kafka infrastructure. This position is part of the Architecture & Enterprise Services department, responsible for maintaining the firms IT infrastructure and supporting seamless business operations. The ideal candidate will have strong expertise in Kafka administration, security, performance tuning, and integration. Your Primary Responsibilities: Kafka Cluster Management Install, configure, and manage Kafka clusters, including Confluent Kafka components. Maintain and optimize existing clusters wh...

Educational Requirements Bachelor of Engineering Service Line Data & Analytics Unit Responsibilities A day in the life of an Infoscion As part of the Infosys delivery team, your primary role would be to interface with the client for quality assurance, issue resolution and ensuring high customer satisfaction. You will understand requirements, create and review designs, validate the architecture and ensure high levels of service offerings to clients in the technology domain. You will participate in project estimation, provide inputs for solution delivery, conduct technical risk planning, perform code reviews and unit test plan reviews. You will lead and guide your teams towards developing opti...

Roles and Responsibilities Client Impact Lead client engagements across a range of industries and problem statements Develop analytics strategies and programs for large, regional, and global clients by leveraging data and technology solutions to unlock client value Team Collaboration & Culture Lead team to creative insights and sound business recommendations, and deliver impactful client presentations while growing team members roles and skills Provide analytical and day-to-day project delivery team leadership, and create a collaborative and inclusive environment for all levels Collaborate with internal Mastercard stakeholders including Product and Business Development to scope projects, cre...

Role & responsibilities Preferred Skillset: Spark, Scala, Linux Based Hadoop Ecosystem (HDFS, Impala, Hive, HBase, etc.), SQL, Jenkins, and Unix Commands . Experience in Big data technologies , real time data processing platform( Spark Streaming - Kafka ) experience in Cloudera would be an advantage . Consistently demonstrates clear and concise written and verbal communication. Ability to multi-task and weekend support for production releases. Hands-on experience on Unix Command. Strong foundation in computer science fundamentals: Data structure, Algorithms, and coding. Experienced in Performance optimization techniques.

Through this role you have an opportunity to develop intuitive software that meets and exceeds the needs of the customer and the company. You also get to collaborate with team members to develop best practices and client requirements for the software. In this role it would be important for you to professionally maintain all codes and create updates regularly to address the customer s and company s concerns You will show your skills in analyzing and testing programs/products before formal launch to ensure flawless performance Troubleshooting coding problems quickly and efficiently will offer you a chance to grow your skills in a high-pace, high-impact environment. Software security is of prim...

• Determine the design and workout strategy for new projects. Lead the Project lifecycle closely following Governance models. • Identify and resolve issues reported within defines SLAs part of routine maintenance of existing applications built in SharePoint online. • Report extraction and automations to support different metrics, that are maintained in the applications. • Work with Stakeholders on the enhancements in the existing applications. Project Planning, Tracking, & Reporting • Workout budgetary estimates during kick off, after understanding the high-level requirements. • Support the team in project planning activities, identify risks proactively, escalating thru appropriate channels ...

Develop and maintain applications using Python , following best coding practices. Build and consume REST APIs , preferably using FastAPI . Collaborate closely with the product development team to co-create features and solutions. Utilize version control systems like GitHub, GitLab, or Azure DevOps effectively. Work with PySpark for large-scale data processing (added advantage). Understand and leverage Databricks architecture, including workflows and pipelines. Connect backend logic with frontend feature sets for seamless application functionality. Implement and maintain CI/CD pipelines using GitHub Actions or Azure DevOps. Ensure code quality, scalability, and maintainability with minimal su...

Lead the data engineering team by driving the design, build, test, and launch of new data pipelines and models on the production data platform Define and implement the processes needed to achieve operational excellence in all areas, including agile development, and data solutions Oversee the design, development and maintenance of data infrastructure, including data warehouses, data lakes, data integration components supporting optimal extraction, transformation, and loading of data Leading the communication with vendors, clients, the leadership team, and other stakeholders to facilitate effective project management and provide ongoing support Collaborate with business owners for roadmap plan...

Position Summary To be a technology expert architecting solutions and mentoring people in BI / Reporting processes with prior expertise in the Pharma domain. Job Responsibilities o Technology Leadership- Lead guide the team independently or with little support to design, implement deliver complex reporting and BI project assignments. o Technical portfolio- Expertise in a range of BI and hosting technologies like the AWS stack (Redshift, EC2), Qlikview, QlikSense, Tableau, Microstrategy, Spotfire o Project Management- Get accurate briefs from the Client and translate into tasks for team members with priorities and timeline plans. Must maintain high standards of quality and thoroughness. Shoul...

Design and scale ML solutions using advanced techniques in NLP, computer vision, and deep learning to solve complex retail problems at Walmart scale. Collaborate cross-functionally with engineering, product, and science teams to build impactful, production-ready models and integrate them into business processes. Drive innovation through rapid experimentation, PoC development, and continuous improvement using the latest research and technologies. Measure and optimize performance by defining model metrics, tracking business impact, and applying statistical methodologies for large-scale data analysis. Contribute to the AI community by publishing research, presenting at top-tier conferences, and...

As a Data Engineer, youll be building and scaling mission-critical data products that power worldwide operations business decisions. In this role, youll drive technical excellence while building data infrastructure and products. Youll be working on scalable solutions that process petabytes of data daily, serving diverse user personas from front-line operators to corporate leadership. We expect you to bring strong technical acumen to solve difficult problems at scale, while maintaining high operational standards and driving innovation in both traditional analytics and AI/ML domains. Design and implement scalable, secure data pipelines and infrastructure using AWS technologies and big data too...

1.Relevant accredited certifications for Azure, AWS, Cloud Engineering, and/or Databricks Data Engineering expertise in Big Data Management using Cloudera, Python, Hive, Scala, Data Warehouse, Data Lake, AWS, Azure 1 Designing and implementing data ingestion pipelines from multiple sources using Azure Databricks 2 Ensure data pipelines run smoothly and efficiently 3 Developing scalable and re-usable frameworks for ingesting of data sets 4 Integrating the end-to-end data pipeline - to take data from source systems to target data repositories ensuring the quality and consistency of data is maintained at all times 5 Working with event based / streaming technologies to ingest and process data 6 ...

Design, develop, and maintain scalable data pipelines and systems. Collaborate with data scientists and software engineers to integrate data solutions. Optimize and improve existing data systems for performance and reliability. Implement data security and privacy measures to protect sensitive information. Monitor and troubleshoot data pipeline issues to ensure data integrity. Required Qualifications Bachelors degree in Computer Science, Engineering, or a related field. Proven experience as a Data Engineer or in a similar role. Strong knowledge of SQL and database management systems. Experience with data pipeline and workflow management tools. Proficiency in programming languages such as Pyth...

Collaborative Solution Development- Engage with a diverse range of stakeholders, including analysts, software engineers, product managers, and customers, to understand their requirements and craft effective solutions. State-of-the-Art Pipelines and Architecture- Design, develop, deploy and operate high-quality production ELT pipelines and data architectures. Integrate data from various sources and formats, ensuring compatibility, consistency, and reliability. Design scalable architectures. Champion Data Best Practices- Proactively establish and advocate for best practices in performance, code quality, data validation, data governance, and discoverability across Adyen through mentoring, train...

Owning the design, development, and maintece of scalable solutions for ongoing metrics, reports, analyses, dashboards, etc. to support analytical and business needs. Development of queries and visualizations for ad-hoc requests and projects, as well as ongoing reporting. Specifically, own all SQL and Tableau ad hoc, recurring, trouble-shooting. Working with team leadership to understand customer needs and concerns as well as define solutions. Translating basic business problem statements into analysis requirements. Work with internal customers to define best output based on expressed stakeholder needs. Using analytical and statistical rigor to solve complex problems and drive business decisi...

Overview Join the Prodapt team in building AI/ML Solutions. If you have 5 - 10 years of experience & would like to work on a cutting-edge ML platform, enabling efficient analysis, testing, and decision-making at scale. Responsibilities Design, develop, and maintain simulation services and tools for ML feature, model, and rule evaluation. Build and optimize data pipelines for point-in-time historical data access and feature backfilling. Containerize and deploy ML models and services using Docker and Seldon on Kubernetes clusters. Integrate with Clients cloud infrastructure (GCP) and internal APIs for seamless simulation workflows. Collaborate with data scientists, ML engineers, and QA to deli...

About The Role : Job Title- Lead Engineer Location- Pune, India Corporate Title- Vice President Role Description As a lead engineer within the Regulatory Transaction Reporting department, you will lead and drive forward critical engineering initiatives and improvements to our application landscape whilst supporting and leading the engineering teams to excel in their roles. You will be closely aligned to the architecture function and development leads, ensuring alignment with planning and correct design and architecture governance is followed for all implementation work. You will lead by example and drive and contribute to automation and innovation initiatives with the engineering teams. What...

About The Role : Job TitleAutomation QA Engineer Corporate TitleAVP LocationPune, India Role Description Quality Assurance (QA) is responsible for ensuring testing strategies, plans and deliverables are executed for all applications and programmes of work within their domain. Work includes Designing and conducting testing procedures and frameworks to verify software and other technical products to ensure expected functional performance Investigating instances where software and technical products do not perform as expected Designing and conducting quality assurance practices to support business development activities Driving the adoption of recognized best practice and policy and contributes...

About The Role : Job TitleSenior Engineer Full Stack Corporate TitleVice President LocationPune, India Role Description Deutsche Bank is looking to expand its internal Technology capability in Pune to provide best in class technology solutions for the Banking industry. You will work as part of a cross-functional agile delivery team, including analysts, developers and testers. You will bring an innovative approach to software development, focusing on using the latest technologies and practices, as part of a relentless focus on business value. You will be someone who sees engineering as team activity, with a predisposition to open code, open discussion and creating a supportive, collaborative ...

About The Role : Job Title Lead AI Engineer, VP LocationPune, India Role Description Innovation and the TDI Chief Strategy Office Deutsche Banks Innovation team identifies, evaluates, and incubates cutting-edge technical innovation. It is part of the Chief Strategy Office of the banks Technology, Data & Innovation (TDI) function and works globally with all business lines and infrastructure functions of the bank. A focus of the team is to create value for clients and the bank using Generative Artificial Intelligence like Large Language Models (LLM), AI agents and other advanced data-driven technologies. As a Lead AI Engineer in the Innovation team of the TDI Chief Strategy Office you will lea...

Job Summary Have you streamed a movie online, accessed an email, or utilized cloud services? If so, youve likely engaged with Equinix data centers in some capacity. Equinix operates International Business Exchange (Data Center) facilities where the information-driven world flourishes. Our Data Centers serve as a platform for many of the worlds leading brands to launch their next groundbreaking innovations. As a Large Language Models (LLM) Researcher, you will be responsible for building and optimizing our Generative AI systems. You will evaluate and create state-of-the-art Generative AI models and develop agent workflows across multiple cloud platforms, including GCP, AWS, and Azure. Respons...

Must-have Skills BS/MS degree or equivalent in related technical field involving coding or equivalent practical experience with 5+ years of overall experience Experience in building and designing microservices and/or cloud native applications. Either strong on databases front or on building big data systems (including ETL pipelines) Being a problem solver with strong can-do attitude and ability to think on the go would be critical for success on this role. Strong fundamentals on OS, networks, distributed systems, designing fault tolerant and high available systems. Strong on at least one of the modern programming languages (Java, Kotlin, Python, C#) along with containers experiences (likes o...

Role: Data Engineer Years of Experience: 5-10 years Key Skills: Pyspark, SQL, Python, Data Bricks, Azure, AWS, GCP, Key responsibilities Design and build data infrastructure: Create and maintain scalable systems, databases, and data pipeline architectures to support data collection, storage, and processing. Transform and prepare data: Develop algorithms and methods to clean, combine, and transform raw data into a structured format suitable for analytics and machine learning. Ensure data quality and security: Implement data validation methods, monitor workflows, and ensure compliance with data governance and security policies. Collaborate with teams: Work with data scientists, analysts, and b...