1617 Data Pipelines Jobs - Page 25

Build and optimize large-scale data platforms on AWS. Must have strong skills in PySpark, AWS Glue, Redshift, S3, and Lambda. Design robust ETL/ELT pipelines in a fast-paced environment. Apply now recruiter2@rankskills.in Perks and benefits Mediclaim policy & Personal Accident Policy

As a Senior AI Scientist at Soothsayer Analytics, you will play a crucial role in leading the development of scalable AI/LLM solutions. Your expertise in ML fundamentals and data engineering will be instrumental in designing, fine-tuning, and deploying models such as LLMs and RAG architectures. Your responsibilities will include: - AI/LLM Development: - Fine-tune and optimize LLMs (e.g., GPT, Llama) and traditional ML models for production. - Implement retrieval-augmented generation (RAG), vector databases, and orchestration tools (e.g., LangChain). - Data Engineering: - Build scalable data pipelines for unstructured/text data using tools like Spark, Kafka, and Airflow. - Optimize storage/re...

Job Description About KPMG in India KPMG entities in India are professional services firm(s). These Indian member firms are affiliated with KPMG International Limited. KPMG was established in India in August 1993. Our professionals leverage the global network of firms, and are conversant with local laws, regulations, markets and competition. KPMG has offices across India in Ahmedabad, Bengaluru, Chandigarh, Chennai, Gurugram, Jaipur, Hyderabad, Jaipur, Kochi, Kolkata, Mumbai, Noida, Pune, Vadodara and Vijayawada. KPMG entities in India offer services to national and international clients in India across sectors. We strive to provide rapid, performance-based, industry-focused and technology-e...

New Opportunity for Data Scientist || Immediate to 30days Joiners Job Description Data Scientist (NLP/ML Research) Position: Data Scientist Location: [Hybrid Noida Office] Experience: 2+ years Team: AI/ML Advanced Research and Product Innovation Reports To: Director Data Science About the Role We're seeking a seasoned Data Scientist to lead NLP/ML research efforts and architect end-to-end solutions. You will be instrumental in driving innovation, mentoring junior team members, and working closely with stakeholders to align AI capabilities with strategic goals. Key Responsibilities Lead the development and deployment of production-grade NLP and ML models. Work with large language models (LLMs...

As a Sr Associate Data Analytics Lead, you will be responsible for the following: - Implementing data pipelines - Hands-on development experience in Spark and DataBricks pipeline - Optimization of DataBricks pipeline - Working with AWS services - Utilizing ElasticSearch effectively - Demonstrating strong SQL knowledge and Query Optimization skills - Familiarity with REST API - Leading individual projects using AGILE methodologies - Managing a team of developers through effective planning and execution - Collaborating with product management and business users to define product roadmap - Brainstorming solutions for team productivity, efficiency, and training - Creating tools and systems to en...

As a Data Quality Engineer, your role will involve ensuring data validation and integrity during migration and data pipeline workflows. You will be responsible for ETL automation testing by validating end-to-end data pipelines. Additionally, you will automate workflows in AWS for scalability and performance optimization. Leveraging CICD for continuous testing and deployment of data pipelines will also be a key aspect of your responsibilities. Furthermore, you will be expected to write complex SQL queries for debugging and data retrieval, as well as develop a framework to support data testing for regression. Qualifications: - Experience with ETL testing and data pipelines. - Proficiency in pr...

Role Overview: Graviton is a high-frequency trading firm operating in international markets, utilizing in-house trade execution strategies. As a Software Developer - Tech Tools at Graviton Research Capital LLP, you will be part of a team dedicated to building cutting-edge tools, services, and visualizations for the Tech and Quants departments. Your role will involve developing robust services that optimize trading operations, improve efficiency, and provide real-time insights on trading data. Key Responsibilities: - Design and maintain scalable, low-latency data pipelines for real-time and historical market data - Build dashboards and alerting systems to monitor system health and detect anom...

Role Overview: You are an experienced Senior Software Data Engineer joining the Data Integrations Team at Addepar. This team is a critical component of the Addepar Platform, which serves as a comprehensive data fabric providing a single source of truth for the product set. Your role involves collaborating with product counterparts in an agile environment to drive business outcomes, contributing to complex engineering projects, and utilizing a modern technology stack. Key Responsibilities: - Partner with multi-functional teams to design, develop, and deploy scalable data solutions that meet business requirements - Build pipelines supporting the ingestion, analysis, and enrichment of financial...

Role Overview: You are sought after for the position of Senior BigQuery Developer at FIS Clouds in Hyderabad. Your role will involve designing, developing, and maintaining robust, scalable data pipelines and advanced analytics solutions using BigQuery and other GCP-native services. Collaborating closely with data scientists, analysts, and business stakeholders, you will ensure efficient and secure access to enterprise data. Key Responsibilities: - Design, develop, and optimize BigQuery data warehouses and data marts to support analytical and business intelligence workloads. - Implement data modeling and best practices for partitioning, clustering, and table design in BigQuery. - Integrate Bi...

Role Overview: As a Solution and Data Architect at Amgen, you will be responsible for designing an enterprise analytics and data mastering solution using Databricks and Power BI. Your expertise in data architecture and analytics will be crucial in creating scalable, reliable, and high-performing solutions to support research cohort-building and advanced research pipelines. You will collaborate with various stakeholders to integrate data from multiple sources and ensure best practices for data governance and security. Key Responsibilities: - Architect scalable enterprise analytics solutions using Databricks, Power BI, and other modern data tools. - Utilize data virtualization, ETL, and semant...

Schaeffler is a dynamic global technology company and its success has been a result of its entrepreneurial spirit and long history of private ownership. Does that sound interesting to you As a partner to all of the major automobile manufacturers, as well as key players in the aerospace and industrial sectors, we offer you many development opportunities. Your Key Responsibilities Develop data pipelines and applies methods and tools to collect, store, process and analyze complex data sets, globally for assigned operations or functions. Design, govern, build and operate solutions for large-scale data architectures and applications across businesses and functions. Select, manage and work hands-o...

The Opportunity As an Engineer of Digital Solutions team, you will be part of Operation Center, India (INOPC-PG), aiming to develop a global value chain, where key business activities, resources and expertise are shared across geographic boundaries in order to optimize value for Hitachi Energy PowerGrid's customers across markets. How You'll Make An Impact Support our clients across the globe in optimizing their maintenance policies and enhancing their equipment availability. Handle configuration of Substation models in our State-of-the-art Digital platform and study the system to support with most efficient Reliability based maintenance plans. Act as an interface between the client and the ...

Hiring an AWS Data Engineer for a 6-month hybrid contractual role based in Bellandur, Bengaluru. The ideal candidate will have 4-6 years of experience in data engineering, with strong expertise in AWS services (S3, EC2, RDS, Lambda, EKS), PostgreSQL, Redis, Apache Iceberg, and Graph/Vector Databases. Proficiency in Python or Golang is essential. Responsibilities include designing and optimizing data pipelines on AWS, managing structured and in-memory data, implementing advanced analytics with vector/graph databases, and collaborating with cross-functional teams. Prior experience with CI/CD and containerization (Docker/Kubernetes) is a plus.

Product Head for Data, Analytics and Platform Role Overview: The Product Head - Data and Analytics will lead the strategic development and execution of our data platform initiatives , ensuring they align with business goals and deliver measurable results. You will be responsible for defining product strategy, setting OKRs , and ensuring the successful execution of programs that drive the product roadmap. You will work cross-functionally with technical teams to integrate advanced analytics, machine learning, and AI capabilities into scalable data platforms. Key Responsibilities: Define and drive the strategic direction for data platforms, aligning technical initiatives with business goals and...

Role Overview: As a Senior Principal Software Engineer - IT at Dell Technologies, you will be a key member of the Software Engineer-IT Team in Bangalore, contributing towards ensuring product integrity and customer satisfaction. Your role will involve turning business requirements into technology solutions through designing, coding, testing/debugging applications, and documenting procedures. You will have the opportunity to work with cutting-edge technologies and make a significant social impact. Key Responsibilities: - Serve as the testing consultant to leader in the IT organization and functional user groups - Lead changes in architecture, methodology, or programming procedures - Assist in...

Role Overview: DataWeave is a leading provider of advanced sales optimization solutions for Digital Commerce businesses, consumer brands, delivery intermediaries, and marketplaces. The Products team at DataWeave focuses on building data products that provide timely insights that are readily consumable and actionable at scale. As a member of the team, you will be responsible for the quality of code, delivering solutions into production, training and mentoring developers, working with technical project management, and providing technical leadership to multiple teams of software engineers. Key Responsibilities: - Take full responsibility for the quality of code and ensure solutions are delivere...

As an AI Data Engineering Lead at our company, you will play a crucial role in accelerating the next generation of intelligent automation across our platform. Your responsibilities will include architecting and implementing AI/ML solutions, building scalable data infrastructure, and staying current with AI/ML advances. Below are the details of what you'll be doing: **Role Overview:** You will lead the AI/ML strategy and execution, design end-to-end solutions, build and deploy production ML models, and define technical roadmaps. Additionally, you will build scalable data infrastructure, design robust data pipelines, and innovate by adopting new technologies. **Key Responsibilities:** - Design...

Role Overview: As a Director Data Strategy Lead at Capgemini Invent India's Enterprise Data Analytics (EDA) team, you will spearhead the Data Strategy capability. Your role involves shaping, selling, and delivering strategic data consulting engagements that drive business transformation through data. A visionary leader with expertise in data strategy, architecture frameworks, cloud modernization, and AI, along with strong customer-facing and storytelling abilities is essential for this position. Key Responsibilities: - Lead strategic conversations with CXOs and senior stakeholders, positioning Capgemini Invent's data strategy offerings. - Define and execute enterprise-wide data strategies al...

Role Overview: As a PL/SQL Data Modeller in this Contract position, you will be responsible for designing database structures, data models, and data frameworks to ensure data can be easily accessed, integrated, and utilized. You will be involved in data profiling and analysis, designing dataflows and data pipelines, and maintaining moderately complex data pipelines. Your role will also include collaborating with business stakeholders to understand their data integration needs, translating business requirements into well-structured data models, and providing recommendations for optimizing existing data systems. Exposure to cloud infrastructure, especially AWS, and familiarity with data visual...

Role Overview: You will collaborate with cross-functional teams (marketing, finance, operations, product) to understand data needs and deliver insights. You will identify trends, opportunities, and potential risks using statistical analysis. Additionally, you will develop and maintain data models, databases, and data pipelines. Providing ad-hoc analysis and reports to support leadership decisions will also be a key responsibility. Ensuring data accuracy, consistency, and integrity across all reporting will be crucial in this role. Key Responsibilities: - Collaborate with cross-functional teams to understand data needs and deliver insights - Identify trends, opportunities, and potential risks...

As a Data Scientist at our company, you will be responsible for supporting the development and deployment of machine learning models and analytics solutions aimed at improving decision-making across the mortgage lifecycle. Your key responsibilities will include: - Developing and maintaining machine learning models and statistical tools for various use cases such as risk scoring, churn prediction, segmentation, and document classification. - Collaborating with Product, Engineering, and Analytics teams to identify data-driven opportunities and support automation initiatives. - Translating business questions into modeling tasks and contributing to the design of experiments and success metrics. ...

Role Overview: As an experienced Data Architect specializing in Databricks, you will be responsible for leading the architecture, design, and migration of enterprise data workloads from on-premises systems to Databricks on Azure or AWS. Your role will involve designing scalable, secure, and high-performing data platforms based on the medallion architecture (bronze, silver, gold layers) to support large-scale ingestion, transformation, and publishing of data. Key Responsibilities: - Define and design cloud-native data architecture on Databricks utilizing Delta Lake, Unity Catalog, and related services. - Develop migration strategies for transferring on-premises data workloads (Oracle, Hadoop,...

**Job Description:** **Role Overview:** You are an experienced developer with over 5 years of experience in developing distributed systems. You have a demonstrated ability to program in Python, utilizing modules such as pandas, numpy, pytest, and pyspark for data manipulation techniques. Additionally, you have a working knowledge of the Flask framework, ORM, and Object Oriented Programming. Your expertise extends to data visualization software like Airflow, API, microservices architecture, and design with Rest or GraphQL. You possess a solid understanding of the development process pipeline, effective code version control & continuous Integration, Deployment. Experience with Kubernetes, Helm...

About AkzoNobel Since 1792, we've been supplying the innovative paints and coatings that help to color people's lives and protect what matters most. Our world class portfolio of brands including Dulux, International, Sikkens and Interpon is trusted by customers around the globe. We're active in more than 150 countries and use our expertise to sustain and enhance the fabric of everyday life. Because we believe every surface is an opportunity. It's what you'd expect from a pioneering and long-established paints company that's dedicated to providing sustainable solutions and preserving the best of what we have today while creating an even better tomorrow. Let's paint the future together. For mo...

About Us We're a small but ambitious startup revolutionizing the e-commerce landscape with cutting-edge AI solutions. Our mission is to empower sellers with the tools and insights they need to thrive in the competitive online marketplace. We're building a suite of AI-powered products and intelligent AI agents designed to streamline advertising, automate research, optimize listings, boost sales, and drive growth for businesses of all sizes. The Role: Senior Machine Learning Engineer (AI/ML & LLMs E-Commerce Ads) We're looking for a Senior Machine Learning Engineer to join our team and play a key role in developing the intelligence behind our AI-driven e-commerce platform. In this role, you'll...