1586 Data Pipeline Jobs - Page 39

In This Role, Your Work Will Involve Technical Implementation & Integration Build proof-of-concepts and MVPs leveraging both the internal AI tech stack and publicly available AI tools Develop and enhance AI-driven solutions using Python Assist in system integration and deployment on cloud platforms (Azure, AWS) Contribute to the internal knowledge base and engineering playbooks Participate in cross-functional projects with product, engineering, and customer success teams Customer Solution Design, Deployment & Success Assist in Client Engagement and Discovery Shadow Solutions Engineers, participate in customer calls, and help analyze requirements to understand complex business workflows and k...

Who we are About Stripe Stripe is a financial infrastructure platform for businesses Millions of companies?from the worlds largest enterprises to the most ambitious startups?use Stripe to accept payments, grow their revenue, and accelerate new business opportunities Our mission is to increase the GDP of the internet, and we have a staggering amount of work ahead That means you have an unprecedented opportunity to put the global economy within everyones reach while doing the most important work of your career, About The Team One of the key drivers to our success is an exceptional emphasis on our people We believe that our competitive advantage lies in our talent and that we can build a sustai...

Position Title Assistant Manager Forest Planning Key Accountabilities / Responsibilities Create customized accelerated plan for obtaining Forest and Wildlife (F&WL) clearance process to ensure on-time project delivery, Planning of budget to maintain Cost efficiency, Cross functional coordination Execution Provide technical support across the Project, supporting Forest and Wildlife team by ensuring robust execution of end-end process as per guidelines and timely availability of Forest & Wildlife Clearances as well as Working Permission achieving First Time Right (FTR), Ensure strict Environment, Forest and Wildlife compliance of states and MoEF&CC, Explore and identify other levers to crunch ...

Senior Data Engineer Cloud & Modern Data Architectures Role Overview: We are looking for a Senior Data Engineer with expertise in ETL/ELT, Data Engineering, Data Warehousing, Data Lakes, Data Mesh, and Data Fabric architectures . The ideal candidate should have hands-on experience in at least one or two cloud data platforms (AWS, GCP, Azure, Snowflake, or Databricks) and a strong foundation in building PoCs, mentoring freshers, and contributing to accelerators and IPs. Must-Have: 5-8 years of experience in Data Engineering & Cloud Data Services . Hands-on with any 2 clouds AWS (Redshift, Glue), GCP (BigQuery, Dataflow), Azure (Synapse, Data Factory), Snowflake, Databricks . Strong SQL, Pytho...

Mitra Infotech is looking for Machine Learning Engineer to join our dynamic team and embark on a rewarding career journey Developing and implementing ML algorithms: The Machine Learning Engineer develops and implements machine learning algorithms to solve specific problems, such as natural language processing, computer vision, or predictive modeling. Building data pipelines: The Machine Learning Engineer is responsible for building data pipelines that collect, store, and preprocess data used in machine learning algorithms. Creating and maintaining ML infrastructure: The Machine Learning Engineer is responsible for creating and maintaining ML infrastructure, including hardware, software, and ...

Senior/Lead Data Engineer (Warehouse Architecture & PySpark) Location: Hyderabad Experience: 6-12 years | Team: Data Platform / Analytics Engineering The opportunity Own the end-to-end build of a modern analytics foundationdesign clean warehouse models, craft high-quality PySpark transformations, and ship reliable pipelines feeding BI/ML at scale. What you’ll do Model the warehouse: Define dimensional/star schemas (SCD1/2, snapshots), conformed dimensions, and clear grains across core domains. Author robust transformations: Build performant PySpark jobs for batch/near-real-time; handle nested JSON, schema evolution, late data, and idempotent re-runs. Ingestion & CDC: Operate change-data-capt...

Job description Key Responsibilities: Data Pipeline Development: Design, develop, and maintain robust and scalable data pipelines to collect, clean, and process large datasets from various sources. Data Integration: Integrate and optimize data from multiple platforms and systems, ensuring smooth flow and accessibility across the organization. Data Quality: Ensure high-quality, accurate, and consistent data is made available for analysis and reporting, applying appropriate data validation and cleaning techniques. ETL Process: Develop, deploy, and maintain Extract, Transform, Load (ETL) processes to move data between systems. Data Warehouse Management: Work on the design and optimization of da...

Job Title: Talent Acquisition Trainee Overview: The Talent Acquisition Trainee will play a crucial role in supporting the recruitment process by assisting experienced recruiters in sourcing, screening, and selecting candidates to meet the organization's hiring needs This position aims to develop the trainee's skills and understanding of recruitment practices, helping them grow into a proficient and effective recruiter, Key Responsibilities: Assist in creating and posting job advertisements across various platforms, Source potential candidates through social media, job boards, and networking, Screen resumes and applications, identifying qualified candidates for further evaluation, Conduct pre...

Role: Python Developer Experience: 5 years to 8 years Location: Coimbatore, Indore, Bhubaneshwar, Nagpur Mandatory Skills: Python, SQL, DBT, Data pipeline Job Description: 5 to 8 years of experience in handle the development integration and optimization of Datawarehouse applications with strong SQL background Hands on exposure using Python and Data Build Tool (DBT) used in ETL and ELT processes Testing ETLs for large scale DW applications Should have exposure working using Agile framework and in a DevOps environment Good experience in Python programming. Experience with Docker or Container based solutions and environments Experience in designing and Testing ETLs that process large volumes of...

5+ years of QA experience with a focus on automation and cloud-native environments. Hands-on expertise in automation tools such as Selenium, Cypress, or Playwright. Experience with ETL/data pipeline testing, including validating source-to-target data mapping, transformations, and aggregation logic. Proficient in writing and executing complex SQL queries for data validation. Working knowledge of GCP services such as BigQuery, Pub/Sub, Cloud Functions, Cloud Storage, etc. Familiarity with API testing using Postman, RestAssured, or similar tools. Scripting knowledge in Python, JavaScript, or Java for test automation. Experience working in CI/CD pipelines and version control (Git). Roles and Res...

Role: Python Data Engineer Experience: 8 years to 12 years Location: Pan India Mandatory skills: Python, SQL, Data Pipeline should reflect in resume. Job description: Extensive experience with Python scripting and Cloud Technologies Minimum 7 years of IT and Data Related Experience needed. Extensive experience with AWS components like S3, Athena, EMR, Glue, Redshift and SageMaker Experience in Automating ETL process with Python and Automation around AWS Data & Infrastructure. Experience with building Python Automation Frameworks around Data Process, Data Analysis and Data Validation.

Role & responsibilities The individual will be responsible for designs, builds, and maintains scalable data solutions including data models, ETL pipelines, and real-time processing systems He/she is also responsible for mentoring junior team members and play a pivotal role in overseeing the technical aspects of data related projects. MUST HAVE Experience on working with ETL on a data warehouse or big data projects. Experience with cloud services like Azure or GCP or AWS with respect to the data related services, could be just ETL ,cleanup or BI but has potential to be groomed in a 1-2 month period. Should have energy and enthusiasm to get into new things, able to stretch. Good SQL skills and...

Role & responsibilities We are recruiting for a technically strong, people-centric Data Engineering Manager to lead a high-impact AI & Data engineering team based in India. This group will build the data platform and products to support the analytics, insights and data science needs for our core subscription business. Data has always been at the heart of the Economist Group, and ultimately we aim to help the business make data driven judgements based on real time, actionable insights How you will contribute in this role: Work with data product managers, analysts, and data scientists to build data and ML pipelines, data warehouse/data lake-house for analytics, reporting and prediction. Build ...

Job Descripon Senior Data Lead Posion Senior Data Lead Loca¢on [Insert Loca¢on / Remote / Hybrid] About the Role We are looking for a Senior Data Lead to work closely with our Data Architect and play a key role in execu¢ng the data strategy, building pipelines, and leading delivery of cri¢cal data projects. This is a hands-on technical leadership role youll design, code, review, and mentor engineers, while ensuring the data pla¢orm is reliable, governed, and future-ready. You will be the bridge between architecture and engineering execu¢on, transla¢ng architectural blueprints into scalable, produc¢on-grade solu¢ons. Key Responsibili¢es Pla¢orm & Pipeline Development ¢ ¢ ¢ ¢ Design and build ...

Overview Annalect is currently seeking a data engineering lead to join our technology team. In this role you will be building data pipelines and developing data set processes. We are looking for people who have a shared passion for technology, design & development, data, and fusing these disciplines together to build cool things. In this role, you will lead teams working on one or more software and data products in the Annalect Engineering Team. You will participate in technical architecture, design and development of software products as well as research and evaluation of new technical solutions. You will also help us drive the vision forward for data engineering projects by helping us exte...

Role & responsibilities Support in preparation of IT intervention requirements and implementation roadmap The Data Scientist will be responsible for supervision of the key ICT interventions under the project including supervising (a) Designing & implementation monitoring of dashboard by leveraging indicators from District Socio-economic Review and SDG District Indicator Framework to consolidated view of fund flow to district and state machinery, (b) Creation of a data dissemination portal and other IT interventions envisaged under the project. (C) Implementation monitoring of existing MIS systems such as MPSIMS, iPAS, BEAMs and GoIs PFMS system Help prepare techno functional requirements for...

IBS Software is looking for Senior Data Engineer to join our dynamic team and embark on a rewarding career journey A Senior Data Engineer is a technical expert responsible for designing, building, and maintaining the systems and infrastructure needed to support data-driven applications and analytics The job responsibilities of a Senior Data Engineer may include: Designing and implementing scalable and reliable data pipelines, data models, and data infrastructure for processing large and complex datasets Developing and maintaining databases, data warehouses, and data lakes that store and manage the organization's data Developing and implementing data integration and ETL (Extract, Transform, L...

Responsibilities: * Develop computer vision models using Python and SQL. * Optimize data pipelines with Tableau and Power BI. * Collaborate on geospatial data projects with cross-functional teams. Provident fund Health insurance

Job Title: Data Engineer | Python & SQL | Intelligence AI Platform Experience: 2-5 Years Job Description:We are looking for a skilled Data Engineer (2-5 years) to join our team working on Intelligine, an AI-powered content creation and marketing platform. The ideal candidate will have strong expertise in Python, SQL, and ETL/ELT pipelines with a proven ability to work on scalable data solutions. Key Responsibilities: * Design, build, and maintain robust ETL/ELT pipelines using Python and SQL * Develop optimized SQL queries (joins, CTEs, window functions) * Integrate data from multiple sources: APIs, flat files, databases, and third-party tools * Optimize workflows, query performance, and dat...

Responsibilities Be a bridge between business and digital organizations and part of a cross-disciplinary team, working closely with other analysts, product managers, data scientists, data engineers, software engineers, data managers and business partners. Provide actionable, data-driven business and product insights by combining deep statistical skills, data manipulation capabilities and business acumen. Maintain metrics and build dashboards. Autonomously execute data analysis. You apply existing data & analytics strategies relevant to your immediate scope You partner with data engineers to define and build simple data models. You integrate existing tools to automate data ingestion, data man...

About the Job : We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. In this pivotal role, you will be instrumental in driving our data engineering initiatives, with a strong emphasis on leveraging Dataiku's capabilities to enhance data processing and analytics. You will be responsible for designing, developing, and optimizing robust data pipelines, ensuring seamless integration of diverse data sources, and maintaining high data quality and accessibility to support our business intelligence and advanced analytics projects. This role requires a unique blend of expertise in traditional data engineering principles, advanced data modeling, and a forward-...

About the Job : We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. In this pivotal role, you will be instrumental in driving our data engineering initiatives, with a strong emphasis on leveraging Dataiku's capabilities to enhance data processing and analytics. You will be responsible for designing, developing, and optimizing robust data pipelines, ensuring seamless integration of diverse data sources, and maintaining high data quality and accessibility to support our business intelligence and advanced analytics projects. This role requires a unique blend of expertise in traditional data engineering principles, advanced data modeling, and a forward-...

About the Job : We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. In this pivotal role, you will be instrumental in driving our data engineering initiatives, with a strong emphasis on leveraging Dataiku's capabilities to enhance data processing and analytics. You will be responsible for designing, developing, and optimizing robust data pipelines, ensuring seamless integration of diverse data sources, and maintaining high data quality and accessibility to support our business intelligence and advanced analytics projects. This role requires a unique blend of expertise in traditional data engineering principles, advanced data modeling, and a forward-...

Role & responsibilities Build and deploy data platforms, data warehouses, and big data solutions across industries (BFSI, Manufacturing, Healthcare, eCommerce, IoT, Digital Twin, etc) Integrating, transforming, and consolidating data from various structured and unstructured data systems. Expert in data ingestion, transformation, storage, and analysis, often using Azure services and migration from legacy on-premise services Essential skills include SQL, Python, R and knowledge of ETL/ELT processes and big data technologies like Apache Spark, Scala, PySpark. Maintain data integrity, resolve data-related issues, and ensure the reliability and performance of data solutions Work with stakeholders...

Architect – Data Engineering Governance & Data Security Req number: R5966 Employment type: Full time Worksite flexibility: Remote Who we are CAI is a global technology services firm with over 8,500 associates worldwide and a yearly revenue of $1 billion+. We have over 40 years of excellence in uniting talent and technology to power the possible for our clients, colleagues, and communities. As a privately held company, we have the freedom and focus to do what is right—whatever it takes. Our tailor-made solutions create lasting results across the public and commercial sectors, and we are trailblazers in bringing neurodiversity to the enterprise. Job Summary We are seeking an accomplished Archi...