874 Data Pipeline Jobs - Page 33

Location: Bengaluru(Embassy Tech Village) Work Mode: Hybrid(2-3 days a week from office) Experience: 5-10 Years Technical capabilities - 5+ yrs of experience as SDET in the data testing of any form (data extract, transform, load) Test Automation experience with non-GUI applications (Backend, API, Data) Experience with testing frameworks like PyTest, JIRA XRAY, Selenium, Cypress, Mable, Postman, etc. Hands-on experience in one or more programming languages like Python, Nodejs Expertise in end-to-end test automation and automation framework using Python Hands-on experience in test, develop and deploy frameworks in AWS Cloud, for data related use cases like data ingestion, data curation, data w...

Role & responsibilities Design, develop, and maintain scalable ETL/ELT data pipelines Work with structured and unstructured data from various sources (APIs, databases, cloud storage, etc.) Optimize data workflows and ensure data quality, consistency, and reliability Collaborate with cross-functional teams to understand data requirements and deliver solutions Maintain and improve our data infrastructure and architecture Monitor pipeline performance and troubleshoot issues in real-time Preferred candidate profile 2-3 years of experience in data engineering or a similar role Proficiency in SQL and Python (or Scala/Java for data processing) Experience with ETL tools (e.g., Airflow, dbt, Luigi) F...

Position: AWS Data Engineer About bluCognition: bluCognition is an AI/ML based start-up specializing in developing data products leveraging alternative data sources and providing servicing support to our clients in financial services sector. Founded in2017, by some very named senior professionals from the financial services industry, the company is headquartered in the US, with the delivery centre based in Pune. We build all our solutions while leveraging the latest technology stack in AI, ML and NLP combined with decades of experience in risk management at some of the largest financial services firms in the world. Our clients are some of the biggest and the most progressive names in the fin...

Refine ambiguous questions and generate new hypotheses about the product through a deep understanding of the data, our customers, and our business Design experiments and interpret the results to draw detailed and impactful conclusions. Define how our teams measure success, by developing Key Performance Indicators and other users/business metrics, in close partnership with Product and other subject areas such as engineering, operations and marketing Collaborate with applied scientists and engineers to build and improve on the availability, integrity, accuracy, and reliability of data logging and data pipelines. Develop data-driven business insights and work with cross-functional partners to f...

--- What the Candidate Will Do ---- Refine ambiguous questions and generate new hypotheses about the product through a deep understanding of the data, our customers, and our business Design experiments and interpret the results to draw detailed and impactful conclusions. Define how our teams measure success, by developing Key Performance Indicators and other users/business metrics, in close partnership with Product and other subject areas such as engineering, operations and marketing Collaborate with applied scientists and engineers to build and improve on the availability, integrity, accuracy, and reliability of data logging and data pipelines. Develop data-driven business insights and work...

---- What the Candidate Will Do ---- Design, develop, and maintain robust and scalable software solutions Find opportunities for quality improvements in the search stack and lead the entire development lifecycle end-to-end, from architecture design and coding to testing and deployment ---- Basic Qualifications ---- Bachelor's degree in Computer Science 3+ years of professional experience in software development with a track record of increasing responsibility and impact. Experience with Go and Python programming languages Demonstrated experience developing sophisticated backend systems and longer-term ownership of critical backend services and infrastructure. Bias to action and proven track ...

Department Overview: augmented Intelligence (augIntel) builds automation, analytics and artificial intelligence solutions that drive success for Cardinal Health by creating material savings, efficiencies and revenue growth opportunities. The team drives business innovation by leveraging emerging technologies and turning them into differentiating business capabilities. Job Overview: Designing, building and operationalizing large-scale enterprise data solutions and applications using one or more of Google Cloud Platform data and analytics services in combination with technologies like Spark, Cloud DataProc, Cloud Dataflow, Apache Beam, Cloud BigQuery, Cloud PubSub, Cloud Functions, Airflow. Re...

Sr. AI/ML Python Developer - 5-8 yrs * cross-functional teams on data analysis & statistics. * Develop ML models - Python, NumPy, Pandas, DLS & NLP. * Imple data pipelines, deploy models on TensorFlow Serving & GCP. Drop to rajkalyan@garudaven.com Food allowance Annual bonus Provident fund Health insurance Office cab/shuttle

Job Title: Sr Data Engineer – Snowflake & Python About the Role: We are seeking a skilled and proactive Sr Data Developer with 5-8 years of hands-on experience in Snowflake , Python , Streamlit , and SQL , along with expertise in consuming REST APIs and working with modern ETL tools like Matillion, Fivetran etc. The ideal candidate will have a strong foundation in data modeling , data warehousing , and data profiling , and will play a key role in designing and implementing robust data solutions that drive business insights and innovation. Key Responsibilities: Design, develop, and maintain data pipelines and workflows using Snowflake and an ETL tool (e.g., Matillion, dbt, Fivetran, or simila...

About ProcDNA: ProcDNA is a global consulting firm. We fuse design thinking with cutting-edge tech to create game-changing Commercial Analytics and Technology solutions for our clients. We're a passionate team of 275+ across 6 offices, all growing and learning together since our launch during the pandemic. Here, you won't be stuck in a cubicle - you'll be out in the open water, shaping the future with brilliant minds. At ProcDNA, innovation isn't just encouraged, it's ingrained in our DNA. What we are looking for: As the Associate Engagement Lead, youll leverage data to unravel complexities, adept at devising strategic solutions that deliver tangible results for our clients. We are seeking a...

About the Role As a Senior Data Engineer you’ll be a core part of our engineering team. You will bring your valuable experience and knowledge, improving the technical quality of our data-focused products. This is a key role in helping us become more mature, deliver innovative new products and unlock further business growth. This role will be part of a newly formed team that will collaborate alongside data team members based in Ireland, USA and India. Following the successful delivery of some fantastic products in 2024, we have embarked upon a data-driven strategy in 2025. We have a huge amount of data and are keen to accelerate unlocking its value to delight our customers and colleagues. You...

Job Title: Senior Data Engineer Experience Required: 3 to 5 Years Location: Baner, Pune Job Type: Full-Time (WFO) Job Summary We are seeking a highly skilled and motivated Senior Data Engineer to join our dynamic team. The ideal candidate will have extensive experience in building and managing scalable data pipelines, working with cloud platforms like Microsoft Azure, AWS and utilizing advanced tools such as Datalakes, PySpark, and Azure Data Factory. The role involves collaborating with cross-functional teams to design and implement robust data solutions that support business intelligence, analytics, and decision-making processes. Key Responsibilities Design, develop, and maintain scalable ...

We're seeking a Data Operations Specialist (1–4 yrs exp.) in Chennai to manage and support data platforms and pipelines on Azure and on-premises, ensuring system reliability, performance, and availability.

Data Engineer Graph Research Data and Analytics What you will do Lets do this. Lets change the world. In this vital role you will be part Researchs Semantic Graph Team is seeking a qualified individual to design, build, and maintain solutions for scientific data that drive business decisions for Research. The successful candidate will construct scalable and high-performance data engineering solutions for extensive scientific datasets and collaborate with Research partners to address their data requirements. The ideal candidate should have experience in the pharmaceutical or biotech industry, leveraging their expertise in semantics, taxonomies, and linked data principles to ensure data harmon...

Seeking skilled & experienced Technical Manager to oversee the successful planning#Lead end-to-end solution delivery, from requirements gathering to implementation#Coordinate with engineering, product management, & QA teams, ensure seamless execution Required Candidate profile Strong technical background#Experience in software engineering, Solution delivery, & Project management#Knowledge on Agile methodologies#Certifications in PMP, Scrum Master, ITIL & RTE certification.

Job Title : Snowflake Developer (4+ years' experience) Location : F, 384, Sector 91 Rd, Phase 8B, Industrial Area, Sector 91, Sahibzada Ajit Singh Nagar, Punjab 160055. Job Type : Fulltime (In-house) Job Overview : We are looking for an experienced Snowflake Developer with 4+ years of hands-on experience in Snowflake Data Warehouse and related tools. You will be responsible for building, managing, and optimizing Snowflake data pipelines, assisting in data integration, and contributing to overall data architecture. The ideal candidate should have a strong understanding of data modeling, ETL processes, and experience working with cloud-based data platforms. Responsibilities : Design, develop, ...

Role & responsibilities Urgent Hiring for one of the reputed MNC Immediate Joiners Only Females Exp - 4-9 Years Bang / Hyd / Pune As a Python Developer with AWS , you will be responsible for developing cloud-based applications, building data pipelines, and integrating with various AWS services. You will work closely with DevOps, Data Engineering, and Product teams to design and deploy solutions that are scalable, resilient, and efficient in an AWS cloud environment. Key Responsibilities: Python Development : Design, develop, and maintain applications and services using Python in a cloud environment. AWS Cloud Services : Leverage AWS services such as EC2 , S3 , Lambda , RDS , DynamoDB , and A...

Role- ML Engineer / ML Ops Engineer Location: Thoraipakkam, Chennai Work mode: Work from Office Task Description: University Degree in Computer Science, Information Technology, or related field 5+ years of experience in the Machine Learning Operations role Design the data pipelines and engineering infrastructure to support our clients' enterprise machine learning systems at scale Take offline models, data scientists build and turn them into a real machine learning production system Develop and deploy scalable tools and services for our clients to handle machine learning training and inference Identify and evaluate new technologies to improve the performance, maintainability, and reliability ...

We are seeking a highly skilled Senior Data Engineer to lead the migration, optimization, and governance of data pipelines in Azure Data Factory (ADF) and SQL. The ideal candidate will have extensive experience in Change Data Capture (CDC), performance tuning, data security, and compliance within a cloud-based architecture. Key Responsibilities : - Architect & optimize CDC pipelines in ADF, Data Sync Services and SQL, ensuring efficient data ingestion. - Implement performance tuning strategies such as parallel processing, indexing, partitioning, and data archiving. - Enforce security & compliance by implementing RBAC, data encryption, audit logging, and access control best practices. - Defin...

Who We Are At Kyndryl, we design, build, manage and modernize the mission-critical technology systems that the world depends on every day. So why work at Kyndryl? We are always moving forward – always pushing ourselves to go further in our efforts to build a more equitable, inclusive world for our employees, our customers and our communities. The Role Are you ready to dive headfirst into the captivating world of data engineering at Kyndryl? As a Data Engineer, you'll be the visionary behind our data platforms, crafting them into powerful tools for decision-makers. Your role? Ensuring a treasure trove of pristine, harmonized data is at everyone's fingertips. As a Data Engineer at Kyndryl, you...

Role & responsibilities Title: Cyber Data Pipeline Engineer Location : Bengaluru Experience: 7 to 14 years Role Profile A successful applicant will contribute to several important initiatives including: Collaborate with Cyber teams to identify, onboard, and integrate new data sources into the platform. Design and implement data mapping, transformation, and routing processes to meet analytics and monitoring requirements. Developing automation tools that integrate with in-house developed configuration management frameworks and APIs Monitor the health and performance of the data pipeline infrastructure. Working as a top-level escalation point to perform complex troubleshoots, working with other...

Role : Scala Spark Developer Location : Pune/Chennai Company : MNC Required Skills & Qualifications: Minimum 5 years of professional experience in Data Engineering, with a strong focus on big data technologies. Proficiency in Scala for developing big data applications and transformations, especially with Apache Spark. Expert-level proficiency in SQL; ability to write complex queries, optimize performance, and understand database internals. Extensive hands-on experience with Apache Spark (Spark SQL, DataFrames, RDDs) for large-scale data processing and analytics. Mandatory - Experience and knowledge in designing, implementing, and managing non-relational data stores (e.g., MongoDB, Cassandra,...

Position: Data Engineer Location: Bangalore, India About Dodge Dodge Construction Network exists to deliver the comprehensive data and connections the construction industry needs to build thriving communities. Our legacy is deeply rooted in empowering our customers with transformative insights, igniting their journey towards unparalleled business expansion and success. We serve decision-makers who seek reliable growth and who value relationships built on trust and quality. By combining our proprietary data with cutting-edge software, we deliver to our customers the essential intelligence needed to excel within their respective landscapes. We propel the construction industry forward by transf...

ML/Data Scientist (8+yrs) Strong in Telecom,Python,ML(TensorFlow/PyTorch), GIS(QGIS/MapInfo/ArcGIS), AWS, Big Data(Spark/Hadoop), SQL, Tableau. Knowledge of 5G & telecom KPIs is a +. C2H via TE Infotech (DishNetworks),Noida. @ssankala@toppersedge.com

Notice Period - Immediate joiner Responsibilities Lead, develop and support analytical pipelines to acquire, ingest and process data from multiple sources Debug, profile and optimize integrations and ETL/ELT processes Design and build data models to conform to our data architecture Collaborate with various teams to deliver effective, high value reporting solutions by leveraging an established DataOps delivery methodology Continually recommend and implement process improvements and tools for data collection, analysis, and visualization Address production support issues promptly, keeping stakeholders informed of status and resolutions Partner closely with on and offshore technical resources Pr...