1569 Data Pipeline Jobs - Page 13

Han Digital Solution (P) is looking for Java , Scala and ADF (Data Engineer) to join our dynamic team and embark on a rewarding career journey We are looking for a skilled Data Engineer with expertise in Java, Scala, and ADF to join our team at Han Digital Solution (P) The ideal candidate will have a strong background in designing and implementing large-scale data systems Roles and Responsibility Design and develop scalable data pipelines using Java, Scala, and ADF Collaborate with cross-functional teams to identify and prioritize project requirements Develop and maintain large-scale data systems that meet business needs Ensure data quality and integrity by implementing robust testing and va...

Han Digital Solution (P) is looking for AI - Data Scientist to join our dynamic team and embark on a rewarding career journey We are looking for a highly skilled AI Data Scientist to join our team at Han Digital Solution (P) The ideal candidate will have a strong background in artificial intelligence and data science, with excellent problem-solving skills and the ability to work collaboratively with cross-functional teams Roles and Responsibility Develop and implement machine learning models and algorithms to solve complex problems Analyze large datasets to identify trends and patterns, and provide insights to stakeholders Design and develop predictive models to forecast future events and ou...

Han Digital Solution (P) is looking for Data Engineer+Python (Aditya POC) to join our dynamic team and embark on a rewarding career journey We are looking for a highly skilled Data Engineer with expertise in Python to join our team at Han Digital Solution (P) The ideal candidate will have experience working with large datasets and developing scalable data pipelines As a Data Engineer, you will play a key role in designing, building, and maintaining the data infrastructure that supports our business operations Roles and Responsibility Design and implement data models and architectures to support business intelligence and analytics Develop and maintain large-scale data pipelines using Python a...

Han Digital Solution (P) is looking for Data Engg-Snowflake-Wipro to join our dynamic team and embark on a rewarding career journey We are looking for a skilled Data Engineer with expertise in Snowflake to join our team at Han Digital Solution (P) The ideal candidate will have a strong background in data engineering and experience working with Snowflake As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data systems Roles and Responsibility Design and implement scalable data pipelines using Snowflake Develop and maintain large-scale data warehouses and data lakes Collaborate with cross-functional teams to identify and prioritize project requireme...

style="background: #fff;">Data Engineer Location: Mumbai (All Areas) Job Description Seeking a Data Engineer with 3-4 years of experience to design, build, and optimize data pipelines and architectures. Responsibilities include data integration, transformation, and ensuring scalability & reliability of data infrastructure. Essential skills include ETL, data modeling, SQL, cloud platforms, and big data technologies

Salary: 15 to 30 LPA Exp: 3 to 8 years Location : Gurgaon/Bangalore/Pune/Chennai Notice: Immediate to 30 days..!! Key Responsibilities & Skillsets: Common Skillsets : 3+ years of experience in analytics, Pyspark, Python, Spark, SQL and associated data engineering jobs. Must have experience with managing and transforming big data sets using pyspark, spark-scala, Numpy pandas Excellent communication & presentation skills Experience in managing Python codes and collaborating with customer on model evolution Good knowledge of data base management and Hadoop/Spark, SQL, HIVE, Python (expertise). Superior analytical and problem solving skills Should be able to work on a problem independently and p...

Key Responsibilities Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark . Write efficient Python code for data processing, transformation, and automation tasks. Integrate data from multiple sources and ensure data quality and consistency. Optimize Spark jobs for performance and cost efficiency in Databricks environment. Collaborate with data scientists, analysts, and business stakeholders to deliver data solutions. Implement best practices for CI/CD , version control, and code deployment in Databricks. Monitor and troubleshoot data workflows and resolve production issues.

Description: We are seeking an technical leader for a hybrid Senior Programmer and Data engineer & Solutions Architect role. This position is for a hands-on expert who not only possesses mastery in Apache Spark development but also has a proven track record of architecting complex, end-to-end data solutions on AWS and on-premise. You will be the technical leader for new data initiatives, responsible for designing the architectural blueprints for our data lake platforms and then leading the hands-on implementation of those designs. We need a strategic thinker who can solve our most complex technical challenges and proactively drive things to add values to the domain.. Requirements: Core Techn...

The Data Engineer is responsible for designing and developing data delivery on our data warehouse on AWS. This data will be consumed by visual dashboards/reports that Sony teams use to better understand trends and insights to improve market share/songwriter deals Experience in data architecture including data modeling, data mining and data ingestion Experience with AWS associated technologies (S3 buckets, Glue, Data Pipeline, DMS, RDS, Redshift, Lambda) Knowledge of creating ETL scripts with languages such as Python, Node.js, SQL Experience in data warehousing and big data Experience with Relational databases (SQL Server) Experience working in Agile/Scrum teams

Skill - GCP , BigQuery, Python Location - Coimbatore Experience - 5.5 to 12 Years Job Summary Data Engineer with a focus on Google Cloud Platform (GCP). Hands-on experience with GCP services: Dataflow Pub/Sub Cloud Storage Cloud Composer. Expertise in workflow orchestration using Apache Airflow. Experience with cloud SQL or other relational databases (PostgreSQL preferred). Solid understanding of data modeling ETL processes and streaming/batch data pipelines. Familiarity with DevOps practices CI/CD and containerization (Docker Kubernetes). Responsibilities Key Responsibilities and Skills Build Data Pipelines on GCP: They need hands-on experience with core GCP services used for data processin...

Roles & Responsibilities Provide technical leadership and mentorship to a team of data engineers. Design, architect, and implement highly scalable, resilient, and performant data pipelines, using GCP technologies is a plus (e.g., Dataproc, Cloud Composer, Pub/Sub, BigQuery). Guide the team in adopting best practices for data engineering, including CI/CD, infrastructure-as-code, and automated testing. Conduct code reviews, design reviews, and provide constructive feedback to team members. Stay up-to-date with the latest technologies and trends in data engineering, Data Pipeline Development: Develop and maintain robust and efficient data pipelines to ingest, process, and transform large volume...

We are looking for a seasoned Data Engineer with strong experience in designing and building modern data platforms from the ground up. This role involves close collaboration with architects, DevOps, and business teams to establish a scalable, secure, and high-performing data ecosystem. The ideal candidate is hands-on, cloud-savvy, and passionate about data infrastructure, governance, and engineering best practices.Job Title: Data Engineer Profilecode : LOC_DATA_ETL_SSA_1 At bpost group, data is more than numbers its a strategic asset that drives critical decisions across our logistics and e-commerce operations. As we continue to evolve into a more data-driven organization, we are investing i...

Responsibilities: * Design, develop, and maintain data pipelines using Spark, Python, Scala, and Java. * Write efficient and optimized SQL queries for data extraction, transformation, and loading (ETL) processes. * Work with DataFrames to manipulate and analyze large datasets. * Implement data storage and processing solutions using cloud technologies (preferably AWS or GCP). * Build and maintain real-time data streaming pipelines using MSK/Kafka. * Utilize S3 for data storage and retrieval. * Work with data lake technologies like Iceberg. * Ensure data quality, integrity, and security. * Collaborate with data scientists and other engineers to understand data requirements and deliver solution...

Lead Data Engineer Data Management Job description Company Overview Accordion is a global financial consulting and technology firm helping CFOs and their teams drive value creation. Rooted in private equity rigor and powered by data, analytics, and AI, we partner with companies across industries to turn insight into impact. Accordion works at the intersection of Private Equity sponsors and portfolio companies management teams across every stage of the investment lifecycle. We provide hands-on, execution-oriented support, driving value through the office of the CFO by building data and analytics capabilities and identifying and implementing strategic work, rooted in data, analytics and AI. Ac...

As a Python Developer at our company, you will be responsible for utilizing your expertise in Python development to contribute to the implementation of AI/ML frameworks and cloud services. Here is a breakdown of the key aspects of the role: **Role Overview:** You will be working with a team of professionals to develop Python-based solutions using frameworks such as FastAPI, Flask, or Django. Your role will involve hands-on experience with AI/ML frameworks like PyTorch, TensorFlow, and LangChain. Additionally, you will be expected to leverage your proficiency in Azure Cloud Services, including Azure ML, Azure OpenAI, Cognitive Services, Azure Storage, and Azure DevOps. **Key Responsibilities:...

We are looking for a highly skilled Network Engineer with expertise in Routing and Switching to join our team at Method-Hub Software Pvt.Ltd. The ideal candidate will have a strong understanding of network protocols and architectures, with the ability to design, implement, and manage complex network systems. Roles and Responsibility Design, implement, and manage network infrastructure, including routers, switches, and firewalls. Configure and troubleshoot network devices such as Cisco routers and switches. Develop and maintain documentation of network architecture and topology. Collaborate with cross-functional teams to ensure seamless integration of network services. Conduct regular network...

Architecting Data Pipelines: Design, build, and maintain highly scalable and efficient data pipelines using cutting-edge AWS services. Driving Code Excellence: Perform significant Code Refactoring on a legacy codebase cleaning, modernizing, and improving readability and long-term maintainability. Mastering Optimization: Apply your deep understanding of Spark SQL and PySpark to continuously optimize code for performance and cost efficiency. Embracing Quality: Champion Test-Driven Development (TDD) and meticulous Unit Testing to ensure functionality, catch bugs early, and maintain a high bar for code quality. Solving the Toughest Challenges: Dive into complex systems to debug and resolve diffi...

You will work as part of a development team that designs, builds and maintains high performant and maintainable code. It is a unique and challenging role that calls for a strong passion for quality and technology, while providing an immense opportunity to design and develop wide reaching code. The role includes work on architecture design, business logic, requirements gathering, data modelling and quality assurance. What were looking for We are looking for creative people with a passion for programming and an eye for detail. A recognized academic background in Computer Science Solid experience with Google Cloud Platform, specifically Dataform, BigQuery and Pub/Sub. We prefer candidates who h...

Key Responsibilities Extract, clean, and load data from various structured and unstructured sources. Design, develop, and optimize data pipelines using Snowflake, SQL, Kafka , and related technologies. Investigate and document new data sets for analytical and operational use cases. Triage incoming bugs, incidents, and data issues , ensuring timely resolution. Perform technical operations tasks to maintain data pipeline performance. Troubleshoot and resolve issues related to data quality, pipeline failures, or latency. Participate in Agile ceremonies sprint refinement, planning, and estimation. Monitor key data flows and proactively raise awareness of potential issues. Define and implement da...

Salary: 15 to 25 LPA Exp: 4 to 7 years Location: Gurgaon/Pune/Bengalore Notice: Immediate to 30 days..!! Job Profile: Experienced Data Engineer with a strong foundation in designing, building, and maintaining scalable data pipelines and architectures. Skilled in transforming raw data into clean, structured formats for analytics and business intelligence. Proficient in modern data tools and technologies such as SQL, T-SQL, Python, Databricks, and cloud platforms (Azure). Adept at data wrangling, modeling, ETL/ELT development, and ensuring data quality, integrity, and security. Collaborative team player with a track record of enabling data-driven decision-making across business units. As a Dat...

SUMMARY Key Requirements: Programming experience with Java 8 or above, Spring Core, Spring MVC, Spring Cloud, RESTful Webservices Minimum 4-5 years of experience in Java Full stack role Experience in Struts 2 Strong experience in JQuery Experience in JSON parsing Strong Experience in React for building responsive and dynamic user interfaces Experience in Java Spring Boot, able to develop scalable API’s and services Minimum 2 years experience in Python, EDA and data processing tasks Experience in Azure Functions Experience in Node.js for specific Azure Function implementations Proficient in MSSQL, PostgreSQL and Snowflake Experience with Azure Functions & AWS Serverless for serverless archite...

About the Role The QA Engineer will own quality assurance across the ML lifecycle from raw data validation through feature engineering checks, model training/evaluation verification, batch prediction/optimization validation, and end-to-end (E2E) workflow testing. The role is hands-on with Python automation, data profiling, and pipeline test harnesses in Azure ML and Azure DevOps. Success means provably correct data, models, and outputs at production scale and cadence. Role & responsibilities Test Strategy & Governance - Define an ML-specific Test Strategy covering data quality KPIs, feature consistency checks, model acceptance gates (metrics + guardrails), and E2E run acceptance (timeliness,...

Duties/ Roles & Responsibilities Lead the design, optimization, and scaling of data pipelines using Data bricks, Spark, and advanced SQL. Drive architectural decisions, enforce best practices, and mentor teams on distributed data processing and lake house patterns. Collaborate with cross-functional partners to deliver high-quality, reliable data solutions that support analytics and business objectives.

Why would you like to join us? TransOrg Analytics specializes in Data Science, Data Engineering and Generative AI, providing advanced analytics solutions to industry leaders and Fortune 500 companies across India, US, APAC and the Middle East. We leverage data science to streamline, optimize, and accelerate our clients' businesses. Visit at www.transorg.com to know more about us. Role Overview: The Product Solutions Architect will be responsible for designing scalable, efficient, and future-ready solution architectures that support product growth. This role requires strong technical depth, structured problem-solving, and the ability to translate complex business needs into actionable archite...

Lead end-to-end data pipeline design and delivery , GCP-native services(Big Query),DBT Cloud, and Python frameworks and AWS Native Services Airflow- good to have Strong data platforms & Pipelines Any dashboarding tool will work (Power BI, Tableau or any other), Any data orchestration tools SQL is must 80% strong technical person who will manage a team of individuals Preferred candidate profile