1569 Data Pipeline Jobs - Page 9

We are looking for a highly skilled Data Modeller to join our team at Barclaycard Payments. The ideal candidate will have a strong background in data modeling and analysis, with excellent problem-solving skills. Roles and Responsibility Design and develop data models to support business intelligence and analytics. Collaborate with stakeholders to identify business requirements and develop solutions. Develop and maintain databases and data systems to support business operations. Analyze complex data sets to identify trends and patterns. Create reports and dashboards to provide insights to stakeholders. Ensure data quality and integrity by implementing data validation and testing procedures. J...

We are looking for a highly skilled Data Scientist ML Ops Engineer Senior Engineer with 3 to 8 years of experience to join our team at Deutsche Bank. The ideal candidate will have a strong background in machine learning and operations engineering, with excellent analytical and problem-solving skills. Roles and Responsibility Design and develop scalable data pipelines using machine learning algorithms. Collaborate with cross-functional teams to identify business problems and develop solutions. Develop and maintain large-scale data systems and infrastructure. Ensure seamless integration of machine learning models into existing systems. Conduct thorough analysis and testing of data quality and ...

Key Responsibilities Pipeline Development & Engineering Build and optimize data pipelines using Dataow (Apache Beam) for batch and streaming workloads Develop Airow DAGs in Cloud Composer for workow orchestration Implement ELT processes using Dataform for SQL-based transformations Design and maintain BigQuery datasets following medallion architecture patterns Create reusable pipeline templates and frameworks for accelerated development Data Platform Implementation BigQuery: Develop optimized schemas, partitioning strategies, and materialized views Dataow: Build Apache Beam pipelines in Python/Java for complex transformations Cloud Composer: Create and maintain Airow workows with proper depen...

skilled in Python, R, SQL/NoSQL, TensorFlow, ML algorithms, Hadoop, Airflow, ETL pipelines, time-series, model deployment, cloud (AWS/GCP), Git, and BI tools like PowerBI/Tableau to build models, automate workflows, and deliver insights.

Role & responsibilities Understand the business requirements for analytics and data interpretation Understands requirements and is involved in the discussions relating to technical and functional design of the sprint/ module/project Design and implement end-to-end data solutions (storage, integration, processing, visualization) in Azure. Used various sources to ingest data into Azure Data Lake Storage (ADLS) such as SQL Server, Excel, Oracle, SQL Azure etc. Extract data from one database and load it into another Build data architecture for ingestion, processing, and surfacing of data for large-scale applications Use many different scripting languages, understanding the nuances and benefits o...

Design and implement data pipelines, data warehouses, and data lakes to ensure accurate, accessible, and secure data. Develop processing and analysis algorithms fit for the intended data complexity and volumes. Collaborate with data scientists to build and deploy machine learning models. Build and maintain systems that collect, store, process, and analyze data. Lead and supervise a team, guiding professional development and allocating work requirements. Demonstrate leadership behaviors to create an environment for colleagues to thrive and deliver excellent results. Perform prescribed activities timely and to a high standard, consistently driving continuous improvement. Utilize in-depth techn...

Retail Media is transforming how advertisers connect with consumers through personalized and targeted campaigns across retailers' digital and physical touchpoints. Retail Media Measurement plays a pivotal role in ensuring the effectiveness of these campaigns, driving value for advertisers, retailers, and consumers alike. This role focuses on designing, building, and scaling solutions that enable the accurate measurement of retail media campaigns across various channels. By providing actionable insights, it empowers stakeholders to optimize media investments, improve ROI, and enhance the overall customer experience. Job Title: Senior Data Engineer Job Summary We are looking for a talented and...

Responsibilities: * Design, develop, and maintain data pipelines using Spark, Scala, and Java. * Write efficient and optimized SQL queries for data extraction, transformation, and loading (ETL) processes. * Work with DataFrames to manipulate and analyze large datasets. * Implement data storage and processing solutions using cloud technologies (preferably AWS or GCP). * Build and maintain real-time data streaming pipelines using MSK/Kafka. * Utilize S3 for data storage and retrieval. * Work with data lake technologies like Iceberg. * Ensure data quality, integrity, and security. * Collaborate with data scientists and other engineers to understand data requirements and deliver solutions. * Par...

About Us DataZymes is a data-driven organization focused on delivering actionable insights through advanced analytics and business intelligence. We are looking for an AWS Data Engineer with strong hands-on experience in building scalable data pipelines and cloud data solutions. Responsibilities Design and implement scalable data pipelines using AWS services such as S3, Glue, PySpark, and EMR . Develop and maintain robust data transformation scripts using Python and SQL . Optimize data storage and retrieval using Redshift, RDS, DynamoDB , and other AWS database services. Build and manage data warehouse layers tailored to business requirements. Apply ETL and data modeling best practices to ens...

Role & Responsibilities As a Forward Deployed Software Engineer (FDSE) at UnifyApps, youll work at the intersection of engineering, problem-solving, and customer impact. You’ll embed with enterprise clients, understand real-world problems, and design and implement solutions using the UnifyApps platform. This role is ideal for high-potential engineers from Tier-1 colleges who want to work on real enterprise-scale problems, gain full-stack exposure, and grow rapidly in a high-impact SaaS environment. Key Responsibilities: Work with client and internal teams to understand complex business processes, legacy systems, and integration needs. Design, build, and deploy solutions using the UnifyApps p...

As a member of the team at MPOWER, you will play a crucial role in building and maintaining the company's data infrastructure. Your responsibilities will include acquiring, organizing, and analyzing data to provide valuable insights for informed business decisions. Your key responsibilities will involve: - Designing, building, and maintaining data pipelines using Python and AWS services such as AWS Redshift, AWS Glue, and Managed Workflows for Apache Airflow (MWAA). - Ensuring data quality and integrity through validation, cleansing, and error handling techniques. - Monitoring, troubleshooting, and optimizing the performance of data pipelines to ensure reliability and scalability. - Identify...

Current Groundtruth looking to ASE for Integration Team, i f you looking for job or change kindly share update resume on laxmi.pal@groundtruth.com A bit about Team GroundTruth seeks a Data Engineering Associate Software Engineer to join our Integration team. The Integration Team connects and consolidates data pipelines across Avails & Inventory Forecast, Identity Graph, and POS Integration systems to ensure accurate, timely insights. We engineer seamless data flows that fuel reliable analytics and decision-making using big data technologies, such as MapReduce, Spark, and Glue. We take pride in building an Engineering Team composed of strong communicators who collaborate with multiple busines...

The Role: The Sr. Data Engineer I will be responsible for designing, building, and maintaining scalable data pipelines and systems to support the organizations data-driven initiatives. This role will work closely with data scientists, analysts, and other stakeholders to ensure data is easily accessible, clean, and reliable for analysis and reporting. The Sr. Data Engineer I will help drive improvements in data architecture and performance, ensuring that data can be leveraged for actionable insights to drive the business forward. The Responsibilities: Data Pipeline Development: Design, build, and maintain scalable and reliable data pipelines to process and transform large volumes of data from...

Key Responsibilities: Perform ETL testing for data extraction, transformation, and loading processes. Validate data accuracy, completeness, and consistency across multiple sources and targets. Write and execute complex SQL queries for data validation and troubleshooting. Work with AWS Redshift for data warehousing and AWS Glue for ETL workflows. (Both are a must) Collaborate with developers, data engineers, and business analysts to understand requirements and design test strategies. Identify defects, log issues, and ensure timely resolution. Prepare and maintain test documentation, including test cases, test plans, and reports. Required Skills: Strong experience in ETL Testing (data validati...

Seeking a Senior Data Engineer to build and optimize ETL pipelines, manage data warehouses, ensure data quality, and work with AWS, SQL, Python, Airflow, and Kafka to support scalable, secure, and high-performance data systems. Required Candidate profile Skilled data engineer with strong SQL, Python, AWS, ETL, warehousing, Airflow, Kafka, and data modeling experience. Capable of troubleshooting issues, optimizing pipelines, and working across teams.

We are looking for a highly skilled Data Scientist to join our team at Medblue Innovations Private Limited. The ideal candidate will have a strong background in data analysis and machine learning, with the ability to develop innovative solutions. Roles and Responsibility Develop and implement advanced data models and algorithms to drive business growth. Collaborate with cross-functional teams to identify and prioritize project requirements. Design and deploy scalable data pipelines and architectures. Analyze complex data sets to extract insights and trends. Develop predictive models and machine learning algorithms to solve real-world problems. Communicate technical findings and recommendatio...

Key Responsibilities Develop and maintain data ingestion pipelines using Azure Data Factory (ADF) and Databricks for structured and unstructured data. Create notebooks to process PDF and Word documents, including extracting text, tables, charts, graphs, and images. Apply NLP / Embedding Models (e.g., OpenAI, Hugging Face, sentence-transformers) to convert extracted content into embeddings. Store embeddings and metadata into Vector Databases (e.g., FAISS, Pinecone, Milvus, Weaviate, ChromaDB). Design and implement semantic search and retrieval workflows to enable prompt-based query capabilities. Optimize ETL pipelines for scalability, reliability, and performance. Collaborate with data scient...

EPAM has presence across 40+ countries globally with 55,000 + professionals & numerous delivery centers, Key locations are North America, Eastern Europe, Central Europe, Western Europe, APAC, Mid East & Development Centers in India (Hyderabad, Pune & Bangalore). Location: Pune/Hyderabad/Bengaluru/Chennai/Coimbatore Work Mode: Hybrid (2-3 days office in a week) Full time Job Description: 5-14 Years of in Big Data & Data related technology experience Expert level understanding of distributed computing principles Expert level knowledge and experience in Apache Spark Hands on programming with Python Experience with building stream-processing systems, using technologies such as Apache Storm or Sp...

5+ years of QA experience with a focus on automation and cloud-native environments. Hands-on expertise in automation tools such as Selenium, Cypress, or Playwright. Experience with ETL/data pipeline testing, including validating source-to-target data mapping, transformations, and aggregation logic. Proficient in writing and executing complex SQL queries for data validation. Working knowledge of GCP services such as BigQuery, Pub/Sub, Cloud Functions, Cloud Storage, etc. Familiarity with API testing using Postman, RestAssured, or similar tools. Scripting knowledge in Python, JavaScript, or Java for test automation. Experience working in CI/CD pipelines and version control (Git). Roles and Res...

? Strong programming skills in Python, PySpark, and SQL. ? Solid experience building and optimizing ETL or ELT workflows using technologies such as Spark, Snowflake, Airflow, and/or Azure Data Factory. ? Strong ability to craft and optimize complex SQL queries, stored procedures, and data transformations. ? Experience developing and maintaining scalable, high-performance data models. ? Hands-on experience with Snowflake, including the use of Snowpark for in-platform data processing. ? Experience integrating external data sources and APIs into data workflows. ? Good experience in Azure cloud infrastructure ? Familiarity with data governance and data quality frameworks (e.g., Microsoft Purview...

Job Title: Data Architect Location: Noida Experience: 8+ years Role Overview: We are seeking a seasoned Data Architect to design and implement robust, scalable, and secure data solutions. The ideal candidate will have deep expertise in data engineering , SQL , PySpark , and data modelling , with exposure of Power BI for reporting and visualization. Key Responsibilities: Define and implement data architecture strategies aligned with business goals. Design and optimize data pipelines and ETL workflows using PySpark and SQL . Develop and maintain data models (dimensional, relational, star schema) for analytics and BI. Ensure data quality, governance, and security across all platforms. Collabora...

Salary: 15 to 25 LPA Exp: 4 to 7 years Location: Gurgaon/Pune/Bengalore Notice: Immediate to 30 days..!! Job Profile: Experienced Data Engineer with a strong foundation in designing, building, and maintaining scalable data pipelines and architectures. Skilled in transforming raw data into clean, structured formats for analytics and business intelligence. Proficient in modern data tools and technologies such as SQL, T-SQL, Python, Databricks, and cloud platforms (Azure). Adept at data wrangling, modeling, ETL/ELT development, and ensuring data quality, integrity, and security. Collaborative team player with a track record of enabling data-driven decision-making across business units. As a Dat...

SUMMARY Wissen Technology is Hirin g for Python + Database Developer About Wissen Technology: At Wissen Technology, we deliver niche, custom-built products that solve complex business challenges across industries worldwide. Founded in 2015, our core philosophy is built around a strong product engineering mindset ensuring every solution is architected and delivered right the first time. Today, Wissen Technology has a global footprint with 2000+ employees across offices in the US, UK, UAE, India, and Australia . Our commitment to excellence translates into delivering 2X impact compared to traditional service providers. How do we achieve this? Through a combination of deep domain knowledge, cut...

SUMMARY Wissen Technology is Hiring for Senior Data Engineer About Wissen Technology: At Wissen Technology, we deliver niche, custom-built products that solve complex business challenges across industries worldwide. Founded in 2015, our core philosophy is built around a strong product engineering mindset ensuring every solution is architected and delivered right the first time. Today, Wissen Technology has a global footprint with 2000+ employees across offices in the US, UK, UAE, India, and Australia. Our commitment to excellence translates into delivering 2X impact compared to traditional service providers. How do we achieve this? Through a combination of deep domain knowledge, cutting-edge...

HIRING- Data Engineer || Hyderabad EXP-4-6Yrs 20LPA & 6-8/9Yrs 25LPA NP-Imm-15 days SHIFTS-Rotational SKILLS-Data pipeline, Data Engineer, Data Workflows, SQL, Data extraction, AWS, Pyspark, Python, Spark Drop your CV at rashibimaginators@gmail.com