874 Data Pipeline Jobs - Page 27

We are seeking a skilled and motivated Data Engineer to join our dynamic team. The ideal candidate will have experience in designing, developing, and maintaining scalable data pipelines and architectures using Hadoop, PySpark, ETL processes , and Cloud technologies . Responsibilities: Design, develop, and maintain data pipelines for processing large-scale datasets. Build efficient ETL workflows to transform and integrate data from multiple sources. Develop and optimize Hadoop and PySpark applications for data processing. Ensure data quality, governance, and security standards are met across systems. Implement and manage Cloud-based data solutions (AWS, Azure, or GCP). Collaborate with data s...

Skill : Java, Spark, Kafka Experience : 10 to 16 years Location : Hyderabad As Data Engineer, you will : Support in designing and rolling out the data architecture and infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources Identify data source, design and implement data schema/models and integrate data that meet the requirements of the business stakeholders Play an active role in the end-to-end delivery of AI solutions, from ideation, feasibility assessment, to data preparation and industrialization. Work with business, IT and data stakeholders to support with data-related technical issues, their data infrastructure needs as we...

Data Engineer Job Description : Jash Data Sciences: Letting Data Speak! Do you love solving real-world data problems with the latest and best techniques? And having fun while solving them in a team! Then come and join our high-energy team of passionate data people. Jash Data Sciences is the right place for you. We are a cutting-edge Data Sciences and Data Engineering startup based in Pune, India. We believe in continuous learning and evolving together. And we let the data speak! What will you be doing? You will be discovering trends in the data sets and developing algorithms to transform raw data for further analytics Create Data Pipelines to bring in data from various sources, with differen...

Job Title: Data & GenAI AWS Specialist Experience: 1-4 Years Location: Bangalore Mandatory Qualification: B.E./ B.Tech/ M.Tech/ MS from IIT or IISc ONLY Job Overview: We are seeking a seasoned Data & GenAI Specialist with deep expertise in AWS Managed Services (PaaS) to join our innovative team. The ideal candidate will have extensive experience in designing sophisticated, scalable architectures for data pipelines and Generative AI (GenAI) solutions leveraging cloud services. Key to this role is the ability to articulate architecture solutions clearly and effectively to customers, helping them conceptualize and implement advanced GenAI-driven applications tailored precisely to their business...

Detailed job description - Skill Set: Strong Knowledge in Databricks. This includes creating scalable ETL (Extract, Transform, Load) processes, data lakes Strong knowledge in Python and SQL Strong experience with AWS cloud platforms is a must Good understanding of data modeling principles and data warehousing concepts Strong knowledge of optimizing ETL jobs, batch processing jobs to ensure high performance and efficiency Implementing data quality checks, monitoring data pipelines, and ensuring data consistency and security Hands on experience with Databricks features like Unity Catalog Mandatory Skills Databricks, AWS

Experience 12yrs only Notice period :- immediate / 15days Location :- Hyderabad Client :- Tech Star Group Please highlight the mandatory sill in resume . Client Feedback :- In short, the client is primarily looking for a candidate with strong expertise in data-related skills, including: SQL & Database Management: Deep knowledge of relational databases (PostgreSQL), cloud-hosted data platforms (AWS, Azure, GCP), and data warehouses like Snowflake. ETL/ELT Tools: Experience with SnapLogic, StreamSets, or DBT for building and maintaining data pipelines. / ETL Tools Extensive Experience on data Pipelines Data Modeling & Optimization: Strong understanding of data modeling, OLAP systems, query opt...

Position Summary: We are seeking a highly skilled ETL QA Engineer with at least 6 years of experience in ETL/data pipeline testing on the AWS cloud stack , specifically with Redshift, AWS Glue, S3 , and related data integration tools. The ideal candidate should be proficient in SQL , capable of reviewing and validating stored procedures , and should have the ability to automate ETL test cases using Python or suitable automation frameworks . Strong communication skills are essential, and web application testing exposure is a plus. Technical Skills Required: SQL Expertise : Ability to write, debug, and optimize complex SQL queries. Validate data across source systems, staging areas, and report...

Job Title: Lead Data Scientist We are seeking a highly skilled and experienced Lead Data Scientist to join our dynamic team. In this role, you will be responsible for leading data-driven projects, mentoring junior data scientists, and guiding the organization in making strategic decisions based on data insights. Key Responsibilities: - Develop and implement advanced statistical models and algorithms to analyze complex data sets. - Collaborate with cross-functional teams to identify business opportunities and translate them into data-driven solutions. - Mentor and oversee a team of data scientists, providing guidance on best practices and techniques in data analysis and modeling. - Communicat...

We are looking for a highly experienced Senior Data Engineer with deep expertise in Snowflake to lead efforts in optimizing the performance of our data warehouse to enable faster, more reliable reporting. You will be responsible for improving query efficiency, data pipeline performance, and overall reporting speed by tuning Snowflake environments, optimizing data models, and collaborating with Application development teams. Roles and Responsibilities Analyze and optimize Snowflake data warehouse performance to support high-volume, complex reporting workloads. Identify bottlenecks in SQL queries, ETL/ELT pipelines, and data models impacting report generation times. Implement performance tunin...

Experience in designing, developing, implementing, and optimizing data solutions on Microsoft Azure. Proven expertise in leveraging Azure services for ETL processes, data warehousing and analytics, ensuring optimal performance and scalability.

About This Role : We are looking for a talented and experienced Data Engineer with Tech Lead with hands-on expertise in any ETL Tool with full knowledge about CI/CD practices with leading a team technically more than 5 and client facing and create Data Engineering, Data Quality frameworks. As a tech lead must ensure to build ETL jobs, Data Quality Jobs, Big Data Jobs performed performance optimization by understanding the requirements, create re-usable assets and able to perform production deployment and preferably worked in DWH appliances Snowflake / redshift / Synapse Responsibilities Work with a team of engineers in designing, developing, and maintaining scalable and efficient data soluti...

Deployment, configuration & maintenance of Databricks clusters & workspaces Security & Access Control Automate administrative task using tools like Python, PowerShell &Terraform Integrations with Azure Data Lake, Key Vault & implement CI/CD pipelines Required Candidate profile Azure, AWS, or GCP; Azure experience is preferred Strong skills in Python, PySpark, PowerShell & SQL Experience with Terraform ETL processes, data pipeline &big data technologies Security & Compliance

Loction :Mumbai Job Description: Looking for Candidates with 8 to 10 years of experience Hands on experience of implementing data pipelines using traditional DWH, Big data & Cloud ecosystem Good exposure to data architecture design, cost and size estimation Good understanding of handling Realtime/streaming pipelines Have experience of Data Quality and Data Governance Have experience of handling & interacting clients and managing vendors Having knowledge on AI/ML, GenAI is a plus Exposure of managing and leading teams

Looking for a Business Analyst with strong SQL, ETL/data warehouse testing, insurance domain knowledge (GuideWire), and experience in defect tracking & issue management tools.

Syensqo is all about chemistry Were not just referring to chemical reactions here, but also to the magic that occurs when the brightest minds get to work together This is where our true strength lies In you In your future colleagues and in all your differences And of course, in your ideas to improve lives while preserving our planets beauty for the generations to come We are looking for: A motivated and experienced Data & Integration Senior Data Engineer who thrives on turning complex data challenges into innovative solutions You will lead service ownership for key data technologies, guide the data engineering team, and collaborate across functions to deliver seamless end-to-end data integra...

BIZMETRIC INDIA PRIVATE LIMITED is looking for DS / ML / Time-Series Engineer to join our dynamic team and embark on a rewarding career journey Data Exploration and Preparation:Explore and analyze large datasets to understand patterns and trends Prepare and clean datasets for analysis and model development Feature Engineering:Engineer features from raw data to enhance the performance of machine learning models Collaborate with data scientists to identify relevant features for model training Model Development:Design and implement machine learning models to solve business problems Work on both traditional statistical models and modern machine learning algorithms Scalable Data Pipelines:Develop...

Who We Are At Kyndryl, we design, build, manage and modernize the mission-critical technology systems that the world depends on every day. So why work at Kyndryl? We are always moving forward – always pushing ourselves to go further in our efforts to build a more equitable, inclusive world for our employees, our customers and our communities. The Role Are you ready to dive headfirst into the captivating world of data engineering at Kyndryl? As a Data Engineer, you'll be the visionary behind our data platforms, crafting them into powerful tools for decision-makers. Your role? Ensuring a treasure trove of pristine, harmonized data is at everyone's fingertips. AWS Data/API Gateway Pipeline Engi...

Technical Lead AI & Data Warehouse (DWH) Pando is a global leader in supply chain technology, building the world's quickest time-to-value Fulfillment Cloud platform. Pandos Fulfillment Cloud provides manufacturers, retailers, and 3PLs with a single pane of glass to streamline end-to-end purchase order fulfillment and customer order fulfillment to improve service levels, reduce carbon footprint, and bring down costs. As a partner of choice for Fortune 500 enterprises globally, with a presence across APAC, the Middle East, and the US, Pando is recognized as a Technology Pioneer by the World Economic Forum (WEF), and as one of the fastest growing technology companies by Deloitte. Role As the Se...

We are hiring for a leading Insurance Consulting organisation for Data Strategy & Governance role Exp: 4 to 6 yyrs Location : BangalorePune Responsibility: Develop and Drive Data Capability Maturity Assessment, Data & Analytics Operating Model & Data Governance exercises for clients Managing Critical Data Elements (CDEs) and coordinating with Finance Data Stewards Overseeing data quality standards and governance implementation Establish processes around effective data management ensuring Data Quality & Governance standards as well as roles for Data Stewards

Data Engineering Pipeline Development Design implement and maintain ETL processes using ADF and ADB Create and manage views in ADB and SQL for efficient data access Optimize SQL queries for large datasets and high performance Conduct end-to-end testing and impact analysis on data pipelines Optimization Performance Tuning Identify and resolve bottlenecks in data processing Optimize SQL queries and Delta Tables for fast data processing Data Sharing Integration Implement Delta Share, SQL Endpoints, and other data sharing methods Use Delta Tables for efficient data sharing and processing API Integration Development Integrate external systems through Databricks Notebooks and build scalable soluti...

To Apply - Submit Details via Google Form - https://forms.gle/8SUxUV2cikzjvKzD9 As a Senior Consultant in our Consulting team, youll build and nurture positive working relationships with teams and clients with the intention to exceed client expectations Seeking experienced AWS Data Engineers to design, implement, and maintain robust data pipelines and analytics solutions using AWS services. The ideal candidate will have a strong background in AWS data services, big data technologies, and programming languages. Role & responsibilities 1. Design and implement scalable, high-performance data pipelines using AWS services 2. Develop and optimize ETL processes using AWS Glue, EMR, and Lambda 3. Bu...

To Apply - Submit Details via Google Form - https://forms.gle/8SUxUV2cikzjvKzD9 As a Senior Consultant in our Consulting team, youll build and nurture positive working relationships with teams and clients with the intention to exceed client expectations Seeking experienced AWS Data Engineers to design, implement, and maintain robust data pipelines and analytics solutions using AWS services. The ideal candidate will have a strong background in AWS data services, big data technologies, and programming languages. Role & responsibilities 1. Design and implement scalable, high-performance data pipelines using AWS services 2. Develop and optimize ETL processes using AWS Glue, EMR, and Lambda 3. Bu...

Senior Data Engineer Cloud & Modern Data Architectures Role Overview: We are looking for a Senior Data Engineer with expertise in ETL/ELT, Data Engineering, Data Warehousing, Data Lakes, Data Mesh, and Data Fabric architectures . The ideal candidate should have hands-on experience in at least one or two cloud data platforms (AWS, GCP, Azure, Snowflake, or Databricks) and a strong foundation in building PoCs, mentoring freshers, and contributing to accelerators and IPs. Must-Have: 5-8 years of experience in Data Engineering & Cloud Data Services . Hands-on with AWS (Redshift, Glue), GCP (BigQuery, Dataflow), Azure (Synapse, Data Factory), Snowflake, Databricks . Strong SQL, Python, or Scala s...

Responsibilities: * Design, develop & maintain data pipelines using Airflow/Data Flow/Data Lake * Optimize performance & scalability of ETL processes with SQL & Python

Overview We are seeking a strategic and hands-on Manager of Business Intelligence (BI) and Data Governance to lead the development and execution of our enterprise-wide data strategy. This role will oversee data governance frameworks, manage modern BI platforms, and ensure the integrity, availability, and usability of business-critical data. Reporting into senior leadership, this role plays a pivotal part in shaping data-informed decision-making across functions including Finance, Revenue Operations, Product, and more. The ideal candidate is a technically proficient and people-oriented leader with a deep understanding of data governance, cloud data architecture, and SaaS KPIs. They will drive...