1578 Data Pipeline Jobs - Page 31

Key Responsibilities: Ensure timely delivery of data to stakeholders. Maintain documentation of data sources, processes, and delivery workflows. Extract, transform, and load (ETL) data from various sources. Maintain and optimize data pipelines to ensure smooth data flow. Ensure data integrity by identifying and resolving inconsistencies. Required Skills: Proficiency in SQL (Snowflake), Excel, EmEditor and Python. Proficiency in AWS. Strong analytical and problem-solving skills. Attention to detail and accuracy in data handling. Ability to work with large datasets efficiently. Good communication skills to liaise with technical and non-technical teams. Strong documentation skills for processes...

**Job Description: Principal Data Architect - Principal Architect** **Company:** Happiest Minds **Location:** Bangalore **Experience Required:** 16 - 20 years **About Happiest Minds:** Happiest Minds is a leading digital transformation and technology services company, empowered by the mission of enhancing the happiness of our customers, employees, and society at large. We provide cutting-edge solutions leveraging advanced technologies to drive meaningful outcomes for our clients. **Position Overview:** We are seeking an experienced and highly skilled Principal Data Architect to join our dynamic team. The ideal candidate will possess a solid background in data architecture, data lakes, and ET...

Job role: AWS Data Engineer Location: PAN India Job description Key Responsibilities Design and build data pipelines & Data lakes to automate ingestion of structured and unstructured data that provide fast, optimized, and robust end-to-end solutions Knowledge about the concepts of data lake and dat warehouse Experience working with AWS big data technologies Improve the data quality and reliability of data pipelines through monitoring, validation and failure detection. Deploy and configure components to production environments Technology: Redshift, S3, AWS Glue, Lambda, SQL, PySpark, SQL.

Required Skills and Qualifications 4+ years of professional experience in a data analyst or support-focused role. Expert-level proficiency in Python, with extensive experience in using libraries like Pandas and NumPy for data cleaning, analysis, and troubleshooting. Strong experience with Tableau for dashboard management and updates. Expert-level proficiency in SQL for data extraction and validation. Exceptional communication and problem-solving skills, with a service-oriented mindset. Proven ability to work independently and manage multiple support requests simultaneously

Experience Required : Minimum 7+ Years (Mandatory) Job Type : Full-time | Day Shift (IST) Job Description : We are looking for a skilled and experienced Databricks Engineer to join our team at our Hyderabad (Madhapur) office. The ideal candidate will have a strong background in data engineering, hands-on experience with Databricks, and a solid understanding of cloud platforms (AWS, Azure, or GCP). This is a great opportunity to work on cutting-edge data projects in a collaborative and fast-paced environment. Key Responsibilities : - Design and implement scalable and efficient data pipelines, ETL/ELT processes, and data integration solutions using Databricks. - Work with large-scale structure...

Job Description 4- 5years experience in IT or professional services experience in IT delivery or large-scale IT analytics projects Candidates must have expertise knowledge of Google Cloud Platform; the other cloud platforms are nice to have. Expert knowledge in SQL development. Experience with Apache Beam/Google Dataflow/Apache Spark in creating end to end data pipelines. Experience in some of the following: Python, Hadoop, Spark, SQL, Big Query, Big Table Cloud Storage, Datastore, Spanner, Cloud SQL, Machine Learning. Experience programming in Java, Python, etc. Identify downstream implications of data loads/migration (e.g., data quality, regulatory, etc.) Implement data pipelines to automa...

Own the overall roadmap, and execution for ServiceNow platform, including ITSM, ITOM, CSM, FSM modules Lead the design, implementation, and support of scalable ServiceNow solutions and workflows. Oversee ServiceNow custom development, scripting, debugging, troubleshooting and high-availability deployments. Ensure robust workflow management, business rules, and ServiceNow best practices are adhered to Drive DevOps and automation practices, including Git version control, CI/CD pipelines, and integration of microservices with Kubernetes Lead configuration, customization, and integration of ServiceNow with enterprise applications Optimize performance, scalability, and security of ServiceNow envi...

We are currently seeking an experienced professional to join our team in the role of Senior Software Engineer. In this role, you will: Design and Develop Scalable Data Pipelines: Architect and implement end-to-end data workflows using Apache Airflow for orchestration, integrating multiple data sources and sinks across cloud and on-prem environments. BigQuery Data Modeling and Optimization: Build and optimize data models in Google BigQuery for performance and cost-efficiency, including partitioning, clustering, and materialized views to support analytics and reporting use cases. ETL/ELT Development and Maintenance: Design robust ETL/ELT pipelines to extract, transform, and load structured and...

Job DescriptionKey Responsibilities:- Design, develop, and maintain scalable, efficient, and reliable systems to support GenAI and machine learning-based applications and use cases- Lead the development of data pipelines, architectures, and tools to support data-intensive projects, ensuring high performance, security, and compliance- Collaborate with other stakeholders to integrate AI and ML models into production-ready systems- Work closely with non-backend expert counterparts, such as data scientists and ML engineers, to ensure seamless integration of AI and ML models into backend systems- Ensure high-quality code, following best practices, and adhering to industry standards and company gu...

Design and develop solutions using Bigdata tools and technologies like Bigquery, Hive, Spark etc. Extensive hands-on experience in object-oriented programming using Python, PySpark APIs etc. Experience in building data pipelines for huge volume of data. Experience in designing, implementing, and managing various ETL job execution flows. Experience in implementing and maintaining Data Ingestion process. Hands on experience in writing basic to advance level of optimized queries using HQL, SQL & Spark. Hands on experience in designing, implementing, and maintaining Data Transformation jobs using most efficient tools/technologies. Ensure the performance, quality, and responsiveness of solutions....

Design and develop solutions using Bigdata tools and technologies like Bigquery, Hive, Spark etc. Extensive hands-on experience in object-oriented programming using Python, PySpark APIs etc. Experience in building data pipelines for huge volume of data. Experience in designing, implementing, and managing various ETL job execution flows. Experience in implementing and maintaining Data Ingestion process. Hands on experience in writing basic to advance level of optimized queries using HQL, SQL & Spark. Hands on experience in designing, implementing, and maintaining Data Transformation jobs using most efficient tools/technologies. Ensure the performance, quality, and responsiveness of solutions....

Sr Data Architect with Banking & Financial Services (BFSI) expertise explicitly should have integrated domain-specific skills into both the technical and strategic areas: Role Overview A Senior Data Engineer & Architect specializing in Azure Data Lake who also brings deep banking and financial services (BFSI) experience. This professional design and governs secure, large-scale data platforms while aligning with the unique compliance, risk, and business intelligence needs of financial institutions. Core Technical Skills Azure Data Lake & Storage > Azure Data Lake Storage Gen2 (ADLS Gen2): Hierarchical namespaces, partitioning strategies, performance tuning. > Delta Lake / Lakehouse: ACID tran...

Job Description Job Title:*Lead Data Engineer Data Company:*Happiest Minds Experience:*8 to 11 years Job Description:** Happiest Minds is seeking a talented and experienced Lead Data Engineer to join our dynamic team The successful candidate will be responsible for designing and implementing robust data pipelines and engineering solutions that empower our analytics and business intelligence initiatives As a key member of our data engineering team, you will work closely with data scientists, analysts, and other stakeholders to turn raw data into actionable insights, Key Responsibilities:** Design, develop, and maintain scalable data pipelines using Azure services and Azure Databricks, Collabo...

Job Description Job Title: Software/Data Engineer AWS Role Overview We are seeking a skilled Data Engineer to develop and maintain a high-performance streaming data platform for a leading healthcare CRM product You will work across technologies like Python, SQL, NodeJS, Kafka, and cloud-native services, contributing to scalable, efficient, and robust data solutions, Key Responsibilities Design, develop, and maintain scalable data pipelines and streaming systems, Work with Python, SQL, Scala, NodeJS, and various data stores (relational, NoSQL, Kafka), Analyze healthcare and marketing datasets using SQL, Refactor and optimize existing codebases for performance improvements, Collaborate closely...

Job Description Job Title:*Lead Data Engineer Data Company:*Happiest Minds Location:*Bangalore Job Type:*Full-time Job Description:** Happiest Minds is seeking a highly skilled and experienced Lead Data Engineer to join our innovative team As a Lead Data Engineer, you will play a vital role in designing, developing, and managing data infrastructure and solutions that drive our data strategy You will lead a team of data engineers, ensuring the delivery of high-quality data pipelines and architecture to support our analytics and business initiatives, Key Responsibilities:** Design and build scalable data pipelines and ETL processes to ingest, process, and store large volumes of structured and ...

Job Description Job Title:*Data Engineer Data Company Name:*Happiest Minds Location:*Bangalore Job Type:*Full-time Experience Required:*3 to 5 years About Happiest Minds Happiest Minds Technologies is a next-generation digital transformation and technology services company, delivering delightful experiences to its clients through a blend of technology and domain expertise Join us to create value and make businesses thrive in the ever-evolving digital landscape, Job Summary We are seeking a talented and motivated Data Engineer to join our dynamic data team The ideal candidate will have a strong foundation in data engineering, specifically in Python, MySQL, SQL, and AWS You will be responsible...

About us: We are building a modern, scalable, end-to-end automated on-premises data platform designed to handle complex data workflows, including data ingestion, ETL processes, physics-based calculations and machine learning predictions. Our platform integrates with multiple data sources, edge devices, and storage systems. We are using core Python as programming language, Docker as deployment technology and Dagster as orchestrator. We are a small cross-functional team sharing a wide range of tasks from database operations to data science. We are looking for a data platform developer with expert Python and Docker knowledge who would help to develop, maintain, and optimize our platform. Key Re...

Role: Senior Member of Technical Staff Experience: 4 to 9 years Location: Manyata Tech Park, Bangalore Work mode: Hybrid Skills: Design Data Analytics, Data Engineering, Data Warehouse, Java, SQL, Spark & Spark-SQL, Autonomous Data Warehouse, Object Store, Data Flow/DIS (or similar tech stack), exposure to cloud Short description : Looking for a Software Developer (Data Engineer) who are willing to perform Data Engineering tasks with skills sets like SQL, Spark, Big Data Development, Java & Cloud experience. Job description: Design, develop, troubleshoot and debug software programs for databases, applications etc. Strong knowledge of SQL is required. You should be having programming experien...

Role & responsibilities We are seeking a highly motivated and technically strong Data Engineering Manager to lead our data engineering team and shape the data foundation of our organization. You will be responsible for architecting, building, and scaling robust data pipelines, storage systems, and real-time infrastructure to support analytics, product intelligence, and AI/ML use cases. This is a hands-on leadership role requiring both technical expertise and people management skills to guide a growing team and ensure delivery of high-impact data solutions. Key Responsibilities Lead and mentor a team of data engineers to design, develop, and maintain high-performance data pipelines and platfo...

What you will do: ACV’s Machine Learning (ML) team is looking to grow its MLOps team. Multiple ACV operations and product teams rely on the ML team’s solutions. Current deployments drive opportunities in the marketplace, in operations, and sales, to name a few. As ACV has experienced hyper growth over the past few years, the volume, variety, and velocity of these deployments has grown considerably. Thus, the training, deployment, and monitoring needs of the ML team has grown as we’ve gained traction. MLOps is a critical function to help ourselves continue to deliver value to our partners and our customers. Successful candidates will demonstrate excellent skill and maturity, be self-motivated...

Proven experience as a Data Solutions Architect or in a similar role. Strong expertise in Azure cloud services (Data Factory, Synapse, Databricks, Cosmos DB, etc.). Data modeling, data integration data governance, security, and frameworks. Health insurance Annual bonus

Data Engineer with a focus on GCP. Minimum 1 year hands-on experience with Python, SQL, DataFlow, DataStream, Cloud Functions, Cloud Run, Pub/Sub, BigQuery, Cloud Storage & data ingestion, CDC techniques, and real-time/batch pipeline orchestration .

Who We Are At Kyndryl, we design, build, manage and modernize the mission-critical technology systems that the world depends on every day. So why work at Kyndryl? We are always moving forward – always pushing ourselves to go further in our efforts to build a more equitable, inclusive world for our employees, our customers and our communities. The Role Are you ready to embark on an exhilarating journey as a Data Consultant? Join Kyndryl and become a driving force behind the transformative power of data! We're seeking an exceptionally talented individual to accelerate the competitive performance of our customers worldwide, establishing us as their unrivaled business and technology consulting p...

Design, build, and maintain scalable data pipelines for finance-related datasets. Partner with finance stakeholders (product & business teams) to understand requirements and translate them into robust technical solutions. Ensure data accuracy, consistency, and timeliness across reporting layers and downstream systems. Optimize SQL queries and Spark jobs for large-scale datasets to improve performance and cost efficiency. Implement scheduling, orchestration, and monitoring of workflows using Airflow (or similar tools). Collaborate with cross-functional teams (Finance, Product, Analytics, Engineering) to deliver business-critical solutions. Adhere to best practices for coding, testing, and dep...

Develop and deploy machine learning models on Azure cloud platform using Python programming language and Azure ML services. Design and implement scalable and efficient data pipelines for model training and inference. Collaborate with data scientists and business stakeholders to understand requirements and translate them into technical solutions. Implement best practices for ML development, including version control, testing, and documentation. Develop and maintain APIs on Azure cloud platform for model deployment and integration with applications. Monitor and optimize model performance, ensuring scalability, reliability, and security. Participate in code reviews, troubleshooting, and optimiz...