1175 Data Pipeline Jobs - Page 22

About the Role : Join TA Digital (Credera), a global consulting firm delivering AI-driven customer experiences. As a Senior Salesforce CPQ Developer , you will design, develop, and implement advanced CPQ solutions tailored to client needs, driving efficiency across product configuration, pricing, and quoting processes. Key Responsibilities: Collaborate with product specialists and the Development Manager to understand complex requirements and translate them into Salesforce solutions Design and develop Salesforce CPQ solutions to streamline the sales process Customize Salesforce CPQ to meet business requirements, including product configuration, pricing, and quoting Collaborate with business ...

My linkedin linkedin.com/in/yashsharma1608. contract period - 6-12month payroll will be - ASV consulting , my company client - will disclose after 1 round Job location - Gurgaon - onsite(WFO) budget - upto 1lpa/month , depending on last (relevant hike) Exprnce - 3+ JD is About the Role We are looking for a skilled AWS Data Engineer with strong hands-on experience in AWS Glue and AWS analytics services. The candidate will be responsible for designing, building, and optimizing scalable data pipelines and ETL processes that support advanced analytics and business intelligence requirements. Key Responsibilities Design and develop ETL pipelines using AWS Glue, PySpark, and AWS services (Lambda, S...

Job Title : Customer Support Engineer/ OPS Engineer Role description :- Customer Support Engineer/ OPS Engineer We are looking for a dynamic executive to join our customer support operations team. The executive will be responsible for technical customer support and manage relationships with customers. This position is ideal for young solar photovoltaic enthusiast in renewable energy sector, who is looking for demonstrating technical understanding in challenges in solar PV power plant O&M and asset management and providing best customer support service. The executive would be closely working directly with customer support manager and operations head on daily basis. The executive will be hands...

Job Description: Complete data ingestion, data pipeline, data lineage , data quality , data wearhouse, data governance and data reconciliation. Essential Skills: Must have Data architect experience and knowledge. Data Architect with over 10 +years of hands-on experience in designing, developing, and managing large-scale data solutions. Proven expertise in building and optimizing ETL pipelines. Strong in data preprocessing, and enhancing data quality.Extracting events, processing large datasets (5 billion+ records) within (a Spark-Hadoop) cluster. Automated data processing tasks for DAAS (Data as a Service) project, streamlining workflow efficiency. Configured file and client setups, ensuring...

YASH Technologies is a leading technology integrator specializing in helping clients reimagine operating models, enhance competitiveness, optimize costs, foster exceptional stakeholder experiences, and drive business transformation. At YASH, you will be part of a team of talented individuals working with cutting-edge technologies. Our purpose is centered around making real positive changes in an increasingly virtual world, transcending generational gaps and disruptions of the future. We are currently seeking AWS Data Pipeline Professionals with 3-5 years of experience in the following areas: - Designing, developing, and implementing cloud solutions on AWS, utilizing a wide range of AWS servi...

Job Functions: You will be a member of our AI Platform Team, supporting the next generation AI architecture for various research and engineering teams within the organization. You'll partner with vendors and the infrastructure engineering team for security and service availability You'll fix production issues with engineering teams, researchers, data scientists, including performance and functional issues Diagnose and solve customer technical problems Participate in training customers and prepare reports on customer issues Be responsible for customer service improvements and recommend product improvements Write support documentation You'll design and implement zero-downtime to monitor and ac...

Join us to do the best work of your career and make a profound social impact as a Data engineer on our Team in Bangalore . We are looking for Data Engineer with strong data analysis skill using platforms such as Gcp and Databricks and have experience in building pipelines using Databricks. The ideal candidate will have hands-on experience with Databricks, Gcp, and python and will ensure data accuracy and consistency. This role requires the ability to extract, analyse, and present data efficiently, enabling data-driven decision-making across the organization. You will: Write and optimize SQL queries for data extraction, aggregation, and reporting. Work with structured and semi-structured data...

Develop and fine-tune generative models (LLMs, GANs, VAEs), optimize prompts, build data pipelines, deploy AI models on cloud/on-prem, ensure scalability, conduct A/B tests, and collaborate across teams using Python, ML frameworks, and MLOps tools.

Role & responsibilities • 5+ years of experience in data engineering or data pipeline development . • Strong hands-on experience with A WS data services, including Redshift, Glue, DynamoDB, Lambda, and API Gateway. • Experience designing high-performance data architectures with both structured and semi-structured data. • Proficiency in Python, SQL, and streaming technologies like Kafka . • Solid understanding of Redis, PostgreSQL, and distributed storage principles. • Familiarity with DevOps tools and practices: CI/CD (GitHub Actions), infrastructure-as-code, monitoring, and automation. • Experience with SonarQube or similar tools for enforcing code quality and maintainability. • Excellent c...

About the Role: Looking for a Principal Data Engineer to lead the design and delivery of scalable data solutions using Azure Data Factory and Azure Data bricks. This is a consulting- focused role that requires strong technical expertise, stakeholder engagement, and architectural thinking. You will work closely with business, functional, and technical teams to define data strategies, Design robust pipelines, and ensure smooth delivery in an Agile environment. Responsibilities Collaborate with business and technology stakeholders to gather and understand data needs Translate functional requirements into scalable and maintainable data architecture Design and implement robust data pipelines Lead...

Summary We are seeking an experienced Data Engineer (5years of relevant experience) with a strong technical background to design, develop, and support cutting-edge data-driven solutions. The ideal candidate will bring expertise in data engineering, cloud technologies, and enterprise-scale solution development, along with exceptional problem-solving, communication, and leadership skills. Key Responsibilities: Design and implement data engineering pipelines using PySpark, Palantir Foundry and Python. Leverage cloud technologies (AWS) to build scalable and secure solutions. Ensure compliance with architecture best practices and develop enterprise-scale solutions. Develop data-driven application...

Role & responsibilities We are seeking a highly skilled and motivated Data Engineer to join our dynamic team. The ideal candidate will have a passion for building and optimizing robust data pipelines, managing real-time data flows, and developing solutions that drive data-driven decision-making. If you thrive in a fast-paced environment and are eager to tackle complex data challenges, we want to hear from you! Role and Responsibilities: Design, develop, and maintain scalable and high-performance ETL/ELT pipelines for batch and real-time data processing from diverse sources. Develop and implement APIs and webhooks for seamless and secure data ingestion and consumption by various internal and ...

1. Software Development: - Design, develop, test, and deploy high-performance and scalable data solutions using Python,PySpark, SQL - Collaborate with cross-functional teams to understand business requirements and translate them into technical specifications. - Implement efficient and maintainable code using best practices and coding standards. 2. Databricks Platform: - Work with Databricks platform for big data processing and analytics. - Develop and maintain ETL processes using Databricks notebooks. - Implement and optimize data pipelines for data transformation and integration. 3. Continuous Learning: - Stay updated on the latest industry trends, tools, and technologies related to Python,...

Join us as a Data Engineer Pyspark at Barclays, where you will spearhead the evolution of our infrastructure and deployment pipelines, driving innovation and operational excellence You will harness cutting-edge technology to build and manage robust, scalable and secure infrastructure, ensuring seamless delivery of our digital solutions, To be successful as a Data Engineer Pyspark you should have experience with: Expertise in PySpark , Python, Strong SQL knowledge, Very good understanding of writing and debugging of code, Quick learner, strong analytical and problem-solving skills and should possess excellent written and verbal communication skills, Some Other Highly Valued Skills May Include...

In This Role, Your Work Will Involve Technical Implementation & Integration Build proof-of-concepts and MVPs leveraging both the internal AI tech stack and publicly available AI tools Develop and enhance AI-driven solutions using Python Assist in system integration and deployment on cloud platforms (Azure, AWS) Contribute to the internal knowledge base and engineering playbooks Participate in cross-functional projects with product, engineering, and customer success teams Customer Solution Design, Deployment & Success Assist in Client Engagement and Discovery Shadow Solutions Engineers, participate in customer calls, and help analyze requirements to understand complex business workflows and k...

Who we are About Stripe Stripe is a financial infrastructure platform for businesses Millions of companies?from the worlds largest enterprises to the most ambitious startups?use Stripe to accept payments, grow their revenue, and accelerate new business opportunities Our mission is to increase the GDP of the internet, and we have a staggering amount of work ahead That means you have an unprecedented opportunity to put the global economy within everyones reach while doing the most important work of your career, About The Team One of the key drivers to our success is an exceptional emphasis on our people We believe that our competitive advantage lies in our talent and that we can build a sustai...

Position Title Assistant Manager Forest Planning Key Accountabilities / Responsibilities Create customized accelerated plan for obtaining Forest and Wildlife (F&WL) clearance process to ensure on-time project delivery, Planning of budget to maintain Cost efficiency, Cross functional coordination Execution Provide technical support across the Project, supporting Forest and Wildlife team by ensuring robust execution of end-end process as per guidelines and timely availability of Forest & Wildlife Clearances as well as Working Permission achieving First Time Right (FTR), Ensure strict Environment, Forest and Wildlife compliance of states and MoEF&CC, Explore and identify other levers to crunch ...

Senior Data Engineer Cloud & Modern Data Architectures Role Overview: We are looking for a Senior Data Engineer with expertise in ETL/ELT, Data Engineering, Data Warehousing, Data Lakes, Data Mesh, and Data Fabric architectures . The ideal candidate should have hands-on experience in at least one or two cloud data platforms (AWS, GCP, Azure, Snowflake, or Databricks) and a strong foundation in building PoCs, mentoring freshers, and contributing to accelerators and IPs. Must-Have: 5-8 years of experience in Data Engineering & Cloud Data Services . Hands-on with any 2 clouds AWS (Redshift, Glue), GCP (BigQuery, Dataflow), Azure (Synapse, Data Factory), Snowflake, Databricks . Strong SQL, Pytho...

Mitra Infotech is looking for Machine Learning Engineer to join our dynamic team and embark on a rewarding career journey Developing and implementing ML algorithms: The Machine Learning Engineer develops and implements machine learning algorithms to solve specific problems, such as natural language processing, computer vision, or predictive modeling. Building data pipelines: The Machine Learning Engineer is responsible for building data pipelines that collect, store, and preprocess data used in machine learning algorithms. Creating and maintaining ML infrastructure: The Machine Learning Engineer is responsible for creating and maintaining ML infrastructure, including hardware, software, and ...

Senior/Lead Data Engineer (Warehouse Architecture & PySpark) Location: Hyderabad Experience: 6-12 years | Team: Data Platform / Analytics Engineering The opportunity Own the end-to-end build of a modern analytics foundationdesign clean warehouse models, craft high-quality PySpark transformations, and ship reliable pipelines feeding BI/ML at scale. What you’ll do Model the warehouse: Define dimensional/star schemas (SCD1/2, snapshots), conformed dimensions, and clear grains across core domains. Author robust transformations: Build performant PySpark jobs for batch/near-real-time; handle nested JSON, schema evolution, late data, and idempotent re-runs. Ingestion & CDC: Operate change-data-capt...

Job description Key Responsibilities: Data Pipeline Development: Design, develop, and maintain robust and scalable data pipelines to collect, clean, and process large datasets from various sources. Data Integration: Integrate and optimize data from multiple platforms and systems, ensuring smooth flow and accessibility across the organization. Data Quality: Ensure high-quality, accurate, and consistent data is made available for analysis and reporting, applying appropriate data validation and cleaning techniques. ETL Process: Develop, deploy, and maintain Extract, Transform, Load (ETL) processes to move data between systems. Data Warehouse Management: Work on the design and optimization of da...

Job Title: Talent Acquisition Trainee Overview: The Talent Acquisition Trainee will play a crucial role in supporting the recruitment process by assisting experienced recruiters in sourcing, screening, and selecting candidates to meet the organization's hiring needs This position aims to develop the trainee's skills and understanding of recruitment practices, helping them grow into a proficient and effective recruiter, Key Responsibilities: Assist in creating and posting job advertisements across various platforms, Source potential candidates through social media, job boards, and networking, Screen resumes and applications, identifying qualified candidates for further evaluation, Conduct pre...

Role: Python Developer Experience: 5 years to 8 years Location: Coimbatore, Indore, Bhubaneshwar, Nagpur Mandatory Skills: Python, SQL, DBT, Data pipeline Job Description: 5 to 8 years of experience in handle the development integration and optimization of Datawarehouse applications with strong SQL background Hands on exposure using Python and Data Build Tool (DBT) used in ETL and ELT processes Testing ETLs for large scale DW applications Should have exposure working using Agile framework and in a DevOps environment Good experience in Python programming. Experience with Docker or Container based solutions and environments Experience in designing and Testing ETLs that process large volumes of...

5+ years of QA experience with a focus on automation and cloud-native environments. Hands-on expertise in automation tools such as Selenium, Cypress, or Playwright. Experience with ETL/data pipeline testing, including validating source-to-target data mapping, transformations, and aggregation logic. Proficient in writing and executing complex SQL queries for data validation. Working knowledge of GCP services such as BigQuery, Pub/Sub, Cloud Functions, Cloud Storage, etc. Familiarity with API testing using Postman, RestAssured, or similar tools. Scripting knowledge in Python, JavaScript, or Java for test automation. Experience working in CI/CD pipelines and version control (Git). Roles and Res...

Role: Python Data Engineer Experience: 8 years to 12 years Location: Pan India Mandatory skills: Python, SQL, Data Pipeline should reflect in resume. Job description: Extensive experience with Python scripting and Cloud Technologies Minimum 7 years of IT and Data Related Experience needed. Extensive experience with AWS components like S3, Athena, EMR, Glue, Redshift and SageMaker Experience in Automating ETL process with Python and Automation around AWS Data & Infrastructure. Experience with building Python Automation Frameworks around Data Process, Data Analysis and Data Validation.