1578 Data Pipeline Jobs - Page 25

Skill: Pyspark Experience: 6 to 9 years Location: Kolkata Job description : Responsibilities Develop and maintain scalable data pipelines using Python and PySpark. Collaborate with data engineers and data scientists to understand and fulfill data processing needs. Optimize and troubleshoot existing PySpark applications for performance improvements. Write clean, efficient, and well-documented code following best practices. Participate in design and code reviews. Develop and implement ETL processes to extract, transform, and load data. Ensure data integrity and quality throughout the data lifecycle. Stay current with the latest industry trends and technologies in big data and cloud computing. ...

As a Product Analyst II, you will play a pivotal role in transforming data into actionable insights to drive product decisions. Partner closely with product managers, engineers, and designers to inform roadmap priorities, improve user experience, and measure product success. This role is ideal for someone with 2-4 years of analytics experience looking to grow into a senior analytics or product domain. - Conduct cohort analysis, funnel analytics, and broader product analytics to deeply understand user behavior, engagement patterns, and drop-off points. - Collaborate with Product Managers to define success metrics for new and existing features. - Perform exploratory deep-dives and structured a...

Experience Required : Minimum 7+ Years (Mandatory) Job Type : Full-time | Day Shift (IST) Job Description : We are looking for a skilled and experienced Databricks Engineer to join our team at our Hyderabad (Madhapur) office. The ideal candidate will have a strong background in data engineering, hands-on experience with Databricks, and a solid understanding of cloud platforms (AWS, Azure, or GCP). This is a great opportunity to work on cutting-edge data projects in a collaborative and fast-paced environment. Key Responsibilities : - Design and implement scalable and efficient data pipelines, ETL/ELT processes, and data integration solutions using Databricks. - Work with large-scale structure...

Position : Data Engineer Senior Consultant Location : Hyderabad (Work From Office) Shift : Night Shifts(08:30 PM to 05:30 AM) Two way cab provided Overall 12+ years of experience with at least 5+ years of experience in working with SQL on MySQL and PostgreSQL databases. Must be skilled in working with advanced SQL functions such as CTEs, window functions, procedural SQL, stored procedures, triggers etc. Significant experience in setting up, securing, and administering databases on premise or cloud (MS Azure). Demonstrated experience in managing data and related infrastructure on MS Azure. Demonstrated experience in building and orchestrating performant, secure, and cost-effective data pipeli...

An exciting opportunity for a skilled and motivated Data Scientist to join a growing group (AXIS, Access eXtract and Integrate Safe Data) within the Clinical Informatics Research Unit (CIRU) This role will continue to establish and grow The Pre-hospital Research and Audit Network (PRANA) PRANA is creating an end to end research ready database Currently there is no method for undertaking research at scale from pre hospital critical care all the way through to patient outcomes Additional connections to other datasets such as traffic collision databases will also be incorporated into PRANA By creating such a resource, the ability to identify life saving trends and treatments in medicine and roa...

Role & responsibilities Understand business processes, a bigger picture and core ideas behind the developed software. Support business analysts and developers with data subject expertise, query building and optimization Educate and evangelize application teams of the best practices on query building and optimization, data modeling and design Develop a thorough understanding of the data definitions, domain values, data relationships, business rules, sources, and data integration for each subject area Advocate and advance toward cutting edge engineering technologies Enhancing data collection procedures to include information that is relevant for building analytic systems Processing, cleansing,...

At Barclays, we dont just adapt to the future we shape it Embark on a transformative journey as Senior AWS Data Engineer Vice President, where you'll spearhead the evolution of our Reference Data Services function You will be part of a team of engineers with accountability and responsibility for the engineering (strategy, architecture, design, build, documentation, and testing) of the software components You will work closely with your colleagues in the wider cloud and middleware organization to bring their products to life as part of a modern developer-focused experience Additionally, your role requires effective stakeholder management, leadership, and decision-making to support business st...

An exciting opportunity for a skilled and motivated Data Scientist to join a growing group (AXIS, Access eXtract and Integrate Safe Data) within the Clinical Informatics Research Unit (CIRU) This role will continue to establish and grow The Pre-hospital Research and Audit Network (PRANA) PRANA is creating an end to end research ready database Currently there is no method for undertaking research at scale from pre hospital critical care all the way through to patient outcomes Additional connections to other datasets such as traffic collision databases will also be incorporated into PRANA By creating such a resource, the ability to identify life saving trends and treatments in medicine and roa...

Role: Senior Snowflake Engineer A Senior Snowflake Developer with Cortex experience is responsible for designing, building, and optimizing data solutions on the Snowflake platform, with a specific focus on leveraging Snowflake Cortex for AI-driven features like RAG (Retrieval- Augmented Generation), LLM functions, and vector search. Key responsibilities include developing scalable ETL/ELT pipelines, creating data models, implementing AI assistants, and coaching other team members, all while adhering to best practices in performance, cost control, and security. Skills: • Bachelors degree in Computer Science, Software Engineering, Information Technology, or related field required. • At least 4...

Salary: 15 to 30 LPA Exp: 3 to 8 years Location : Gurgaon/Bangalore/Pune/Chennai Notice: Immediate to 30 days..!! Key Responsibilities & Skillsets: Common Skillsets : 3+ years of experience in analytics, Scala, Pyspark, Python, Spark, SQL and associated data engineering jobs. Must have experience with managing and transforming big data sets using pyspark, spark-scala, Numpy pandas Excellent communication & presentation skills Experience in managing Python codes and collaborating with customer on model evolution Good knowledge of data base management and Hadoop/Spark, SQL, HIVE, Python (expertise). Superior analytical and problem solving skills Should be able to work on a problem independentl...

Job Title: AWS Data Engineer Company Name: NCSi Job Description: NCSi is seeking a skilled AWS Data Engineer to join our dynamic team. The ideal candidate will be responsible for designing and building robust data pipelines, developing data integration processes, and optimizing database performance in cloud environments. You will work collaboratively with data scientists, analysts, and other stakeholders to ensure data accuracy, reliability, and accessibility across the organization. Your role will also involve implementing best practices in data governance and security, as well as leveraging AWS services to support data analytics and reporting needs. Key Responsibilities: - Design, develop,...

skilled and motivated Data Engineer with hands-on experience in cloud platforms like AWS, Azure, or GCP to join our growing data team. have strong knowledge of data architecture, pipeline development, and cloud-based data processing.

Seeking a Senior Python Developer with 4+ years of experience,skilled in Flask/Django/FastAPI,REST APIs and data pipelines,SQL.Ensure scalable backend design,conduct code reviews and mentor teams for data and analytics-driven solutions. Health insurance Provident fund

About The Opportunity : A fast-scaling technology company operating in the Enterprise AI / Generative AI sector, building production-grade LLM-driven products and intelligent automation for global clients. We deliver secure, low-latency generative systems that power conversational AI, summarization, code generation, and retrieval-augmented applications across cloud-native environments. We are hiring a Senior Generative AI Engineer (7+ years) to own architecture, model development, and production deployment of advanced generative systems. This is a fully remote role for candidates based in India. Role & Responsibilities : - Lead end-to-end design and delivery of Generative AI/LLM solutionsdat...

Role Responsibilities : - Design and implement data pipelines using MS Fabric. - Develop data models to support business intelligence and analytics. - Manage and optimize ETL processes for data extraction, transformation, and loading. - Collaborate with cross-functional teams to gather and define data requirements. - Ensure data quality and integrity in all data processes. - Implement best practices for data management, storage, and processing. - Conduct performance tuning for data storage and retrieval for enhanced efficiency. - Generate and maintain documentation for data architecture and data flow. - Participate in troubleshooting data-related issues and implement solutions. - Monitor and...

About the Job : We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. In this pivotal role, you will be instrumental in driving our data engineering initiatives, with a strong emphasis on leveraging Dataiku's capabilities to enhance data processing and analytics. You will be responsible for designing, developing, and optimizing robust data pipelines, ensuring seamless integration of diverse data sources, and maintaining high data quality and accessibility to support our business intelligence and advanced analytics projects. This role requires a unique blend of expertise in traditional data engineering principles, advanced data modeling, and a forward-...

About The Opportunity : A fast-scaling technology company operating in the Enterprise AI / Generative AI sector, building production-grade LLM-driven products and intelligent automation for global clients. We deliver secure, low-latency generative systems that power conversational AI, summarization, code generation, and retrieval-augmented applications across cloud-native environments. We are hiring a Senior Generative AI Engineer (7+ years) to own architecture, model development, and production deployment of advanced generative systems. This is a fully remote role for candidates based in India. Role & Responsibilities : - Lead end-to-end design and delivery of Generative AI/LLM solutionsdat...

Role Responsibilities : - Design and implement data pipelines using MS Fabric. - Develop data models to support business intelligence and analytics. - Manage and optimize ETL processes for data extraction, transformation, and loading. - Collaborate with cross-functional teams to gather and define data requirements. - Ensure data quality and integrity in all data processes. - Implement best practices for data management, storage, and processing. - Conduct performance tuning for data storage and retrieval for enhanced efficiency. - Generate and maintain documentation for data architecture and data flow. - Participate in troubleshooting data-related issues and implement solutions. - Monitor and...

Role & Responsibilities : - Lead end-to-end design and delivery of Generative AI/LLM solutionsdata ingestion, pre-processing, model training/fine-tuning, evaluation, and scalable inference. - Develop and productionize transformer-based models (instruction-tuning, LoRA, quantization) using PyTorch/TensorFlow and Hugging Face tooling. - Architect and implement RAG pipelines integrating vector databases (FAISS/Milvus/Chroma), dense / sparse retrieval, and scalable embedding workflows. - Optimize inference throughput and latency using ONNX/TorchScript/TensorRT, autoscaling, and cost-efficient deployment patterns on cloud infra. - Define MLOps best practices : CI/CD for models, containerization, ...

About the Job : We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. In this pivotal role, you will be instrumental in driving our data engineering initiatives, with a strong emphasis on leveraging Dataiku's capabilities to enhance data processing and analytics. You will be responsible for designing, developing, and optimizing robust data pipelines, ensuring seamless integration of diverse data sources, and maintaining high data quality and accessibility to support our business intelligence and advanced analytics projects. This role requires a unique blend of expertise in traditional data engineering principles, advanced data modeling, and a forward-...

Role Responsibilities : - Design and implement data pipelines using MS Fabric. - Develop data models to support business intelligence and analytics. - Manage and optimize ETL processes for data extraction, transformation, and loading. - Collaborate with cross-functional teams to gather and define data requirements. - Ensure data quality and integrity in all data processes. - Implement best practices for data management, storage, and processing. - Conduct performance tuning for data storage and retrieval for enhanced efficiency. - Generate and maintain documentation for data architecture and data flow. - Participate in troubleshooting data-related issues and implement solutions. - Monitor and...

Salary: 15 to 25 LPA Exp: 4 to 7 years Location: Gurgaon/Pune/Bengalore Notice: Immediate to 30 days..!! Job Profile: Experienced Data Engineer with a strong foundation in designing, building, and maintaining scalable data pipelines and architectures. Skilled in transforming raw data into clean, structured formats for analytics and business intelligence. Proficient in modern data tools and technologies such as SQL, T-SQL, Python, Databricks, and cloud platforms (Azure). Adept at data wrangling, modeling, ETL/ELT development, and ensuring data quality, integrity, and security. Collaborative team player with a track record of enabling data-driven decision-making across business units. As a Dat...

Data Reporting & Dashboarding team responsible for designing, developing, and maintaining interactive dashboards and data reports to support business decision-making. The team will focus on data lookup, validation, visualization, and performance reporting across multiple business domains. We are seeking professionals at different levels (A3B1) with strong skills in Power BI or Tableau , solid understanding of data analysis and reporting logic , and hands-on experience working with large datasets. Key Responsibilities: Develop and maintain interactive dashboards and reports using Power BI or Tableau . Perform data validation, quality checks, and consistency analysis to ensure accurate reporti...

About Happiest Minds:** Happiest Minds is a leading digital transformation and technology services company, empowered by the mission of enhancing the happiness of our customers, employees, and society at large. We provide cutting-edge solutions leveraging advanced technologies to drive meaningful outcomes for our clients. Location: Manyata Tech Park JD: We seek a Senior Data Scientist with 5+ years of Machine Learning and database expertise to develop, deploy, and maintain advanced models in the security domain. Responsibilities include designing scalable data pipelines, performing exploratory data analysis, engineering features, and collaborating with security and IT teams to integrate thre...

Role Proficiency: This role requires proficiency in data pipeline development including coding and testing data pipelines for ingesting wrangling transforming and joining data from various sources. Must be skilled in ETL tools such as Informatica Glue Databricks and DataProc with coding expertise in Python PySpark and SQL. Works independently and has a deep understanding of data warehousing solutions including Snowflake BigQuery Lakehouse and Delta Lake. Capable of calculating costs and understanding performance issues related to data solutions. Outcomes: Act creatively to develop pipelines and applications by selecting appropriate technical options optimizing application development mainten...