749 Data Pipelines Jobs - Page 6

Role Overview: As a skilled developer, you will be responsible for developing and maintaining scalable data pipelines utilizing Python and PySpark. Your role involves collaborating closely with data engineers and data scientists to comprehend and meet data processing requirements efficiently. Your tasks will also include optimizing and resolving issues in existing PySpark applications to enhance performance. It is essential to write clean, efficient, and well-documented code adhering to industry best practices. Furthermore, you will actively participate in design and code reviews to ensure quality and consistency across projects. Additionally, you will be involved in developing and executing...

As a Cloud Platform Administrator in Bengaluru, you will be responsible for various key tasks related to user and access management, cluster and resource management, job management, monitoring and troubleshooting, platform maintenance, collaboration, and cost optimization. Your primary responsibilities will include: - Creating users, setting roles, and managing access controls for the cloud platform. - Provisioning clusters, monitoring resource usage, and optimizing performance for efficient operations. - Scheduling, managing, and optimizing jobs and workflows. Furthermore, you will play a vital role in: - Monitoring system performance, troubleshooting issues, managing updates, backups, inte...

As a skilled data engineering professional with over 5 years of experience, you will be responsible for designing and implementing end-to-end ETL solutions using Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Azure Databricks, and SSIS. Your strong proficiency in SQL, REST API integration, and automation through CI/CD pipelines using Azure DevOps and Git will be crucial for this role. You must have a solid understanding of maintaining and optimizing data pipelines, warehouses, and reporting within the Microsoft SQL stack. Key Responsibilities: - Design and implement comprehensive data solutions using Microsoft Azure Data Factory (ADF) and Azure Data Lake Storage (ADLS), Azure Data...

As a Senior Data Engineer, your role will involve developing scalable data pipelines using Microsoft Fabric. Your responsibilities will include: - Developing and managing data pipelines through Microsoft Fabric Data Factory and OneLake - Designing and creating ingestion and transformation pipelines for structured and unstructured data - Establishing frameworks for metadata tagging, version control, and batch tracking to ensure security, quality, and compliance of data pipelines - Contributing to CI/CD integration, observability, and documentation - Collaborating with data architects and analysts to align with business requirements effectively To qualify for this role, you should have: - At l...

Job Description: Job Title: Software Engineer (Python, AWS, and JavaScript) We seek out curious minds. We value attention to detail, and we care deeply about outcomes. We're looking above all for passionate people, eager to learn, willing to share, establishing innovative ways of working and influencing culture change. Are you ready to share this exciting challenge with us Qualifications & Experience: Education: Bachelor's or Master's degree in Computer or Information Science, Engineering, IT, or a related field. Experience: 6-8 years of experience in full-stack development. Required Skills & Qualifications: Strong experience in Python and PySpark (minimum 6 years). Good understanding of Jav...

Over 50,000 customers globally trust our end-to-end, cloud-driven networking solutions. They rely on our top-rated services and support to accelerate their digital transformation efforts and deliver unprecedented progress. With double-digit growth year over year, no provider is better positioned to deliver scalable outcomes than Extreme. Inclusion is one of our core values and in our DNA. We are committed to fostering an inclusive workplace that embraces our differences and creates an atmosphere where all our employees thrive because of their differences, not in spite of them. Become part of Something big with Extreme! As a global networking leader, learn why there's no better time to join t...

Seeking a highly motivated and experienced Data Scientist to join our team. This role involves a blend of hands-on technical expertise, strategic planning, and leadership, focusing on designing and implementing innovative AI/ML cloud-native solutions across various business units. You will be responsible for the full project lifecycle, from scoping requirements with stakeholders to delivering and maintaining robust applications in production. This role also involves mentoring team members, managing vendor relationships, and contributing to the ongoing development of our existing AI/ML cloud infrastructure and governance practices. The position will report into Vice President & Head - Innovat...

As a Data Engineer Intern at our company, you will have the opportunity to kickstart your career in Artificial Intelligence by gaining hands-on experience in building and managing data pipelines that drive AI-driven solutions. You will be located in Gota and expected to work from the office from 10:00 AM to 7:00 PM, Monday to Friday. **Role Overview:** - Collaborate with our AI team to design, build, and optimize data pipelines. - Work on real-world data challenges to ensure smooth data flow and reliability. - Assist in preparing and structuring datasets for machine learning models. - Gain exposure to modern tools, frameworks, and best practices in data engineering. **Key Responsibilities:**...

Role Overview: You are a forward-thinking Product Manager with expertise in AI-native product development, automation, agentic workflows, and self-improving systems. Your main focus will be on owning the vision for AI-powered, autonomous decision-making features across HVAC, energy optimization, and IoT platforms. You will work at the intersection of GenAI, IoT, and intelligent automation, driving end-to-end product ownership from ideation to AI-first experience design, deployment, and delivery. Your responsibilities will include focusing on customer benefits through customer-first product innovation, feature adoption, activation rates, and time to value. Key Responsibilities: - Define the p...

As an Artificial Intelligence Specialist at Gen AI, your role involves driving customer conversations to explore Gen AI opportunities. You will need to understand specific customer requirements and design acceptable Gen AI solution architectures tailored to their needs. Additionally, you will be responsible for creating customer proposals and responding to RFPs. Guiding solution engineers in developing Gen AI POCs and solutions for various industry verticals is also a key aspect of your role. Your proactive approach to problem-solving is vital as you drive topics to resolution with a focus on quality, speed, reliability, and efficiency. Staying updated on the latest technology advancements a...

Role Overview: As a Full Stack Developer at U.S. Pharmacopeial Convention (USP), you will be an integral part of the Digital & Innovation group, contributing to the creation of innovative digital products using cutting-edge cloud technologies. Your role will involve building scalable applications and platforms, ensuring adherence to governance principles and security policies, and actively participating in code reviews, architecture discussions, and agile development processes to maintain high-quality, maintainable, and scalable code. Additionally, you will provide technical guidance and mentorship to junior developers and effectively communicate technical designs and solutions to both techn...

Role Overview: You are a Senior Machine Learning Engineer responsible for designing, developing, and deploying Computer Vision solutions in production environments. Your role involves leading the design, development, and deployment of advanced computer vision systems that automate AI across various operational workflows. You will be accountable for the entire solution lifecycle, from problem scoping to deployment and monitoring, working alongside a team of ML engineers. Key Responsibilities: - Lead the design and deployment of computer vision models for applications such as object detection, tracking, human activity analysis, and visual quality control. - Build and optimize image and video d...

**Job Description:** As a Python Azure Data Factory (ADF) professional, your role involves working with technologies such as Azure Databricks, PySpark, Delta Lake, ETL/ELT, data pipelines, and data lakehouse architecture. **Key Responsibilities:** - **Technical Delivery:** - Develop robust ETL/ELT workflows using Informatica to ingest, transform, and load data into our data warehouse, Azure Synapse Analytics. - Conduct exploratory data analysis (EDA) to validate pipeline outputs and ensure data accuracy. - Ensure data quality by implementing proactive checks and monitoring mechanisms. - Write and optimize complex SQL queries to support analytical and reporting team requirements. - Collaborat...

As an MLOps Engineer, your role will involve designing, implementing, and maintaining scalable machine learning pipelines and infrastructure. You should have a strong understanding of ML lifecycle management, CI/CD for ML models, and cloud platforms to ensure seamless integration between data science and production environments. Key Responsibilities: - Build, deploy, and manage end-to-end ML pipelines from experimentation to production. - Develop and maintain CI/CD workflows for ML models. - Automate training, testing, monitoring, and retraining processes. - Manage model versioning, deployment, and scaling in production environments. - Ensure observability, logging, and monitoring for ML mod...

Role Overview: You should have a Bachelor's degree in computer science or data analytics and a minimum of 2 years of professional software development experience. Your expertise in Python for solving data analytics-related challenges is crucial, along with hands-on experience in data visualization tools like Matplotlib, Tableau, PowerBI, or similar. A solid understanding of data pipelines, analysis workflows, and process automation is necessary. Strong problem-solving skills and the ability to thrive in fast-paced environments are key qualities for this role. Key Responsibilities: - Design, develop, and maintain data analytics tooling to monitor, analyze, and enhance system performance and s...

As a qualified candidate for the role, you should possess the following attributes: Role Overview: You will be proficient in JavaScript and Python, showcasing strong skills in both front-end and back-end development. Your experience with React and Angular, combined with Redux for state management, will enable you to create efficient and dynamic web applications. Additionally, your expertise in Node.js will allow you to build scalable server-side applications, while your knowledge of Serverless architectures will help in the creation of optimized and cost-effective solutions. You should also be familiar with data pipelines and have experience in managing RDBMS for structured data storage and ...

As a Python Developer at Exela, you will be part of a team responsible for designing, developing, testing, and maintaining Python applications and services. Your role will involve working on various aspects such as REST APIs, data pipelines, automation scripts, LLM integration, LangGraph, LangChain, preprocessing, and reporting. Key Responsibilities: - Design, develop, test, and maintain Python applications and services - Work with REST APIs, data pipelines, and automation scripts - Work on LLM integration, LangGraph, LangChain, preprocessing, and reporting - Write clean, maintainable, and efficient code - Conduct unit testing and code reviews - Optimize performance and troubleshoot producti...

As a Solution Architect at Dbiz.AI, you will play a crucial role in designing and implementing scalable, secure, and efficient data architectures for enterprise solutions using AWS. Your expertise in microservices architecture and cloud technologies will be essential in leading the development of AWS-based data ecosystems. You will collaborate with cross-functional teams to align data architecture with business goals, drive the adoption of new tools and methodologies, and provide technical guidance to data engineers and developers. **Key Responsibilities:** - **Data Architecture Design:** - Design and implement scalable, secure, and efficient data architectures for enterprise solutions using...

About the Role: We are seeking a skilled Python Backend Engineer with 3+ years of professional experience to design, build, and optimize backend services powering our AI-driven applications. The ideal candidate will have a solid foundation in backend engineering and exposure to AI/ML workflows. Key Responsibilities: Design, develop, and maintain scalable backend services using Python (FastAPI/Flask/Django) Build and optimize APIs, microservices, and data pipelines Collaborate with AI/ML/UI engineers to integrate models into production Implement best practices in testing, CI/CD, observability, and security Optimize performance, reliability, and cost in cloud environments (AWS/Azure/GCP) Requi...

We are hiring a Senior Data Engineer for a 6-month remote contract position. The ideal candidate is highly skilled in building scalable data pipelines and working within the Azure cloud ecosystem, especially Databricks, ADF, and PySpark. You'll work closely with cross-functional teams to deliver enterprise-level data engineering solutions. #KeyResponsibilities Build scalable ETL pipelines and implement robust data solutions in Azure. Manage and orchestrate workflows using ADF, Databricks, ADLS Gen2, and Key Vaults. Design and maintain secure and efficient data lake architecture. Work with stakeholders to gather data requirements and translate them into technical specs. Implement CI/CD pipeli...

Job Opening: Software Tester (Manual + Automation) Contract Role Position: Software Tester (Manual + Automation) Engagement: Contract (69 Months) Location: Remote Experience Required: 4+ Years Key Responsibilities: Develop and execute test plans, test cases, and regression test suites for ReactJS front-end, Python FastAPI backend, and data pipelines. Perform API testing using Postman, Swagger, and validate backend components (PostgreSQL, MongoDB, Kafka). Conduct UI functional testing for internal dashboards and web platforms. Validate data workflows, including Azure Data Factory and Databricks jobs. Identify, log, and track bugs using JIRA or Azure Boards; coordinate with dev teams for fixes...

We are looking for an experienced Software Developer with 5+ years of expertise in Java or Python for data processing and automation. The ideal candidate should have strong proficiency in Java, Spring, Microservices, and REST API development. Experience with cloud platforms, specifically Google Cloud Platform (GCP), is required. The candidate must have hands-on experience in creating IDE plugins for PyCharm, VS Code, IntelliJ, and web consoles. Additionally, proficiency in Python and data pipeline development is mandatory. The role demands a deep understanding of software development principles, automation, and scalable system design.

Notice Period: Immediate About the role: We are hiring a Senior Snowflake Data Engineer with 10+ years of experience in cloud data warehousing and deep expertise on the Snowflake platform. The ideal candidate should have strong skills in SQL, ETL/ELT, data modeling, and performance tuning, along with a solid understanding of Snowflake architecture, security, and cost optimization. Roles & Responsibilities: Collaborate with data engineers, product owners, and QA teams to translate business needs into efficient Snowflake-based data models and pipelines. Design, build, and optimize data solutions leveraging Snowflake features such as virtual warehouses, data sharing, cloning, and time travel. D...

We are looking for a Senior Data Engineer who will design, build, and maintain scalable data pipelines and ingestion frameworks. The ideal candidate must have experience with DBT, orchestration tools like Airflow or Prefect, and cloud platforms such as AWS. Responsibilities include developing ELT pipelines, optimizing queries, implementing CI/CD, and integrating with AWS services. Strong SQL, Python, and data modeling skills are essential. The role also involves working with real-time and batch processing, ensuring high performance and data integrity.

We are seeking a skilled Big Data Developer with 3+ years of experience to develop, maintain, and optimize large-scale data pipelines using frameworks like Spark, PySpark, and Airflow. The role involves working with SQL, Impala, Hive, and PL/SQL for advanced data transformations and analytics, designing scalable data storage systems, and integrating structured and unstructured data using tools like Sqoop. The ideal candidate will collaborate with cross-functional teams to implement data warehousing strategies and leverage BI tools for insights. Proficiency in Python programming, workflow orchestration with Airflow, and Unix/Linux environments is essential. Location: Remote- Delhi / NCR,Banga...