1586 Data Pipeline Jobs - Page 43

About the Role: We are seeking a highly skilled and motivated Backend Developer with 2 to 5 years of experience to design and implement a high-performance, secure, and scalable server-side architecture for our trading terminal. In this role, you will develop systems capable of processing large volumes of real-time financial data, ensuring low latency and exceptional reliability for mission-critical applications. Your expertise will be central to empowering data-driven trading experiences for our users. Key Responsibilities: Service Architecture & Development: Design, develop, and maintain high-performance backend services, RESTful APIs, and microservices. Architect systems that efficiently p...

Role & responsibilities We are seeking a skilled Data Engineer to maintain robust data infrastructure and pipelines that support our operational analytics and business intelligence needs. Candidates will bridge the gap between data engineering and operations, ensuring reliable, scalable, and efficient data systems that enable data-driven decision making across the organization. Strong proficiency in Spark SQL, hands-on experience with realtime Kafka, Flink Databases: Strong knowledge of relational databases (Oracle, MySQL) and NoSQL systems Proficiency with Version Control Git, CI/CD practices and collaborative development workflow Strong operations management and stakeholder communication s...

Critical Skills to Possess: 5+ years of experience in data engineering or ETL development. 5+ years of hands-on experience with Informatica. Experience in production support , handling tickets, and monitoring ETL systems. Strong SQL skills with experience in querying large datasets. Familiarity with data warehousing concepts and design (e.g., star schema, snowflake schema). Experience with relational databases such as Oracle, SQL Server, or PostgreSQL. Knowledge of cloud platforms such as AWS, Azure, or GCP is a plus. Preferred Qualifications: BS degree in Computer Science or Engineering or equivalent experience Roles and Responsibilities Roles and Responsibilities: Design, develop, and main...

3+ yrs in BI operations Strong in BI monitoring and cross-functional collaboration. Monitor & Support BI systems Resolve incidents, Optimize workflows Ensure data reliability across tools Example - (Qlik, Power BI, Tableau, and SAP Analytics Cloud.) Required Candidate profile 3+ yrs in BI operations with hands-on in Power BI, Qlik & Tableau. Skilled in troubleshooting, data modeling, ETL, and incident resolution. Strong in BI monitoring and cross-functional collaboration.

YipitData is a leading market research and analytics firm specializing in the disruptive economy, having recently secured a significant investment from The Carlyle Group valued over $1B. Recognized for three consecutive years as one of Inc's Best Workplaces, we are a rapidly expanding technology company with offices across various locations globally, fostering a culture centered on mastery, ownership, and transparency. As a potential candidate, you will have the opportunity to collaborate with strategic engineering leaders and report directly to the Director of Data Engineering. This role involves contributing to the establishment of our Data Engineering team presence in India and working wi...

Bachelor s or master s degree in computer science, Data Science, Engineering, or a related field. EsyCommerce is seeking a highly experienced Data Engineer to join our growing team in either Mumbai or Pune. This role requires a strong foundation in data engineering principles, coupled with experience in application development and data science techniques. The ideal candidate will be responsible for designing, developing, and maintaining robust data pipelines and applications, as well as leveraging analytical skills to transform data into valuable insights. This position calls for a blend of technical expertise, problem-solving abilities, and effective communication skills to drive data-drive...

To Apply - Mandatory to submit Details via Google Form - https://forms.gle/cCa1WfCcidgiSTgh8 Position : Senior Data Engineer - Total 8+ years Required Relevant 6+ years in Databricks, AWS, Apache Spark & Informatica (Required Skills) As a Senior data Engineer in our team, youll build and nurture positive working relationships with teams and clients with the intention to exceed client expectations Seeking experienced data Engineer to design, implement, and maintain robust data pipelines and analytics solutions using databricks & AWS services. The ideal candidate will have a strong background in data services, big data technologies, and programming languages. Role & responsibilities Technical ...

IENERGY is a leading provider of EHS and ESG software platforms that are specifically crafted to equip organizations with cutting-edge digital systems for Environment, Health & Safety, Sustainability, and ESG reporting. With a strong focus on leveraging advanced technology and extensive domain expertise, our platform seamlessly integrates real-time data to facilitate intuitive workflows and robust system integration. Trusted by a diverse user base of over 15,000 clients, which includes Fortune 500 companies, IENERGY is dedicated to delivering tangible business impact through swift deployment and scalable innovation. Our suite of offerings comprises IENERGY AURORA for IoT-driven remote monito...

You will be working as a Monitoring Team Lead for a Data Pipeline L1 team, overseeing the daily operations to ensure the health and stability of data pipelines, and managing incident response. Your role will involve leading the team, monitoring performance, and escalating issues as needed. As a Team Leader, you will guide and mentor the L1 monitoring team to ensure proficiency in data pipeline monitoring, troubleshooting, and escalation procedures. You will manage team performance, distribute tasks effectively, and resolve conflicts. Acting as a point of contact for the team, you will represent them to stakeholders and advocate for their needs. Your responsibilities will also include develop...

Tech stack- Database: Mongodb: S3 Postgres Strong experience on Data pipelines; mapping React; Node; Python Aws; Lambda About the job Summary We are seeking a detail-oriented and proactive Data Analyst to lead our file and data operations, with a primary focus on managing data intake from our clients and ensuring data integrity throughout the pipeline. This role is vital to our operational success and will work cross-functionally to support data ingestion, transformation, validation, and secure delivery. The ideal candidate must have hands-on experience with healthcare datasets, especially medical claims data, and be proficient in managing ETL processes and data operations at scale. Responsi...

Hi Candidates, we have job openings in one of our MNC Company Interested candidates can apply here and share details to chandrakala.c@i-q.co Note: NP-0-15 days only serving Role & responsibilities We are looking for Data Managers Work Exp: Min 5 yrs. (mandatory) Location: Remote (India) JD: The data modeler designs, implements, and documents data architecture and data modeling solutions, which include the use of relational, dimensional, and NoSQL databases. These solutions support enterprise information management, business intelligence, machine learning, data science, and other business interests. The successful candidate will: - Be responsible for the development of the conceptual, logical...

We are looking for an experienced data engineer to join our team. You will use various methods to transform raw data into useful data systems. For example, youll create algorithms and conduct statistical analysis. Overall, youll strive for efficiency by aligning data systems with business goals. To succeed in this data engineering position, you should have strong analytical skills and the ability to combine data from different sources. Data engineer skills also include familiarity with several programming languages and knowledge of learning machine methods. Responsibilities Analyze and organize raw data Build data systems and pipelines Interpret trends and patterns Conduct complex data analy...

Role Summary: We are looking for a Data Engineer who will be responsible for designing and developing scalable data pipelines, managing data staging layers, and integrating multiple data sources through APIs and SQL-based systems. You'll work closely with analytics and development teams to ensure high data quality and availability. Key Responsibilities: Design, build, and maintain robust data pipelines and staging tables. Develop and optimize SQL queries for ETL processes and reporting. Integrate data from diverse APIs and external sources. Ensure data integrity, validation, and version control across systems. Collaborate with data analysts and software engineers to support analytics use cas...

You will be working as an IoT Solution Architect at IENERGY, a leading provider of EHS and ESG software solutions. Your primary responsibility will involve designing and implementing IoT solutions, providing consulting services, and ensuring seamless integration with existing systems. You will collaborate with cross-functional teams to develop software solutions, manage business processes, and optimize IoT architecture for improved performance and efficiency. To excel in this role, you should have proficiency in Solution Architecture and Integration, along with experience in Consulting, Business Process, and Software Development. Additionally, you should possess 2-3 years of experience in se...

Acuity Knowledge Partners (Acuity) is a leading provider of bespoke research, analytics and technology solutions to the financial services sector, including asset managers, corporate and investment banks, private equity and venture capital firms, hedge funds and consulting firms. Its global network of over 6,000 analysts and industry experts, combined with proprietary technology, supports more than 600 financial institutions and consulting companies to operate more efficiently and unlock their human capital, driving revenue higher and transforming operations. Acuity is headquartered in London and operates from 10 locations worldwide. The company fosters a diverse, equitable and inclusive wor...

Job Summary As a Software Engineer at NetApp India’s R&D division, you will be responsible for the design, development and validation of software for Big Data Engineering across both cloud and on-premises environments. You will be part of a highly skilled technical team named NetApp Active IQ. The Active IQ DataHub platform processes over 10 trillion data points per month that feeds a multi-Petabyte DataLake. The platform is built using Kafka, a serverless platform running on Kubernetes, Spark and various NoSQL databases. This platform enables the use of advanced AI and ML techniques to uncover opportunities to proactively protect and optimize NetApp storage, and then provides the insights a...

We are seeking a highly skilled and motivated Data Engineer to join our dynamic team. The ideal candidate will have a strong background in designing, building, and optimizing data pipelines and architectures to support our growing data-driven initiatives. Knowledge of machine learning techniques and frameworks is a significant advantage and will allow you to collaborate closely with our data science team. Key Responsibilities: - Design, implement, and maintain scalable data pipelines for collecting, processing, and analyzing large datasets. - Build and optimize data architectures to support business intelligence, analytics, and machine learning models. - Collaborate with data scientists, ana...

Job Title: GCP Data Engineer Location: Pune, India Experience: 4 to 7 Years Job Type: Full-Time Job Summary: We are looking for a highly skilled GCP Data Engineer with 4 to 7 years of experience to join our data engineering team in Pune . The ideal candidate should have strong experience working with Google Cloud Platform (GCP) , including Dataproc , Cloud Composer (Apache Airflow) , and must be proficient in Python , SQL , and Apache Spark . The role involves designing, building, and optimizing data pipelines and workflows to support enterprise-grade analytics and data science initiatives. Key Responsibilities: Design and implement scalable and efficient data pipelines on GCP , leveraging D...

Job Responsibilities: Technical Leadership: • Provide technical leadership and mentorship to a team of data engineers. • Design, architect, and implement highly scalable, resilient, and performant data pipelines, using GCP technologies is a plus (e.g., Dataproc, Cloud Composer, Pub/Sub, BigQuery). • Guide the team in adopting best practices for data engineering, including CI/CD, infrastructure-as-code, and automated testing. • Conduct code reviews, design reviews, and provide constructive feedback to team members. • Stay up-to-date with the latest technologies and trends in data engineering, Data Pipeline Development: • Develop and maintain robust and efficient data pipelines to ingest, proc...

Hi, Greetings from Peoplefy Infosolutions !!! We are hiring for one of our reputed MNC client based in Pune. We are looking for candidates with 10+ years of experience who is currently working as a Data Architect. Job Description: We are seeking a highly skilled and experienced Cloud Data Architect to design, implement, and manage scalable, secure, and efficient cloud-based data solutions. The ideal candidate will possess a strong combination of technical expertise, analytical skills, and the ability to collaborate effectively with cross-functional teams to translate business requirements into technical solutions. Key Responsibilities: Design and implement data architectures, including data ...

Salary: 20 to 35 LPA Exp: 3 to 7 years Location: Gurgaon/Pune/Bengalore Notice: Immediate to 30 days..!! Job Profile: Experienced Data Engineer with a strong foundation in designing, building, and maintaining scalable data pipelines and architectures. Skilled in transforming raw data into clean, structured formats for analytics and business intelligence. Proficient in modern data tools and technologies such as SQL, T-SQL, Python, Databricks, and cloud platforms (Azure). Adept at data wrangling, modeling, ETL/ELT development, and ensuring data quality, integrity, and security. Collaborative team player with a track record of enabling data-driven decision-making across business units. As a Dat...

You will need hands-on experience in converting SAS to Python, along with strong mathematics and statistics skills to excel in this role. Additionally, proficiency in AI-specific utilities like ChatGPT, Hugging Face Transformers, etc., is essential. Your ability to comprehend business requirements and derive use cases and solutions from structured/unstructured data will be crucial. Your responsibilities will include converting SAS to Python code, acquiring the necessary skills for building and deploying machine learning models for production, and engaging in feature engineering, exploratory data analysis, pipeline creation, model training, and hyperparameter tuning. You will also be expected...

You should have 5+ years of experience in core Java and the Spring Framework. Additionally, you must have at least 2 years of experience in Cloud technologies such as GCP, AWS, or Azure, with a preference for GCP. It is required to have experience in big data processing on a distributed system and in working with databases including RDBMS, NoSQL databases, and Cloud natives. You should also have expertise in handling various data formats like Flat file, JSON, Avro, XML, etc., including defining schemas and contracts. Furthermore, you should have experience in implementing data pipelines (ETL) using Dataflow (Apache Beam) and in working with Microservices and integration patterns of APIs with...

The successful candidate for the Full Stack Developer position at USP will demonstrate an understanding of the organization's mission and a commitment to excellence through inclusive and equitable behaviors. As a key member of the Digital & Innovation group, you will be responsible for developing innovative digital products using cutting-edge cloud technologies. Your role will involve building scalable applications and platforms, ensuring compliance with governance principles and security policies, participating in code reviews and agile development processes, and providing technical guidance to junior developers. Effective communication of technical designs and solutions to both technical a...

Role & responsibilities: Design and implement scalable data pipelines for feature extraction, transformation, and loading (ETL) using technologies such as Pyspark, Scala, and relevant big data frameworks. Govern and optimize data pipelines to ensure high reliability, efficiency, and data quality across on-premise and cloud environments. Collaborate closely with data scientists, ML engineers, and business stakeholders to understand data requirements and translate them into technical solutions. Implement best practices for data governance, metadata management, and compliance with regulatory requirements. Lead a team of data engineers, providing technical guidance, mentorship, and fostering a c...