1617 Data Pipelines Jobs - Page 39

Role Overview: You will be responsible for conducting thorough data analysis using tools like Excel and SQL to uncover trends, patterns, and anomalies. Additionally, you will apply statistical methods for in-depth interpretation of complex datasets and design data pipelines to support strategic decision-making. Generating business intelligence reports and actionable insights to guide data-driven decision-making will also be part of your role. Key Responsibilities: - Conduct thorough data analysis using Excel and SQL - Apply statistical methods for in-depth interpretation of complex datasets - Design and develop data pipelines and models for strategic decision-making - Generate business intel...

Job Description: You will be joining LeanQubit, a Certified Integrator of Ignition, dedicated to crafting custom Ignition-based solutions for manufacturing, energy, and utility industries. Based in Mumbai, LeanQubit invites potential partners to connect with them. Role Overview: As a Java Team Lead for a hybrid/remote role in Mumbai, you will lead a team of developers, oversee Java-based applications" design and implementation, and ensure best practices are followed. Your responsibilities will include managing microservices architecture, applying design patterns, and driving the software development lifecycle. Key Responsibilities: - Lead the design, development, and deployment of complex so...

Total Experience : 1+ years Mode of Hire: Permanent Required Skills Set (Mandatory): JavaScript/Node.js/Python, Web Crawling (Puppeteer/Playwright), Strong knowledge of web technologies Desired Skills (Good if you have): HTML, Distributed Systems, Data pipelines, Proxy Management, Anti-bot Techniques Job Responsibilities Design scalable web crawling solutions using modern technologies like Puppeteer and Playwright. Develop optimal and cost-effective methods for efficiently crawling diverse websites at scale. Implement robust proxy management and sophisticated anti-bot techniques. Create comprehensive debugging and monitoring systems for crawler performance. Actively contribute to technical d...

Joining: Immediate to 15 days Key Skills: Strong hands-on experience in PySpark Expertise in Big Data technologies Solid understanding of SQL, Data Pipelines, and ETL Interested candidates can share their resume with the following details to navaneetha@suzva.com Current CTC Expected CTC Location

Your IT Future, Delivered. Associate DevOps Engineer Open to candidates PAN India (Offices in Chennai, Mumbai, Hyderabad, Bengaluru & Indore). With a global team of 6000+ IT professionals, DHL IT Services connects people and keeps the global economy running by continuously innovating and creating sustainable digital solutions. We work beyond global borders and push boundaries across all dimensions of logistics. You can leave your mark shaping the technology backbone of the biggest logistics company of the world. Our offices in Cyberjaya, Prague, and Chennai have earned #GreatPlaceToWork certification, reflecting our commitment to exceptional employee experiences. Digitalization. Simply deliv...

We are looking out for Postgres Lead/ Manager Role. Experience - 10 Plus Work Location - Hyderabad/ Chennai Work Mode -05 days Office Notice Period - Maximum 30 days Responsibilities Strong knowledge of Postgres database systems, data modeling techniques, and SQL proficiency. Strong experience in integrations: Build and maintain integrations with internal and external data sources and APIs. Experienced in handling JDBC-based connections, API calls, and implementing microservices. Implement RESTful APIs and web services for data access and consumption. Design, implement, and optimize end-to-end data pipelines for ingesting, processing, and transforming large volumes of structured and unstruct...

The Job Design / Implementation of Data Pipelines for processing large volumes of data Ingesting batch and streaming data from various data sources. Writing complex SQL using any RDBMS (Oracle, PostgreSQL, SQL Server etc.) Data Modeling: Proficiency in creating both normalized and denormalized database schemas. Developing applications in Python. Developing ETL, OLAP based and Analytical Applications. Working experience in Azure / AWS services Working in Databricks, Snowflake or other cloud data platforms. Working in Agile / Scrum methodologies Good knowledge of cloud security concepts and implementation of different types of authentication methods working on Azure DevOps; Create and manage G...

As a DataOps Engineer, your primary responsibility will be designing and maintaining scalable ML model deployment infrastructure using Kubernetes and Docker. You will need to implement CI/CD pipelines for ML workflows, ensure adherence to security best practices, and set up monitoring tools for tracking system health, model performance, and data pipeline issues. Collaboration with cross-functional teams to optimize the end-to-end lifecycle of data products and identify performance bottlenecks and data reliability issues in the ML infrastructure will also be part of your role. Key Responsibilities: - Design and maintain scalable ML model deployment infrastructure using Kubernetes and Docker -...

As a VP - Reporting & Analytics at Barclays, you will spearhead the evolution of the Risk function by leading the Reporting Production Team for Operational Risk and Risk Oversight. You will provide high-quality MI and reporting to support colleagues across Operational Risk & Risk Oversight, enable senior management in risk management decision making, and ensure compliance with external and regulatory reporting commitments. You will work closely with the Reporting Change Team and receive local and functional leadership support from respective leads in India and globally. **Key Responsibilities:** - Provide leadership to the production reporting team - Set day-to-day direction for team members...

About us: At Articul8 AI, we relentlessly pursue excellence and create exceptional AI products that exceed customer expectations. We are a team of dedicated individuals who take pride in our work and strive for greatness in every aspect of our business. Job Description: Articul8 AI is seeking an exceptional Senior/Staff AI Researcher to join us in shaping the future of Generative Artificial Intelligence (GenAI). As a member of our Applied Research team, you will be responsible for conducting cutting-edge research to advance the capabilities of our AI systems. Your role will involve designing, implementing, and evaluating novel approaches to improve our GenAI models, working at the intersecti...

As a Technical Architect at Quadance Technologies Private Limited, your role will involve leading the design and development of scalable, enterprise-level applications and AI/ML solutions. You will be responsible for collaborating with stakeholders to translate requirements into maintainable architectures, implementing end-to-end solutions, providing technical leadership to development teams, and ensuring adherence to architectural standards. Additionally, you will participate in technical discussions with client teams, evaluate emerging technologies, and drive innovation in architecture and solutioning. Your key responsibilities will include: - Lead the architectural design and development ...

Role Overview: You will be working on end-to-end development of features for a major Technology company. The project consists of different teams focusing on various aspects of the search features development process. Specific roles and responsibilities for this profile include: Key Responsibilities: - Extracting data from external websites using Python and other internal applications and ingesting it into the Clients databases - Data Modelling and Schema Creation / Maintenance - Creating and maintaining data pipelines using internal applications, utilizing 5-10% Python - Performing data analysis, Data visualisation, creating SQL reports and dashboards using internal applications - Creating c...

Role Overview: As a Data Engineer at our company, you will play a crucial role in simplifying our organization by developing innovative data-driven solutions through data pipelines, modeling, and ETL design. Your main goal will be to ensure commercial success while prioritizing customer safety and data security. You will be responsible for driving customer value by solving complex business problems and implementing the most suitable tools to build data solutions. Additionally, you will engage with the data engineering community to deliver opportunities and build a scalable data architecture. Key Responsibilities: - Build advanced automation for data engineering pipelines by eliminating manua...

Key Responsibilities Design & implement data models, warehouses, and databases using Microsoft Fabric, Azure Synapse Analytics, and Azure Data Lake. Develop ETL pipelines with SQL Server Integration Services (SSIS), Azure Synapse Pipelines, and Azure Data Factory. Leverage Fabric Lakehouse for Power BI reporting, real-time analytics, and automation. Optimize data integration, governance, security, and performance. Collaborate with cross-functional teams to develop scalable data solutions. Implement Medallion Architecture for efficient data processing. Work in an Agile environment, using DevOps principles for automation & CI/CD. Required Skills Microsoft Fabric & Azure Lakehouse, OneLake, Dat...

As the Tech Lead - Quality Assurance in the Digital Business division at Sony Pictures Networks, your primary responsibility will be to establish and lead the quality engineering and automation framework for the SonyLIV platform's analytics product. This role necessitates a comprehensive understanding of data analytics testing, data pipelines, ETL processes, and data warehousing. **Key Responsibilities:** - Design and implement automation frameworks for front-end and back-end systems using industry-standard tools like Selenium, Appium, Pytest, Android Studio, Xcode, Eclipse, Jenkins, Jira, etc. - Establish quality processes and metrics for applications across platforms - Develop automation d...

Role Overview: At PwC, as a member of the data and analytics engineering team, your focus will be on leveraging advanced technologies and techniques to design and develop robust data solutions for clients. You will play a crucial role in transforming raw data into actionable insights, enabling informed decision-making and driving business growth. Your primary responsibility will be in data engineering, where you will concentrate on designing and building data infrastructure and systems to facilitate efficient data processing and analysis. Your role will involve developing and implementing data pipelines, data integration, and data transformation solutions. Key Responsibilities: - Oversee the...

Role Overview: As a Technical Lead specializing in System Integration, you will play a crucial role in integrating Benchling ELN into Pharma systems. Your expertise in ELN workflows, data interoperability, and system integration best practices will be instrumental in designing and implementing efficient integration solutions. Your responsibilities will include developing API-based integrations, middleware, and data pipelines to facilitate seamless data exchange between Benchling ELN and other lab systems such as LIMS, CDS, and SDMS. Key Responsibilities: - Develop API-based integrations, middleware, and data pipelines for seamless data exchange - Ensure compliance with regulatory standards (...

Role Overview: You have extensive experience in analytics and large-scale data processing across diverse data platforms and tools. Your responsibilities will include managing data storage and transformation across AWS S3, DynamoDB, Postgres, and Delta Tables with efficient schema design and partitioning. You will develop scalable analytics solutions using Athena and automate workflows with proper monitoring and error handling. Ensuring data quality, access control, and compliance through robust validation, logging, and governance practices will be a crucial part of your role. Additionally, you will design and maintain data pipelines using Python, Spark, Delta Lake framework, AWS Step functio...

Data Architect / Data Engineer Location: Chennai Experience: 46 years Your Role As a Data Architect / Data Engineer, you will be responsible for creating and maintaining digital twin models and integrating real-time data streams to support simulation and analytics in maritime systems. You will work with advanced tools and platforms to build scalable, high-performance data architectures that power intelligent digital solutions. In this role, you will: Design and maintain digital twin models for maritime systems Integrate real-time data from various sources into simulation environments Work with AWS TwinMaker and other cloud-native services for digital twin development Collaborate with simulat...

Role Overview: As an Applications Development Senior Manager, you will be responsible for managing a team or department to establish and implement new or revised application systems and programs in coordination with the Technology team. Your main objective will be to drive applications systems analysis and programming activities. Key Responsibilities: - Manage one or more Applications Development teams to accomplish established goals and conduct personnel duties for the team, such as performance evaluations, hiring, and disciplinary actions - Utilize in-depth knowledge and skills across multiple Applications Development areas to provide technical oversight across systems and applications - R...

Role Overview: As a Data Engineer at our company, you will be responsible for designing, developing, and optimizing data pipelines and architectures to ensure efficient data processing. Your role will involve working with complex datasets, integrating various data sources, and maintaining high data quality and reliability standards. Key Responsibilities: - Develop and maintain efficient ETL (Extract, Transform, Load) pipelines to handle large datasets from different sources. - Build and manage data storage solutions using both relational and non-relational databases, focusing on high availability, performance, and scalability. - Collaborate with APIs, third-party data sources, and internal d...

As a Deep Learning Engineer at CureBay, you will be an integral part of the team working on AI-driven solutions that leverage multi-modal deep learning techniques for tasks such as image, video, object detection, and audio classification. Your role will involve designing and implementing deep learning models, integrating attention mechanisms, and optimizing models for speed, accuracy, and robustness. Collaboration with cross-functional teams to deploy models into production will also be a key aspect of your responsibilities. **Key Responsibilities:** - Design and implement deep learning models for various tasks including image, video, object detection, and audio classification. - Integrate a...

Key Responsibilities Lead the design, development, and implementation of enterprise-scale AI/ML solutions optimized for real-world applications. Collaborate with cross-functional teams to identify business requirements and translate them into robust AI/ML strategies. Develop and optimize generative AI solutions using tools and frameworks such as LangChain, LangSmith, LlamaIndex, MCP, Semantic Kernel, Autogen, and Agents. Utilize Python along with frameworks such as PyTorch and TensorFlow to build sophisticated machine learning models and algorithms. Conduct analysis of data and identify trends, providing actionable insights to stakeholders. Mentor and guide junior data scientists, promoting ...

We are looking for an Analytics Engineering Manager to grow and lead a small team of 1-3 Analytics Engineers, co-own our emerging analytics engineering practice, and play a critical role in further developing our rapidly growing Data Analytics organization. Partnering with people leaders and ICs within the analytics team and around the business, this role will facilitate the production of datasets used to surface key insights and drive business decision making. This role requires someone with strong analytical and technical skills, and who is comfortable working across multiple functional areas in a fast-paced, challenging environment. We are looking to speak to candidates who are based in G...

Role Overview: You will be responsible for designing, developing, and deploying solutions on the Salesforce Data Cloud platform. Your role will involve collaborating with stakeholders to gather requirements and translating them into technical specifications. Additionally, you will develop and optimize data models to support business processes and reporting needs, ensuring data governance and security best practices for data integrity and compliance. As part of your responsibilities, you will be required to perform troubleshooting, debugging, and performance tuning of Salesforce Data Cloud solutions. It is essential to stay updated with Salesforce Data Cloud updates, best practices, and indus...