491 Data Pipeline Jobs - Page 8

Job Description for the position of Solution Manager At Nielsen, division Media Planning, we develop creative analytical solutions to support companies in the optimization of their marketing and communications budgets. Analytical and econometric approaches form the backbone of these solutions, answering questions like "What media mix works best to communicate my brand message?, What budget should I invest in media and marketing?". Our software tools and other decision-support solutions combine market research, data, modeling results and technical business intelligence. ABOUT THE JOB: In this position, you contribute to the successful implementation of our software solutions for advertisers, media owners and media agencies. Our solutions help these companies in making decisions around their marketing communication budgets. Based upon research data combined with analytical and econometric approaches our solutions answer questions like "What budget is needed to generate an increase of 10% in brand preference? What media mix contributes the most to the revenue?" You will be working in (small) teams on the design, execution, implementation and operational management of projects. This involves analyzing data, setting up surveys, designing a good way to integrate data (survey data, modeling and industry knowledge) into our software tools and maintaining contact with clients both internal and external. In your role, you will act as a liaison between various parts of our business: analytics, data processing and software development, and have a chance to be involved in the entire project from client request and solution design to tool delivery and training. YOUR PROFILE Bachelors of Technology in Computer Science (preferred) with 5+ years of experience. Experience working in geographically distributed teams. Affinity with data, software and quantitative techniques. Ability to reverse engineer & trouble shoot issues end to end. Experience in implementing and refactoring ETL pipelines (e.g. Apache Airflow). Analytical thinking combined with good social and communication skills. Experience using common scripting languages (e.g., Python). Experience working with Amazon Web Services (e.g. Athena). Experience with Jira, Git, Confluence,Google Suite. Well-organized and able to handle and prioritize multiple projects/tasks. Able to communicate effectively both orally and in writing. Proactive and self learner Fluent in English. WHAT WE OFFER Competitive salary and benefits. Personal and professional development opportunities. Exciting projects and clients. Continuous Learning Environment An open, respectful and multicultural atmosphere.

Job Description Job Duties and Responsibilities: We are looking for a self-starter to join our Data Engineering team. You will work in a fast-paced environment where you will get an opportunity to build and contribute to the full lifecycle development and maintenance of the data engineering platform. With the Data Engineering team you will get an opportunity to - Design and implement data engineering solutions that is scalable, reliable and secure on the Cloud environment Understand and translate business needs into data engineering solutions Build large scale data pipelines that can handle big data sets using distributed data processing techniques that supports the efforts of the data science and data application teams Partner with cross-functional stakeholder including Product managers, Architects, Data Quality engineers, Application and Quantitative Science end users to deliver engineering solutions Contribute to defining data governance across the data platform Basic Requirements: A minimum of a BS degree in computer science, software engineering, or related scientific discipline is desired 3+ years of work experience in building scalable and robust data engineering solutions Strong understanding of Object Oriented programming and proficiency with programming in Python (TDD) and Pyspark to build scalable algorithms 3+ years of experience in distributed computing and big data processing using the Apache Spark framework including Spark optimization techniques 2+ years of experience with Databricks, Delta tables, unity catalog, Delta Sharing, Delta live tables(DLT) and incremental data processing Experience with Delta lake, Unity Catalog Advanced SQL coding and query optimization experience including the ability to write analytical and nested queries 3+ years of experience in building scalable ETL/ ELT Data Pipelines on Databricks and AWS (EMR) 2+ Experience of orchestrating data pipelines using Apache Airflow/ MWAA Understanding and experience of AWS Services that include ADX, EC2, S3 3+ years of experience with data modeling techniques for structured/ unstructured datasets Experience with relational/columnar databases - Redshift, RDS and interactive querying services - Athena/ Redshift Spectrum Passion towards healthcare and improving patient outcomes Demonstrate analytical thinking with strong problem solving skills Stay on top of emerging technologies and posses willingness to learn.

Role & responsibilities Proficient: Languages/Framework: Fast API, Azure UI Search API (React) tabases and ETL: Cosmos DB (API for MongoDB), Data Factory Data Bricks Proficiency in Python and R Cloud: Azure Cloud Basics (Azure DevOps) Gitlab: Gitlab Pipeline o Ansible and REX: Rex Deployment Data Science: Prompt Engineering + Modern Testing Data mining and cleaning ML (Supervised/unsupervised learning) NLP techniques, knowledge of Deep Learning techniques include RNN, transformers End-to-end AI solution delivery AI integration and deployment AI frameworks (PyTorch) MLOps frameworks Model deployment processes Data pipeline monitoring Expert: (in addition to proficient skills) Languages/Framework: Azure Open AI Data Science: Open AI GPT Family of models 4o/4/3, Embeddings + Vector Search Databases and ETL: Azure Storage Account Expertise in machine learning algorithms (supervised, unsupervised, reinforcement learning) Proficiency in deep learning frameworks (TensorFlow, PyTorch) Strong mathematical foundation (linear algebra, calculus, probability, statistics) Research methodology and experimental design Proficiency in data analysis tools (Pandas, NumPy, SQL) Strong statistical and probabilistic modelling skills Data visualization skills (Matplotlib, Seaborn, Tableau) Knowledge of big data technologies (Spark, Hive) Experience with AI-driven analytics and decision-making systems Note: ***Notice Period should not be more than 10-15 days. ***

Cloud & Data Engineering Specialist Remote Work Contract Duration: 6 months Exp Level: 3 - 4 years (Must be able to work according to JD) Work Timings: (2:30 pm to 11:30 pm IST) We are seeking two highly skilled Cloud & Data Engineering Specialists to join our dynamic team. These roles will focus on designing, building, and optimizing scalable cloud-based solutions, data pipelines, and analytics platforms. The ideal candidates will have strong expertise in cloud platforms, data engineering, and modern technologies, with a focus on delivering robust, secure, and efficient data solutions. Position 1: Cloud & Data Engineering Specialist (Resource 1) Key Responsibilities: Design, develop, and maintain cloud-based solutions on Azure or AWS. Implement and manage real-time data streaming and messaging systems using Kafka. Develop scalable applications and services using Java and Python. Deploy, manage, and monitor containerized applications using Kubernetes. Build and optimize big data processing pipelines using Databricks. Manage and maintain databases, including SQL Server and Snowflake, and write complex SQL scripts. Work with Unix/Linux commands to manage and monitor system operations. Collaborate with cross-functional teams to ensure seamless integration of cloud-based solutions. Key Skills: Expertise in Azure or AWS cloud platforms. Proficiency in Kafka, Java, Python, and Kubernetes. Hands-on experience with Databricks for big data processing. Strong database management skills with SQL Server, Snowflake, and advanced SQL scripting. Solid understanding of Unix/Linux commands. Position 2: Cloud & Data Engineering Specialist (Resource 2) Key Responsibilities: Design and implement cloud solutions across Azure, AWS, and GCP platforms. Develop and optimize data pipelines using PySpark, Python, and SQL. Build and manage ETL workflows using Azure Data Factory (ADF). Work with big data technologies such as Apache Spark and Databricks to process large datasets. Design and deliver dashboards and reports using Tableau and Power BI. Implement DevOps practices, including version control with Git, CI/CD pipelines, and containerization using Docker. Collaborate with stakeholders to gather requirements and deliver scalable data solutions. Key Skills: Proficiency in Azure, AWS, and GCP cloud platforms. Strong programming skills in Python, SQL, and PySpark. Experience with Snowflake and SQL Server databases. Expertise in ETL tools like Azure Data Factory (ADF). Hands-on experience with Apache Spark and Databricks for big data processing. Proficiency in reporting tools such as Tableau and Power BI. Knowledge of DevOps practices, including Git, CI/CD pipelines, and Docker. General Requirements for Both Roles: Bachelors degree in Computer Science, Engineering, or a related field (or equivalent experience). 3+ years of experience in cloud and data engineering roles. Strong problem-solving and analytical skills. Excellent communication and collaboration abilities. Proven ability to work in a fast-paced, agile environment. Send resume to kalaivanan.balasubramaniam@covalensedigital.com Thanks kalai 8015302990

Job Summary : We are seeking an experienced Data Engineer with strong expertise in Databricks, Python, PySpark, and Power BI, along with a solid background in data integration and the modern Azure ecosystem. The ideal candidate will play a critical role in designing, developing, and implementing scalable data engineering solutions and pipelines. Key Responsibilities : - Design, develop, and implement robust data solutions using Azure Data Factory, Databricks, and related data engineering tools. - Build and maintain scalable ETL/ELT pipelines with a focus on performance and reliability. - Write efficient and reusable code using Python and PySpark. - Perform data cleansing, transformation, and migration across various platforms. - Work hands-on with Azure Data Factory (ADF) for at least 1.5 to 2 years. - Develop and optimize SQL queries, stored procedures, and manage large data sets using SQL Server, T-SQL, PL/SQL, etc. - Collaborate with cross-functional teams to understand business requirements and provide data-driven solutions. - Engage directly with clients and business stakeholders to gather requirements, suggest optimal solutions, and ensure successful delivery. - Work with Power BI for basic reporting and data visualization tasks. - Apply strong knowledge of data warehousing concepts, modern data platforms, and cloud-based analytics. - Adhere to coding standards and best practices, including thorough documentation and testing (unit, integration, performance). - Support the operations, maintenance, and enhancement of existing data pipelines and architecture. - Estimate tasks and plan release cycles effectively. Required Technical Skills : - Languages & Frameworks : Python, PySpark - Cloud & Tools : Azure Data Factory, Databricks, Azure ecosystem - Databases : SQL Server, T-SQL, PL/SQL - Reporting & BI Tools : Power BI (PBI) - Data Concepts : Data Warehousing, ETL/ELT, Data Cleansing, Data Migration - Other : Version control, Agile methodologies, good problem-solving skills Preferred Qualifications : - Experience with coding in Pysense within Databricks (added advantage) - Solid understanding of cloud data architecture and analytics processes - Ability to independently initiate and lead conversations with business stakeholders

Job Title: Senior Data Engineer Location: Remote Job Type: Fulltime YoE: 7 to 10 years relevant experience Shift: 6.30pm to 2.30am IST Job Purpose: The Senior Data Engineer designs, builds, and maintains scalable data pipelines and architectures to support the Denials AI workflow under the guidance of the Team Lead, Data Management. This role ensures data is reliable, compliant with HIPAA, and optimized. Duties & Responsibilities: Collaborate with the Team Lead and crossfunctional teams to gather and refine data requirements for Denials AI solutions. Design, implement, and optimize ETL/ELT pipelines using Python, Dagster, DBT, and AWS data services (Athena, Glue, SQS). Develop and maintain data models in PostgreSQL; write efficient SQL for querying and performance tuning. Monitor pipeline health and performance; troubleshoot data incidents and implement preventive measures. Enforce data quality and governance standards, including HIPAA compliance for PHI handling. Conduct code reviews, share best practices, and mentor junior data engineers. Automate deployment and monitoring tasks using infrastructure-as-code and AWS CloudWatch metrics and alarms. Document data workflows, schemas, and operational runbooks to support team knowledge transfer. Qualifications: Bachelors or Masters degree in Computer Science, Data Engineering, or related field. 5+ years of handson experience building and operating productiongrade data pipelines. Solid experience with workflow orchestration tools (Dagster) and transformation frameworks (DBT) or other similar tools such (Microsoft SSIS, AWS Glue, Air Flow). Strong SQL skills on PostgreSQL for data modeling and query optimization or any other similar technologies (Microsoft SQL Server, Oracle, AWS RDS). Working knowledge with AWS data services: Athena, Glue, SQS, SNS, IAM, and CloudWatch. Basic proficiency in Python and Python data frameworks (Pandas, PySpark). Experience with version control (GitHub) and CI/CD for data projects. Familiarity with healthcare data standards and HIPAA compliance. Excellent problemsolving skills, attention to detail, and ability to work independently. Strong communication skills, with experience mentoring or leading small technical efforts.

About the Team The Data Platform Tech Team at Navi is instrumental in enabling data-driven decision-making across the organization. We build and manage the core infrastructure and tools required to collect, store, process, and analyze data at scale. Our platforms support self-serve capabilities for both batch and real-time data processing. We work closely with Analytics, Data Science, and Product teams to power a wide range of data use cases across Navi. About the Role As an SDE-3 on the Data Platform team at Navi, you'll design and build large-scale systems powering web events, analytics, and real-time data pipelines. You'll lead backend development, contribute to platform architecture, and solve complex data problems. This is a high-impact IC role with strong cross-functional collaboration and mentorship opportunities. What We Expect From You Design, develop, and maintain backend services, data pipelines, and batch/realtime datasets related to web events and analytics. Strong proficiency in at least one of the following languages: Java, Python, Scala. Expertise in object-oriented design, design patterns, and data structures. Lead the development of new foundational capabilities in support of enabling our users to interact with, analyze, and derive insights from their data. Solve complex and challenging problems in the intersection between low latency, high correctness, and full determinism. Participate in code reviews, provide mentorship to junior team members, and enforce coding standards. Investigate, diagnose, and resolve software defects and issues, ensuring a high level of product quality. Contribute to the overall architecture and design of data platform frameworks. Strong interpersonal skills, showcasing effective stakeholder management with product and design teams. A minimum of 5 years of software development experience. Must Haves Familiarity with modern data lakehouse architectures and related technologies (e.g. Spark, Flink, Kafka, Trino). Prior experience in on-prem data-platform is preferred. Demonstrated ability to quickly adapt to new and complex development environments, along with strong deep-dive analytical skills. Previous success in mentoring and guiding junior engineers. Inside Navi We are shaping the future of financial services for a billion Indians through products that are simple, accessible, and affordable. From Personal & Home Loans to UPI, Insurance, Mutual Funds, and Gold we’re building tech-first solutions that work at scale, with a strong customer-first approach. Founded by Sachin Bansal & Ankit Agarwal in 2018, we are one of India’s fastest-growing financial services organisations. But we’re just getting started! Our Culture The Navi DNA Ambition. Perseverance. Self-awareness. Ownership. Integrity. We’re looking for people who dream big when it comes to innovation. At Navi, you’ll be empowered with the right mechanisms to work in a dynamic team that builds and improves innovative solutions. If you’re driven to deliver real value to customers, no matter the challenge, this is the place for you. We chase excellence by uplifting each other—and that starts with every one of us. Why You'll Thrive at Navi At Navi, it’s about how you think, build, and grow. You’ll thrive here if: You’re impact-driven : You take ownership, build boldly, and care about making a real difference. You strive for excellence : Good isn’t good enough. You bring focus, precision, and a passion for quality. You embrace change : You adapt quickly, move fast, and always put the customer first.

We are seeking a skilled Data Engineer with expertise in MuleSoft to join our dynamic team In this role, you will be responsible for designing, developing, and maintaining robust data integration solutions that leverage MuleSoft's powerful capabilities You will collaborate closely with cross-functional teams to gather requirements and translate them into scalable data architectures Our ideal candidate is not only proficient in data engineering but also has a strong understanding of API-led connectivity and microservices architecture You will work on various projects that involve data extraction, transformation, and loading (ETL) processes, as well as ensuring the integrity and accessibility of data across different systems Your analytical mindset and problem-solving skills will be crucial in optimizing data flows and enhancing performance Additionally, you will be involved in the automation of data processes, implementing best practices for data management, and ensuring compliance with data governance policies By joining our team, you will have the opportunity to work with a variety of technologies, contribute to innovative projects, and grow your skills in a collaborative environment Responsibilities: Design and implement ETL processes using MuleSoft to integrate data from various sources. Collaborate with stakeholders to gather and understand data integration requirements. Develop and maintain data pipelines and workflows to ensure efficient data transfer and processing. Optimize data models for performance and scalability across different applications and environments. Monitor and troubleshoot data integration processes, addressing any issues that arise. Ensure data quality and integrity by implementing validation and cleansing procedures. Document data flows, processes, and integration designs to maintain comprehensive records. Requirements: Bachelor's degree in Computer Science, Engineering, or a related field. Proven experience as a Data Engineer with a strong focus on MuleSoft technologies. Hands-on experience with API development and integration using MuleSoft Anypoint Platform. Strong understanding of data modeling concepts and database management systems. Proficiency in programming languages such as Java, Python, or SQL. Experience with cloud services such as AWS, Azure, or Google Cloud Platform. Excellent problem-solving skills and attention to detail, with the ability to work collaboratively.

-Design, develop & maintain data pipelines using GCP services: Dataflow, BigQuery, and Pub/Sub -Provisioning infrastructure on GCP using IaC with Terraform -Implement & manage data warehouse solutions -Monitor and resolve issues in data workflows Required Candidate profile -Expertise in GCP, Apache Beam, Dataflow, & BigQuery -Pro in Python, SQL, PySpark -Worked with Cloud Composer for orchestration -Solid understanding of DWH, ETL pipelines, and real-time data streaming

ETL / Data Analysis + AWS / Azure Immediate Joiner 5+ years Work From Home Contract to Hire Azure/AWS Spark Terraform Python SQL ETL / Data Pipelines Large Data Handling Data Validation/Transformation Annual bonus Provident fund Health insurance

Who We Are At Kyndryl, we design, build, manage and modernize the mission-critical technology systems that the world depends on every day. So why work at Kyndryl? We are always moving forward – always pushing ourselves to go further in our efforts to build a more equitable, inclusive world for our employees, our customers and our communities. The Role Are you ready to dive headfirst into the captivating world of data engineering at Kyndryl? As a Data Engineer, you'll be the visionary behind our data platforms, crafting them into powerful tools for decision-makers. Your role? Ensuring a treasure trove of pristine, harmonized data is at everyone's fingertips. As a Data Engineer at Kyndryl, you'll be at the forefront of the data revolution, crafting and shaping data platforms that power our organization's success. This role is not just about code and databases; it's about transforming raw data into actionable insights that drive strategic decisions and innovation. An ELK(Elastic, Logstash & Kibana) Data Engineer is responsible for developing, implementing, and maintaining the ELK stack-based solutions within an organization. The engineer plays a crucial role in developing efficient and effective log processing, indexing, and visualization for monitoring, troubleshooting, and analysis purposes. In this role, you'll be engineering the backbone of our data infrastructure, ensuring the availability of pristine, refined data sets. With a well-defined methodology, critical thinking, and a rich blend of domain expertise, consulting finesse, and software engineering prowess, you'll be the mastermind of data transformation. Your journey begins by understanding project objectives and requirements from a business perspective, converting this knowledge into a data puzzle. You'll be delving into the depths of information to uncover quality issues and initial insights, setting the stage for data excellence. But it doesn't stop there. You'll be the architect of data pipelines, using your expertise to cleanse, normalize, and transform raw data into the final dataset—a true data alchemist. Armed with a keen eye for detail, you'll scrutinize data solutions, ensuring they align with business and technical requirements. Your work isn't just a means to an end; it's the foundation upon which data-driven decisions are made – and your lifecycle management expertise will ensure our data remains fresh and impactful. So, if you're a technical enthusiast with a passion for data, we invite you to join us in the exhilarating world of data engineering at Kyndryl. Let's transform data into a compelling story of innovation and growth. Your Future at Kyndryl Every position at Kyndryl offers a way forward to grow your career. We have opportunities that you won’t find anywhere else, including hands-on experience, learning opportunities, and the chance to certify in all four major platforms. Whether you want to broaden your knowledge base or narrow your scope and specialize in a specific sector, you can find your opportunity here. Who You Are Who You Are You’re good at what you do and possess the required experience to prove it. However, equally as important – you have a growth mindset; keen to drive your own personal and professional development. You are customer-focused – someone who prioritizes customer success in their work. And finally, you’re open and borderless – naturally inclusive in how you work with others. Required Skills and Experience BS or MS degree in Computer Science or a related technical field 10+ years overall IT Industry Experience. 5+ years of Python or Java development experience 5+ years of SQL experience (No-SQL experience is a plus) 4+ years of experience with schema design and dimensional data modelling 3+ years of experience with Elastic, Logstash and Kibana Ability in managing and communicating data warehouse plans to internal clients. Experience designing, building, and maintaining data processing systems. Experience working with Machine Learning model is a plus. Knowledge of cloud platforms (e.g., AWS, Azure, GCP) and containerization technologies (e.g., Docker, Kubernetes) is a plus. Elastic Certification is preferrable. Preferred Skills and Experience • Experience working with Machine Learning model is a plus. • Knowledge of cloud platforms (e.g., AWS, Azure, GCP) and containerization technologies (e.g., Docker, Kubernetes) is a plus. • Elastic Certification is preferrable. Being You Diversity is a whole lot more than what we look like or where we come from, it’s how we think and who we are. We welcome people of all cultures, backgrounds, and experiences. But we’re not doing it single-handily: Our Kyndryl Inclusion Networks are only one of many ways we create a workplace where all Kyndryls can find and provide support and advice. This dedication to welcoming everyone into our company means that Kyndryl gives you – and everyone next to you – the ability to bring your whole self to work, individually and collectively, and support the activation of our equitable culture. That’s the Kyndryl Way. What You Can Expect With state-of-the-art resources and Fortune 100 clients, every day is an opportunity to innovate, build new capabilities, new relationships, new processes, and new value. Kyndryl cares about your well-being and prides itself on offering benefits that give you choice, reflect the diversity of our employees and support you and your family through the moments that matter – wherever you are in your life journey. Our employee learning programs give you access to the best learning in the industry to receive certifications, including Microsoft, Google, Amazon, Skillsoft, and many more. Through our company-wide volunteering and giving platform, you can donate, start fundraisers, volunteer, and search over 2 million non-profit organizations. At Kyndryl, we invest heavily in you, we want you to succeed so that together, we will all succeed. Get Referred! If you know someone that works at Kyndryl, when asked ‘How Did You Hear About Us’ during the application process, select ‘Employee Referral’ and enter your contact's Kyndryl email address.

Lead Data Engineer Data Management Job description Company Overview Accordion works at the intersection of sponsors and management teams throughout every stage of the investment lifecycle, providing hands-on, execution-focused support to elevate data and analytics capabilities. So, what does it mean to work at Accordion? It means joining 1,000+ analytics, data science, finance & technology experts in a high-growth, agile, and entrepreneurial environment while transforming how portfolio companies drive value. It also means making your mark on Accordions futureby embracing a culture rooted in collaboration and a firm-wide commitment to building something great, together. Headquartered in New York City with 10 offices worldwide, Accordion invites you to join our journey. Data & Analytics (Accordion | Data & Analytics) Accordion's Data & Analytics (D&A) team delivers cutting-edge, intelligent solutions to a global clientele, leveraging a blend of domain knowledge, sophisticated technology tools, and deep analytics capabilities to tackle complex business challenges. We partner with Private Equity clients and their Portfolio Companies across diverse sectors, including Retail, CPG, Healthcare, Media & Entertainment, Technology, and Logistics. D&A team delivers data and analytical solutions designed to streamline reporting capabilities and enhance business insights across vast and complex data sets ranging from Sales, Operations, Marketing, Pricing, Customer Strategies, and more. Location: Hyderabad Role Overview: Accordion is looking for Lead Data Engineer. He/she will be responsible for the design, development, configuration/deployment, and maintenance of the above technology stack. He/she must have in-depth understanding of various tools & technologies in the above domain to design and implement robust and scalable solutions which address client current and future requirements at optimal costs. The Lead Data Engineer should be able to evaluate existing architectures and recommend way to upgrade and improve the performance of architectures both on-premises and cloud-based solutions. A successful Lead Data Engineer should possess strong working business knowledge, familiarity with multiple tools and techniques along with industry standards and best practices in Business Intelligence and Data Warehousing environment. He/she should have strong organizational, critical thinking, and communication skills. What You will do: Partners with clients to understand their business and create comprehensive business requirements. Develops end-to-end Business Intelligence framework based on requirements including recommending appropriate architecture (on-premises or cloud), analytics and reporting. Works closely with the business and technology teams to guide in solution development and implementation. Work closely with the business teams to arrive at methodologies to develop KPIs and Metrics. Work with Project Manager in developing and executing project plans within assigned schedule and timeline. Develop standard reports and functional dashboards based on business requirements. Conduct training programs and knowledge transfer sessions to junior developers when needed. Recommend improvements to provide optimum reporting solutions. Curiosity to learn new tools and technologies to provide futuristic solutions for clients. Ideally, you have: Undergraduate degree (B.E/B.Tech.) from tier-1/tier-2 colleges are preferred. More than 5 years of experience in related field. Proven expertise in SSIS, SSAS and SSRS (MSBI Suite.) In-depth knowledge of databases (SQL Server, MySQL, Oracle etc.) and data warehouse (any one of Azure Synapse, AWS Redshift, Google BigQuery, Snowflake etc.) In-depth knowledge of business intelligence tools (any one of Power BI, Tableau, Qlik, DOMO, Looker etc.) Good understanding of Azure (OR) AWS: Azure (Data Factory & Pipelines, SQL Database & Managed Instances, DevOps, Logic Apps, Analysis Services) or AWS (Glue, Aurora Database, Dynamo Database, Redshift, QuickSight). Proven abilities to take on initiative and be innovative. Analytical mind with problem solving attitude. Why Explore a Career at Accordion: High growth environment: Semi-annual performance management and promotion cycles coupled with a strong meritocratic culture, enables fast track to leadership responsibility. Cross Domain Exposure: Interesting and challenging work streams across industries and domains that always keep you excited, motivated, and on your toes. Entrepreneurial Environment : Intellectual freedom to make decisions and own them. We expect you to spread your wings and assume larger responsibilities. Fun culture and peer group: Non-bureaucratic and fun working environment; Strong peer environment that will challenge you and accelerate your learning curve. Other benefits for full time employees: Health and wellness programs that include employee health insurance covering immediate family members and parents, term life insurance for employees, free health camps for employees, discounted health services (including vision, dental) for employee and family members, free doctors consultations, counsellors, etc. Corporate Meal card options for ease of use and tax benefits. Team lunches, company sponsored team outings and celebrations. Cab reimbursement for women employees beyond a certain time of the day. Robust leave policy to support work-life balance. Specially designed leave structure to support woman employees for maternity and related requests. Reward and recognition platform to celebrate professional and personal milestones. A positive & transparent work environment including various employee engagement and employee benefit initiatives to support personal and professional learning and development.

Primine Software Private Limited is looking for Machine Learning Engineer to join our dynamic team and embark on a rewarding career journeyA Machine Learning Engineer is responsible for designing, building, and maintaining systems that use artificial intelligence (AI) and machine learning (ML) algorithms to solve complex problems. 1.Developing and implementing ML algorithms: The Machine Learning Engineer develops and implements machine learning algorithms to solve specific problems, such as natural language processing, computer vision, or predictive modeling.2.Building data pipelines: The Machine Learning Engineer is responsible for building data pipelines that collect, store, and preprocess data used in machine learning algorithms.3.Creating and maintaining ML infrastructure: The Machine Learning Engineer is responsible for creating and maintaining ML infrastructure, including hardware, software, and cloud platforms, that support the development and deployment of ML models.4.Testing and validating ML models: The Machine Learning Engineer tests and validates ML models, ensuring that they are accurate, robust, and scalable.5.Troubleshooting ML systems: The Machine Learning Engineer troubleshoots ML systems, identifying and resolving issues related to performance, accuracy, and scalability.6.Deploying ML models: The Machine Learning Engineer deploys ML models in production environments, integrating them with other software systems and ensuring that they are reliable and scalable.

Responsibilities : Collaborate with the sales team to understand customer challenges and business objectives and propose solutions, POC etc. Develop and deliver impactful technical presentations and demos showcasing the capabilities of GCP Data and AI , GenAI Solutions. Conduct technical proof of concepts (POCs) to validate the feasibility and value proposition of GCP solutions. Collaborate with technical specialists and solution architects from COE Team to design and configure tailored cloud solutions. Manage and qualify sales opportunities, working closely with the sales team to progress deals through the sales funnel. Stay up to date on the latest GCP offerings, trends, and best practices. Experience : Design and implement a comprehensive strategy for migrating and modernizing existing relational on premise databases to scalable and cost effective solution on Google Cloud Platform ( GCP). Design and Architect the solutions for DWH Modernization and experience with building data pipelines in GCP. Strong Experience in BI reporting tools ( Looker, PowerBI and Tableau). In depth knowledge of Google Cloud Platform (GCP) services, particularly Cloud SQL, Postgres, Alloy DB, BigQuery, Looker Vertex AI and Gemini (GenAI). Strong knowledge and experience in providing the solution to process massive datasets in real time and batch process using cloud native/open source Orchestration techniques. Build and maintain data pipelines using Cloud Dataflow to orchestrate real time and batch data processing for streaming and historical data. Strong knowledge and experience in best practices for data governance, security, and compliance. Excellent Communication and Presentation Skills with ability to tailor technical information as per customer needs. Strong analytical and problem solving skills. Ability to work independently and as part of a team.

Exp in dashboard story development, dashboard creation, and data engineering pipelines. Manage and organize large volumes of application log data using Google Big Query Exp with log analytics, user engagement metrics, and product performance metrics Required Candidate profile Exp with tool like Tableau Power BI, or ThoughtSpot AI . Understand log data generated by Python-based applications. Ensure data integrity, consistency, and accessibility for analytical purposes.

Salary: 8 to 24 LPA Exp: 3 to 7 years Location: Gurgaon/Pune/Bengalore Notice: Immediate to 30 days..!! Job Profile: Experienced Data Engineer with a strong foundation in designing, building, and maintaining scalable data pipelines and architectures. Skilled in transforming raw data into clean, structured formats for analytics and business intelligence. Proficient in modern data tools and technologies such as SQL, T-SQL, Python, Databricks, and cloud platforms (Azure). Adept at data wrangling, modeling, ETL/ELT development, and ensuring data quality, integrity, and security. Collaborative team player with a track record of enabling data-driven decision-making across business units. As a Data engineer, Candidate will work on the assignments for one of our Utilities clients. Collaborating with cross-functional teams and stakeholders involves gathering data requirements, aligning business goals, and translating them into scalable data solutions. The role includes working closely with data analysts, scientists, and business users to understand needs, designing robust data pipelines, and ensuring data is accessible, reliable, and well-documented. Regular communication, iterative feedback, and joint problem-solving are key to delivering high-impact, data-driven outcomes that support organizational objectives. This position requires a proven track record of transforming processes, driving customer value, cost savings with experience in running end-to-end analytics for large-scale organizations. Design, build, and maintain scalable data pipelines to support analytics, reporting, and advanced modeling needs. Collaborate with consultants, analysts, and clients to understand data requirements and translate them into effective data solutions. Ensure data accuracy, quality, and integrity through validation, cleansing, and transformation processes. Develop and optimize data models, ETL workflows, and database architectures across cloud and on-premises environments. Support data-driven decision-making by delivering reliable, well-structured datasets and enabling self-service analytics. Provides seamless integration with cloud platforms (Azure), making it easy to build and deploy end-to-end data pipelines in the cloud Scalable clusters for handling large datasets and complex computations in Databricks, optimizing performance and cost management. Must to have Client Engagement Experience and collaboration with cross-functional teams Data Engineering background in Databricks Capable of working effectively as an individual contributor or in collaborative team environments Effective communication and thought leadership with proven record. Candidate Profile: Bachelors/masters degree in economics, mathematics, computer science/engineering, operations research or related analytics areas 3+ years experience must be in Data engineering. Hands on experience on SQL, Python, Databricks, cloud Platform like Azure etc. Prior experience in managing and delivering end to end projects Outstanding written and verbal communication skills Able to work in fast pace continuously evolving environment and ready to take up uphill challenges Is able to understand cross cultural differences and can work with clients across the globe.

Salary: 15 to 30 LPA Exp: 3 to 8 years Location : Gurgaon/Bangalore/Pune/Chennai Notice: Immediate to 30 days..!! Key Responsibilities & Skillsets: Common Skillsets : 3+ years of experience in analytics, Pyspark, Python, Spark, SQL and associated data engineering jobs. Must have experience with managing and transforming big data sets using pyspark, spark-scala, Numpy pandas Excellent communication & presentation skills Experience in managing Python codes and collaborating with customer on model evolution Good knowledge of data base management and Hadoop/Spark, SQL, HIVE, Python (expertise). Superior analytical and problem solving skills Should be able to work on a problem independently and prepare client ready deliverable with minimal or no supervision Good communication skill for client interaction Data Management Skillsets: Ability to understand data models and identify ETL optimization opportunities. Exposure to ETL tools is preferred Should have strong grasp of advanced SQL functionalities (joins, nested query, and procedures). Strong ability to translate functional specifications / requirements to technical requirements

Role & responsibilities The individual will be responsible for designs, builds, and maintains scalable data solutions including data models, ETL pipelines, and real-time processing systems He/she is also responsible for mentoring junior team members and play a pivotal role in overseeing the technical aspects of data related projects. MUST HAVE Experience on working with ETL on a data warehouse or big data projects. Experience with cloud services like Azure or GCP or AWS with respect to the data related services, could be just ETL ,cleanup or BI but has potential to be groomed in a 1-2 month period. Should have energy and enthusiasm to get into new things, able to stretch. Good SQL skills and able to solve complex problems in database SQL language efficiently. Able to identify and resolve performance and scalability issues. Can work as an individual contributor. Basic understanding of reporting tools and basic visuals. Actively participate and manage design and code reviews. Help in estimation the task with assistance. Guide team members on the project, including structured problem solving, development of best practices, etc. Good communication and analytical skills.

Role & responsibilities What You'll Do Build the underneath data platform and maintain data processing pipelines using best in class technologies. Special focus on R&D to challenge status-quo and build the next generation data mesh that is efficient and cost effective. Translate complex technical and functional requirements into detailed designs. Who You Are Strong programming skills (Python, Java, and Scala) Experience writing SQL , structuring data, and data storage practices Experience with data modeling Knowledge of data warehousing concepts Experienced building data pipelines and micro services Experience with Spark , Airflow and other streaming technologies to process incredible volumes of streaming data A willingness to accept failure, learn and try again An open mind to try solutions that may seem impossible at first Strong understanding of data structures, algorithms, multi-threaded programming, and distributed computing concepts Experience working on Amazon Web Services- AWS ( EMR, Kinesis, RDS, S3 , SQS and the like) Preferred candidate profile At least 6+ years of professional experience as a software engineer or data engineer Education: Bachelor's degree or higher in Computer Science, Data Science, Engineering, or a related technical field.

Key Responsibilities: - Architect and Develop GPT Models:** Design, build, and optimize custom GPT models, including architecture and fine-tuning, to meet the specific needs of the legal domain. - Cloud Architecture: Develop and manage cloud-based infrastructure with a deep understanding of cloud services and architecture to deploy and scale AI solutions efficiently. - Integration and Deployment: Oversee the deployment of GenAI models and ensure seamless integration with the KonProz platform. - Vector DB Utilization: Work with Vector DBs to enhance data retrieval, storage, and interaction with AI models. - Performance Optimization: Monitor and optimize the performance of AI models and cloud services to ensure reliability and efficiency. - Innovation and Collaboration: Collaborate with cross-functional teams to drive innovation and integrate AI capabilities into the product offering. - Technical Leadership: Provide technical guidance and mentorship to junior engineers, fostering a culture of learning and growth within the team. - Stay Current: Keep abreast of the latest advancements in AI and cloud technologies and apply new insights to improve our solutions. Qualifications: - 2-5 years of experience in developing and deploying GPT models, with a strong understanding of creating custom models and their architectures. - Deep knowledge of cloud platforms (e.g., AWS, Azure, Google Cloud) and cloud architecture for scalable AI solutions. - Proficiency in Python and relevant ML libraries (e.g., TensorFlow, PyTorch). - Hands-on experience with Vector DBs and cloud-based data pipelines. - Strong problem-solving skills and a proactive approach to learning and development. - Excellent communication skills and the ability to work effectively in a collaborative team environment. - Ability to adapt to a startup pace and thrive in a dynamic work setting.

Company Overview We are a fast-growing research firm with a global footprint, addressing major market, strategic, economic, scientific, and technological developments for business leaders in industrial, pharmaceutical, and technology organizations. We provides comprehensive analysis of global market sizing, forecasting, and industry intelligence, covering markets where advances in science and technology are improving the quality, standard, and sustainability of businesses, economies, and lives. We are in the process of retooling the business for next-generation innovative online and AI-powered products. Job Summary We are seeking a skilled Agentic AI Technical Engineer to design, develop, and implement AI-powered solutions that enhance our market research and industry intelligence offerings. This role requires hands-on expertise in agentic AI systems, proficiency in machine learning frameworks, and the ability to collaborate on innovative online platforms. The ideal candidate will build autonomous, goal-oriented AI agents to deliver actionable insights for clients in industrial, pharmaceutical, and technology sectors. Key Responsibilities System Development : Design and develop agentic AI systems, implementing reinforcement learning, natural language processing, and multi-agent coordination to support market analysis and forecasting. Model Implementation : Build, train, and deploy AI models that process complex datasets, delivering accurate and actionable insights for market research applications. Technical Collaboration : Work closely with AI engineers, data scientists, and product teams to integrate AI solutions into client-facing platforms and ensure seamless functionality. Research and Innovation : Apply advancements in agentic AI and machine learning to create cutting-edge tools for market sizing, forecasting, and industry intelligence. Performance Optimization : Optimize AI models for scalability, accuracy, and real-time performance, ensuring efficient data processing in production environments. Data Pipeline Development : Build and maintain data pipelines to support AI-driven analytics, ensuring data quality and compatibility with research objectives. Ethics and Compliance : Implement AI systems that adhere to ethical guidelines, data privacy regulations (e.g., GDPR, CCPA), and industry-specific compliance standards. Technical Support : Provide technical expertise and troubleshooting for AI systems, collaborating with teams to resolve issues and improve system performance. Qualifications Education : Masters or Ph.D. in Computer Science, Artificial Intelligence, Machine Learning, or a related field. Experience : 3–5 years building AI/ML-powered systems with at least one project involving agentic workflows or LLM-based automation Demonstrated experience building with tools like LangChain, LangGraph, or similar frameworks for multi-agent coordination Hands-on use of LLM APIs (e.g., OpenAI, Claude, Mistral), including prompt design, chaining, and tool integration Worked with vector databases (e.g., Pinecone, FAISS, Weaviate) for retrieval-augmented generation tasks Familiarity with deploying or integrating AI solutions in cloud environments (AWS, GCP, or serverless hosting) Technical Skills : Ability to design and optimize multi-step agent workflows using LLMs and external tools Strong Python developer with experience working in async, event-driven systems or streaming architectures Comfortable working with Neo4j or other graph databases to represent and query structured relationships Experience building or integrating lightweight ETL pipelines to ingest and transform structured/unstructured content Understanding of secure multi-tenant context handling and basic MLOps concepts like model versioning or feedback loops Communication and Collaboration Skills : Comfortable working cross-functionally with researchers, engineers, and product leads to turn workflows into agent behavior Able to clearly communicate system logic, capabilities, and limitations to both technical and non-technical stakeholders Experience working in distributed teams or async-first environments is a plus Self-motivated, organized, and able to manage ambiguity while delivering high-leverage contributions Preferred Qualifications : Experience with AI applications in market research, forecasting, or industry intelligence. Knowledge of industrial, pharmaceutical, or technology markets. Contributions to AI research, open-source projects, or publications in relevant fields. Nice to Have: Experience building domain-specific agents (e.g.,for research, finance, regulatory analysis, etc.) Exposure to reinforcement learning, RLHF, or retrieval scoring techniques Familiarity with streaming search, data federation, or client data integration pipelines Contributions to open-source agentic or LLM tooling, or AI writing/thought leadership Background in decision-support, research automation, or SaaS AI platforms ____________________________________________________________________________________ Ideal Candidate Profile Innovative Developer : Passionate about building AI solutions that transform market research and deliver impactful insights. Technical Problem Solver : Skilled at implementing and optimizing complex AI systems for real-world applications. Team Player : Thrives in collaborative environments, contributing to shared goals with a focus on quality and innovation. Client-Focused : Committed to developing AI tools that empower clients with actionable, data-driven insights.

Experience: 4-8 Years Job Location: Remote No. of Position: Multiple Qualifications: B Tech / M Tech/ MCA or Higher Work Timings: 1:30 PM IST to 10:30 PM IST Functional Area: Data Engineering Job Description: We are seeking a skilled Data Engineer with 4 to 8 years of experience to join our team. The ideal candidate will have a strong background in Python programming, along with expertise in AWS or Azure services. The candidate should also possess solid SQL skills and be proficient in web scraping techniques. Role and responsibilities: Develop and maintain data pipelines using Python, PySpark, and SQL to extract, transform, and load data from various sources. Implement and optimize data processing workflows on AWS or Azure cloud platforms. Utilize Databricks or Azure data factory for efficient data storage and processing. Develop and maintain web scraping scripts to gather data from online sources. Collaborate with cross-functional teams to design and implement API endpoints for data access. Work on UI Path automation projects to streamline data extraction and processing tasks. Develop and maintain Django or Flask web applications for internal data management and visualization. Leverage Pandas and other data manipulation libraries for data analysis and preprocessing. Enhance API development skills for integrating data services with external systems. Stay updated with the latest industry trends and technologies, such as Flask, PyTorch, etc., to continuously improve data engineering processes. Skills, Knowledge, Experience: Bachelor's degree in Computer Science, Engineering, or related field. 4 to 8 years of experience in data engineering roles. Proficiency in Python programming language. Strong understanding of AWS or Azure cloud services. Solid SQL skills for querying and manipulating data. Previous experience with web scraping techniques and tools. Hands-on experience with Django web framework. Knowledge of API development and integration. Experience with PySpark for big data processing. Proficiency in Pandas for data manipulation and analysis. Familiarity with UI Path for automation or Power Automate is advantageous. Experience with Databricks. Familiarity with Flask and PyTorch is a plus. Experience working with USA or European clients is a plus. Experience working with multi-vendor, multi-culture, distributed offshore, and onshore development teams in a dynamic and complex environment will be helpful in day-to-day working. Must have excellent written and verbal communication skills. The candidate should be able to present his suggestions and explain the technical approach.

Overview We are looking for a Senior Data Engineer with deep hands-on expertise in PySpark, Databricks, and distributed data architecture. This individual will play a lead role in designing, developing, and optimizing data pipelines critical to our Ratings Modernization, Corrections, and Regulatory implementation programs under PDB 2.0. The ideal candidate will thrive in fast-paced, ambiguous environments and collaborate closely with engineering, product, and governance teams. Responsibilities Design, develop, and maintain robust ETL/ELT pipelines using PySpark and Databricks . Own pipeline architecture and drive performance improvements through partitioning, indexing, and Spark optimization . Collaborate with product owners, analysts, and other engineers to gather requirements and resolve complex data issues. Perform deep analysis and optimization of SQL queries , functions, and procedures for performance and scalability. Ensure high standards of data quality and reliability via robust validation and cleansing processes. Lead efforts in Delta Lake and cloud data warehouse architecture , including best practices for data lineage and schema management. Troubleshoot and resolve production incidents and pipeline failures quickly and thoroughly. Mentor junior team members and guide best practices across the team. Qualifications Bachelor's degree in Computer Science, Engineering, or a related technical field. 6+ years of experience in data engineering or related roles. Advanced proficiency in Python, PySpark, and SQL . Strong experience with Databricks , BigQuery , and modern data lakehouse design. Hands-on knowledge of Azure or GCP data services. Proven experience in performance tuning and large-scale data processing . Strong communication skills and the ability to work independently in uncertain or evolving contexts What we offer you Transparent compensation schemes and comprehensive employee benefits, tailored to your location, ensuring your financial security, health, and overall wellbeing. Flexible working arrangements, advanced technology, and collaborative workspaces. A culture of high performance and innovation where we experiment with new ideas and take responsibility for achieving results. A global network of talented colleagues, who inspire, support, and share their expertise to innovate and deliver for our clients. Global Orientation program to kickstart your journey, followed by access to our Learning@MSCI platform, LinkedIn Learning Pro and tailored learning opportunities for ongoing skills development. Multi-directional career paths that offer professional growth and development through new challenges, internal mobility and expanded roles. We actively nurture an environment that builds a sense of inclusion belonging and connection, including eight Employee Resource Groups. All Abilities, Asian Support Network, Black Leadership Network, Climate Action Network, Hola! MSCI, Pride & Allies, Women in Tech, and Women’s Leadership Forum. At MSCI we are passionate about what we do, and we are inspired by our purpose – to power better investment decisions. You’ll be part of an industry-leading network of creative, curious, and entrepreneurial pioneers. This is a space where you can challenge yourself, set new standards and perform beyond expectations for yourself, our clients, and our industry. MSCI is a leading provider of critical decision support tools and services for the global investment community. With over 50 years of expertise in research, data, and technology, we power better investment decisions by enabling clients to understand and analyze key drivers of risk and return and confidently build more effective portfolios. We create industry-leading research-enhanced solutions that clients use to gain insight into and improve transparency across the investment process. MSCI Inc. is an equal opportunity employer. It is the policy of the firm to ensure equal employment opportunity without discrimination or harassment on the basis of race, color, religion, creed, age, sex, gender, gender identity, sexual orientation, national origin, citizenship, disability, marital and civil partnership/union status, pregnancy (including unlawful discrimination on the basis of a legally protected parental leave), veteran status, or any other characteristic protected by law. MSCI is also committed to working with and providing reasonable accommodations to individuals with disabilities. If you are an individual with a disability and would like to request a reasonable accommodation for any part of the application process, please email Disability.Assistance@msci.com and indicate the specifics of the assistance needed. Please note, this e-mail is intended only for individuals who are requesting a reasonable workplace accommodation; it is not intended for other inquiries. To all recruitment agencies MSCI does not accept unsolicited CVs/Resumes. Please do not forward CVs/Resumes to any MSCI employee, location, or website. MSCI is not responsible for any fees related to unsolicited CVs/Resumes. Note on recruitment scams We are aware of recruitment scams where fraudsters impersonating MSCI personnel may try and elicit personal information from job seekers. Read our full note on careers.msci.com

5-9yrs exp in Pipeline Monitoring, working data, ETL, Data warehouse & BI Projects. Coding exp, Develop & maintain Dashboards. Knowledge in AWS Glue, S3, Athena, JIRA, Programming- Python, Java, Scala etc. Bangalore/ Pune/ Hyderabad/ Greater Noida.

Job Title: Automation EngineerDatabricks Job Type: Full-time, Contractor Location: Hybrid Hyderabad | Pune| Delhi About Us: Our mission at micro1 is to match the most talented people in the world with their dream jobs If you are looking to be at the forefront of AI innovation and work with some of the fastest-growing companies in Silicon Valley, we invite you to apply for a role By joining the micro1 community, your resume will become visible to top industry leaders, unlocking access to the best career opportunities on the market Job Summary: We are seeking a detail-oriented and innovative Automation EngineerDatabricks to join our customer's team In this critical role, you will design, develop, and execute automated tests to ensure the quality, reliability, and integrity of data within Databricks environments If you are passionate about data quality, thrive in collaborative environments, and excel at both written and verbal communication, we'd love to meet you Key Responsibilities: Design, develop, and maintain robust automated test scripts using Python, Selenium, and SQL to validate data integrity within Databricks environments Execute comprehensive data validation and verification activities to ensure accuracy and consistency across multiple systems, data warehouses, and data lakes Create detailed and effective test plans and test cases based on technical requirements and business specifications Integrate automated tests with CI/CD pipelines to facilitate seamless and efficient testing and deployment processes Work collaboratively with data engineers, developers, and other stakeholders to gather data requirements and achieve comprehensive test coverage Document test cases, results, and identified defects; communicate findings clearly to the team Conduct performance testing to ensure data processing and retrieval meet established benchmarks Provide mentorship and guidance to junior team members, promoting best practices in test automation and data validation Required Skills and Qualifications: Strong proficiency in Python, Selenium, and SQL for developing test automation solutions Hands-on experience with Databricks, data warehouse, and data lake architectures Proven expertise in automated testing of data pipelines, preferably with tools such as Apache Airflow, dbt Test, or similar Proficient in integrating automated tests within CI/CD pipelines on cloud platforms (AWS, Azure preferred) Excellent written and verbal communication skills with the ability to translate technical concepts to diverse audiences Bachelors degree in Computer Science, Information Technology, or a related discipline Demonstrated problem-solving skills and a collaborative approach to teamwork Preferred Qualifications: Experience with implementing security and data protection measures in data-driven applications Ability to integrate user-facing elements with server-side logic for seamless data experiences Demonstrated passion for continuous improvement in test automation processes, tools, and methodologies