76 Etl Frameworks Jobs

Required Skills & Experience 57 years of experience in data engineering, with at least 23 years of Databricks hands-on experience. Strong expertise in Apache Spark (PySpark/Scala/SQL) and distributed data processing. Solid experience with Delta Lake, Lakehouse architecture, and data modeling. Hands-on experience with at least one cloud platform: Azure Data Lake, AWS S3, or GCP BigQuery/Storage. Strong proficiency in SQL for data manipulation and performance tuning. Experience with ETL frameworks, workflow orchestration tools (Airflow, ADF, DBX Workflows). Good understanding of CI/CD, Git-based workflows, and DevOps practices. Exposure to MLOps and MLflow is a strong plus. Knowledge of data g...

Who are we and What do we do ShareChat (https://sharechat.com/about) is India's largest homegrown social media company, with 200 million monthly active users across all its platforms including Moj, a leading short video app which was launched in a record 30 hours. Founded in October 2015, with a vision of building an inclusive community that encourages & empowers each individual to share their unique journey and valuable experiences with confidence. We are spearheading India's internet revolution by building products through world-class AI & tech to evangelize the content ecosystem for India in regional languages. We believe in complete ownership of problem-solving while committing to speed ...

As a Data Engineer, you will be responsible for managing and optimizing data pipelines for efficient data processing. Your role will involve working with a variety of technologies and platforms to ensure smooth data workflows. Key Responsibilities: - Utilize your 5+ years of experience in data engineering, including at least 2+ years of hands-on experience with Databricks. - Demonstrate strong expertise in Apache Spark using PySpark, Scala, and SQL for efficient distributed data processing. - Implement and maintain Delta Lake and Lakehouse architecture for effective data modeling. - Work with cloud platforms such as Azure Data Lake, AWS S3, or GCP BigQuery/Storage to manage data infrastructu...

Role Overview: Yethi Consulting is looking for an experienced Data Migration Lead / Manager to oversee and manage comprehensive data migration initiatives for core banking systems. As the Lead / Manager, you will be responsible for guiding legacy-to-new application migrations, version upgrades, and client-side transition projects. Your primary focus will be to ensure precision, efficiency, and regulatory compliance throughout the entire migration process. Key Responsibilities: - Lead and manage end-to-end data migration projects for Core Banking Systems such as Flexcube, Finacle, TCS BaNCS, T24, OBDX, etc. - Define and implement data migration strategy, roadmap, and methodology including ETL...

Responsibilities Job Requirements Translate business needs into technical specifications. Design, build, and deploy BI solutions (e.g., reporting tools) for clinical, operational, and financial analytics. Develop interactive and scalable data visualizations for executives, clinical, and operational stakeholders. Optimize DAX expressions, Power Query, and data models for high-performance reporting. Ensure Role-Based Access Control (RBAC) and Row-Level Security for data governance. Integrate Power BI with SQL databases, cloud data lakes (AWS/Azure), and REST APIs. Maintain and support data analytics platforms. Conduct unit testing and troubleshooting. Collaborate with teams to integrate system...

The Company PayPal has been revolutionizing commerce globally for more than 25 years. Creating innovative experiences that make moving money, selling, and shopping simple, personalized, and secure, PayPal empowers consumers and businesses in approximately 200 markets to join and thrive in the global economy. We operate a global, two-sided network at scale that connects hundreds of millions of merchants and consumers. We help merchants and consumers connect, transact, and complete payments, whether they are online or in person. PayPal is more than a connection to third-party payment networks. We provide proprietary payment solutions accepted by merchants that enable the completion of payments...

Role Overview: We are seeking a Data Migration Expert with deep expertise in architecting and executing complex data migrations across hybrid and cloud ecosystems. This is not a DBA role this is a hands-on migration architect who speaks the language of data engineering and thrives on solving high-scale data movement challenges with precision and agility. Key Responsibilities: - Design and implement large-scale data migration strategies across on-prem, hybrid, and cloud environments. - Build and orchestrate real-time data pipelines using CDC and ETL frameworks. - Ensure seamless schema evolution, data lineage tracking, and zero-downtime transfers. - Collaborate with cross-functional teams to ...

Role Overview: As a Data Engineering Lead at the company, your main responsibility will be to lead the architecture and development of data pipelines, warehouses, and AI-ready infrastructures. You will be responsible for designing and overseeing machine learning workflows supporting OCR, NLP, and predictive analytics in products like Scribe, Quanta, and Compass. Additionally, you will guide the implementation of Enterprise Data Warehouse (EDW) solutions for global clients using AWS, GCP, or Azure. Collaboration with client stakeholders to translate business challenges into robust data strategies and architectures will also be a key part of your role. Furthermore, you will mentor and manage a...

Role Description We are looking for a Senior Data Warehouse Engineer with good designing skills to work with Conversant Engineering Data Warehousing teams which deals in data of petabyte scale. This role will help to define solutions for given product and engineering initiatives , and drive solutions to delivery by effectively engaging with team members across the globe. The person in this role will be working closely with key stakeholders throughout the organization , so he or she must be able to communicate and keep them informed on overall health on projects impacting Conversant Data Warehouse platforms. He should also be able to mentor Juniors in the Teams. What you ll need Lead , design...

Role Description We are looking for a Senior Data Warehouse Engineer with good designing skills to work with Conversant Engineering Data Warehousing teams which deals in data of petabyte scale. This role will help to define solutions for given product and engineering initiatives , and drive solutions to delivery by effectively engaging with team members across the globe. The person in this role will be working closely with key stakeholders throughout the organization , so he or she must be able to communicate and keep them informed on overall health on projects impacting Conversant Data Warehouse platforms. He should also be able to mentor Juniors in the Teams. What you ll need Lead , design...

Role Description We are looking for a Senior Data Warehouse Engineer with good designing skills to work with Conversant Engineering Data Warehousing teams which deals in data of petabyte scale. This role will help to define solutions for given product and engineering initiatives , and drive solutions to delivery by effectively engaging with team members across the globe. The person in this role will be working closely with key stakeholders throughout the organization , so he or she must be able to communicate and keep them informed on overall health on projects impacting Conversant Data Warehouse platforms. He should also be able to mentor Juniors in the Teams. What you ll need Lead , design...

Role Description We are looking for a Senior Data Warehouse Engineer with good designing skills to work with Conversant Engineering Data Warehousing teams which deals in data of petabyte scale. This role will help to define solutions for given product and engineering initiatives , and drive solutions to delivery by effectively engaging with team members across the globe. The person in this role will be working closely with key stakeholders throughout the organization , so he or she must be able to communicate and keep them informed on overall health on projects impacting Conversant Data Warehouse platforms. He should also be able to mentor Juniors in the Teams. What you ll need Lead , design...

Role Overview: As a Project Manager/Senior Project Manager at GormalOne LLP, you will play a crucial role in driving data-centric product initiatives for the DairyTech ecosystem. Your focus will be on building scalable data platforms, enabling advanced analytics, and delivering actionable insights through intuitive dashboards and visualization tools to empower decision-making across supply chain, operations, sales, and customer engagement. Key Responsibilities: - Define and own the product strategy for the Data & Analytics platform to support business growth in the DairyTech domain. - Identify data-driven opportunities in collaboration with business stakeholders, ensuring alignment with orga...

Visa is a world leader in payments and technology, with over 259 billion payments transactions flowing safely between consumers, merchants, financial institutions, and government entities in more than 200 countries and territories each year. Our mission is to connect the world through the most innovative, convenient, reliable, and secure payments network, enabling individuals, businesses, and economies to thrive while driven by a common purpose to uplift everyone, everywhere by being the best way to pay and be paid. Make an impact with a purpose-driven industry leader. Join us today and experience Life at Visa. Job Description Visa's Cybersecurity Assurance team is seeking an Entry-Level Ass...

About Allvue We are Allvue Systems, the leading provider of software solutions for the Private Capital and Credit markets. Whether a client wants an end-to-end technology suite, or independently focused modules, Allvue helps eliminate the boundaries between systems, information, and people. We're looking for ambitious, smart, and creative individuals to join our team and help our clients achieve their goals. Working at Allvue Systems means working with pioneers in the fintech industry. Our efforts are powered by innovative thinking and a desire to build adaptable financial software solutions that help our clients achieve even more. With our common goals of growth and innovation, whether you'...

As a Back-end Engineer at Turgon, you will be responsible for playing a crucial role in scaling the product and AI agents. This position offers a unique opportunity to work on cutting-edge AI agentic frameworks alongside experienced founders and make a significant impact on the company's growth. Key Responsibilities: - Designing and implementing scalable backend services for LLM applications - Developing data pipelines for ingestion, preprocessing, and storage (both batch and real-time) - Integrating and orchestrating LLM frameworks such as LangChain, LangGraph, and Hugging Face using Python - Collaborating with the DevOps team to containerize, monitor, and automate deployments utilizing Doc...

As a Lead Data Engineer at the company, you will play a crucial role in designing, developing, and optimizing data pipelines and architectures to meet the increasing data and analytics requirements. Collaborating with data scientists, analysts, and software engineers, you will ensure efficient data flow, storage, and processing. **Key Responsibilities:** - Design, build, and maintain scalable, efficient, and reliable data pipelines. - Develop and optimize ETL/ELT processes for transforming raw data into usable formats. - Implement data warehousing solutions and manage data lakes. - Ensure data quality, consistency, and security across different platforms. - Collaborate with cross-functional ...

Data Engineering Tech Lead Azure Databricks About Lingaro: Lingaro Group is the end-to-end data services partner to global brands and enterprises. We lead our clients through their data journey, from strategy through development to operations and adoption, helping them to realize the full value of their data. Since 2008, Lingaro has been recognized by clients and global research and advisory firms for innovation, technology excellence, and the consistent delivery of highest-quality data services. Our commitment to data excellence has created an environment that attracts the brightest global data talent to our team. Requirements: A bachelor's or master's degree in Computer Science, Informatio...

Role Summary & Role Description: Technical Manager with specific Oracle, PL/SQL and design, develop, and optimize data workflows on the Databricks platform. The ideal candidate will have deep expertise in Apache Spark, PySpark, Python, job orchestration, and CI/CD integration to support scalable data engineering and analytics solutions. Analyzes, designs, develops and maintains software applications to support business units. Expected to spend 80% of the time on hands-on development, design and architecture and remaining 20% on guiding the team on technology and removing other impediments Capital Markets Projects experience preferred Provides advanced technical expertise in analyzing, design...

Role Overview: You will be part of the 2nd Line Assurance function at VOIS, responsible for monitoring and mitigating key technology risks to drive a risk-aware culture within the organization. Your role will involve promoting a risk-sensitive approach to processes, architectures, applications, and platforms within the Technology department. The key technology risks you will focus on include Cyber Security, Technology Resilience, IT Transformation, Shadow IT, and Lifecycle Management. Your responsibilities will include working closely with local markets and Group functions, testing key controls, identifying corrective actions, and reporting to key stakeholders, including the Technology leade...

Role Overview: As a Senior AWS Data Engineer - Vice President at Barclays, you will lead the evolution of the Reference Data Services function by spearheading the engineering of software components. Your role will involve collaborating with a team of engineers to design, build, and test software components. Effective stakeholder management, leadership, and decision-making will be crucial to support the business strategy and risk management. Your primary focus will be on AWS cloud services, data engineering, and automation using Python, ETL frameworks, data transformation, and data quality tools. Key Responsibilities: - Build and maintain data architectures pipelines to ensure the transfer an...

You will be responsible for architecting, building, and optimizing scalable data pipelines and platforms to drive critical business decisions. Your expertise in PySpark, ETL frameworks, SQL Stored Procedures, and Data Modeling will be crucial for performance tuning across large datasets. - Lead the design and development of scalable data pipelines and ETL workflows using PySpark and SQL - Develop and maintain Stored Procedures for batch processes and data transformations - Design and implement robust data models (Star, Snowflake, and normalized models) for data warehouse and lake environments - Collaborate with BI, Analytics, and Product teams to understand data requirements and ensure data ...

As the Value Cluster Lead for Stores (Techno Functional Stores) in the Retail domain, your role involves owning the strategy, growth, and delivery excellence for a store cluster. This includes analytics and operations solutions, bridging business and technology aspects. You will lead pre-sales, solution architecture, and program delivery while building a high-performing team of data engineers, analysts, and consultants. - Cluster Strategy & Roadmap: Define the stores analytics value cluster vision, identifying use cases across inventory optimization, footfall analysis, planogram compliance, and workforce scheduling using various technology solutions. - Techno Functional Leadership: Translate...

As a Data Engineer, your role will involve designing, building, and maintaining scalable data ingestion pipelines to onboard structured and semi-structured data from various sources. You will work with Azure-native services like Data Factory and Azure Data Lake, along with tools like Databricks or Apache Spark for data ingestion and transformation. Your responsibilities will include developing metadata-driven ingestion frameworks, collaborating with source system owners, and ensuring data quality, lineage, and governance principles are followed. Additionally, you will optimize ingestion processes for performance, reliability, and cloud cost efficiency, supporting both batch and real-time ing...

As a Jr. SQL Developer at AutoFlowTech, your role will involve: - Proven working experience as a Data Analyst or Business Data Analyst - Technical expertise in data models, database design development, data mining, and segmentation techniques - Strong knowledge and experience with databases such as SQL, ETL frameworks, Excel, and SSIS - Strong analytical skills to collect, organize, analyze, and disseminate information with precision - Proficiency in queries, report writing, and presenting findings Qualifications required for this role: - Master's / bachelor's degree in computer science or a related field - 1 to 3 years of experience as a Data Analyst or Business Data Analyst - Familiarity w...