774 Etl Pipelines Jobs - Page 5

Skills: Databricks, azure, PySpark, Data Engineering, Azure Data Services, Delta Lake, ETL Pipelines, DevOps, Title: Senior Databricks Architect Location: Mumbai, Gurgaon, Hyderabad About The Role We are seeking a Senior Databricks Architect to lead the design and implementation of scalable, secure, and cost-optimized data platforms on Azure using Databricks. You will own end-to-end architecture for data ingestion, transformation, governance, and analytics, and mentor engineering teams to deliver high-quality data solutions. Key Responsibilities Architect and implement enterprise-grade data platforms on Azure Databricks (DLT, Workflows, Unity Catalog) with strong security, governance, and re...

At EY, youll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And were counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all. EY-Consulting-Microsoft-Fabri-Manager As part of our EY-DnA team, you will be responsible for designing, developing, and maintaining distributed systems using Microsoft Fabric, including One Lake, Azure Data Factory (ADF), Azure Synapse, Notebooks, Data Warehouse, and Lakehouse. You will play a crucial role in architecting and implementing ente...

EY_ Consulting_Microsoft_Fabric- Manager As part of our EY-DnA team, you will be responsible for designing, developing, and maintaining distributed systems using Microsoft Fabric, including One Lake, Azure Data Factory (ADF), Azure Synapse, Notebooks, Data Warehouse, and Lakehouse. You will play a crucial role in architecting and implementing enterprise data platforms and data management practices, ensuring the delivery of high-quality solutions that meet business requirements. You will collaborate with system architects, business analysts, and stakeholders to understand their requirements and convert them into technical designs. Your role will involve designing, building, testing, deploying...

Role & responsibilities Bachelor's degree in Computer Science, Engineering, or related field. 8+ years of experience in data engineering or related roles. Proficiency in programming languages such as Python, Java, or Scala. Strong SQL skills and experience with relational databases (e.g., MySQL, PostgreSQL). Experience with data warehousing concepts and technologies (e.g., Snowflake, Redshift). Familiarity with big data processing frameworks (e.g., Apache Spark, Hadoop). Hands-on experience with ETL tools and data integration platforms. Knowledge of cloud platforms such as AWS, Azure, or Google Cloud Platform. Understanding of data modeling principles and data warehousing design patterns. Ex...

Sr. Data Engineer About the Role The Senior Data Engineer will play a pivotal role in designing, developing, and maintaining scalable, high-performance data pipelines and infrastructure. You will lead multiple data engineering initiatives, collaborate with cross-functional teams, and deliver enterprise-grade data solutions aligned with business goals. This role is ideal for a proactive problem solver with strong leadership skills, hands-on expertise in PySpark, AWS Glue, Databricks, and S3 , and the ability to translate complex technical concepts into actionable insights for business stakeholders. Key Responsibilities Lead functional teams and manage multiple data engineering projects simult...

Database Reliability Engineer NoSQL & Cloud Data Platforms Desired Experience Range: 5 - 7 years Location of Requirement: Bangalore Must Have 5+ years of experience managing Snowflake and MongoDB databases. Strong experience designing and implementing scalable data models for business and analytical use cases. Proven expertise in building and maintaining ETL/ELT pipelines using tools like Airflow, dbt, or similar frameworks. Proficiency in Python for scripting and automation . Advanced SQL skills with the ability to write, tune,and maintain queries for large-scale datasets. Experience with cloud platforms (AWS, Azure) and infrastructure-as-code/configuration management tools (Terraform, Ansi...

As a Mobile Test Automation Developer, your main responsibilities will include: - Designing and developing mobile test automation (iOS/Android) using Java, Appium, TestNG - Enhancing existing automation frameworks and scripts - Executing regression and automated test plans - Defining testing scope for stories and features - Collaborating across teams and reporting defects - Advocating CI/CD and testing best practices - Ensuring high-quality releases and test coverage To be successful in this role, you must have the following qualifications: - Minimum of 2 years of experience in mobile test automation using Appium, TestNG, and Java - Experience with XCUI Test, Selenium WebDriver, Xcode Instru...

As a Data Engineer Offshore at PwC, you will be part of the managed services team focusing on outsourced solutions and supporting clients across various functions. Your role will involve streamlining operations, reducing costs, and improving efficiency for organizations by managing key processes and functions on their behalf. Utilizing your skills in project management, technology, and process optimization, you will deliver high-quality services to clients. Your main responsibilities will include transitioning and running services, managing delivery teams, programs, commercials, performance, and delivery risk. Continuous improvement and optimization of managed services processes, tools, and ...

Job Purpose and Impact The Professional, Data Engineering job designs, builds and maintains moderately complex data systems that enable data analysis and reporting. With limited supervision, this job collaborates to ensure that large sets of data are efficiently processed and made accessible for decision making. Key Accountabilities DATA & ANALYTICAL SOLUTIONS : Develops moderately complex data products and solutions using advanced data engineering and cloud based technologies, ensuring they are designed and built to be scalable, sustainable and robust. DATA PIPELINES: Maintains and supports the development of streaming and batch data pipelines that facilitate the seamless ingestion of data ...

Technical / Behavioral - You must have proven track record of working in collaborative teams to deliver high quality data solutions in a multi-developer agile environment following design & coding best practices. You must be an expert in using SQL queries with UNIX shell /Python scripting skills. You should have experience of using AWS services like EC2, S3, EMR to move data onto cloud platform and experience with Enterprise data lake strategy and use of Snowflake (AWS) is a big plus. You must have knowledge to develop ELT/ETL pipelines to move data to and from Snowflake data store using combination of Python and Snowflake SnowSQL is a big plus. You should have experience in SQL performance ...

The Value You Deliver Communication Analytics to provide end-to-end insight for all multi-channel communications sent via Martech owned products. The BI capability include: Operational reporting/visualization across channels and BUs Rationalize and port relevant reporting capability Legacy Marketing communications App Insights & reporting dashboards Out of the box reporting capability Ad hoc/Self Service capability Export functionality of reporting data for further analysis by individual teams The Skills that are Key to this role Technical Experience in Reporting tool like Tableau/PowerBI/Qlik Hands-on experience in Database development , data analysis , building data pipelines , data wrangl...

Key Responsibilities: Develop, configure, and optimize Apache DolphinScheduler workflows for data processing and automation. Design and implement ETL pipelines, job scheduling, and workflow orchestration solutions. Troubleshoot and resolve performance, scalability, and reliability issues in DolphinScheduler. Integrate DolphinScheduler with big data tools such as Hadoop, Spark, Flink, Hive, and Kafka. Work closely with data engineers, DevOps, and software teams to enhance workflow automation. Develop custom plugins and extensions for DolphinScheduler as needed. Monitor and optimize job execution, resource allocation, and workflow efficiency. Maintain best practices for CI/CD, version control,...

Role & responsibilities Job DescriptionJob Description ommon Skils - SQL, GCP BQ, ETL pipelines using Python/Airflow,Experience on Spark/Hive/HDFS,Data modeling for Data conversion Resources (4) Prior experience working on a conv/migration HR project is additional skill needed along with above mentioned skills Common Skils - SQL, GCP BQ, ETL pipelines using Pythin/Airflow, Experience on Spark/Hive/HDFS, Data modeling for Data conversion Resources (4) Prior experience working on a conv/migration HR project is additional skill needed along with above mentioned skills Data Engineer - Knows HR Knowledge , all other requirement from Functional Area given by UBER

Responsibilities: * Design, develop, and implement performant ETL pipelines using PySpark on AWS EMR * Integrate data storage solutions (especially AWS S3) with Spark * Performance tune Spark jobs and troubleshoot issues

Role Overview: As a member of the Gen AI team at BambooBox, you will play a pivotal role in building a set of intelligent agents that assist marketing teams in their day-to-day operations. Your responsibilities will include: Key Responsibilities: - Designing and integrating next-gen AI features, such as LLM-powered chatbots and RAG-based services - Developing backend systems using Node.js and Python - Creating ETL pipelines and deploying APIs and cloud functions - Collaborating closely with ML engineers and product teams to translate generative AI concepts into tangible SaaS features Qualifications Required: - Minimum 4 years of experience in developing products with a strong background in L...

Job Description: As a Qualcomm Software Engineer, you will be responsible for designing, developing, creating, modifying, and validating embedded and cloud edge software, applications, and specialized utility programs to launch cutting-edge, world-class products exceeding customer needs. You will collaborate with various teams to design system-level software solutions and gather performance requirements and interfaces. Key Responsibilities: - Design and implement robust ETL pipelines and data integration workflows using SQL, NoSQL, and big data technologies for Data Engineering & Infrastructure. - Build, fine-tune, and deploy ML/NLP models using frameworks like TensorFlow, PyTorch, and Sciki...

Primary Responsibilities: Lead all phases of data engineering, including requirements analysis, data modeling, pipeline design, development, and testing Design and implement performance and operational enhancements for scalable data systems Develop reusable data components, frameworks, and patterns to accelerate team productivity and innovation Conduct code reviews and provide feedback aligned with data engineering best practices and performance optimization Ensure data solutions meet standards for quality, scalability, security, and maintainability through rigorous design and code reviews Actively participate in Agile/Scrum ceremonies to deliver high-quality data solutions Collaborate with ...

Job Description Summary As Technical Product Manager for our Data Products, you will join our GridOS Data Fabric product management team who are delivering solutions designed to accelerate decarbonization by managing DERs at scale and proactively manage disruptions from climate change. Specifically, you will be accountable for managing technical product lifecycle activities around our core Data Products in partnership with our own & partner development teams to build trusted data products for our GridOS ADMS applications. Job Description Roles and Responsibilities Technical product management, responsible for delivering Data Products in partnership with both GE Vernova & partner development ...

Title: Senior Databricks Architect Location: Mumbai, Gurgaon, Hyderabad About the Role We are seeking a Senior Databricks Architect to lead the design and implementation of scalable, secure, and cost-optimized data platforms on Azure using Databricks. You will own end-to-end architecture for data ingestion, transformation, governance, and analytics, and mentor engineering teams to deliver high-quality data solutions. Key Responsibilities Architect and implement enterprise-grade data platforms on Azure Databricks (DLT, Workflows, Unity Catalog) with strong security, governance, and reliability. Design end-to-end data pipelines: batch and streaming (Structured Streaming), CDC, and near real-ti...

At EY, youll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you And were counting on your unique voice and perspective to help EY become even better, too Join us and build an exceptional experience for yourself, and a better working world for all, EY_ Consulting_Microsoft FabricSenior As part of our EY-DnA team, you will be responsible for designing, developing, and maintaining distributed systems using Microsoft Fabric, including One Lake, Azure Data Factory (ADF), Azure Synapse, Notebooks, Data Warehouse, and Lakehouse You will play a crucial role in architecting and implementing enterpr...

Role Overview: As a skilled Python developer at BeautifulCode in Hyderabad, your role will involve writing clean, maintainable, and well-documented Python code. You will collaborate with team members and stakeholders to understand business requirements and implement effective software solutions. Additionally, you will be responsible for building and maintaining software components with data processing and manipulation, working with cloud platforms like Google Cloud Platform for application deployment and scaling, and building ETL pipelines and data integrations. You will also play a crucial role in ensuring application security and compliance with company policies and industry regulations. K...

Responsibilities: Design, develop & maintain ETL pipelines using Snowflake, Python & Power BI. Collaborate with cross-functional teams on data projects from ideation to implementation. Eager to learn and adapt to new data technologies.

Sr. Full-Stack Developer Location: Bangalore. Experience: 6 TO 10 years Full-stack developer with expertise in building scalable web applications across front-end and back-end technologies working with data-centric systems and analytics platforms

skilled and motivated Data Engineer with hands-on experience in cloud platforms like AWS, Azure, or GCP to join our growing data team. have strong knowledge of data architecture, pipeline development, and cloud-based data processing.

As a Business Intelligence Consultant II at Allstate, you will be responsible for leveraging data and analytics to answer complex questions and influence business strategy through communication of findings to stakeholders. Your strong background in SQL, Data Modeling, Report Development, and Data visualization will be crucial in working closely with stakeholders to understand business requirements and translate them into interactive reports and analytical solutions. - Design, develop, and maintain Power BI dashboards and reports that provide actionable insights - Write efficient, optimized, and advanced SQL queries to extract and manipulate data from relational databases (e.g., Oracle, Dremi...