122 Etlelt Pipelines Jobs - Page 2

Role Overview: As a Data and Technical Architect for Salesforce Data Cloud at Horizontal Digital, you will be a demonstrated leader in technical and data architecture aspects of customer and partner engagements that lead to the successful delivery of Data Cloud Projects. In this key leadership role, you will play a critical role in setting customers up for success by prescriptively helping to shape and then lead the project teams within the Salesforce Data Cloud space. Your main responsibilities will include collaborating with stakeholders to define technical vision, driving solution and data architecture, and ensuring seamless integration of Salesforce Data Cloud with enterprise systems, al...

You will be responsible for architecting and implementing scalable offline data pipelines for manufacturing systems, including AMHS, MES, SCADA, PLCs, vision systems, and sensor data. You will design and optimize ETL/ELT workflows using Python, Spark, SQL, and orchestration tools like Airflow to transform raw data into actionable insights. Leading database design and performance tuning across SQL and NoSQL systems will be part of your responsibilities, optimizing schema design, queries, and indexing strategies for manufacturing data. It will be essential for you to enforce robust data governance by implementing data quality checks, lineage tracking, access controls, security measures, and re...

As the Head of Data at our company, you will have the responsibility to oversee and manage the data processes to ensure efficiency and accuracy. Your role will involve the following key responsibilities: - Design, develop, and optimize ETL/ELT pipelines for structured and unstructured data processing. - Maintain a modern data warehouse/lakehouse architecture. - Implement data partitioning, indexing, and performance tuning strategies for improved efficiency. - Establish robust cataloging, data quality rules, validation checks, and monitoring systems with a focus on data privacy and security guidelines. - Take ownership of business metrics from data extraction to analysis, hypothesis generatio...

As a Software Engineer II - Data, your role involves contributing to the design and development of data systems, including pipelines, APIs, analytics, AI, and machine learning at scale. You will be a key member of the Data Products team, responsible for building and maintaining production-grade pipelines and platform components that drive business and product outcomes. Your focus will be on writing high-quality, scalable, and maintainable code across data pipelines, APIs, and data platform services. Key Responsibilities: - Write high-quality, scalable, and maintainable code across data pipelines, APIs, and data platform services. - Collaborate with senior engineers, analysts, and product sta...

As an experienced Data Engineer with 6+ years of experience, your role will involve: - Designing and implementing modern data lakehouse architectures (Delta Lake, or equivalent) on cloud platforms like AWS, Azure, or GCP. - Defining data modeling, schema evolution, partitioning, and governance strategies for high-performance and secure data access. - Owning the technical roadmap for scalable data platform solutions, aligned with enterprise needs and future growth. - Providing architectural guidance and conducting code/design reviews across data engineering teams. - Building and maintaining reliable, high-throughput data pipelines for ingestion, transformation, and integration of structured, ...

As a Senior Data Engineer focusing on Azure Databricks, your role involves designing, building, and optimizing scalable data pipelines and solutions using Azure Databricks and related technologies. Your primary goal is to enable faster, data-driven decisions as part of the data transformation journey. You will be based in Nagpur, Pune, Chennai, or Bangalore. Key Responsibilities: - **Data Pipeline Development:** Design and implement robust batch and streaming data pipelines using Azure Databricks and Spark. - **Data Architecture Implementation:** Apply Medallion Architecture to structure data layers (raw, enriched, curated). - **Data Quality & Governance:** Ensure data accuracy, consistency,...

As a Data Engineer at IXLY Technologies, you will play a crucial role in building, optimizing, and maintaining scalable data pipelines and analytics solutions using modern data engineering tools and cloud technologies. You will work with Microsoft Fabric and Azure Synapse Analytics to design, develop, and maintain data pipelines and ETL processes. Your responsibilities will include implementing data models, data lakes, and data warehouses, optimizing data storage and processing performance, developing complex SQL queries, and collaborating with cross-functional teams to translate business requirements into technical solutions. Additionally, you will ensure data quality, governance, and secur...

You will be responsible for leading the Data Center of Excellence (CoE) and driving growth in Data Engineering & Analytics areas. Your key responsibilities will include: - Driving the growth of the Data CoE by developing capabilities and differentiators in data architecture, data engineering, data analytics, data quality, and governance. - Scaling the Data Engineering practice by establishing foundational standards, hiring top talent, and building NextGen data solutions, methodologies, and frameworks. - Defining and executing data modernization strategies to enable real-time analytics, AI/ML adoption, and enterprise-wide digital transformation. - Designing and delivering modern data platform...

Role Overview: Join an exciting initiative with our customer's Marketing Performance Team, where your engineering expertise will play a key role in building a next-generation Marketing Analytics Platform. This ecosystem will consolidate marketing data across various formats to drive intelligent decision-making and deliver AI-powered insights. We are seeking a seasoned AdTech Engineer with strong data infrastructure experience and a passion for performance-driven marketing technology. Key Responsibilities: - Data Integration & Transformation: Design and build robust ETL/ELT pipelines to unify data across disparate AdTech platforms and marketing tools. - Platform Development: Contribute to bui...

As an Industrial Software Developer at AVEVA, you will be responsible for the following key tasks: - Commercial Data Intelligence: You will play a crucial role in delivering and advancing the commercial data strategy to ensure a single version of the truth for reporting, analytics, and AI/ML initiatives. This will involve contributing to data warehouse and data lake initiatives to enable advanced analytics and AI-ready datasets. - Data Engineering Pipelines: Your responsibilities will include designing and maintaining ETL/ELT pipelines using Microsoft-based tools (e.g., Data Factory, Synapse, Fabric pipelines) to ensure data quality, integrity, and timeliness. You will also be driving the ad...

As a highly skilled Software Engineer with strong Python expertise and a solid understanding of data engineering principles, you will join the team to develop and optimize scalable applications and data workflows, integrating diverse data sources, and supporting the development of data-driven products. Your role will involve hands-on experience in software development, data modeling, ETL/ELT pipelines, APIs, and cloud-based data systems. **Key Responsibilities:** - Design, develop, and maintain Python-based applications, APIs, and microservices focusing on performance, scalability, and reliability. - Write clean, modular, and testable code following best software engineering practices. - Par...

As an Azure Databricks+ MS Fabrics+ Python Data Engineer at PWC AC, you will play a crucial role in designing and implementing scalable ETL/ELT pipelines using Databricks and Apache Spark. Your responsibilities will include: - Proficiency with Azure Data Services, Databricks, and Snowflake - Deep understanding of traditional and modern data architecture and processing concepts - Hands-on experience with Snowflake utilities, SnowSQL, SnowPipe, ETL data pipelines, and Big Data model techniques using Python / Java - Building stream-processing systems using solutions such as Storm or Spark-Streaming - Experience with Big Data ML toolkits like Mahout, SparkML, or H2O - Knowledge in Python and cer...

As a Technical Lead-AI ML Expert at ZF, you will play a crucial role in shaping the future of mobility by leveraging your expertise in artificial intelligence and machine learning. Here is what you can expect in this role: **Role Overview:** You will lead the development process by owning the design, implementation, deployment, and maintenance of traditional ML and Generative AI solutions. Additionally, you will translate business requirements into data-driven and GenAI-driven use cases, scope features, estimates, and timelines. Your role will also involve mentoring and coaching junior/mid-level engineers on best practices in ML, MLOps, and GenAI. **Key Responsibilities:** - Lead end-to-end ...

As a Technical Architect, you will play a pivotal role in defining, designing, and overseeing the technical architecture of projects while maintaining strong hands-on coding capabilities. You will work across Frontend, Backend, DevOps, and Data Engineering domains, ensuring best practices, scalability, and efficiency in all solutions. Additionally, you will act as a mentor, guiding engineers at different levels while staying current with emerging technologies including AI/ML and LLM stacks. Key Responsibilities: - Define technical architecture and lead design discussions for scalable, secure, and high-performance applications - Provide hands-on coding and technical expertise across the full ...

As a Data Engineer at our company, you will play a crucial role in designing, developing, and maintaining scalable data infrastructure. You will collaborate with data scientists, analysts, and cross-functional teams to ensure efficient data flow and support analytics and business intelligence initiatives. Key Responsibilities: - Design, build, and maintain scalable ETL/ELT pipelines and data integration workflows. - Develop and optimize data models for analytics and reporting. - Ensure data reliability, accuracy, and security across multiple systems. - Collaborate with data scientists, analysts, and engineers to support business needs. - Manage and optimize data warehouses, data lakes, and r...

As a Data Engineer with 5+ years of experience, your role will involve working with data engineering, data analysis, and data architecture. You will be expected to demonstrate strong expertise in SQL and Python. Additionally, you should be familiar with tools such as Databricks, Snowflake, and Power BI. Experience with distributed data systems like Hadoop and Spark, as well as Cloud platforms such as Snowflake, BigQuery, and Redshift, will be crucial for this role. Key Responsibilities: - Develop and maintain ETL/ELT pipelines using technologies like Apache Spark, Airflow, and dbt. - Utilize APIs and data services for real-time and batch processing. - Ensure data governance, security, and co...

Role Overview: You will be responsible for driving the strategy, roadmap, and execution of data products and capabilities within Teradata's Vantage platform, focusing on meeting customer needs in the India market. Your role will involve defining requirements for data ingestion, storage, governance, security, and accessibility for analytics, AI/ML, and operational workloads. Your impact will be enabling customers to derive more value from their data faster and with greater trust. Key Responsibilities: - Define requirements for data ingestion, storage, governance, security, and accessibility within Teradata's Vantage platform - Prioritize data features such as data pipelines, connectors, gover...

As a skilled Node.js Developer with a strong foundation in data engineering, you will be joining the engineering team at Affle 3i Limited in Gurgaon. Your primary responsibilities will include building scalable backend systems using modern Node.js frameworks and tools, designing and maintaining robust data pipelines and integrations, and collaborating with various teams to ship end-to-end features. Key Responsibilities: - Build and maintain performant APIs and backend services using Python and FastAPI. - Develop and manage ETL/ELT pipelines, data models, schemas, and data transformation logic for analytics and operational use. - Ensure data quality, integrity, and consistency through validat...

You will be a critical part of the data engineering team, working at the intersection of software engineering, DevOps, and data analytics. Your responsibilities will include: - Designing, developing, and managing robust, scalable, and efficient ETL/ELT pipelines using Python and modern DataOps methodologies. - Implementing data quality checks, pipeline monitoring, and error handling mechanisms. - Building data solutions using cloud-native services on AWS (e.g., S3, ECS, Lambda, CloudWatch). - Containerizing applications using Docker and orchestrating using Kubernetes for scalable deployments. - Working with infrastructure-as-code tools and CI/CD pipelines to automate deployments. - Designing...

As a Senior Data Engineer at our company, you will play a crucial role in designing, developing, and optimizing scalable data pipelines and applications using Apache Spark and Databricks. Your responsibilities will include: - Designing, developing, and implementing scalable data pipelines and applications with a focus on adhering to industry best practices. - Performing in-depth performance tuning and optimization of Spark applications within the Databricks environment. - Troubleshooting complex issues related to data ingestion, transformation, and pipeline execution. - Collaborating with cross-functional teams to deliver end-to-end data solutions. - Continuously evaluating and adopting new ...

As a highly experienced GCP Data Engineer (AdTech), your role will involve leading the analytics team, building scalable data pipelines, and mentoring analysts. Your responsibilities will include: - Leading and mentoring a team of data analysts to ensure quality delivery and technical upskilling. - Designing, developing, and maintaining scalable ETL/ELT pipelines using GCP tools such as BigQuery, Dataflow, Cloud Composer, Cloud Functions, and Pub/Sub. - Ingesting and processing log-level data from platforms like Google Ad Manager, Google Analytics, DV360, and other advertising and marketing tech sources. - Building and optimizing data pipelines from diverse sources via APIs, cloud connectors...

As an experienced and highly skilled GCP Data Engineer, your role will involve designing, developing, and maintaining robust data pipelines and cloud-based solutions on Google Cloud Platform. You will collaborate closely with data analysts, architects, and stakeholders to build scalable and efficient data systems. Key Responsibilities: - Design, build, and maintain scalable ETL/ELT pipelines using GCP tools such as Cloud Dataflow, BigQuery, Cloud Composer, and Cloud Storage. - Develop and optimize data models for analytics and reporting using BigQuery. - Implement data quality, data governance, and metadata management best practices. - Collaborate with data scientists, analysts, and other en...

As a Senior Data Engineer at our company, your main role is to design, build, and maintain scalable data pipelines and architectures to support the Denials AI workflow. You will work under the guidance of the Team Lead, Data Management to ensure that the data is reliable, compliant with HIPAA, and optimized. **Key Responsibilities:** - Collaborate with the Team Lead and cross-functional teams to gather and refine data requirements for Denials AI solutions. - Design, implement, and optimize ETL/ELT pipelines using Python, Dagster, DBT, and AWS data services (Athena, Glue, SQS). - Develop and maintain data models in PostgreSQL; write efficient SQL for querying and performance tuning. - Monitor...

You will be responsible for gathering, cleaning, and analyzing large datasets from various structured and unstructured sources. Your role will involve performing statistical analysis to identify trends, patterns, and anomalies. You will define, track, and report on key performance indicators (KPIs) and business metrics. Additionally, you will translate business requirements into technical data models and reports and communicate insights clearly through visualizations and presentations to stakeholders. Automation of recurring reports and ad-hoc analysis using SQL, Python, or Excel will also be part of your responsibilities. - Gather, clean, and analyze large datasets from various sources - Pe...

As a Palantir Data Engineer Senior Associate at PwC, your role involves architecting, developing, and optimizing end-to-end data solutions on Palantir platforms, while mentoring junior teammates. You will have a voice in designing, building, and operating the next generation of software and services leveraging the power of Palantir. Your work will focus on transitioning and running services, managing delivery teams, programs, commercials, performance, and delivery risk. Your responsibilities also include continuous improvement and optimization of managed services processes, tools, and services. Key Responsibilities: - Design and maintain Foundry ontologies, datasets, and code workbooks on Pa...