924 Elt Jobs - Page 5

Technical / Behavioral - You must have proven track record of working in collaborative teams to deliver high quality data solutions in a multi-developer agile environment following design & coding best practices. You must be an expert in using SQL queries with UNIX shell /Python scripting skills. You should have experience of using AWS services like EC2, S3, EMR to move data onto cloud platform and experience with Enterprise data lake strategy and use of Snowflake (AWS) is a big plus. You must have knowledge to develop ELT/ETL pipelines to move data to and from Snowflake data store using combination of Python and Snowflake SnowSQL is a big plus. You should have experience in SQL performance ...

6+ years of experience in Data Engineer to design and implement scalable data solutions. Deep expertise in cloud data warehousing, ETL/ELT processes, data modeling, and business intelligence. Support design and implementation of end-to-end data solutions leveraging AWS Redshift, Apache Airflow, dbt, and other modern data tools including DataBricks. Experience in Azure Data Stack can also be considered. Develop data models and implement data pipelines to ingest, transform, and load data from various sources into data warehouse. Create and maintain Apache Airflow DAGs to orchestrate complex data workflows and ETL processes. Implement data transformations and modeling using dbt to ensure data q...

Develop and maintain data pipelines, ELT processes, and workflow orchestration using Apache Airflow, Python and PySpark to ensure the efficient and reliable delivery of data. Design and implement custom connectors to facilitate the ingestion of diverse data sources into our platform, including structured and unstructured data from various document formats . Collaborate closely with cross-functional teams to gather requirements, understand data needs, and translate them into technical solutions. Implement DataOps principles and best practices to ensure robust data operations and efficient data delivery. Design and implement data CI/CD pipelines to enable automated and efficient data integrati...

Skill: AWS Glue Experience: 6 to 9 years Location: Kolkata Job description Technical Skills : AWS Glue: 3+ years of hands-on experience in AWS Glue ETL development Python/PySpark: Strong programming skills in Python and PySpark for data transformation AWS Services: Proficiency in S3, Redshift, Athena, Lambda, and EMR Data Formats: Experience with Parquet, Avro, JSON, CSV, and ORC file formats SQL: Advanced SQL skills for data querying and transformation ETL Concepts: Deep understanding of ETL/ELT design patterns and best practices Data Modeling: Knowledge of dimensional modeling, star/snowflake schemas Version Control: Experience with Git/Bitbucket for code management Preferred Skills: Exper...

Will be responsible for the design development and administration of data management solutions that align with the business needs of the organization. Will also respond to support Operations specially on production tickets and business requests for access and reporting. Overall 5+ years of experience and has at least 2-3 years of experience in Data Engineering especially ADF Python PySpark SparkSql Sql and Databricks programming and cloud data ingestion data pipelines handling structured and semi-structured data formats like JSON XML. Collaborate with internal and external personnel including senior management system architects software developers database administrators design analysts and ...

Description: Will be responsible for the design development and administration of data management solutions that align with the business needs of the organization. Design relational databases to support business enterprise applications and physical data modeling according to project requirements for data acquisition and security as well as customer-defined deliverables. Has at least 2 years of experience in Data Engineering using ADF to orchestrate and development of data pipelines speically for ingestion harmonisation and curation of the source data. This includes good hands on experience on writing Stored procedures complex views performance tuning of the SQL good underastanding and manage...

Will be responsible for the design development and administration of data management solutions that align with the business needs of the organization. Design relational databases to support business enterprise applications and physical data modeling according to project requirements for data acquisition and security as well as customer-defined deliverables. Has at least 5-6 years of relevant experience in Data Engineering especially ADF Python PySpark SparkSql Sql and Databricks programming and cloud data ingestion data pipelines handling structured and semi-structured data formats like JSON XML. Must have experience in building ETL/ELT Data warehouses/lakes for Real time event driven scenar...

We are seeking a highly skilled and experienced Azure Engineer with strong expertise in Business Intelligence (BI) and hands-on experience with Microsoft Fabric. The ideal candidate will have a solid background in cloud-based data solutions extensive experience in designing and implementing BI platforms and a passion for driving analytics and data-driven decision-making. Key Responsibilities: Azure Platform Management: Design implement and maintain scalable Azure solutions including Azure Data Lake Azure Synapse Analytics and Azure Data Factory. Manage cloud resources including compute storage and networking ensuring optimal performance and cost efficiency. Business Intelligence Solutions: D...

We are seeking a highly skilled and experienced Azure Engineer with strong expertise in Business Intelligence (BI) and hands-on experience with Microsoft Fabric. The ideal candidate will have a solid background in cloud-based data solutions extensive experience in designing and implementing BI platforms and a passion for driving analytics and data-driven decision-making. Key Responsibilities: Azure Platform Management: Design implement and maintain scalable Azure solutions including Azure Data Lake Azure Synapse Analytics and Azure Data Factory. Manage cloud resources including compute storage and networking ensuring optimal performance and cost efficiency. Business Intelligence Solutions: D...

EXP 4 to 6 yrs Location Any PSL Location Rate below 14$ JD - DBT/AWS Glue/Python/Pyspark Hands-on experience in data engineering, with expertise in DBT/AWS Glue/Python/Pyspark. Strong knowledge of data engineering concepts, data pipelines, ETL/ELT processes, and cloud data environments (AWS) Technology DBT, AWS Glue, Athena, SQL, Spark, PySpark Good understanding of Spark internals and how it works. Goot skills in PySpark Good understanding of DBT basically should be to understand DBT limitations and when it will end-up in model explosion Good hands-on experience in AWS Glue AWS expertise should know different services and should know how to configure them and infra-as-code experience Basic ...

Following skillsets are expected out of the joining candidates 1.Ability to lead a technical team and provide mentorship and leadership to junior team members 2.Experience implementing modern data pipelines using Snowflake 3.Strong experience of creating and managing all SQL Server database objects (jobs, tables, views, indices, stored procedures, UDFs, triggers, etc) 4.Experience building and supporting data sources for BI/Analytical tools like SSRS and Power BI 5.Expert in all components of SQL Server (Database Engine, SSIS, SSRS) 6.Expert in ETL and ELT processes within SQL Server or Azure7.Experience implementing and supporting Microsoft BI stack (i.e. SSIS, SSAS)

We are seeking a highly skilled Data Architect/Lead with expertise in Azure Data Systems to join our team. As a Data Architect/Lead, you will be responsible for designing and implementing scalable, high-performance data solutions using Azure data services. This role offers an exciting opportunity to work on cutting-edge projects and play a key role in shaping our data architecture strategy. Responsibilities: Design, develop, and implement scalable and robust data architectures using Azure data services such as Azure SQL Database, Azure Synapse Analytics, Azure Data Lake, and Azure Data Factory. Collaborate with cross-functional teams to gather requirements and translate them into data archit...

We are seeking a Data Integration Engineer with 58 years of experience in building and orchestrating data pipelines using Apache Airflow and integrating data into Snowflake The role involves designing and maintaining Airflow DAGs, integrating structured and unstructured data via JDBC connectors, REST APIs, and flat file ingestion The candidate should have strong hands-on experience with Postman for API validation, SQL encryption/decryption techniques for securing sensitive datasets, and Python for transformation and ETL logic Responsibilities include implementing detailed logging, monitoring, and alerting for pipeline reliability, conducting data quality checks such as row-level validation a...

We are seeking a Data Integration Engineer with 5- 8 years of experience in building and orchestrating data pipelines using Apache Airflow and integrating data into Snowflake The role involves designing and maintaining Airflow DAGs, integrating structured and unstructured data via JDBC connectors, REST APIs, and flat file ingestion The candidate should have strong hands-on experience with Postman for API validation, SQL encryption/decryption techniques for securing sensitive datasets, and Python for transformation and ETL logic Responsibilities include implementing detailed logging, monitoring, and alerting for pipeline reliability, conducting data quality checks such as row-level validation...

As a skilled Data Architect, you will tackle intricate data challenges from various structured and unstructured sources. Your expertise in crafting micro data lakes and spearheading data strategies at an enterprise level will be crucial for this role. You will design and construct tailored micro data lakes specifically catered to the lending domain, defining and executing enterprise data strategies encompassing modeling, lineage, and governance. Your responsibilities will include architecting robust data pipelines for both batch and real-time data ingestion, as well as devising strategies for extracting, transforming, and storing data from diverse sources like APIs, PDFs, logs, and databases...

Project description We are looking for a skilled Data Engineer with strong experience in AWS and Apache Airflow to join a dynamic data team. You will be responsible for building and maintaining scalable data pipelines, orchestrating workflows, and ensuring data quality and availability across platforms. Responsibilities ?Data Pipeline DevelopmentDesign, build, and maintain robust, scalable, and efficient ETL/ELT pipelines for ingesting, transforming, and loading data from various sources into our data lake and data warehouse. ?AWS ExpertiseDevelop and manage data solutions using a wide range of AWS services, including but not limited to ?StorageS3 (for data lakes), RDS (PostgreSQL, MySQL), R...

About The Role Project Role : Data Platform Engineer Project Role Description : Assists with the data platform blueprint and design, encompassing the relevant data platform components. Collaborates with the Integration Architects and Data Architects to ensure cohesive integration between systems and data models. Must have skills : Data Modeling Techniques and Methodologies Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Platform Engineer, you will assist with the data platform blueprint and design, encompassing the relevant data platform components. Your typical day will involve collaborating wit...

As a Data Engineering Manager within the Enterprise Data Platform team at PitchBook, you will lead a team of skilled data engineers responsible for building and optimizing data pipelines, managing large-scale data models, and ensuring the quality, availability, and performance of enterprise data assets. This leadership role combines technical depth, strategic thinking, and people management to enable PitchBook’s data-driven decision-making and analytics capabilities. You’ll collaborate closely with cross-functional partners across Technology & Engineering, Product, Sales, Marketing, Research, Finance, and Administration to deliver robust, scalable data solutions. The ideal candidate is a han...

Role Overview: As a Senior Data Engineer at iSOCRATES, you will have a pivotal role in uncovering patterns in data sets and creating algorithms to enhance the usability of raw data for the organization. Your position necessitates a broad array of technical competencies, encompassing a profound understanding of SQL database design, proficiency in multiple programming languages, and adeptness in team management. Your primary responsibilities will include devising and implementing production data pipelines within a big data framework, managing data migration initiatives, crafting clean code following Test-Driven Development (TDD) principles, and innovating contemporary data architecture strateg...

Factspan Overview: Factspan is a pure play data and analytics services organization. We partner with fortune 500 enterprises to build an analytics center of excellence, generating insights and solutions from raw data to solve business challenges, make strategic recommendations and implement new processes that help them succeed. With offices in Seattle, Washington and Bengaluru, India; we use a global delivery model to service our customers. Our customers include industry leaders from Retail, Financial Services, Hospitality, and technology sectors. Work Timing: 2.00 PM IST to 10.00 PM IST Position Summary: We are seeking a skilled Data Engineer with 7+ years of experience to join our growing ...

Perform end-to-end reconciliation between source systems, canonical entities (Order, Shipment, Freight Invoicing, Cost Allocation, Claims, Complaints), and dashboard outputs. Ensure data lineage traceability from the Silver layer to the dashboard metrics. Collaborate with ETL Developers, Data Architects, and Product Owners to resolve discrepancies. Document validation results, maintain defect logs, and provide QA sign-off for data releases. Support automation of validation processes using SQL, Python (Pandas), or other frameworks. Participate in sprint ceremonies, provide QA estimates, and ensure data readiness for UAT (if applicable Skill Strong SQL skills for data validation, reconciliatio...

Job Title: Engineer Work Location: ~BANGALORE~ Skill Required - Digital : Amazon Web Service(AWS) Cloud Computing Experience Range - 6 to 8 Job Description Key Accountabilities: • Experience building data pipelines for various heterogenous data sources. • Identifying, designing and implementing scalable data delivery pipelines and automating manual processes • Building required infrastructure for optimal data extraction, transformation and loading of data using cloud technologies like AWS, Azure etc., • Develop end to end processes on the enterprise level for use by the clinical data configuration specialist to prepare data extraction and transformations of raw data quickly and efficiently f...

Job Description: Data Engineer Databricks Platform Position Overview We are looking for an experienced Data Engineer to design and implement a scalable data platform using Databricks . The ideal candidate will have strong expertise in building modern data architectures, integrating diverse data sources (SQL Server, MongoDB, InfluxDB), and enabling analytics and reporting use cases. Key Responsibilities Design, build, and maintain a robust data platform on Databricks. Develop efficient ETL/ELT pipelines to ingest data from SQL Server, MongoDB, and InfluxDB into Databricks Delta Lake. Implement real-time and batch data ingestion strategies using tools such as Kafka, Azure Event Hubs, or simila...

About Zscaler Serving thousands of enterprise customers around the world including 45% of Fortune 500 companies, Zscaler (NASDAQ: ZS) was founded in 2007 with a mission to make the cloud a safe place to do business and a more enjoyable experience for enterprise users. As the operator of the world’s largest security cloud, Zscaler accelerates digital transformation so enterprises can be more agile, efficient, resilient, and secure. The pioneering, AI-powered Zscaler Zero Trust Exchange™ platform, which is found in our SASE and SSE offerings, protects thousands of enterprise customers from cyberattacks and data loss by securely connecting users, devices, and applications in any location. Named...

About The Role Project Role : Data Architect Project Role Description : Define the data requirements and structure for the application. Model and design the application data structure, storage and integration. Must have skills : Google Cloud Platform Architecture Good to have skills : NA Minimum 15 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Architect, you will design and deliver end-to-end data architecture solutions for platforms, products, or engagements on Google Cloud. You will define architectures that meet performance, scalability, security, and compliance requirements while ensuring data integrity and accessibility. Yo...