988 Glue Jobs - Page 13

Understands the process flow and the impact on the project module outcome. Works on coding assignments for specific technologies basis the project requirements and documentation available Debugs basic software components and identifies code defects. Focusses on building depth in project specific technologies. Expected to develop domain knowledge along with technical skills. Effectively communicate with team members, project managers and clients, as required. A proven high-performer and team-player, with the ability to take the lead on projects. Design and create S3 buckets and folder structures (raw, cleansed_data, output, script, temp-dir, spark-ui) Develop AWS Lambda functions (Python/Boto...

As a Technical Lead - Cloud Data Platform (AWS) at Incedo, you will be responsible for designing, deploying and maintaining cloud-based data platforms on the AWS platform. You will work with data engineers, data scientists and business analysts to understand business requirements and design scalable, reliable and cost-effective solutions that meet those requirements. Roles & Responsibilities: Designing, developing and deploying cloud-based data platforms using Amazon Web Services (AWS) Integrating and processing large amounts of structured and unstructured data from various sources Implementing and optimizing ETL processes and data pipelines Developing and maintaining security and access con...

Roles & Responsibilities : • We are looking for a strong Senior Data Engineering who will be majorly responsible for designing, building and maintaining ETL/ ELT pipelines . • Integration of data from multiple sources or vendors to provide the holistic insights from data. • You are expected to build and manage Data Lake and Data warehouse solutions, design data models, create ETL processes, implementing data quality mechanisms etc. • Perform EDA (exploratory data analysis) required to troubleshoot data related issues and assist in the resolution of data issues. • Should have experience in client interaction oral and written. • Experience in mentoring juniors and providing required guidance t...

About The Role Project Role : Application Lead Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills : Amazon Web Services (AWS) Good to have skills : NAMinimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Lead, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. Your typical day will involve collaborating with various teams to ensure that application development aligns with business objectives, overseeing project timelines, and facilitating communication among stakehold...

AWS-Specific Skills AWS Storage & Compute Services S3: Deep understanding of bucket policies, lifecycle rules, and data organization Glue: For ETL jobs, crawlers, and data cataloging Athena: Querying data directly from S3 using SQL EMR: For big data processing using Spark/Hadoop Lakehouse Technologies Apache Hudi: Knowledge of transactional data lakes, schema evolution, etc Data Cataloging & Governance AWS Glue Data Catalog Integration with Lake Formation Security & Compliance IAM roles and policies Encryption Data masking and tokenization strategies Mandatory Competencies Cloud - AWS - AWS S3, S3 glacier, AWS EBS Big Data - Big Data - Hadoop Big Data - Big Data - SPARK Database - Database P...

Key Responsibilities: Design and implement automated test strategies for validating ETL pipelines and data integration workflows. Develop automated test scripts using industry-standard tools/frameworks to validate data ingestion, transformation, and delivery. Test data flows involving Kafka topics, ensuring data consistency and schema validation across producers/consumers. Validate data correctness between source systems (DB2) and downstream targets (e.g., S3, Redshift, etc.). Perform regression, performance, and functional testing of data-centric applications and APIs. Build and maintain reusable automation test cases and frameworks aligned with DevOps and CI/CD. Collaborate closely with ET...

Tech Lead - Azure/Snowflake & AWS Migration Key Responsibilities - Design and develop scalable data pipelines using Snowflake as the primary data platform, integrating with tools like Azure Data Factory, Synapse Analytics, and AWS services. - Build robust, efficient SQL and Python-based data transformations for cleansing, enrichment, and integration of large-scale datasets. - Lead migration initiatives from AWS-based data platforms to a Snowflake-centered architecture, including: o Rebuilding AWS Glue pipelines in Azure Data Factory or using Snowflake-native ELT approaches. o Migrating EMR Spark jobs to Snowflake SQL or Python-based pipelines. o Migrating Redshift workloads to Snowflake with...

The ideal candidate for the Data QA position should have 6 to 10 years of experience in ETL+ DWH Testing and Cloud Testing with expertise in AWS (S3 & Glue) and Azure, along with proficiency in SQL Server, including writing SQL queries. Additionally, familiarity with Snowflake, ADF, and API Testing is essential. Key technical skills required for this role include a strong understanding of Boomi or similar integration tools like MuleSoft, Informatica, Talend, or Workato, experience with Data Bricks, in-depth knowledge of ETL Testing, proficiency in writing SQL Queries, and good technical documentation skills. The candidate should also possess strong communication skills to effectively interac...

A career in our Advisory Acceleration Centre is the natural extension of PwC's leading class global delivery capabilities. We provide premium, cost-effective, high-quality services that support process,

You should have a minimum of 3 years of experience in data engineering, with a specific emphasis on AWS technologies. Your expertise should include proficiency in Python, SQL, PySpark, and data modeling. Additionally, you must have hands-on experience working with AWS services like Glue, Redshift, Lambda, S3, and Airflow. It is preferred that you have prior experience dealing with data sourced from energy systems, smart grids, or industrial IoT platforms. Strong problem-solving skills and keen attention to detail are essential for this role. Moreover, excellent communication and collaboration abilities will be highly beneficial in effectively working with the team.,

We are seeking a Data Engineer Intern or Trainee with the following key skills: - Proficient in SQL Database tuning and performance optimization - Experience with Airflow implementation using Python or Scala - Strong knowledge of Python and PySpark - Familiarity with AWS Redshift, Snowflake, or Databricks for data warehousing - Ability to work with ETL services in AWS such as EMR, GLUE, S3, Redshift, or similar services in GCP or Azure. This opportunity is open to both freshers and individuals with up to 1 year of experience. Comprehensive on-the-job training will be provided for freshers. Candidates with a B.Tech background and no prior IT experience are also encouraged to apply. Job Types:...

As a Data Engineer - Pyspark at Barclays, you will play a pivotal role in advancing our infrastructure and deployment pipelines, fostering innovation and operational excellence. Your responsibility will include leveraging state-of-the-art technology to develop and manage robust, scalable, and secure infrastructure, ensuring the seamless delivery of our digital solutions. To excel in this role, you should possess expertise in PySpark and Python, along with a strong foundation in SQL. You must exhibit proficiency in writing and debugging code, coupled with being a quick learner with exceptional analytical and problem-solving skills. Effective written and verbal communication skills are essenti...

As a Project Engineer at one of the top Australian banks, you will play a key role in modernizing the data and analytics platform. Working closely with IT and business stakeholders in the Data and Platform team, you will contribute to implementing the bank's data strategy to position it as the best AI bank worldwide. Your responsibilities will include creating technology blueprints and engineering roadmaps for multi-year data transformational programs. You should have a strong understanding of architectural patterns, cloud-native solutions on AWS, microservices architecture, solutions integration, and containerization. Extensive knowledge of AWS services, especially those related to data sto...

You have a fantastic opportunity to join us as a DAML Presales Engineer with a rich experience of 10-16 years. In this role, you will serve as a trusted advisor to our customers, guiding them towards effective DAML-based solutions integrated seamlessly with modern AWS data architectures. Your expertise in AWS data analytics platforms will be instrumental in solving intricate business challenges. Your main responsibilities will include collaborating closely with sales teams and AWS stakeholders to gain insights into client issues and crafting innovative DAML-based smart contract solutions that work harmoniously with AWS Data Lake and analytics services. You will be expected to deliver impactf...

As a Principal Architect - Data & Cloud at Quantiphi, you will bring your 14-20 years of experience in Technical, Solutioning, and Analytical roles to lead the way in architecting, designing, and implementing end-to-end data pipelines and data integration solutions for structured and unstructured data sources and targets. With a focus on Cloud platforms such as GCP, AWS, and Azure, you will be responsible for building and managing Data Lakes, Data Warehouse, Data Integration, Data Migration, and Business Intelligence/Artificial Intelligence solutions. Your role will involve understanding business requirements and translating them into functional and non-functional areas, defining boundaries ...

Join a dynamic leader in the cloud data engineering sector, specializing in advanced data solutions and real-time analytics for enterprise clients. This role offers an on-site opportunity in India to work on cutting-edge AWS infrastructures where innovation is at the forefront of business transformation. The ideal candidate for this role is a professional with 4+ years of proven experience in AWS data engineering, Python, and PySpark. You will play a crucial role in designing, optimizing, and maintaining scalable data pipelines that drive business intelligence and operational efficiency. As part of your responsibilities, you will design, develop, and maintain robust AWS-based data pipelines ...

As a Data Engineer (AWS) at our company, you will be utilizing your 3-5 years of experience, with a minimum of 3 years specifically focused on AWS cloud services. Your main responsibilities will revolve around providing technical support for data engineering systems, including troubleshooting issues related to data pipelines, AWS services, Snowflake, Hadoop, and Spark. Your day-to-day tasks will include investigating and resolving issues in data pipelines, optimizing Spark jobs, managing incidents related to data processing, and collaborating with cross-functional teams to address critical issues promptly. You will also need to possess a solid understanding of big data architectures such as ...

The job of an AWS & DevOps Developer involves utilizing their sound knowledge of AWS to create and deploy applications. Prior experience in AWS applications is essential for this role. Strong proficiency in DevOps practices, Cloud formation, and Python programming is also required. The ideal candidate should have experience in AWS data analytics tools such as EMR and Glue. Additionally, being comfortable with data engineering tasks using pyspark is a key requirement. A good understanding of AWS IAM, networking, security, and API gateway is necessary for this role.,

Join us as a Data Engineer responsible for supporting the successful delivery of Location Strategy projects to plan, budget, agreed quality and governance standards. You'll spearhead the evolution of our digital landscape, driving innovation and excellence. You will harness cutting-edge technology to revolutionise our digital offerings, ensuring unparalleled customer experiences. To be successful as a Data Engineer, you should have experience with hands-on experience in pyspark and strong knowledge of Dataframes, RDD, and SparkSQL. Additionally, experience in developing, testing, and maintaining applications on AWS Cloud is crucial. A strong hold on the AWS Data Analytics Technology Stack (G...

At Lilly, the global healthcare leader headquartered in Indianapolis, Indiana, we are dedicated to uniting caring with discovery to make life better for people around the world. Our 35,000 employees are committed to discovering and delivering life-changing medicines, improving disease management, and contributing to our communities through philanthropy and volunteerism. At Lilly, we prioritize putting people first and giving our best effort to our work. The LCCI Tech Team is currently seeking a motivated leader for the Enterprise Data Organization. This individual will play a crucial role in optimizing data extraction, enhancing data infrastructure, and aligning operations with organizationa...

Job description Requires deep knowledge and hands-on experience with Terraform Requires in-depth knowledge and hands-on experience with AWS services like S3 bucket, Ec2 instance, load balancer, IAM role, Athena, IAM policy, Secret Manager, Parameter Store, lambda, and AWS Code Build Must have knowledge of at least one CICD tool like Jenkins Basic knowledge in AWS Data services like Redshift, MySQL, Glue, Athena, S3, EBS is required Experience in infrastructure support of any data analytic tool like Databricks, EMR, or Glue is desirable Experience in infrastructure support of any data lake or data warehouse tool is a plus.

Join us as a Cloud Data Engineer (AWS) at Barclays, where you will be responsible for supporting the successful delivery of location strategy projects to plan, budget, agreed quality, and governance standards. You'll spearhead the evolution of our digital landscape, driving innovation and excellence. You will harness cutting-edge technology to revolutionize our digital offerings, ensuring unparalleled customer experiences. To be successful as a Cloud Data Engineer (AWS) you should have experience with AWS cloud services such as S3, Glue, Athena, Lake Formation, CloudFormation, etc. You should also have a strong SQL knowledge, proficiency in PySpark, a very good understanding of writing and d...

The AVP, Principal Product Engineer at Synchrony plays a pivotal role in modernizing workloads by leading vendor refactoring efforts, executing break-fix tasks, and devising user enablement strategies. This role requires a profound understanding of AWS analytics services such as EMR Studio, S3, Redshift, Glue, and Tableau, coupled with robust skills in user engagement, training development, and change management. Collaboration with vendors, business users, and cloud engineering teams is essential to refactor legacy code, ensure seamless execution of fixes, and create comprehensive training materials and user job aids. Additionally, as the Principal Product Engineer, overseeing user testing, ...

As a DAML Presales Engineer with 10-16 years of experience, your primary role will be to act as a trusted advisor to customers, advocating for DAML-based solutions integrated with modern AWS data architectures. Your responsibilities will include collaborating with sales teams and AWS stakeholders to design smart contract solutions using DAML and AWS Data Lake and analytics services. You will be expected to deliver compelling demos, PoCs, and technical presentations showcasing DAML solutions and AWS data platforms like Glue, EMR, and Redshift. Your expertise in AWS data lake components such as Glue, EMR, and Redshift will be crucial in designing solutions that balance performance, cost, and r...

As a QA Engineer at Capco, a Wipro company, you will play a vital role in ensuring the quality and reliability of ETL pipelines and data integration workflows. With a focus on automation and industry-standard tools, you will be responsible for designing and implementing automated test strategies to validate data ingestion, transformation, and delivery processes. Your expertise will be crucial in performing regression, performance, and functional testing of data-centric applications and APIs. With 6+ years of experience in QA Engineering, you will leverage your hands-on experience with Apache Kafka and proficiency in validating data pipelines on AWS (S3, Lambda, Glue, Redshift, or EMR). Your ...