1330 Aws Glue Jobs - Page 11

Skills: 3-5 years experience on AWS Glue, Python and Snowflake. Technical Skills - AWS Glue: Proficiency in developing and managing Spark-based Glue jobs. - Snowflake: Strong experience in Snowflake data warehousing, including schema design, query optimization, and performance tuning. - Kafka: Hands-on experience with Kafka for real-time data streaming and integration. - AWS Lambda: Experience in building serverless functions for event-driven workflows. - Programming: Proficiency in Python - SQL: Advanced SQL skills for data transformation and analysis. - Cloud Infrastructure: Familiarity with AWS services such as S3, CloudWatch, SES and IAM. - CI/CD: Familiarity with Terraform and Git actio...

Roles and Responsibilities Design, develop, and maintain data governance frameworks, policies, and procedures to ensure compliance with regulatory requirements. Collaborate with stakeholders to identify business needs and implement solutions using AWS Glue, Pyspark, SQL, and other tools. Develop automated data quality checks using AWS services such as S3 Bucket Policies and Lambda Functions. Ensure adherence to company standards for data management best practices through regular audits and monitoring. Provide training and support to team members on new technologies and processes. Desired Candidate Profile 4-7 years of experience in Data Management or related field (Data Engineering). B.Tech/...

Role Overview: At EY, you'll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture, and technology to become the best version of you. We are counting on your unique voice and perspective to help EY become even better. Join us to build an exceptional experience for yourself and contribute to creating a better working world for all. Key Responsibilities: - Develop, deploy, and monitor machine learning models in production environments. - Automate ML pipelines for model training, validation, and deployment. - Optimize ML model performance, scalability, and cost efficiency. - Implement CI/CD workflows for ML model versioning, testing, and deplo...

As an AWS Developer at Barclays, you will be responsible for supporting the successful delivery of Location Strategy projects within the planned budget, agreed quality, and governance standards. You will play a key role in evolving the digital landscape, driving innovation, and excellence. By harnessing cutting-edge technology, you will revolutionize digital offerings to ensure unparalleled customer experiences. - Experience with AWS cloud services such as Lambda, S3, Step Function, data analytics services like Athena, Redshift, Snowflake. - Proficiency in programming languages like Python/Pandas, Spark. - Hands-on experience in handling high volumes of data processing using AWS Glue, PySpar...

As a Data Engineer, you will be responsible for managing and optimizing data processes to ensure efficient data flow within the organization. You will utilize your 4+ years of experience to design and implement data solutions using SQL. Working hands-on with modern cloud data warehousing solutions such as Snowflake, Big Query, and Redshift is a key part of your role. Additionally, you will demonstrate expertise in ETL/ELT processes for batch and streaming data processing, troubleshoot data issues effectively, and propose solutions for data optimization. You will also apply your knowledge of AWS services including S3, DMS, Glue, and Athena. Familiarizing yourself with DBT for data transformat...

Roles and responsibilities Design AWS architectures based on business requirements. Create architectural diagrams and documentation. Present cloud solutions to stakeholders. Skills and Qualifications: Strong knowledge of AWS services: EC2, S3, RDS, Lambda, IAM, VPC, CloudFormation, CloudWatch, etc. Experience with serverless architectures (AWS Lambda, API Gateway, Step Functions). Experience of AWS networking (VPC, Route 53, ELB, Security Groups, etc.). Experience with AWS CloudFormation for automating infrastructure. Proficiency in scripting languages such as Python or Bash. Experience with automation tools (AWS Systems Manager, AWS Lambda) Experience of containerization (Docker, Kubernetes...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

About The Role Project Role : Application Lead Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills : AWS BigData Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Lead, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. Your typical day will involve collaborating with various teams to ensure project milestones are met, facilitating discussions to address challenges, and guiding your team in implementing effective solutions. You will also en...

About The Role Project Role : Application Lead Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills : AWS BigData Good to have skills : NA Minimum 7.5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Lead, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. Your typical day will involve collaborating with various teams to ensure project milestones are met, facilitating discussions to address challenges, and guiding your team in implementing effective solutions. You will also ...

About The Role Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : AWS Glue Good to have skills : Microsoft SQL Server, Python (Programming Language), Data Engineering Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education About The Role :Developing a customer insights platform that will provide an ID graph and Digital customer view to help drive improvements in marketing decisions. Responsibilities:Design, build, and maintain data pipelines using AWS services (Glue, Neptune, S3).Participate in code reviews, testing, and ...

Job Summary Seeking an experienced ETL Developer to design, build, and maintain data pipelines and integrations across cloud-native environments. The role focuses on developing robust ETL processes, ensuring data quality, performance, and scalability using AWS services and Python. Priority Skills ETL knowledge AWS tech stack (AWS Lambda, AWS S3, AWS Terraform) AWS Glue Python Secondary Skills (Good to Have) GraphQL PostgreSQL GitHub Actions Location: Chennai Interested? Send your updated CV to VimalaP3@hexaware.com If this excites you, drop in your details through the form below https://lnkd.in/daMht8-A . Key Responsibilities Design, develop, test, and deploy ETL pipelines to ingest, transfo...

Requirements for the candidate: Data Engineer with a minimum of 7+ years of experience of data engineering experience. The role will require deep knowledge of data engineering techniques to create data pipelines and build data assets. At least 4+ years of Strong hands on programming experience with Pyspark / Python / Boto3 including Python Frameworks, libraries according to python best practices. Strong experience in code optimization using spark SQL and pyspark. Understanding of Code versioning, Git repository, JFrog Artifactory. AWS Architecture knowledge specially on S3, EC2, Lambda, Redshift, CloudFormation etc and able to explain benefits of each Code Refactorization of Legacy Codebase:...

Description Are you a highly skilled data engineer and project leaderDo you think big, enjoy complexity and building solutions that scaleAre you curious to know what you could achieve in a company that pushes the boundaries of modern technologyIf you answered yes and you have a background in FinTech youll love this role and Amazons data obsessed culture, Amazon Devices and Services Fintech is the global team that designs and builds the ficial planning and analysis tools for wide variety of Amazons new and established organizations From Kindle to Ring and even new and exciting companies like Kuiper (our new interstellar satellite play) this team enjoys a wide variety of complex and interestin...

Job Title: AWS Senior Data Engineer Experience Required: Minimum 5+ years Job Summary : We are seeking a skilled Data Engineer with a strong background in data ingestion, processing, and storage. The ideal candidate will have experience working with various data sources and technologies, particularly in a cloud environment. You will be responsible for designing and implementing data pipelines, ensuring data quality, and optimizing data storage solutions. Key Responsibilities: Design, develop, and maintain scalable data pipelines for data ingestion and processing using Python, Spark, and AWS services. Work with on-prem Oracle databases, batch files, and Confluent Kafka for data sourcing. Impl...

Role & responsibilities Develop and maintain data pipelines using AWS Glue , Lambda , EMR , and Snowflake . Implement real-time ingestion using Snowpipe and Streams for CDC (Change Data Capture). Write efficient PySpark or Scala Spark jobs for large-scale data processing. Automate workflows and orchestrate jobs using AWS Step Functions , Airflow , or similar tools. Optimize Snowflake queries and warehouse performance. Collaborate with Data Scientists, Analysts, and DevOps teams to deliver reliable data solutions. Monitor and troubleshoot data pipeline failures and latency issues. Preferred candidate profile Strong experience with Snowflake architecture, SQL, and performance tuning. Hands-on ...

As a skilled Data Engineer at BigThinkCode Technologies, your role involves designing, building, and maintaining robust data pipelines and infrastructure to optimize data flow and ensure scalability. Your technical expertise in Python, SQL, ETL/ELT frameworks, and cloud data warehouses, along with strong collaboration skills, will be crucial in partnering with cross-functional teams to enable seamless access to structured and unstructured data across the organization. **Key Responsibilities:** - Design, develop, and maintain scalable ETL/ELT pipelines for structured and unstructured data processing. - Optimize and manage SQL queries for performance and efficiency in handling large-scale data...

Job Summary: We are seeking a highly skilled Data Engineer to design, build, and optimize scalable data pipelines using Snowflake, AWS services, and Apache Spark. You will be responsible for real-time and batch data ingestion, transformation, and orchestration across cloud platforms. Key Responsibilities: Develop and maintain data pipelines using AWS Glue , Lambda , EMR , and Snowflake . Implement real-time ingestion using Snowpipe and Streams for CDC (Change Data Capture). Write efficient PySpark or Scala Spark jobs for large-scale data processing. Automate workflows and orchestrate jobs using AWS Step Functions , Airflow , or similar tools. Optimize Snowflake queries and warehouse performa...

Role & responsibilities Develop and maintain data pipelines using AWS Glue , Lambda , EMR , and Snowflake . Implement real-time ingestion using Snowpipe and Streams for CDC (Change Data Capture). Write efficient PySpark or Scala Spark jobs for large-scale data processing. Automate workflows and orchestrate jobs using AWS Step Functions , Airflow , or similar tools. Optimize Snowflake queries and warehouse performance. Collaborate with Data Scientists, Analysts, and DevOps teams to deliver reliable data solutions. Monitor and troubleshoot data pipeline failures and latency issues. Required Skills: Strong experience with Snowflake architecture, SQL, and performance tuning. Hands-on expertise i...

Job seekers, We have openings for Snowflake developer, Client:Carelon Global solutions Payroll: Sp soft(C2H) NP:Immediate to 10 days Locarion: Bangalore/Gurgoan Workmode:Hybrid Key Responsibilities: Develop and maintain data pipelines using AWS Glue , Lambda , EMR , and Snowflake . Implement real-time ingestion using Snowpipe and Streams for CDC (Change Data Capture). Write efficient PySpark or Scala Spark jobs for large-scale data processing. Automate workflows and orchestrate jobs using AWS Step Functions , Airflow , or similar tools. Optimize Snowflake queries and warehouse performance. Collaborate with Data Scientists, Analysts, and DevOps teams to deliver reliable data solutions. Monito...

Job Summary: We are seeking a highly skilled Data Engineer to design, build, and optimize scalable data pipelines using Snowflake, AWS services, and Apache Spark. You will be responsible for real-time and batch data ingestion, transformation, and orchestration across cloud platforms. Key Responsibilities: Develop and maintain data pipelines using AWS Glue , Lambda , EMR , and Snowflake . Implement real-time ingestion using Snowpipe and Streams for CDC (Change Data Capture). Write efficient PySpark or Scala Spark jobs for large-scale data processing. Automate workflows and orchestrate jobs using AWS Step Functions , Airflow , or similar tools. Optimize Snowflake queries and warehouse performa...

Job Summary: We are seeking a highly skilled Data Engineer to design, build, and optimize scalable data pipelines using Snowflake, AWS services, and Apache Spark. You will be responsible for real-time and batch data ingestion, transformation, and orchestration across cloud platforms. Key Responsibilities: Develop and maintain data pipelines using AWS Glue , Lambda , EMR , and Snowflake . Implement real-time ingestion using Snowpipe and Streams for CDC (Change Data Capture). Write efficient PySpark or Scala Spark jobs for large-scale data processing. Automate workflows and orchestrate jobs using AWS Step Functions , Airflow , or similar tools. Optimize Snowflake queries and warehouse performa...

Job Title: AWS Data Engineer Company Name: NCSi Job Description: NCSi is seeking a skilled AWS Data Engineer to join our dynamic team. The ideal candidate will be responsible for designing and building robust data pipelines, developing data integration processes, and optimizing database performance in cloud environments. You will work collaboratively with data scientists, analysts, and other stakeholders to ensure data accuracy, reliability, and accessibility across the organization. Your role will also involve implementing best practices in data governance and security, as well as leveraging AWS services to support data analytics and reporting needs. Key Responsibilities: - Design, develop,...

Role Overview: You will be responsible for building and deploying the infrastructure for ingesting high-volume support data from consumer interactions, devices, and apps. Additionally, you will design and implement processes to turn data into insights, model and mine the data to describe the system's behavior, and predict future actions. Your role will involve enabling data-driven change by creating effective visualizations and reports for internal and external stakeholders. You will also develop and maintain data-related code in an automated CI/CD build/test/deploy environment, generate specific reports for various tenants, and conduct research individually and collaboratively on problem-so...

Role & responsibilities Designs and develops robust data pipelines for ingesting, transforming, and loading (ETL/ELT) data from multiple sources into analytics platforms. Mastery of SQL, Python (Pandas, NumPy, PySpark),Experience with modern ETL/ELT frameworks, expertise in Databricks/ Delta Lake / Glue / Redshift Preferred candidate profile

Develop, test, and maintain ETL pipelines from TMS source systems to canonical layers. Implement data transformation, cleansing, and validation logic. Support QA by fixing data defects and ensuring completeness of load. Mandatory Competencies ETL - ETL - Tester Beh - Communication ETL - ETL - AWS Glue Data Science and Machine Learning - Data Science and Machine Learning - Databricks