1333 Aws Glue Jobs - Page 31

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

We are looking for skilled Data Engineer to join our team. This role involves robust ETL/ELT pipelines, developing dashboards using Power BI. Work in AWS & Conduct data profiling, validation & QC. Follow best practices in data architecture. Health insurance Provident fund

AWS-Specific Skills AWS Storage & Compute Services S3: Deep understanding of bucket policies, lifecycle rules, and data organization Glue: For ETL jobs, crawlers, and data cataloging Athena: Querying data directly from S3 using SQL EMR: For big data processing using Spark/Hadoop Lakehouse Technologies Apache Hudi: Knowledge of transactional data lakes, schema evolution, etc Data Cataloging & Governance AWS Glue Data Catalog Integration with Lake Formation Security & Compliance IAM roles and policies Encryption Data masking and tokenization strategies Mandatory Competencies Cloud - AWS - AWS S3, S3 glacier, AWS EBS Big Data - Big Data - Hadoop Big Data - Big Data - SPARK Database - Database P...

Genpact (NYSE: G) is a global professional services and solutions firm delivering outcomes that shape the future. Our 125,000+ people across 30+ countries are driven by our innate curiosity, entrepreneurial agility, and desire to create lasting value for clients. Powered by our purpose the relentless pursuit of a world that works better for people – we serve and transform leading enterprises, including the Fortune Global 500, with our deep business and industry knowledge, digital operations services, and expertise in data, technology, and AI. Inviting applications for the role of Lead Consultant-Data Engineer, AWS+Python, Spark, Kafka for ETL! Responsibilities Develop, deploy, and manage ETL...

Build data pipelines using Spark, PySpark, Python. Strong in Data Lake/Warehouse, SQL, EMR, Iceberg, Parquet. Expertise in NoSQL/Relational DBs, data modeling, AWS, APIs, CI/CD (GitLab), Docker, Kubernetes, microservices, and messaging systems.

Role: AWS Glue Data Engineer Location: Chennai/Bangalore/Hyderabad Mode Of Hire: Permanent Work Mode: Work from office / Hybrid Notice Period: Immediate joiners preferred. Required Skills: 510 years of overall data engineering experience Minimum 3+ years hands-on with AWS Glue and PySpark Proficiency in SQL, Python, and AWS data tools Strong understanding of data lake architecture Good communication and problem-solving skills Experience with AWS Lake Formation Familiarity with CI/CD, Terraform, or CloudFormation AWS Certifications (Data Analytics, Solutions Architect)

Job details: Title AWS Data Engineer Type Hybrid Location Chennai Key Skills AWS Glue, RedShift, S3, Lambda Athena. Hands on experience in Data Engineer with AWS, Glue, Lambda, SQL, Python, Redshift. Must have working knowledge in designing and implementing data pipelines on any of the cloud providers (AWS is preferred). Must be able to work with large volumes of data coming from various sources. Perform data cleansing, data validation etc Hands on ETL developer who is good at python, SQL. AWS services like glue, glue crawlers, lambda, red shift, athena, s3, EC2, IAM, Monitoring and Logging mechanisms- AWS cloudwatch, setting up alerts. Deployment knowledge on cloud. Integrate CI/CD pipeline...

As a Senior Cloud Data Platform (AWS) Specialist at Incedo, you will be responsible for designing, deploying and maintaining cloud-based data platforms on the AWS platform. You will work with data engineers, data scientists and business analysts to understand business requirements and design scalable, reliable and cost-effective solutions that meet those requirements. Roles & Responsibilities: Designing, developing and deploying cloud-based data platforms using Amazon Web Services (AWS) Integrating and processing large amounts of structured and unstructured data from various sources Implementing and optimizing ETL processes and data pipelines Developing and maintaining security and access co...

Role Description As a Senior Cloud Data Platform (AWS) Specialist at Incedo, you will be responsible for designing, deploying and maintaining cloud-based data platforms on the AWS platform. You will work with data engineers, data scientists and business analysts to understand business requirements and design scalable, reliable and cost-effective solutions that meet those requirements. Roles & Responsibilities: Designing, developing and deploying cloud-based data platforms using Amazon Web Services (AWS) Integrating and processing large amounts of structured and unstructured data from various sources Implementing and optimizing ETL processes and data pipelines Developing and maintaining secur...

Role Description As a Senior Cloud Data Platform (AWS) Specialist at Incedo, you will be responsible for designing, deploying and maintaining cloud-based data platforms on the AWS platform. You will work with data engineers, data scientists and business analysts to understand business requirements and design scalable, reliable and cost-effective solutions that meet those requirements. Roles & Responsibilities: Designing, developing and deploying cloud-based data platforms using Amazon Web Services (AWS) Integrating and processing large amounts of structured and unstructured data from various sources Implementing and optimizing ETL processes and data pipelines Developing and maintaining secur...

Technical Skills Experience building data transformation pipelines using DBT and SSIS Moderate programming experience with Python Moderate experience with AWS Glue Strong experience with SQL and ability to write efficient code and manage it through GIT repositories Nice-to-have skills Experience working with SSIS Experience working in a Wealth management industry Experience in agile development methodologies

As a Technical Lead - Cloud Data Platform (AWS) at Incedo, you will be responsible for designing, deploying and maintaining cloud-based data platforms on the AWS platform. You will work with data engineers, data scientists and business analysts to understand business requirements and design scalable, reliable and cost-effective solutions that meet those requirements. Roles & Responsibilities: Designing, developing and deploying cloud-based data platforms using Amazon Web Services (AWS) Integrating and processing large amounts of structured and unstructured data from various sources Implementing and optimizing ETL processes and data pipelines Developing and maintaining security and access con...

Job Requirements Mandatory Skills Bachelors degree in computer science, Data Science, engineering, mathematics, information systems, or a related technical discipline 7+ years of relevant experience in data engineering roles Detailed knowledge of data warehouse technical architectures, data modelling, infrastructure components, ETL/ ELT and reporting/analytic tools and environments, data structures and hands-on SQL coding Proficient in at least one or more programming languages: Java, Python, Ruby, Scala Experienced with AWS services such as Redshift, S3, EC2, Lambda, Athena, EMR, AWS Glue, Datapipeline. Exposure to data visualization and reporting with tools such as Amazon QuickSight, Metab...

AWS-Specific Skills AWS Storage & Compute Services S3: Deep understanding of bucket policies, lifecycle rules, and data organization Glue: For ETL jobs, crawlers, and data cataloging Athena: Querying data directly from S3 using SQL EMR: For big data processing using Spark/Hadoop Lakehouse Technologies Apache Hudi: Knowledge of transactional data lakes, schema evolution, etc Data Cataloging & Governance AWS Glue Data Catalog Integration with Lake Formation Security & Compliance IAM roles and policies Encryption Data masking and tokenization strategies Mandatory Competencies Cloud - AWS - AWS S3, S3 glacier, AWS EBS Big Data - Big Data - Hadoop Big Data - Big Data - SPARK Database - Database P...

Roles and Responsibilities Design, develop, and maintain large-scale data pipelines using AWS services such as S3, Lambda, Step Functions, etc. Develop ETL processes using PySpark and Redshift to extract insights from NoSQL databases like DynamoDB. Ensure high availability and scalability of the data warehousing infrastructure on AWS. Troubleshoot complex issues related to data processing, storage, and retrieval. Collaborate with cross-functional teams to identify business requirements and design solutions that meet those needs.

We are seeking an experienced AWS PySpark Developer with 2-8 years of experience to design, build, and optimize our data pipelines and analytics architecture. The ideal candidate will have a strong background in data wrangling and analysis, with a deep understanding of AWS data services. Key Responsibilities: Design, build, and optimize robust data pipelines and data architecture on the AWS cloud platform. Wrangle, explore, and analyze large datasets to identify trends, answer business questions, and pinpoint areas for improvement. Develop and maintain a next-generation analytics environment, providing a self-service, centralized platform for all data-centric activities. Formulate and implem...

Job Title: AWS Data Engineer Location: Gurugram, India No of openings: Experience Required: Company: PwC India Job Description: PwC India is seeking a talented AWS Data Engineer to join our team in Gurgaon. The ideal candidate will have 2-7 years of experience with a strong focus on AWS services, data engineering, and analytics. This role offers an exciting opportunity to work on cutting-edge projects for global clients while leveraging your expertise in cloud technologies and data management. Key Responsibilities: 1. AWS Service Implementation: Design, develop, and maintain data solutions using AWS services, with a particular focus on S3, Athena, Glue, EMR, and Redshift. Implement and optim...

At PwC, the focus in data and analytics engineering is on leveraging advanced technologies and techniques to design and develop robust data solutions for clients. You play a crucial role in transforming raw data into actionable insights, enabling informed decision-making and driving business growth. In data engineering at PwC, you will concentrate on designing and building data infrastructure and systems to enable efficient data processing and analysis. Your responsibilities include developing and implementing data pipelines, data integration, and data transformation solutions. As an AWS Architect / Manager at PwC - AC, you will interact with Offshore Manager/Onsite Business Analyst to under...

The ideal candidate for the Data QA position should have 6 to 10 years of experience in ETL+ DWH Testing and Cloud Testing with expertise in AWS (S3 & Glue) and Azure, along with proficiency in SQL Server, including writing SQL queries. Additionally, familiarity with Snowflake, ADF, and API Testing is essential. Key technical skills required for this role include a strong understanding of Boomi or similar integration tools like MuleSoft, Informatica, Talend, or Workato, experience with Data Bricks, in-depth knowledge of ETL Testing, proficiency in writing SQL Queries, and good technical documentation skills. The candidate should also possess strong communication skills to effectively interac...

As an AWS Developer at PwC's Advisory Acceleration Center, you will collaborate with the Offshore Manager and Onsite Business Analyst to comprehend requirements and take charge of implementing Cloud data engineering solutions on AWS, such as Enterprise Data Lake and Data hub. With a focus on architecting and delivering scalable cloud-based enterprise data solutions, you will bring your expertise in end-to-end implementation of Cloud data engineering solutions using tools like Snowflake utilities, SnowSQL, SnowPipe, ETL data Pipelines, and Big Data model techniques using Python/Java. Your responsibilities will include loading disparate data sets, translating complex requirements into detailed...

As a Snowflake Data Engineer with 3-5 years of experience, you will be responsible for designing, developing, and optimizing cloud-based data warehousing solutions. This is an exciting opportunity to work on a flagship data initiative for a premier Big 4 consulting client, offering ample scope for technical innovation, learning, and career growth. Your key responsibilities will include: - Designing and developing high-performance data pipelines in Snowflake for data ingestion, transformation, and storage. You will focus on external tables, semi-structured data handling, and transformation logic. - Optimizing Snowflake workloads to ensure optimal query execution and cost-effective utilization...

As an AWS Developer at PwC's Acceleration Center in Bangalore, you will be responsible for the end-to-end implementation of Cloud data engineering solutions like Enterprise Data Lake and Data hub in AWS. You will collaborate with Offshore Manager/Onsite Business Analyst to understand requirements and architect scalable, distributed, cloud-based enterprise data solutions. Your role will involve hands-on experience with Snowflake utilities, SnowSQL, SnowPipe, ETL data Pipelines, and Big Data model techniques using Python/Java. You must have a deep understanding of relational and NoSQL data stores, methods, and approaches such as star and snowflake dimensional modeling. Strong expertise in AWS ...

As an AWS Data Engineer with 1.6 to 6 years of experience, you will be responsible for designing, developing, and maintaining scalable data pipelines and ETL workflows on AWS. Your expertise in Snowflake, Python, and SQL will be crucial as you process, transform, and analyze data. You will work towards optimizing existing data architectures to ensure high performance and cost efficiency while integrating data from multiple sources into a unified data warehouse. Collaboration with data analysts, data scientists, and business stakeholders is essential to meet data requirements effectively. You will implement data governance, quality checks, and best practices to enhance data engineering proces...

As a software developer, you will be working in a constantly evolving environment driven by technological advances and the strategic direction of the organization you are employed by. Your primary responsibilities will include creating, maintaining, auditing, and enhancing systems to meet specific needs, often based on recommendations from systems analysts or architects. You will be tasked with testing both hardware and software systems to identify and resolve system faults. Additionally, you will be involved in writing diagnostic programs and designing and developing code for operating systems and software to ensure optimal efficiency. In situations where necessary, you will also provide re...