1330 Aws Glue Jobs - Page 14

Role Description :As a Principle Engineer - Cloud Data Platform (AWS) at Incedo, you will be responsible for designing, deploying and maintaining cloud-based data platforms on the AWS platform. You will work with data engineers, data scientists and business analysts to understand business requirements and design scalable, reliable and cost-effective solutions that meet those requirements. Roles & Responsibilities: Designing, developing and deploying cloud-based data platforms using Amazon Web Services (AWS) Integrating and processing large amounts of structured and unstructured data from various sources Implementing and optimizing ETL processes and data pipelines Developing and maintaining s...

Role Description As a Technical Lead - Cloud Data Platform (AWS) at Incedo, you will be responsible for designing, deploying and maintaining cloud-based data platforms on the AWS platform. You will work with data engineers, data scientists and business analysts to understand business requirements and design scalable, reliable and cost-effective solutions that meet those requirements. Roles & Responsibilities: Designing, developing and deploying cloud-based data platforms using Amazon Web Services (AWS) Integrating and processing large amounts of structured and unstructured data from various sources Implementing and optimizing ETL processes and data pipelines Developing and maintaining securi...

DATA ENGINEER (Databricks & AWS) Overview: As a Data Engineer, you will work with multiple teams to deliver solutions on the AWS Cloud using core cloud data engineering tools such as Databricks on AWS, AWS Glue, Amazon Redshift, Athena, and other Big Data-related technologies. This role focuses on building the next generation of application-level data platforms and improving recent implementations. Hands-on experience with Apache Spark (PySpark, SparkSQL), Delta Lake, Iceberg, and Databricks is essential. Responsibilities : • Define, design, develop, and test software components/applications using AWS-native data services: Databricks on AWS, AWS Glue, Amazon S3, Amazon Redshift, Athena, AWS ...

Role Overview: At PwC, your role as an Associate in the Tower of Data, Analytics & Specialist Managed Service will involve working as part of a team of problem solvers, helping to solve complex business issues from strategy to execution. Your responsibilities at this management level will include but are not limited to: Key Responsibilities: - Use feedback and reflection to develop self-awareness, personal strengths, and address development areas. - Be flexible to work in stretch opportunities/assignments. - Demonstrate critical thinking and the ability to bring order to unstructured problems. - Review Ticket Quality and deliverables, provide Status Reporting for the project. - Adhere to SLA...

Role Overview: You will be responsible for migrating SSIS packages to AWS Glue, focusing on ETL processes and cloud computing. This role will involve automating the migration using tools like AWS Schema Conversion Tool (AWS SCT) and developing custom connectors. Key Responsibilities: - Plan and analyze the migration process - Create AWS Glue jobs - Develop custom connectors - Perform data transformation and validation - Monitor and maintain the migrated packages Qualifications Required: - Strong knowledge of AWS Glue - Hands-on experience with AWS Cloud - Proficiency in ETL Concepts - Familiarity with SSIS - Scripting and programming skills - Additional skills in AWS Redshift and SQL profici...

Role: AWS + PySpark Data Engineer Experience: 4-8 yrs Must Have Skills: 4 Years Relevant in AWS PySpark distributed data processing, job optimization, troubleshooting. AWS Data Lake S3, Glue, Athena. Redshift – SQL queries, integration with PySpark. Strong SQL (advanced). Python coding for data manipulation & pipelines. Good to Have: SAP S/4HANA Data Extraction (OData, RFC, connectors). Pandas/NumPy for Python-based data transformation.

The role involves leading the design and management of data solutions in a cloud environment with a focus on AWS services. Responsibilities: Lead the design, implementation, and optimization of scalable data pipelines and architectures utilizing AWS Glue, Elastic MapReduce (EMR), Lambda, Redshift, Athena, DynamoDB, OpenSearch, and S3. Use Spark on AWS for data transformation and processing across large datasets. Develop and maintain efficient data workflows with SQS for task queueing and orchestration. Integrate, transform, and manage data using Mulesoft and Talend for seamless data integration. Ensure high-performance data storage, retrieval, and analytics across Redshift, DynamoDB, and Ath...

Join us as a Data Engineer Youll be the voice of our customers, using data to tell their stories and put them at the heart of all decision-making Well look to you to drive the build of effortless, digital first customer experiences If youre ready for a new challenge and want to make a far-reaching impact through your work, this could be the opportunity youre looking for We're offering this role at vice president level What you'll do As a Data Engineer, youll be looking to simplify our organisation by developing innovative data driven solutions through data pipelines, modelling and ETL design, inspiring to be commercially successful while keeping our customers, and the banks data, safe and se...

Role & responsibilities Job Summary: We are looking for a highly skilled Python Developer with strong experience in AWS serverless technologies, particularly AWS Lambda. The ideal candidate will be proficient in building automation scripts, handling file transfers and transformations, and developing event-driven workflows using AWS services. Key Responsibilities: 1. Python Development & Automation Write efficient Python scripts for file processing, transformations, and data workflows. Develop reusable, scalable code for AWS Lambda functions. 2. AWS Serverless Solutions Build and deploy serverless applications using AWS Lambda Integrate Lambda with services like S3, SNS, SQS, and EventBridge....

Were Hiring – Data Architect | Data Engineer Location: Bangalore | CTC: Up to 35 LPA Interview: F2F | Experience: 12+ Years Key Skills: Data Architecture & Modelling (Star & Snowflake schemas) AWS: S3, RedShift, Glue Big Data: Spark, Hive, Presto SQL (Strong query writing & procedures) | Python | GitHub Data Lakes, ETL (Informatica preferred) DevSecOps, CI/CD, Data Pipeline Automation Responsibilities: Design & deliver scalable data engineering solutions on AWS Build & optimize ingestion pipelines (Spark, Presto, RedShift) Ensure data quality, governance & security best practices Collaborate with Data Modelers & BI/Analytics teams Troubleshoot, monitor & enhance performance of data systems D...

Key Responsibilities AWS Glue job development and optimization Lake Formation security and governance implementation Data pipeline development and maintenance Platform feature development and testing Documentation and runbook creation Required Technical Skills AWS Glue ETL development and optimization AWS Lake Formation security and governance Python and SQL programming Data modeling and schema design Amazon S3 optimization and performance tuning Required Experience 4-6 years in data engineering with focus on AWS platforms Strong Python and SQL skills AWS Glue, Lake Formation, and S3 optimization experience Experience with data modeling and schema design Knowledge of data governance and secu...

About The Role Project Role : Application Lead Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills : Amazon Web Services (AWS), AWS Glue Good to have skills : AWS Administration, AWS Database Migration Service (DMS) Minimum 12 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Lead, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. Your typical day will involve collaborating with various teams to ensure that application development aligns with organizational goals, addressin...

As a Data Engineer at our company, you will play a critical role in designing, developing, and maintaining data pipeline architecture to ensure the efficient flow of data across the organization. **Key Responsibilities:** - Develop robust and scalable data pipelines using AWS Glue, Apache Airflow, and other relevant technologies. - Integrate various data sources, including SAP HANA, Kafka, and SQL databases, for seamless data flow and processing. - Optimize data pipelines for performance and reliability. **Data Management And Transformation:** - Design and implement data transformation processes to clean, enrich, and structure data for analytical purposes. - Utilize SQL and Python for data e...

About The Role Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : AWS Glue Good to have skills : NAMinimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will design, build, and configure applications to meet business process and application requirements. A typical day involves collaborating with various teams to understand their needs, developing solutions that align with business objectives, and ensuring that applications are optimized for performance and usability. You will al...

Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : AWS Glue Good to have skills : NAMinimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will design, build, and configure applications to meet business process and application requirements. A typical day involves collaborating with team members to understand project needs, developing application features, and ensuring that the applications function seamlessly within the existing infrastructure. You will also engage in troubleshoo...

About The Role Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Python (Programming Language) Good to have skills : DevOps, Data Engineering, AWS Glue, REST API, LambdaMinimum 2 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will engage in the design, construction, and configuration of applications tailored to fulfill specific business processes and application requirements. Your typical day will involve collaborating with team members to understand project needs, developing inno...

About The Role Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Python (Programming Language) Good to have skills : DevOps, Data Engineering, AWS Glue, REST API, LambdaMinimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will engage in the design, construction, and configuration of applications tailored to fulfill specific business processes and application requirements. Your typical day will involve collaborating with team members to understand project needs, developing inno...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

Skills: Skills: Minimum of 6-9 years of experience hands on experience onAWS Glue, Python, Pyspark, good in writing codes. Exp. withAirflow and SQL. Authoring ETL processes using Python and PySpark ETL process monitoring using CloudWatch events Experience with cloud infrastructure management tools (AWS CloudFormation, Azure Resource Manager). Proficiency in scripting or programming languages (Python, PowerShell, etc.) for automation and infrastructure-as-code. Knowledge of networking concepts and technologies, including virtual private clouds (VPCs), subnets, and load balancers. Familiarity with containerization technologies (Docker, Kubernetes) and microservices architecture. Understanding ...