1847 Aws Glue Jobs - Page 35

Role & responsibilities Job Summary: We are looking for a highly skilled Python Developer with strong experience in AWS serverless technologies, particularly AWS Lambda. The ideal candidate will be proficient in building automation scripts, handling file transfers and transformations, and developing event-driven workflows using AWS services. Key Responsibilities: 1. Python Development & Automation Write efficient Python scripts for file processing, transformations, and data workflows. Develop reusable, scalable code for AWS Lambda functions. 2. AWS Serverless Solutions Build and deploy serverless applications using AWS Lambda Integrate Lambda with services like S3, SNS, SQS, and EventBridge....

Were Hiring – Data Architect | Data Engineer Location: Bangalore | CTC: Up to 35 LPA Interview: F2F | Experience: 12+ Years Key Skills: Data Architecture & Modelling (Star & Snowflake schemas) AWS: S3, RedShift, Glue Big Data: Spark, Hive, Presto SQL (Strong query writing & procedures) | Python | GitHub Data Lakes, ETL (Informatica preferred) DevSecOps, CI/CD, Data Pipeline Automation Responsibilities: Design & deliver scalable data engineering solutions on AWS Build & optimize ingestion pipelines (Spark, Presto, RedShift) Ensure data quality, governance & security best practices Collaborate with Data Modelers & BI/Analytics teams Troubleshoot, monitor & enhance performance of data systems D...

Key Responsibilities AWS Glue job development and optimization Lake Formation security and governance implementation Data pipeline development and maintenance Platform feature development and testing Documentation and runbook creation Required Technical Skills AWS Glue ETL development and optimization AWS Lake Formation security and governance Python and SQL programming Data modeling and schema design Amazon S3 optimization and performance tuning Required Experience 4-6 years in data engineering with focus on AWS platforms Strong Python and SQL skills AWS Glue, Lake Formation, and S3 optimization experience Experience with data modeling and schema design Knowledge of data governance and secu...

About The Role Project Role : Application Lead Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills : Amazon Web Services (AWS), AWS Glue Good to have skills : AWS Administration, AWS Database Migration Service (DMS) Minimum 12 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Lead, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. Your typical day will involve collaborating with various teams to ensure that application development aligns with organizational goals, addressin...

As a Data Engineer at our company, you will play a critical role in designing, developing, and maintaining data pipeline architecture to ensure the efficient flow of data across the organization. **Key Responsibilities:** - Develop robust and scalable data pipelines using AWS Glue, Apache Airflow, and other relevant technologies. - Integrate various data sources, including SAP HANA, Kafka, and SQL databases, for seamless data flow and processing. - Optimize data pipelines for performance and reliability. **Data Management And Transformation:** - Design and implement data transformation processes to clean, enrich, and structure data for analytical purposes. - Utilize SQL and Python for data e...

About The Role Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : AWS Glue Good to have skills : NAMinimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will design, build, and configure applications to meet business process and application requirements. A typical day involves collaborating with various teams to understand their needs, developing solutions that align with business objectives, and ensuring that applications are optimized for performance and usability. You will al...

Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : AWS Glue Good to have skills : NAMinimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will design, build, and configure applications to meet business process and application requirements. A typical day involves collaborating with team members to understand project needs, developing application features, and ensuring that the applications function seamlessly within the existing infrastructure. You will also engage in troubleshoo...

About The Role Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Python (Programming Language) Good to have skills : DevOps, Data Engineering, AWS Glue, REST API, LambdaMinimum 2 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will engage in the design, construction, and configuration of applications tailored to fulfill specific business processes and application requirements. Your typical day will involve collaborating with team members to understand project needs, developing inno...

About The Role Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Python (Programming Language) Good to have skills : DevOps, Data Engineering, AWS Glue, REST API, LambdaMinimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will engage in the design, construction, and configuration of applications tailored to fulfill specific business processes and application requirements. Your typical day will involve collaborating with team members to understand project needs, developing inno...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

Skills: Skills: Minimum of 6-9 years of experience hands on experience onAWS Glue, Python, Pyspark, good in writing codes. Exp. withAirflow and SQL. Authoring ETL processes using Python and PySpark ETL process monitoring using CloudWatch events Experience with cloud infrastructure management tools (AWS CloudFormation, Azure Resource Manager). Proficiency in scripting or programming languages (Python, PowerShell, etc.) for automation and infrastructure-as-code. Knowledge of networking concepts and technologies, including virtual private clouds (VPCs), subnets, and load balancers. Familiarity with containerization technologies (Docker, Kubernetes) and microservices architecture. Understanding ...

We are looking for a skilled AWS Datalake Lead with 4 to 8 years of experience to lead the technical design and architecture of AWS datalake and its related services, ensuring alignment with customer requirements, industry best practices, and project objectives. This position is based in Hyderabad. Roles and Responsibility Lead the technical design and architecture of AWS datalake and its related services. Conduct reviews to ensure adherence to coding standards and best practices. Provide technical support and resolve incidents, service requests, and minor enhancements. Manage datasets in AWS Data Lake, including adding, updating, or deleting them. Monitor storage usage and handle capacity p...

Job Title :AWS Data Engineer Location :Bangalore Notice Period : Immediate to 60 Days Preferred. Job Description: We are seeking skilled and dynamic Cloud Data Engineers specializing in AWS, Databricks. The ideal candidate will have a strong background in data engineering, with a focus on data ingestion, transformation, and warehousing. They should also possess excellent knowledge of PySpark or Spark, and a proven ability to optimize performance in Spark job executions. Key Responsibilities: - Design, build, and maintain scalable data pipelines for a variety of cloud platforms including AWS. - Implement data ingestion and transformation processes to facilitate efficient data warehousing. - U...

Role & responsibilities Job Description: Job Summary: We are looking for a skilled Data Engineer to join our data team. The ideal candidate will have hands-on experience in building robust data pipelines using Airflow, managing data warehouses in Snowflake and Redshift, and working with cloud platforms such as AWS and Azure (Databricks, Glue). You will work closely with Data Scientists, Analysts, and Engineers to ensure data availability, quality, and reliability across the organization. Key Responsibilities: l Design, develop, and maintain scalable ETL/ELT pipelines using Apache Airflow, AWS Glue, and Databricks. l Integrate data from multiple sources into Snowflake and Redshift data wareho...

Role Description: As a Senior Technical Lead - Cloud Data Platform (AWS) at Incedo, you will be responsible for designing, deploying and maintaining cloud-based data platforms on the AWS platform. You will work with data engineers, data scientists and business analysts to understand business requirements and design scalable, reliable and cost-effective solutions that meet those requirements. Roles & Responsibilities : Designing, developing and deploying cloud-based data platforms using Amazon Web Services (AWS) Integrating and processing large amounts of structured and unstructured data from various sources Implementing and optimizing ETL processes and data pipelines Developing and maintaini...

Hi, Greetings from the IDES LABS, Job Posting Title: Senior Platform Engineer Description: Bachelors degree (Computer Science), masters degree or Technical Diploma or equivalent At least 8 years of experience in a similar role At least 5 years of experience on AWS and/or Azure At least 5 years of experience on Databricks At least 5 years of experience on multiples Azure and AWS PaaS solutions: Azure Data Factory, MSSQL, Azure storage, AWS S3, Cognitive search, CosmosDB, Event Hub, AWS glue Strong knowledge of AWS and Azure architecture design best practices Knowledge of ITIL & AGILE methodologies (certifications are a plus) Experience working with DevOps tools such as Git, CI/CD pipelines, A...

Role Description As a Technical Lead - Cloud Data Platform (AWS) at Incedo, you will be responsible for designing, deploying and maintaining cloud-based data platforms on the AWS platform. You will work with data engineers, data scientists and business analysts to understand business requirements and design scalable, reliable and cost-effective solutions that meet those requirements. Roles & Responsibilities: Designing, developing and deploying cloud-based data platforms using Amazon Web Services (AWS) Integrating and processing large amounts of structured and unstructured data from various sources Implementing and optimizing ETL processes and data pipelines Developing and maintaining securi...

Role Description As a Principle Engineer - Cloud Data Platform (AWS) at Incedo, you will be responsible for designing, deploying and maintaining cloud-based data platforms on the AWS platform. You will work with data engineers, data scientists and business analysts to understand business requirements and design scalable, reliable and cost-effective solutions that meet those requirements. Roles & Responsibilities: Designing, developing and deploying cloud-based data platforms using Amazon Web Services (AWS) Integrating and processing large amounts of structured and unstructured data from various sources Implementing and optimizing ETL processes and data pipelines Developing and maintaining se...

Role Description As a Technical Lead - Cloud Data Platform (AWS) at Incedo, you will be responsible for designing, deploying and maintaining cloud-based data platforms on the AWS platform. You will work with data engineers, data scientists and business analysts to understand business requirements and design scalable, reliable and cost-effective solutions that meet those requirements. Roles & Responsibilities: Designing, developing and deploying cloud-based data platforms using Amazon Web Services (AWS) Integrating and processing large amounts of structured and unstructured data from various sources Implementing and optimizing ETL processes and data pipelines Developing and maintaining securi...