Freelance AWS Data Engineer / Python Developer

Company Description

ThreatXIntel is a startup cyber security company focused on protecting businesses and organizations from cyber threats. Our services include cloud security, web and mobile security testing, cloud security assessment, and DevSecOps. We provide customized, affordable solutions to meet the specific needs of our clients, offering proactive monitoring to identify vulnerabilities before they can be exploited.

Role Description

This is a part-time remote role for a Freelance AWS Data Engineer / Python Developer at ThreatXIntel. The role involves tasks such as data engineering, data modeling, ETL (Extract Transform Load), data warehousing, and data analytics.

Technical Skill Set Required

• AWS Glue

  – ETL job development, Glue Catalog, Crawlers, job optimization.

• AWS Lambda

  – Serverless functions for automation and orchestration.

• AWS Step Functions

  – Workflow design, state machine orchestration, error handling.

• PySpark

  – Data processing, transformations, schema handling, performance tuning.

• Pytest

  – Unit testing, integration testing for data pipelines and automation scripts.

• Python

  – Strong coding and debugging skills, reusable function development.

• AWS Core Services

  – S3, IAM, CloudWatch for monitoring, logging, and access control.

• Documentation

  – Ability to prepare clear and concise technical documentation for pipelines, workflows, and processes.

Role Expectations

• Build and maintain

  data pipelines

  using AWS Glue & PySpark.
• Create

  serverless automation

  with Lambda and Step Functions.
• Ensure pipelines are

  tested

  with Pytest and optimized for performance.
• Develop

  robust error handling

  and logging mechanisms.
• Deliver

  documentation

  covering architecture, workflows, and deployment.