AWS Infrastructure Consultant

Key Responsibilities

Infrastructure Design and Management:

• Design and provision scalable, secure, and cost-efficient infrastructure onAWSusingTerraform(mandatory).
• Deploy and manage services such asDatabricks (on AWS),Amazon Redshift,EMR, andRDSusing Infrastructure-as-Code.
• Maintain version-controlled, reusable infrastructure modules in alignment with IaC best practices.

DevOps and CI/CD Automation:

• Develop and maintainCI/CD pipelinesusingJenkins(mandatory) integrated with Git-based workflows.
• Automate infrastructure provisioning, testing, and deployment processes.
• Support CI/CD for data orchestration tools such asAirflow,Databricks, andRedshift.

Data Pipeline Platform Management:

• Collaborate with data engineers to build and deploy pipelines usingAWS Glue,Kinesis,Lambda,S3, andRedshift.
• Implement orchestration usingApache Airflow(open-source and AWS MWAA).
• Ensure reliability, scalability, and security throughout the data lifecycle.

Monitoring, Logging Optimization:

• Set up monitoring, logging, and alerting tools usingCloudWatch,Prometheus,Grafana, andCloudTrail.
• Perform root-cause analysis, troubleshoot issues, and optimize infrastructure and pipeline performance.

Security and Compliance:

• Apply AWS security best practices across environments, includingIAM,VPC,KMS, encryption, and network segmentation.
• Conduct audits and implement governance policies.

Documentation Collaboration:

• Work cross-functionally with DevOps, data engineering, and cloud security teams.
• Document architecture, configurations, workflows, and reusables to drive team efficiency and knowledge sharing.

Required Skills Qualifications

Cloud Infrastructure-as-Code (IaC):

• Strong hands-on experience with AWS (EC2, S3, Lambda, RDS, Glue, Redshift, IAM, VPC, Kinesis).
• Expertise in Terraformfor multi-account, multi-service AWS infrastructure.

DevOps CI/CD:

• Proficiency with Jenkins(mandatory).
• Experience with GitLab CI/CD, GitHub Actions, or AWS CodePipeline is a plus.
• Familiarity with Docker andAmazon EKSis desirable.

Data Platform Experience:

• Experience provisioning and operating platforms likeDatabricks,Amazon Redshift, and optionallySnowflake.
• Experience building and managing data pipelines usingGlue,Airflow, andStep Functions.

Monitoring Troubleshooting:

• Strong understanding of observability tools and frameworks (e.g., CloudWatch, Grafana, Prometheus).

Scripting Automation:

• Proficient inPython,Bash, or similar scripting languages.

Security Governance:

• Hands-on with IAM roles/policies, VPC security,KMS encryption, and compliance controls.

Preferred Qualifications

• Certifications:AWS Solutions Architect, DevOps Engineer, or Data Analytics Specialty.
• Databricks, Redshift, or Snowflake certifications are a plus.
• Experience with other IaC tools (Ansible, CloudFormation) and workflow engines (Apache Airflow, MWAA).
• Exposure to big data platforms like EMR, Spark, or Hadoop.