Python Developer

Job Title: Python Developer

Location: Jaipur, India

Role Overview

Build and operate a scalable, headless browser automation service

Core Responsibilities

• Automation adapters:

   Implement Playwright/Selenium flows per publisher (login, navigate to location_url, fuzzy-match review, post reply, verify).

• Job orchestration:

   Use 

  Celery

   (Redis broker) for distributed workers; 

  Celery Beat

   for scheduled tasks (retries, health checks, DOM drift probes).

• Scalability & throughput:

• Configure worker pools & autoscaling to comfortably handle 10–100+ concurrent jobs

  .
• Rate-limit per publisher/domain; backoff on throttling.
• Idempotency (dedupe by publisher_identifier + location_identifier + review_identifier).

• Observability:

   Emit structured logs/metrics/traces; 

  Datadog

   dashboards + alerts (success rate, latency, error taxonomy, captcha/MFA rate).

• Blockers & challenges:

   Detect captchas/MFA/layout changes; implement 

  human-in-the-loop

   resolution (no bypassing); resilient locators/waits.

• Reliability:

   Health checks, circuit breakers, canary releases, and feature flags to disable a publisher adapter quickly.

Required Experience

• 3+ years building 

  production browser automation

   (Playwright preferred) for authenticated flows.
• Strong 

  Python

   (async, typing, testing) and 

  Celery + Redis

   expertise (routing, acks/retries, ETA/Countdown, result backends).

• AWS

   (ECS/EKS or EC2, S3 for artifacts, CloudWatch/ALB, IAM) and 

  Docker

  .

• Datadog

   (metrics, logs, APM traces, monitors, SLOs).
• Practical handling of 

  rate limits, CAPTCHA, MFA

  , session lifecycle, and SRF.

Nice to Have

• Fuzzy matching/text similarity (e.g., rapidfuzz) for robust review matching.
• Terraform/IaC; blue/green or canary deploys.
• Postgres for job/state storage (unique constraints for dedupe).

Minimal Target Architecture

• API (FastAPI)

   receives JSON job → validates schema → enqueues Celery task (Redis broker).

• Celery workers

   (Docker) run Playwright in headless mode; store artifacts (S3); emit metrics/logs (Datadog).

• Celery Beat

   schedules DOM-probe jobs, reprocessing, and key rotation checks.

• Result sink

   (Postgres/Redis/S3) stores typed outcome: status, error_code, timestamps, artifact URIs.

• Feature flags

   (env/DB) to enable/disable publisher adapters instantly.

Core Technical Skills

Browser Automation

• Strong hands-on with 

  Playwright

   (preferred) or 

  Selenium

   for headless automation.
• Experience handling 

  logged-in user flows

   (authentication, sessions, CSRF tokens, cookies).
• Robust 

  DOM locator strategies

   (ARIA roles, CSS/XPath, page objects).
• Ability to manage 

  dynamic pages

  , lazy loads, and infinite scrolls.
• Captcha/MFA detection and 

  human-in-the-loop integration

   (not bypass).

Backend & Job Orchestration

• Python

   (async/await, typing, testing, logging).

• Celery

   (task queues, retries, routing, scheduling with Celery Beat).

• Redis

   (as Celery broker & caching layer).
• Designing 

  idempotent and fault-tolerant job pipelines

  .

Scalability & Systems Design

• Architecting distributed worker pools that scale for 

  10–100+ concurrent jobs

  .
• Rate limiting, throttling, and backoff strategies.
• Experience with 

  message queues

   and asynchronous task execution.
• Familiarity with 

  multi-tenant architectures

   (per-publisher adapters).

Cloud & Infrastructure

• AWS

  :
• ECS/EKS or EC2 for containerized workers.
• S3 for artifact storage (screenshots, logs, videos).
• Secrets Manager/KMS for secure credential handling.
• CloudWatch/ALB for monitoring/logging.

• Docker

  : building, running, and deploying containerized workers.
• CI/CD pipelines for automation deployments.

Observability & Monitoring

• Datadog

   (APM, metrics, logs, custom dashboards, alerting).
• OpenTelemetry/structured logging for traceability.
• Error taxonomy and health monitoring for automation jobs.

Security & Compliance

• Secure storage and rotation of credentials (Vault/Secrets Manager).
• Encrypted communication between services.
• Audit trails (job logs, screenshots, structured events).

Data & Matching

• Fuzzy text matching

   (rapidfuzz/fuzzywuzzy, token set ratio) to identify reviews reliably.
• String normalization (case, whitespace, punctuation, emoji handling).
• Postgres/MySQL basics for job state/result persistence.