Senior Python Developer

Position: Python Developer

Location: Remote (India)

Salary: 80-90k pm

Experience: 4+ years

We’re looking for a Python Developer with solid hands-on experience in job and web data scraping,

AI-enabled data processing, and building scalable ETL pipelines. You’ll design, implement, and operate scraping and enrichment systems that feed recruitment and analytics products. Familiarity with Aspen/Webspidermount and Kadoa is a plus. QA-minded engineers who care about data accuracy and observability will thrive in this role.

Key Responsibilities:

• Design, develop, and maintain AI-powered web and job-scraping pipelines.
• Extract structured and unstructured data from job boards, company sites, APIs, and other sources.
• Build and operate ETL processes to clean, normalize, enrich, and load data into analytics stores.
• Ensure data accuracy, completeness, and consistency via automated validation and QA checks.
• Implement and maintain monitoring, alerting, and observability for scraper health and data quality.
• Optimize scraping and pipeline performance for reliability and scale (rate-limiting, retries, distributed workers).
• Integrate scraped and enriched data with internal systems, analytics platforms, and downstream consumers.
• Collaborate with product, data science, and QA teams to meet business and compliance requirements.
• Stay current with AI, scraping techniques, legal/ethical scraping best practices, and anti-bot mitigations.

Required Skills:

Proficient in Python

PostgreSQL

BeautifulSoup, Scrapy, Selenium, Pandas

pytest

multiprocessing

·        Experience building robust data pipelines, working with APIs, rate limits, pagination, and proxying.

·        Proven experience building data scrapers and AI-assisted scraping/enrichment workflows.

·        Familiarity with data QA processes, validation frameworks, and creating automated data checks.

·        Excellent debugging skills and experience with logging, metrics, and alerting for production jobs.

·        Strong problem-solving, communication skills, and ability to work independently and in teams.

Customer- and user-oriented

Preferred Skills:

Aspen / Webspidermount

·        Hands-on experience with machine learning or AI-driven data processing and enrichment.

·        Familiarity with recruitment data, job-board ecosystems, and common job taxonomies (titles, skills, locations).

·        Experience with distributed task queues (e.g., Celery, RQ) or orchestration (Airflow, Prefect).

·        Experience with cloud platforms (AWS/GCP) and containerization (Docker, Kubernetes).

·        Knowledge of legal/compliance aspects of web scraping and data privacy.

What we offer:

·  Challenging problems at the intersection of data engineering, AI, and recruitment analytics.

·  Collaborative, fast-paced engineering culture that values quality and ownership.

·  Opportunity to influence data architecture and product direction.