Python Developer (Scrapy)

Senior Python Developer — Web Scraping (1.5 years industry experience)

Location: Ahemdabad ( Work from Office )
Employment type: Full-time
Company: Actowiz Solutions

Role overview

We are looking for a pragmatic, hands-on Senior Python Developer with ~1.5 years of direct experience in the web-scraping industry. You’ll design, build and maintain high-quality, reliable data extraction systems (scrapers, crawlers, parsers) and the API layers that serve that data to downstream services. The ideal candidate is comfortable working across the full stack of a scraping product — headless browsers, HTTP clients, extraction pipelines, storage, and REST APIs — and can write production-ready code, tests, and deployment pipelines.

Key responsibilities

• Design, develop, and maintain scalable scrapers and crawlers (headless browser & HTTP-based) to extract structured data from web and app sources.
• Build and maintain backend APIs (Django REST / FastAPI) that serve processed data to clients and internal services.
• Implement robust request flows using requests, curl_cffi (or equivalent), and HTTP/2 where appropriate.
• Develop and operate browser-based scraping using Selenium and Playwright (including stealth variants / evasion best-practices at a high level).
• Parse and normalize diverse data formats (HTML, JSON, XML), using CSS selectors, XPath, regex, and DOM traversal.
• Implement retry, back-off, rate limiting, queueing and monitoring for scraping pipelines.
• Work with data stores (MongoDB, PostgreSQL, Redis, S3 or equivalent) and write efficient ingestion/ETL jobs.
• Write unit/integration tests, CI/CD pipelines, and containerize services (Docker).
• Perform code reviews, mentor junior developers, and collaborate with product, QA, and ops teams.
• Monitor, debug and resolve production issues, and continuously improve reliability and performance.
• Ensure respectful, legal and ethical data collection practices (observe robots policies, rate limits, data usage constraints as applicable).

Required (must-have) technical skills

• Python — Strong, idiomatic Python (3.8+).
• Scraping tools & HTTP clients: requests (advanced), curl_cffi (or similar low-level cURL bindings).
• Browser automation: Selenium (WebDriver), Playwright (including experience with stealth/anti-detection approaches).
• APIs: Design & implement REST APIs using Django (Django REST Framework) and FastAPI.
• Data parsing: HTML parsing (BeautifulSoup, lxml), XPath/CSS selectors, JSON/XML handling.
• Asynchronous programming: asyncio, aiohttp or equivalent for high-throughput crawlers.
• Containers & devops basics: Docker, Git, basic CI/CD (GitHub Actions / GitLab CI / Jenkins).
• Datastores: MongoDB and/or PostgreSQL; familiarity with Redis for queues/caching.
• Debugging & monitoring: Logging best practices, error tracking, metrics, alerting.
• Linux & networking fundamentals: Comfort working on Linux servers, HTTP internals, headers, proxies.
• Code quality: Tests (unit/integration), code reviews, clear documentation.

Nice-to-have

• Experience with curl_cffi in production scraping stacks (performance/streaming use-cases).
• Familiarity with anti-bot/anti-detection tradeoffs at a high level (without providing instructions to break laws) — e.g., fingerprinting awareness, browser fingerprint mitigation strategies, polite rate-limiting.
• Hands-on experience with cloud platforms (AWS/GCP/Azure) and managed services (S3, ECS/EKS, Lambda).
• Experience with queueing systems (RabbitMQ, Celery, Kafka) and orchestration.
• Knowledge of proxies, proxy management, and CAPTCHA solving integrations (high-level architectural experience).
• Prior product/enterprise experience delivering SLAs for data pipelines.

Soft skills & behavioral

• Strong problem-solving and debugging skills — comfortable diagnosing flaky scrapers and transient production issues.
• Good communication: explain technical tradeoffs to non-engineers and write clear design docs.
• Team player: experience mentoring junior engineers and participating in code reviews.
• Ownership mindset: you ship, operate, and iterate on what you build.

Job Types: Full-time, Permanent

Pay: ₹15,000.00 - ₹40,000.00 per month

Benefits:

• Flexible schedule
• Paid time off

Application Question(s):

• Where do you stay?
• What is your Current Salary ?
• What is your Expected Salary ?
• Notice Period ?
• Are you okay to relocate to Ahemdabad?

Work Location: In person

Speak with the employer
+91 7862049428