Senior Web Scraping Engineer

About us

We’re building a product that relies heavily on collecting structured data from a number of known websites. We need someone experienced who can own this part of the system end-to-end; from writing scrapers to making sure they scale and stay reliable.

This is a senior role. We’re looking for someone who has already dealt with the real-world problems of web scraping and can work independently.

What you'll do

• Build and maintain scrapers for 24-30 websites (starting with 6).
• Handle authenticated sites, sessions, cookies, tokens, etc.
• Work with search endpoints and dynamic content.
• Architect the data ingestion pipeline, caching, retries, and queues.
• Make the scrapers stable and resilient even as websites change.
• Collaborate closely with our backend team to define and use APIs.
• Handle scaling, rate limits, and performance.

What we’re looking for (hard requirements)

• 5 - 7+ years of building production-grade web scrapers
• Strong experience with both API-based scraping and browser automation (Playwright/Puppeteer)
• Familiarity with proxies/IP rotation and basic anti-bot handling
• Comfortable with auth flows (cookies, sessions, tokens)
• Ability to design pipelines: ingestion → clean → store → cache
• Can work independently and own the scraper system end-to-end

Nice to have (not required)

• Familiarity with Cloudflare’s ecosystem.
• Python, Node.js (TypeScipt only), or Go; any backend language is fine, TypeScript is preferred.

Who will do well here

• You enjoy scrapers that work reliably, and predictably.
• You think in terms of systems, not quick patches.
• You can work remotely and manage your own time.
• You appreciate straightforward communication and minimal bureaucracy.

Compensation

• Base: ₹3,00,000 - 4,00,000 per month (36 -

• 
  

• 48 LPA).Equit

• y/options are available if you choose to take a lower cash component.