Web Crawling & Data Extraction Engineer (1 to 6 yrs)

Web Crawling & Data Extraction Engineer (WFH)

Experience:

Location:

Mode of Engagement:

No of Positions:

Educational Qualification:

Industry:

Notice Period:

What We Are Looking For

• Strong hands-on experience in web crawling, scraping, and automated data extraction.
• Experience working with Playwright, Puppeteer, Selenium, and Python Requests for scraping JS-heavy and dynamic websites.
• Good understanding of HTML DOM, XPaths, JSON APIs, and handling pagination, infinite scroll, and dynamic page loads.
• Ability to manage sessions, cookies, headers, token-based flows, and bypass basic anti-bot protections.
• Experience building automated extraction scripts that run on schedules or pipelines.
• Ability to write clean, structured, and optimized Python code for crawling and automation.

Responsibilities

• Build and maintain web crawlers & automated extraction scripts using Playwright/Puppeteer/Selenium/Requests.
• Extract data from complex, JavaScript-rendered, or protected websites.
• Handle dynamic elements, redirects, headers, authentication, and anti-bot mitigations.
• Clean, validate, and structure extracted data (JSON, CSV, DB).
• Use SQL/NoSQL databases to store extracted content.
• Maintain automation scripts, scheduling, retries, logging, and monitoring for stable crawling.
• Work closely with internal teams to deliver accurate and timely data.

Qualifications

• 1–4 years of hands-on experience in Python-based web crawling and data extraction.
• Experience with Playwright, Puppeteer, Selenium, and Python Requests/BeautifulSoup.
• Understanding of proxies, user-agents, sessions, and anti-bot workarounds.
• Experience with SQL or MongoDB.
• Strong debugging, analytical, and logical reasoning skills.
• Good English communication skills.