Data Scraping Expert

We are seeking a highly skilled Scraping Expert to join our data team and drive automated extraction of tender information from various Indian portals, including the Government e-Marketplace (GeM). The ideal candidate will leverage advanced scraping techniques, Optical Character Recognition (OCR), and data validation methods to collect, process, and deliver high-quality tender datasets for our stakeholders.

Key Responsibilities

• Develop and maintain robust web-scraping pipelines to extract tender information from GeM, state/central portals, and private aggregators.
• Integrate OCR tools to accurately extract text from scanned PDFs and tender-related images.
• Parse, normalize, and structure scraped tender data (e.g., tender ID, project title, department, dates, values).
• Monitor scraping jobs with automated alerts and update scripts for GeM UI changes.
• Collaborate with analysts and product teams to align data outputs with business requirements.

Required Qualifications

• 3–5 years of hands-on experience building and deploying web-scraping solutions using Python frameworks (BeautifulSoup, Scrapy, Selenium).
• Proficiency with OCR libraries (Tesseract, OpenCV) and techniques for processing scanned documents.
• Prior experience working with or scraping data from the GeM portal or other Indian tender websites.
• Strong data cleaning, transformation, and storage skills using SQL/noSQL databases or data warehouses.
• Excellent problem-solving abilities to troubleshoot IP blocks, dynamic content, and anti-bot measures.

Technical Skills

• Python
• BeautifulSoup
• Apache Airflow
• AWS (e.g., EC2, S3, Lambda)

Job Type: Full-time

Pay: ₹50,000.00 - ₹70,000.00 per month

Schedule:

• Day shift

Experience:

• Apache Airflow: 2 years (Required)

Work Location: In person