Python Web Scraper Developer (Instagram Data Extraction)

About the Role: We are looking for an experienced Python Web Scraper / Automation Engineer with strong expertise in building reliable scraping systems for large-scale data extraction. Your primary responsibility will be to develop scripts and APIs capable of scraping public Instagram influencer data, including profile information, demographic indicators, static details, and recent posts.

note: This role requires deep knowledge of web scraping techniques, anti-bot evasion, proxy rotation, and handling modern web security mechanisms.

Key Responsibilities

1. Develop high-quality Python scraping scripts to extract data from public Instagram profiles.

2. Scrape and structure influencer data including: Username, full name, bio, profile picture, Follower/following counts, Country, category, niche (from public indicators), Recent posts (images, reels metadata, captions, likes, comments), Engagement-related statistics

3. Build APIs or data pipelines to deliver scraped data as clean JSON.

4. Implement proxy rotation, user-agent rotation, and anti-bot bypass techniques to maintain scraping stability.

5. Handle authentication cookies, session management, and dynamic content scraping.

6. Ensure scraping runs efficiently at scale with low error rates.

7. Maintain and monitor scraping performance, solve captchas or rate-limit issues when needed.

8. Write clean, maintainable code with proper documentation. (for the backend team)

9. Keep track of changes in Instagram’s structure and update scrapers accordingly.

Required Skills & Experience

1. Strong expertise in Python and libraries such as: Requests, aiohttp, BeautifulSoup, lxml, Selenium / Playwright, Async programming (bonus)

2. Deep understanding of web scraping fundamentals, including: HTML parsing, Reverse engineering API endpoints, Handling JavaScript-rendered pages, Browser automation

3. Experience with anti-bot evasion techniques: Proxy networks (residential/datacenter), User-agent spoofing, Session / cookie reuse, Captcha solving APIs (bonus)

4. Experience building APIs to expose scraped data (for main backend)

5. Familiarity with Instagram structure, public data endpoints, and common scraping patterns.

6. Ability to work with datasets and output well-structured JSON.

7. Strong debugging skills and ability to handle rate-limit scenarios.

8. Knowledge of secure data handling, error recovery, and scaling scrapers.

Job Type: Full-time

Pay: ₹25,000.00 - ₹30,000.00 per month

Work Location: In person