Python Developer – Web Scraping & Automation (3 to 5 yrs)

Python Developer – Web Scraping & Automation (WFH)

Experience:

Location:

Mode of Engagement:

No of Positions:

Educational Qualifications:

Industry:

Notice Period:

What We Are Looking For:

• Strong hands-on experience in 

  Python

   for large-scale web scraping and automation. 

• Expert in web scraping using Scrapy, BeautifulSoup, Selenium, Playwright, and Requests.

   
• Excellent understanding of 

  HTML, CSS, DOM structure

  , and browser-rendered content. 
• Experience in backend/API development using 

  FastAPI, Django, or Flask

  . 
• Ability to handle proxies, user-agent rotation, cookies, sessions, and anti-bot mechanisms. 
• Strong debugging, error-handling, and optimization skills. 
• Ability to independently deliver reliable and scalable data extraction solutions. 

Key Responsibilities:

• Develop, optimize, and maintain 

  web scraping scripts and automation pipelines

   in Python. 
• Extract data from static, dynamic, and JavaScript-heavy websites using browser automation. 
• Manage 

  pagination, authentication of workflows, cookies, sessions

  , and rendering challenges. 
• Build and maintain 

  REST APIs

   to support data access and processing. 
• Implement end-to-end pipelines for data 

  cleaning, transformation, and storage

   (CSV, JSON, DB). 
• Work with 

  SQL/NoSQL databases

   to store and manage scraped datasets. 
• Use 

  Docker

   for containerization and deploy scraping tools to cloud environments. 
• Troubleshoot scraping failures, automate recovery, and ensure high data accuracy. 
• Maintain logs, monitoring, and documentation for scraping workflows. 

Qualifications:

• Strong programming experience in 

  Python

  . 
• Proficiency with 

  Scrapy, Beautiful Soup, Selenium, Playwright, and Requests

  . 
• Understanding HTTP, APIs, headers, cookies, sessions, and web protocols. 
• Experience with backend frameworks: 

  FastAPI / Django / Flask

  . 
• Experience working with 

  Docker

   and basic CI/CD workflows 

  is good to have

  . 
• Knowledge of relational and non-relational databases. 
• Strong debugging, optimization, and troubleshooting abilities.