Web Scraping with Python & JavaScript (3 to 6 years)

Web Scraping with Python & JavaScript (3 to 6 years)

Experience: 3+ Years 

Location: Remote

Mode of Engagement: Full-time

No of Positions: 3

Educational Qualifications: Bachelor's degree in Computer Science, Information Technology

Industry: IT / Software Development

Notice Period: Immediate Joiners Preferred

What We Are Looking For:

• Strong expertise in web scraping with hands-on experience in large-scale data collection systems.
• Proficient in Python, including Scrapy, Requests, Playwright, and Selenium.
• Skilled in JavaScript for web scraping and automation tasks.
• Capable of designing, building, and maintaining robust and scalable scrapers for a variety of websites.
• Experience with AI-driven techniques to improve scraping efficiency, data extraction accuracy, and automation workflows.
• Passionate about leveraging advanced technologies to solve complex data challenges

Key Responsibilities

• Design, develop, and maintain large-scale web scrapers to collect structured and unstructured data from multiple sources.
• Optimize scraping pipelines for scalability, reliability, and performance.
• Work with Scrapy and related Python frameworks/libraries for crawler development.
• Write JavaScript-based scrapers or automation scripts for complex websites with dynamic content.
• Monitor, debug, and maintain scrapers, ensuring high availability and fault tolerance.
• Implement AI/ML-based solutions in scraping workflows, such as:
• Using LLMs for content parsing/cleaning.
• Automating CAPTCHA solving and anti-bot evasion.
• Improving data matching, deduplication, and entity recognition.
• Collaborate with data engineers and product teams to ensure scraped data quality and usability.
• Maintain proper documentation, testing, and CI/CD practices for scraper code.

Qualifications

• Bachelor's degree in computer science, IT, or related field.
• Strong programming skills in Python (Scrapy, Requests, Playwright/Selenium).
• Proficiency in JavaScript for handling browser automation and dynamic sites.
• Experience with large-scale scraping systems (distributed crawling, proxy rotation, request throttling).
• Solid understanding of web technologies (HTTP, HTML, CSS, JSON, AJAX).
• Experience working with databases (SQL/NoSQL) for storing and processing scraped data.
• Hands-on experience in AI/ML tools (e.g., OpenAI, Hugging Face, spaCy, Transformers, or LangChain) applied to data extraction/processing.
• Familiarity with containerization and orchestration (Docker, Kubernetes) for scraper deployments.
• Strong problem-solving skills for dealing with anti-bot measures and rate limits.
• Excellent communication and collaboration skills.