Job
Description
Job Title: Sr. Python Developer – Web Scraper Location: Ahmedabad, Gujarat Job Type: Full Time Experience: 3+ Years Department: Python About Simform: Simform is a premier digital engineering company specializing in Cloud, Data, AI/ML, and Experience Engineering to create seamless digital experiences and scalable products. Simform is a strong partner for Microsoft, AWS, Google Cloud, and Databricks. With a presence in 5+ countries, Simform primarily serves North America, the UK, and the Northern European market. Simform takes pride in being one of the most reputed employers in the region, having created a thriving work culture with a high work-life balance that gives a sense of freedom and opportunity to grow. Role Overview: The Sr. Python Developer – Web Scraper will be responsible for building scalable, resilient, and intelligent web scraping solutions to extract structured and unstructured data from websites, APIs, and digital platforms. This role involves handling anti-bot mechanisms, dynamic content rendering, and real-time data extraction pipelines, while ensuring high data quality and performance. The ideal candidate will have strong Python skills, deep understanding of scraping techniques, and experience working with headless browsers and cloud platforms. Key Responsibilities: Design and implement scalable web scraping frameworks to collect data from complex and dynamic websites. Develop custom spiders/crawlers using Python libraries like Playwright, Puppeteer, Selenium, Scrapy, or BeautifulSoup. Apply advanced anti-bot evasion strategies such as CAPTCHA solving, IP rotation, user-agent spoofing, browser fingerprinting, and session/cookie management. Automate scraping tasks across distributed systems using tools like Celery, Airflow, cron, and ETL orchestration platforms. Optimize scraper performance for speed, accuracy, and resilience to website structure changes. Implement network interception, DOM traversal, WebSocket handling, and headless browser control. Store and manage scraped data in cloud or local storage using PostgreSQL, MongoDB, or S3. Integrate scraping systems with APIs or microservices for data consumption and downstream workflows. Monitor scraper reliability and handle retry logic, error logging, and dynamic throttling. Write modular, well-documented, and testable Python code with proper unit testing and version control. Collaborate with engineers, data scientists, and stakeholders to understand scraping goals and align them with business needs. Required Skills & Qualifications: Bachelor’s/Master’s degree in Computer Science, Engineering, or a related field. 3+ years of experience in Python development with specialization in web scraping. Deep understanding of modern anti-scraping defenses and bypass techniques (e.g., CAPTCHA, IP bans, dynamic rendering). Proficiency with headless browser tools like Playwright, Puppeteer, or Selenium. Strong grasp of DOM manipulation, JavaScript execution, network inspection, and asynchronous scraping using asyncio, aiohttp, etc. Experience in handling large-scale data extraction and storage using SQL and NoSQL databases. Hands-on experience deploying scrapers and automation workflows on AWS, GCP, or Azure. Familiarity with containerization using Docker and optional experience with Kubernetes. Comfortable with REST API integration, job scheduling, and microservices-based environments. Strong debugging, optimization, and testing skills. Clear understanding of legal and ethical scraping boundaries. Why Join Us: Young Team, Thriving Culture Flat-hierarchical, friendly, engineering-oriented, and growth-focused culture Well-balanced learning and growth opportunities Free health insurance Office facilities with a game zone, in-office kitchen with affordable lunch service, and free snacks Sponsorship for certifications/events and library service Flexible work timing, leaves for life events, WFH, and hybrid options
