Python Developer - Web Crawling/Scraping

Role & Responsibilities

• Collaborate with cross-functional teams to define, design, and implement new features.
• Ensure the performance, quality, and responsiveness of web crawling systems.
• Identify and correct bottlenecks and fix bugs in web crawling processes.
• Help maintain code quality, organization, and automation.
• Stay up-to-date with the latest industry trends and technologies.
  

Skills

• 3+ years of experience in Web Scraping or Crawling through Scrapy, Selenium or other frameworks and related libraries (like beautifulsoup, puppeteer).
• Should be expert on latest version of Python.
• Should have very good experience on fetching data from multiple online sources, cleanse it and build APIs on top of it.
• Good understanding of data structure and algorithms, as well as how they affect system performance in real world applications
• Sound knowledge in bypassing Bot Detection techniques.
• Web RestFul APIs / Microservices Development Experience.
• Think deeply about developing large scale scraping tools including data integrity, health and monitoring systems.
• Develop a deep understanding of our vast data sources on the web and know exactly how, when, and which data to scrape, parse and store.
• Work with SQL and NoSQL databases to store raw data.
• Develop frameworks for automating and maintaining constant flow of data from multiple sources.
• Good knowledge of distributed technologies, real-time systems of high throughput, low latency, and highly scalable systems.
• Work independently with little supervision to research and test innovative solutions.
• Should have a strong passion for coding.
• Must take quality, security and performance seriously.
• Ability to pair with other engineers and cross-team as needed.
• Excellent communication skills, including the ability to present effectively to both business and technical
  

Why Join BYLD

• Learn directly from industry experts with 20+ years of experience
• Opportunity to work in a dynamic and collaborative environment.
• Professional development and growth opportunities.
• Gain experience in world class management practices
• Insurance Benefits (Medical and Accidental) for all employees
• Multi-level Rewards programs for all employees