Lead Data Scraping

Job Title

Location

Experience Level

Employment Type

Job Summary:

We are seeking a highly skilled and experienced Lead Data Scraping Engineer to join our team. The ideal candidate will have a minimum of 4 years of hands-on experience in IT scraping, with at least 2 years leading a team of 5+ developers. This role requires deep technical knowledge in advanced scraping techniques, reverse engineering, automation, and leadership skills to drive the team towards success.

Key Responsibilities:

• Design and develop scalable data scraping solutions using tools like Scrapy and

Python libraries.

• Lead and mentor a team of 5+ developers, managing project timelines and

deliverables.

• Implement advanced blocking and captcha-solving techniques to bypass scraping restrictions.

• Conduct source code reverse engineering and automate web and app interactions.

• Manage proxies, IP rotation, and SSL unpinning to ensure effective scraping.

• Maintain and improve API integrations and data pipelines.

• Ensure code quality through effective version control, error handling, and documentation.

• Collaborate with cross-functional teams for project planning and execution.

• Monitor performance and provide solutions under high-pressure environments.

Required Skills and Experience:

• Data Scraping: Minimum 4 years in IT scraping industry

• Leadership: Minimum 2 years leading a team of 5+ developers

• Scraping Tools: Scrapy, Threading, requests, web automation

• Technical Proficiency:

o Advanced Python

o Captcha solving and blocking handling

o Source reverse engineering

o Proxy management & IP rotation

o App automation, SSL Unpin, Frida

o API Management, Version Control Systems

o Error Handling, SQL, MongoDB, Pandas

Leadership Skills:

• Basic project management

• Moderate documentation

• Team handling

• Pressure management

• Flexibility and adaptability

• High accountability

Preferred (Good to Have):

• Experience with Linux

• Knowledge of Appium, Fiddler, Burp Suite