Jr. Web Scraping Engineer

Role & responsibilities

Job Overview:

We are looking for a highly motivated Junior Data Engineer with a passion for web scraping and web crawling to join our team. The ideal candidate will have strong Python programming skills and experience with web scraping frameworks and libraries like Requests, BeautifulSoup, Selenium, Playwright or URLlib. You will be responsible for building efficient and scalable web scrapers, extracting valuable data, and ensuring data integrity.

This role requires a keen eye for problem-solving, the ability to work with complex data structures, and a strong understanding of web technologies like HTML, CSS, DOM, XPATH, and Regular Expressions. Knowledge of JavaScript would be an added advantage.

Responsibilities:

• As a Web Scraper, your role is to apply your knowledge set to fetch data from multiple
• online sources
• Developing highly reliable web Scraper and parsers across various websites
• Extract structured/unstructured data and store them into SQL/No SQL data store
• Work closely with Project/Business/Research teams to provide scrapped data for analysis
• Maintain the scraping projects delivered to production
• Develop frameworks for automating and maintaining constant flow of data from multiple
• sources
• Work independently with minimum supervision
• Develop a deep understanding of the data sources on the web and know exactly how,

when, and which data to scrap, parse and store this data

Required Skills and Experience:

• Experience as Web Scraper of 1 to 2 years.
• Proficient knowledge in Python language and working knowledge

of Web Crawling/Web scraping in Python Requests, Beautifulsoup or URLlib and

Selenium, Playwright.

• Must possess strong knowledge of basic Linux commands for system navigation,

management, and troubleshooting.

• Must have expertise in proxy usage to ensure secure and efficient network operations.
• Must have experience with captcha-solving techniques for seamless automation and

data extraction.

• Experience with data parsing - Strong knowledge of Regular expression, HTML, CSS,

DOM, XPATH. Knowledge of Javascript would be a plus

Preferred candidate profile

• Must be able to access, manipulate, and transform data from a variety of database and flat file sources. MongoDB & MYSQL skills are essential.

• Must possess strong knowledge of basic Linux commands for system navigation,

management, and troubleshooting.

• Must be able to develop reusable code-based scraping products which can be used by

others.

• GIT knowledge is mandatory for version control and collaborative development

workflows.

• Must have experience handling cloud servers on platforms like AWS, GCP, and

LEAPSWITCH for scalable and reliable infrastructure management.

• Ability to ask the right questions and deliver the right results in a way that is

understandable and usable to your clients.

• A track record of digging in to the tough problems, attacking them from different angles,

and bringing innovative approaches to bear is highly desirable. Must be capable of selfteaching new techniques.

Behavioural expectations:

• Be excited by and have positive outlook to navigate ambiguity

• Passion for results and excellence

• Team player

• Must be able to get the job done by working collaboratively with others

• Be inquisitive and an analytical mind; out-of-the-box thinking

• Prioritize among competing opportunities, balance consumer needs with business and

product priorities, and clearly articulate the rationale behind product decisions

• Straightforward and professional

• Good communicator

• Maintain high energy and motivate

• A do-it-yourself orientation, consistent with the companys roll-up the- sleeves culture

• Proactive