Software Engineer

Software Engineer - Content Parsing

The Opportunity

data quality, comprehensive testing, and the ability to implement effective alerting and notification systems.

Responsibilities :

• Design, develop, and maintain robust and scalable HTML parsing solutions to extract diverse web content.
• Implement advanced content categorization logic to accurately classify and tag extracted data based on predefined schemas and business rules, incorporating AI/ML techniques where applicable.
• Develop and integrate alerting and notification systems to monitor parsing performance, identify anomalies, and report on data quality issues.
• Write comprehensive unit, integration, and end-to-end test cases to ensure the accuracy, reliability, and robustness of parsing logic, covering all boundary conditions and edge cases.
• Optimize parsing performance and efficiency to handle large volumes of data.
• Troubleshoot and resolve parsing issues, adapting to changes in website structures and content formats.
• Contribute to the continuous improvement of our parsing infrastructure and methodologies, including the research and adoption of new AI-driven parsing techniques.
• Manage and deploy parsing solutions in a Linux environment.
• Collaborate with DevOps engineers to improve the scaling, deployment, and operational efficiency of parsing solutions.
• This role requires occasional weekend work as content changes are typically deployed on weekends, necessitating monitoring and immediate adjustments.

Qualifications :

• Bachelor's degree in Computer Science or a closely related technical field is required.
• Experience in software development with a strong focus on data extraction and parsing.
• Proficiency in Python and its ecosystem, particularly with libraries for web scraping and parsing (e.g., Beautiful Soup, lxml, Scrapy, Playwright, Selenium).
• Demonstrated experience in building or parsing complex and unstructured HTML content into structured data formats.
• Understanding and practical experience with content categorization techniques (e.g., keyword extraction, rule-based classification, basic NLP concepts).
• Proven ability to design and implement effective alerting and notification systems (e.g., integrating with Slack, PagerDuty, email, custom dashboards).
• Attention to details with unit testing skills, with a meticulous approach to covering all boundary conditions, error cases, and edge scenarios.
• Experience working in a Linux environment, including shell scripting and command-line tools. Familiarity with data storage solutions (e.g., SQL databases) and data serialization formats (e.g., JSON, XML.
• Experience with version control systems (e.g., Git). Excellent problem-solving skills.
• Strong communication and collaboration abilities.