eMedEvents - Python Developer - Web Scraping

Python Developer Web Scraping & Data Processing

Experience :

Employment Type :

Job Overview

Key Responsibilities

• Develop scalable Python scripts to effectively scrape and parse biomedical data from a wide range of web sources, including websites, pre-print servers, citation indexes, scientific journals, and treatment guidelines.
• Build robust modules specifically for splitting multi-record documents (such as PDFs, HTML, and other formats) into individual, manageable content units.
• Implement NLP-based field extraction pipelines utilizing libraries like spaCy, NLTK, or advanced regex for precise metadata tagging.
• Design and automate complex data acquisition workflows using schedulers and orchestrators like cron, Celery, or Apache Airflow for periodic scraping and content updates.
• Store parsed and processed data efficiently in both relational (PostgreSQL) and NoSQL (MongoDB) databases, ensuring optimal schema design for performance and scalability.
• Ensure robust logging, comprehensive exception handling, and rigorous content quality validation across all data processing and scraping workflows.
  

Required Skills And Qualifications

• 3+ years of hands-on experience in Python, particularly focused on data extraction, transformation, and loading (ETL).
• Strong command over web scraping libraries, including :
• BeautifulSoup
• Scrapy
• Selenium
• Playwright
• Proficiency in PDF parsing libraries, such as :
• PyMuPDF
• pdfminer.six
• PDFPlumber
• Experience with HTML/XML parsers: lxml, XPath, html5lib.
• Familiarity with regular expressions, NLP concepts, and advanced field extraction techniques.
• Working knowledge of SQL and/or NoSQL databases (MySQL, PostgreSQL, MongoDB).
• Understanding of API integration (RESTful APIs) for interacting with structured data sources.
• Experience with task schedulers and workflow orchestrators (cron, Apache Airflow, Celery).
• Proficiency in version control using Git/GitHub and comfort working in collaborative development environments.
  

Good To Have

• Exposure to biomedical or healthcare data parsing (scientific abstracts, clinical trials data, drug labels).
• Familiarity with cloud environments like AWS (specifically Lambda, S3 for data storage and processing).
• Experience with data validation frameworks and building robust QA rules for data quality.
• Understanding of ontologies and taxonomies (UMLS, MeSH) for structured content tagging.