315 Scrapy Jobs - Page 10

About Sellmark Sellmark is committed to creating brands that foster memories and traditions by producing industry-leading outdoor lifestyle products. We promote a healthy outdoor lifestyle and drive innovation through positive leadership, strong ethics, and unwavering dedication. Our team-oriented culture encourages self-growth, mutual respect, and a passion for excellence—both at work and beyond. We seek individuals who bring passion to everything they do, instill confidence, trust, and respect, and inspire success while building strong relationships. If you're looking for a dynamic, professional, and supportive team, we’d love to have you join us. Job Summary We are seeking a hands-on Web Scraping & Automation Engineer to join our India-based analytics team. You’ll build proprietary scrapers to extract business data from public data sources to support US market lead generation. Over time, your skills will also be used in marketing, supply chain, new product development, and more per the needs of the organization. This is a fast-paced, outcome-oriented role, with direct business impact and a focus on innovation and reliability. - Develop modular and scalable web scraping tools (Python preferred) for structured data collection - Target platforms: Maps (Google, Apple, Bing), Government websites, Review Websites (Yelp), Independent retailer / competitor websites, E-commerce (Amazon, Scheels, Academy, Ebay), etc. - Integrate proxy rotation, captcha handling, and anti-blocking techniques - Export clean, structured data (CSV, JSON) for enrichment and analysis - Collaborate with Data Analyst for QA and reporting - Optimize scrapers for speed, efficiency, and long-term maintainability - Provide weekly progress reports and error logs to stakeholders Qualifications - Bachelor’s degree or diploma in any discipline. Data Science, Statistics, Economics preferred - 1–3 years of hands-on experience in scraping (BeautifulSoup, Selenium, Playwright, Scrapy, etc. - Familiarity with anti-bot strategies (user agents, proxies, time delays). - Comfort with Git, API requests, and basic Linux environments - Ability to troubleshoot and adapt to dynamic site structures - Experience scraping map-based platforms) - Exposure to scheduling tools (Airflow, CRON jobs) - Interest in U.S. retail/Distribution or tactical gear sectors - High attention to detail, consistency, and ability to meet deadlines. - Excellent written and verbal communication skills Work Environment & Physical Requirements While performing the duties of this job, the employee may be required to sit or stand for extended periods of time. The employee may be required to bend, twist, reach, push, pull and operate office machinery. Must be able to lift up to (50) pounds. Specific work assignments may change without notice. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions of the job. This role is based out of our Bangalore office and follows a standard Monday to Friday, 10:30 AM to 7:30 PM schedule. We do not offer hybrid or remote-first arrangements. We value the energy and collaboration that come from working together in person, and believe it plays a key role in building team culture, sparking creativity, and maintaining focus. Schedules may vary minimally depending on business needs and may occasionally require flexibility outside normal business hours. Benefits Competitive salary based on experience. Growth path within Sellmark’s global Anaytics, Insights and Data function. Wellness and development-focused company culture. Paid time off and holiday policy aligned with local regulations. Tools and support to succeed in a fast-paced, data-focused environment. Disclaimer The above information is intended to describe the general nature and level of work being performed. It is not intended to be an exhaustive list of responsibilities, duties, or skills required. Requirements for this job may be subject to change to meet business needs. Sellmark Corporation is an Equal Opportunity Employer. We do not discriminate based on race, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, genetic information, veteran status, or any other legally protected status. We are committed to fostering an inclusive and diverse workplace where all individuals feel valued and respected. Job Type: Full-time Pay: ₹40,000.00 - ₹50,000.00 per month Benefits: Health insurance Life insurance Provident Fund Schedule: Day shift Monday to Friday Ability to commute/relocate: Bangalore, Karnataka: Reliably commute or planning to relocate before starting work (Preferred) Application Question(s): What is the earliest date that you are able to begin working in the office? Experience: Web scraping: 3 years (Preferred) Work Location: In person

About Us: Upscrape is a fast-growing data automation and web scraping company building advanced scraping systems, custom data pipelines, and API-driven solutions for enterprise clients. We work on complex real-world challenges that require precision, scale, and expertise. As we continue to grow, we are looking to bring on an experienced developer to join our core technical team. Position Overview: We are hiring a full-time Python developer with strong experience in web scraping, browser automation, and backend API development. The ideal candidate has previously built production-level scraping systems, understands anti-bot protections, and can independently manage end-to-end data extraction workflows. This is a highly focused technical role, ideal for someone who enjoys solving real-world scraping challenges and working on meaningful projects that deliver immediate impact. Key Responsibilities: Build and maintain robust web scraping pipelines for dynamic, heavily protected websites. Develop backend APIs to serve and manage scraped data. Handle browser-based scraping using tools such as Playwright, Selenium, or Puppeteer. Implement advanced proxy management, IP rotation, and anti-blocking mechanisms. Ensure efficient error handling, retries, and system stability at scale. Collaborate closely with the founder and technical team to deliver client projects on time. Required Experience & Skills: 1+ years of hands-on scraping experience . Python (Playwright, Selenium, Requests, Async/Aiohttp, Scrapy, etc.). Experience bypassing anti-bot protections (Cloudflare, captchas, WAFs, bot detection). Proxy management at scale (residential, rotating proxies, IP pools). REST API development (Flask / FastAPI preferred). Database experience (MongoDB, PostgreSQL). Version control (Git), Docker, and Linux-based environments. Strong debugging and problem-solving ability. Clear, consistent communication. Bonus (Nice to Have): Experience with AI-powered data enrichment (LLMs, OCR, GPT-4 integrations). Familiarity with large-scale scraping architectures (millions of records). Previous work in SaaS, APIs, or productized data services. The Right Fit: We are looking for a developer who is: Self-driven - takes full ownership of tasks. Technically strong - has delivered real-world scraping solutions. Highly responsive - available for fast-paced collaboration across time zones. Outcome-focused - understands that clean, working systems matter more than theory. What We Offer: Remote full-time position. Stable long-term role with growth potential. Direct, efficient communication, no corporate bureaucracy. Work on meaningful projects with direct client impact. Competitive compensation based on skill and experience. How to Apply (Important Filter): In your application, please include: Links or code samples of scraping projects you’ve built Which tools and libraries you are most comfortable with A short explanation of how you approach extracting data from modern, highly dynamic websites that require advanced automation and protection handling.

Responsibilities: * Design, develop, test and maintain scalable Python applications using Scrapy, Selenium and Requests. * Implement anti-bot systems and data pipeline solutions with Airflow and Kafka. Share CV on recruitment@fortitudecareer.com Flexi working Work from home

Job Title: Data Engineer Location: Chennai, India Experience: 5+ years Work Mode: Full-time (9am-6:30pm), In-office (Monday to Friday) Department: Asign Data Sciences About Us At Asign, we are revolutionizing the art sector with our innovative digital solutions. We are a passionate and dynamic startup dedicated to enhancing the art experience through technology. Join us in creating cutting-edge products that empower artists and art enthusiasts worldwide. Role Overview We are looking for an experienced Data Engineer with a strong grasp of ELT architecture and experience to help us build and maintain robust data pipelines. This is a hands-on role for someone passionate about structured data, automation, and scalable infrastructure. The ideal candidate will be responsible for sourcing data, ingesting, transforming, storing, and making data accessible and reliable for data analysis, machine learning, and reporting. You will play a key role in maintaining and evolving our data architecture and ensuring that our data flows efficiently and securely. Key Responsibilities ● Design, develop, and maintain efficient and scalable ELT data pipelines. ● Work closely with the data science and backend teams to understand data needs and transform raw inputs into structured datasets. ● Integrate multiple data sources including APIs, web pages, spreadsheets, and databases into a central warehouse. ● Monitor, test, and continuously improve data flows for reliability and performance. ● Create documentation and establish best practices for data governance, lineage, and quality. ● Collaborate with product and tech teams to plan data models that support business and AI/ML applications. Required Skills ● Minimum 5 years of hands-on experience in data engineering. ● Solid understanding and experience with ELT pipelines and modern data stack tools. ● Practical knowledge of one or more orchestrators (Dagster, Airflow, Prefect, etc.). ● Proficiency in Python and SQL. ● Experience working with APIs and data integration from multiple sources. ● Familiarity with one or more cloud data warehouses (e.g., Snowflake, BigQuery, Redshift). ● Strong problem-solving and debugging skills. Qualifications: Must-have: ● Bachelor’s/Master’s degree in Computer Science, Engineering, Statistics, or a related field ● Proven experience (5+ years) in data engineering, data integration, and data management ● Hands-on experience in data sourcing tools and frameworks (e.g. Scrapy, Beautiful Soup, Selenium, Playwright) ● Proficiency in Python and SQL for data manipulation and pipeline development ● Experience with cloud-based data platforms (AWS, Azure, or GCP) and data warehouse tools (e.g. Redshift, BigQuery, Snowflake) ● Familiarity with workflow orchestration tools (e.g. Airflow, Prefect, Dagster) ● Strong understanding of relational and non-relational databases (PostgreSQL, MongoDB, etc.) ● Solid understanding of data modeling, ETL best practices, and data governance principles ● Systems knowledge and experience working with Docker. ● Strong and creative problem-solving skills and the ability to think critically about data engineering solutions. ● Effective communication and collaboration skills ● Ability to work independently and as part of a team in a fast-paced, dynamic environment. Good-to-have: ● Experience working with APIs and third-party data sources ● Familiarity with version control (Git) and CI/CD processes ● Exposure to basic machine learning concepts and working with data science teams ● Experience handling large datasets and working with distributed data systems Why Join Us? ● Innovative Environment: Be part of a forward-thinking team that is dedicated to pushing the boundaries of art and technology. ● Career Growth: Opportunities for professional development and career advancement. ● Creative Freedom: Work in a role that values creativity and encourages new ideas. ● Company Culture: Enjoy a dynamic, inclusive, and supportive work environment.

Note: Please don't apply if you do not have at least 5 years of Scrapy experience Location: KOLKATA ------------ We are seeking a highly experienced Web Scraping Expert (Python) specialising in Scrapy-based web scraping and large-scale data extraction. This role is focused on building and optimizing web crawlers, handling anti-scraping measures, and ensuring efficient data pipelines for structured data collection. The ideal candidate will have 6+ years of hands-on experience developing Scrapy-based scraping solutions, implementing advanced evasion techniques, and managing high-volume web data extraction. You will collaborate with a cross-functional team to design, implement, and optimize scalable scraping systems that deliver high-quality, structured data for critical business needs.Key Responsibilities Scrapy-based Web Scraping Development Develop and maintain scalable web crawlers using Scrapy to extract structured data from diverse sources. Optimize Scrapy spiders for efficiency, reliability, and speed while minimizing detection risks. Handle dynamic content using middlewares, browser-based scraping (Playwright/Selenium), and API integrations. Implement proxy rotation, user-agent switching, and CAPTCHA solving techniques to bypass anti-bot measures. Advanced Anti-Scraping Evasion Techniques Utilize AI-driven approaches to adapt to bot detection and prevent blocks. Implement headless browser automation and request-mimicking strategies to mimic human behavior. Data Processing & Pipeline Management Extract, clean, and structure large-scale web data into structured formats like JSON, CSV, and databases. Optimize Scrapy pipelines for high-speed data processing and storage in MongoDB, PostgreSQL, or cloud storage (AWS S3). Code Quality & Performance Optimization Write clean, well-structured, and maintainable Python code for scraping solutions. Implement automated testing for data accuracy and scraper reliability. Continuously improve crawler efficiency by minimizing IP bans, request delays, and resource consumption. Required Skills and Experience Technical Expertise 5+ years of professional experience in Python development with a focus on web scraping. Proficiency in using Scrapy based scraping Strong understanding of HTML, CSS, JavaScript, and browser behavior. Experience with Docker will be a plus Expertise in handling APIs (RESTful and GraphQL) for data extraction. Proficiency in database systems like MongoDB, PostgreSQL Strong knowledge of version control systems like Git and collaboration platforms like GitHub. Key Attributes Strong problem-solving and analytical skills, with a focus on efficient solutions for complex scraping challenges. Excellent communication skills, both written and verbal. A passion for data and a keen eye for detail Why Join Us? Work on cutting-edge scraping technologies and AI-driven solutions. Collaborate with a team of talented professionals in a growth-driven environment. Opportunity to influence the development of data-driven business strategies through advanced scraping techniques. Competitive compensation and benefits.

Job Title: Data Analyst (with Data Visualization & Web Scraping Skills) Location: Hyderabad Experience Required: 1-2 Years Employment Type: Full-time J ob Summary: We are seeking a highly motivated and detail-oriented Data Analyst with strong expertise in data visualization, data analysis, and web scraping . The ideal candidate will be responsible for collecting, processing, and analyzing large sets of data from multiple sources and presenting them in meaningful formats to support business decisions. K ey Responsibilities: Perform end-to-end data analysis and reporting using structured and unstructured data. Build insightful and interactive dashboards and reports using tools like Power BI, Tableau, or Google Data Studio. Conduct data scraping/web scraping using tools like Python (BeautifulSoup, Selenium, Scrapy), APIs, or browser automation. Clean, transform, and validate data using tools such as Excel, SQL, or Python (pandas, numpy). Collaborate with cross-functional teams to understand data requirements and deliver actionable insights. Monitor data quality, consistency, and security in automated scraping workflows. Provide ad hoc data reports and support business teams in decision-making through analytical insights. Key Skills & Tools: Data Visualization: Power BI, Tableau, Looker, or Google Data Studio Data Analysis: SQL, Excel, Python (pandas, matplotlib, seaborn) Web/Data Scraping: BeautifulSoup, Selenium, Scrapy, Python scripting, API integration Database Knowledge: MySQL, PostgreSQL, or similar Other Tools (optional): Google Sheets, Jupyter Notebook, Git, Airflow Qualifications: Bachelor’s degree in Computer Science, Statistics, Engineering, or related field 2–5 years of hands-on experience in data analytics and scraping projects Strong understanding of data integrity, compliance, and ethical scraping practices Preferred Attributes: Strong analytical and problem-solving mindset Ability to handle large datasets efficiently Experience working in Agile/fast-paced environments Familiarity with data warehouses or cloud data platforms (e.g., AWS, BigQuery) Job Type: Full-time Pay: ₹250,000.00 - ₹400,000.00 per year Benefits: Provident Fund Schedule: Day shift Supplemental Pay: Overtime pay Work Location: In person

Interested candidates may share their profiles at gauri.shedge@ambitsoftware.com Below is the Job Description for your reference: Roles & Responsibilities: Design, develop and maintain Scrapy web crawlers. Leverage the platform and open-source projects to perform distributed information extraction, retrieval, and data processing. Identify and resolve performance and scalability issues with distributed crawling at scale. Help identify, debug and fix problems with open source projects, including Scrapy. Requirements: 3+ years of software development experience in Python. Solid Python knowledge. Familiarity with Linux/UNIX, HTTP, HTML, JavaScript, and Networking. Good communication skills in written and spoken English. Availability to work full time. Bonus points for: Scrapy experience is a big plus. Familiarity with techniques and tools for crawling, extracting, and processing data (e.g., Scrapy, NLTK, pandas, scikit-learn, mapreduce, NoSQL, etc). Proficient English. Tech Stack Elements: Python / Javascript, Scrapy, AI Spider Templates, Page Objects, Spidermon, API(+ Extraction) and Scrapy Cloud. HTML/CSS, HTTP and web protocols, Javascript and Dynamic content. Data cleaning and processing, data storage, workflow automation. IP rotation and proxies, user-agent rotation, captcha handling, rate limiting / throttling, device fingerprinting obfuscation.

Designation- Python Developer & Web Crawling Experience- 3+ Years Location- Gurgaon, Haryana ABOUT US: Founded in 1998, BYLD is the largest group in the South Asian region, offering technology-enabled HR and business productivity solutions. We have served 5,00,000+ individuals, worked with more than 50% of the Fortune 500 and over 60% of the Business World top 1000 companies. Please read about us www.byldgroup.com Role & Responsibilities: Collaborate with cross-functional teams to define, design, and implement new features. Ensure the performance, quality, and responsiveness of web crawling systems. Identify and correct bottlenecks and fix bugs in web crawling processes. Help maintain code quality, organization, and automation. Stay up-to-date with the latest industry trends and technologies. Skills: 3+ years of experience in Web Scraping or Crawling through Scrapy, Selenium or other frameworks and related libraries (like BeautifulSoup, puppeteer). Should be an expert on the latest version of Python. Should have very good experience on fetching data from multiple online sources, cleanse it and build APIs on top of it. Good understanding of data structure and algorithms, as well as how they affect system performance in real world applications Sound knowledge in bypassing Bot Detection techniques. Web RestFul APIs / Microservices Development Experience. Think deeply about developing large scale scraping tools including data integrity, health and monitoring systems. Develop a deep understanding of our vast data sources on the web and know exactly how, when, and which data to scrape, parse and store. Work with SQL and NoSQL databases to store raw data. Develop frameworks for automating and maintaining constant flow of data from multiple sources. Good knowledge of distributed technologies, real-time systems of high throughput, low latency, and highly scalable systems. Work independently with little supervision to research and test innovative solutions. Should have a strong passion for coding. Must take quality, security and performance seriously. Ability to pair with other engineers and cross-team as needed. Excellent communication skills, including the ability to present effectively to both business and technical Interested candidates can share updated cv at talentacquisition.aad@byldgroup.com

We are seeking a highly skilled Scraping Expert to join our data team and drive automated extraction of tender information from various Indian portals, including the Government e-Marketplace (GeM). The ideal candidate will leverage advanced scraping techniques, Optical Character Recognition (OCR), and data validation methods to collect, process, and deliver high-quality tender datasets for our stakeholders. Key Responsibilities Develop and maintain robust web-scraping pipelines to extract tender information from GeM, state/central portals, and private aggregators. Integrate OCR tools to accurately extract text from scanned PDFs and tender-related images. Parse, normalize, and structure scraped tender data (e.g., tender ID, project title, department, dates, values). Monitor scraping jobs with automated alerts and update scripts for GeM UI changes. Collaborate with analysts and product teams to align data outputs with business requirements. Required Qualifications 3–5 years of hands-on experience building and deploying web-scraping solutions using Python frameworks (BeautifulSoup, Scrapy, Selenium). Proficiency with OCR libraries (Tesseract, OpenCV) and techniques for processing scanned documents. Prior experience working with or scraping data from the GeM portal or other Indian tender websites. Strong data cleaning, transformation, and storage skills using SQL/noSQL databases or data warehouses. Excellent problem-solving abilities to troubleshoot IP blocks, dynamic content, and anti-bot measures. Technical Skills Python BeautifulSoup Apache Airflow AWS (e.g., EC2, S3, Lambda) Job Type: Full-time Pay: ₹50,000.00 - ₹70,000.00 per month Schedule: Day shift Experience: Apache Airflow: 2 years (Required) Work Location: In person

Job Description: This position is for a senior software engineer who is passionate about solving challenging business problems and has - ✓ Strong experience in developing ETL pipeline using SQL, T-SQL, stored procedures, SQL server, performance tuning of queries and stored procedures. ✓ Experience in troubleshooting and resolving database integrity issues, performance issues, deadlock issues, connectivity issues, etc. ✓ Hands-on and strong knowledge of data frames and pandas in Python. Reading data from excel, CSV, SQL, and JSON and writing to SQL and JSON. Sql ✓ Excellent communication skills and client-facing experience. Strong experience in DDL and DML commands, UNIONS, JOINS, Subqueries, execution plan, CTE, Pivot/unpivot and indexes. Strong experience in sorting & filtering data, grouping & functions for data processing (Aggregation functions, ranking functions…etc.). Strong experience in creating tables, views, triggers, partitions, complex stored procedures, functions, indexes, and other database objects for data extraction loading and transformations. Performing routine data quality checks and validations to ensure data integrity and consistency. Python Strong Proficiency in Python Programming: hands-on experience. JSON Data Parsing: Extensive experience in handling JSON data formats and implementing parsing techniques. RESTful API & Google Analytics Integration: Skilled in development, integration, and optimization. Secure File Transfer Protocols: Proficient in working with SFTP for secure data transfers. Web Scraping: Experience using tools like Beautiful Soup, Scrapy, and Selenium for data extraction. Python Libraries: Familiar with libraries like NumPy and Pandas for data manipulation and analysis. Performance Optimization: Expertise in debugging and optimizing code for scalability and efficiency. Machine Learning Models (Added Advantage): Experience in developing, training, and deploying machine learning models for predictive analytics. Skills: sql server,scrapy,t-sql,etl pipeline development,sql,pandas,performance tuning,etl development,numpy,sftp,machine learning,selenium,python,data parsing,beautiful soup,stored procedures,data frames,restful api,google analytics integration,rest api integration,web scraping

Job Summary We are seeking a highly skilled and experienced Senior Data Scraping Engineer to design, develop, and orchestrate robust web scraping frameworks. The ideal candidate will have 8-10 years of experience in ethical web scraping, including navigating login-protected websites, solving CAPTCHAs, and managing proxies or third-party services. You will be responsible for building scalable, efficient, and compliant scraping pipelines using industry-standard programming languages and tools, ensuring data integrity and adherence to legal and ethical guidelines. Key Responsibilities Framework Development: Design and implement end-to-end web scraping frameworks to extract structured data from diverse web sources, including those requiring authentication (e.g., behind logins). CAPTCHA Handling: Develop and integrate solutions to bypass or solve CAPTCHAs (e.g., reCAPTCHA, hCaptcha) using ethical tools, services, or machine learning techniques. Proxy & Service Management: Configure and manage proxy services (e.g., rotating proxies, residential proxies) and third-party APIs (e.g., CAPTCHA-solving services) to ensure uninterrupted and anonymous scraping operations. Ethical Compliance: Ensure all scraping activities comply with website terms of service, data privacy regulations (e.g., GDPR, CCPA), and industry best practices for ethical data collection. Data Quality & Validation: Implement robust data validation and cleaning processes to ensure the accuracy, completeness, and consistency of scraped data. Scalability & Optimization: Build scalable scraping pipelines capable of handling large volumes of data with optimized performance, minimal latency, and efficient resource utilization. Monitoring & Maintenance: Develop monitoring tools to track scraping performance, detect failures (e.g., IP bans, structural changes in websites), and maintain scraping scripts to adapt to website updates. Collaboration: Work closely with data engineers, analysts, and product teams to understand data requirements and deliver high-quality datasets for downstream applications. Documentation: Maintain comprehensive documentation for scraping workflows, tools, and processes to ensure transparency and reproducibility. Required Qualifications Experience: 8-10 years of professional experience in web scraping, data extraction, or related fields, with a proven track record of handling complex scraping projects. Programming Languages: - Primary: Proficiency in Python (e.g., Scrapy, BeautifulSoup, Selenium, Requests) for building scraping scripts and frameworks. - Secondary (Preferred): Familiarity with JavaScript/Node.js (e.g., Puppeteer, Cheerio) for dynamic website scraping or Go for high-performance tasks. Tools & Technologies: - Scraping Frameworks: Expertise in Scrapy, Selenium, Puppeteer, or equivalent tools for scraping static and dynamic web content. - CAPTCHA Solutions: Experience with CAPTCHA-solving services (e.g., 2Captcha, Anti- CAPTCHA) or custom ML-based solutions. - Proxy Management: Hands-on experience with proxy services like Bright Data, Oxylabs, Smartproxy, or ScrapingBee for IP rotation and anonymity. - Headless Browsers: Proficiency in using headless browsers (e.g., Chrome, Firefox) for scraping JavaScript-heavy websites. - Databases: Knowledge of SQL (e.g., PostgreSQL, MySQL) and NoSQL (e.g., MongoDB) for storing and querying scraped data. - Cloud Platforms (Preferred): Familiarity with AWS, GCP, or Azure for deploying scraping pipelines or managing infrastructure. Orchestration & Automation: - Experience with workflow orchestration tools like Apache Airflow, Prefect, or Celery for scheduling and managing scraping tasks. - Knowledge of containerization (e.g., Docker) and CI/CD pipelines for deploying scraping scripts. Ethical & Legal Knowledge: Strong understanding of web scraping ethics, website terms of service, and data privacy regulations (e.g., GDPR, CCPA). Problem-Solving: Exceptional ability to troubleshoot issues like IP bans, rate limits, and website structural changes. Communication: Strong verbal and written communication skills to collaborate with cross-functional teams and document processes effectively. Preferred Qualifications Experience with machine learning or AI-based techniques for CAPTCHA solving or dynamic content extraction.

eGrove Systems Pvt Ltd is looking for Senior Python Developer to join its team of experts. Skill: - Senior Python Developer Exp: 4+Yrs NP: Immediate to 15days Location: Chennai/Madurai Interested candidates can send your resume to annie@egrovesys.com Skills Requirement: - Hands-on software development skills, deep technical expertise across the entire software delivery process. Forward-thinking skilled individual. Structured, organized, and a good communicator. Write reusable, Testable, and Efficient code. Required Skills: - 3+ years of Strong experience in Python & 2 years in Django Web framework. Experience or Knowledge in implementing various Design Patterns. Good Understanding of MVC framework & Object-Oriented Programming. Experience in PGSQL / MySQL and MongoDB. Good knowledge in different frameworks, packages & libraries Django/Flask, Django ORM, Unit Test, NumPy, Pandas, Scrapy etc., Experience developing in a Linux environment, GIT & Agile methodology. Good to have knowledge in any one of the JavaScript frameworks: jQuery, Angular, ReactJS. Good to have experience in implementing charts, graphs using various libraries. Good to have experience in Multi-Threading, REST API management. About Company: - eGrove Systems is a leading IT solutions provider specializing in eCommerce, enterprise application development, AI-driven solutions, digital marketing, and IT consulting services. Established in 2008, we are headquartered in East Brunswick, New Jersey, with a global presence. Our expertise includes custom software development, mobile app solutions, DevOps, cloud services, AI chatbots, SEO automation tools, and workforce learning systems. We focus on delivering scalable, secure, and innovative technology solutions to enterprises, start-ups, and government agencies. At eGrove Systems, we foster a dynamic and collaborative work culture driven by innovation, continuous learning, and teamwork. We provide our employees with cutting-edge technologies, professional growth opportunities, and a supportive work environment to thrive in their careers.

We are seeking an experienced and motivated Data Scraper / Lead Generator to join our fast-growing team in Mumbai. The ideal candidate will have a strong background in generating leads through web scraping and online research, specifically targeting the Europe, UK, USA and other international markets.. Key Responsibilities:. Conduct in-depth online research to identify potential leads in targeted geographies. Use advanced web scraping tools and techniques to extract accurate contact and business data from various sources.. Validate and verify collected data to ensure quality and relevance.. Maintain and manage a structured database of leads for outreach and tracking.. Collaborate closely with the sales and marketing teams to deliver a steady pipeline of high-quality leads.. Stay up to date with industry trends, tools, and best practices in data scraping and lead generation.. Requirements:. Proven experience in data scraping lead generation, especially in international markets (UK preferred).. Proficiency in web scraping tools and methods (e.g., Python/BeautifulSoup, Scrapy, Octoparse, or similar).. Strong attention to detail, organizational skills, and data accuracy.. Ability to manage time efficiently and handle multiple tasks.. Excellent communication and coordination skills.. Preferred:. Immediate availability or short notice period.. Show more Show less

Client : Our client is a leading Software as a Service (SaaS) company that specializes in the transformation of data in the US healthcare industry through cutting-edge Artificial Intelligence (AI) solutions. Requirements : Our client is looking for Python Web Scraper, who should continually strive to advance engineering excellence and technology innovation. The mission is to power the next generation of digital products and services through innovation, collaboration, and transparency. You will be a technology leader and doer who enjoys working in a dynamic, fast-paced environment. Responsibilities : - Design and build scalable, reliable web scraping solutions using Python/PySpark. - Develop enterprise-grade scraping services that are robust, fault-tolerant, and production-ready. - Work with large volumes of structured and unstructured data; parse, clean, and transform as required. - Implement robust data validation and monitoring processes to ensure accuracy, consistency, and availability. - Write clean, modular code with proper logging, retries, error handling, and documentation. - Automate repetitive scraping tasks and optimize data workflows for performance and scalability. - Optimize and manage databases (SQL/NoSQL) to ensure efficient data storage, retrieval, and manipulation for both structured and unstructured data. - Analyze and identify data sources relevant to business needs. - Collaborate with data scientists, analysts, and engineers to integrate data from disparate sources and ensure smooth data flow between systems. Desired Profile : - Bachelor's or Master's degree in Computer Science, Information Technology, or a related field. - 2-4 years of experience in web scraping, data crawling, or data engineering. - Proficiency in Python with web scraping tools and libraries (e.g., Beautiful Soup, Scrapy, or Selenium). - Basic working knowledge of PySpark and data pipelines. - Experience with cloud-based platforms (AWS, Google Cloud, Azure) and familiarity with cloud-native data tools like Apache Airflow and EMR. - Expertise in SQL and NoSQL databases (e.g., MySQL, PostgreSQL, MongoDB, Cassandra). - Understanding of data governance, data security best practices, and data privacy regulations (e.g., GDPR, HIPAA). - Familiarity with version control systems like Git.

Job Title: MERN stack Developer Location: Mohali, India (On-site) Experience Required: Minimum 3 Years Job Summary: We are seeking a highly skilled Software Engineer with hands-on experience in the MERN stack (MongoDB, Express.js, React.js, Node.js) and strong scripting and automation capabilities using Python. The ideal candidate should be comfortable working on server deployments (VPS), automation tasks, and have a solid understanding of cloud services and DevOps tools. Key Responsibilities: Develop and maintain scalable web applications using the MERN stack. Write and maintain scripts for automation tasks using Python and relevant frameworks. Manage server deployments on VPS environments, ensuring performance, uptime, and security. Work with Git for version control and collaborative development. Collaborate with the team to build, test, and deploy new features quickly and efficiently. Monitor and improve backend performance. Bonus: Contribute to cloud integration and containerization using Docker, Azure, or Kubernetes. Required Skills: Strong expertise in Node.js and backend logic. Solid experience with MongoDB, Express.js Proficient in Python and automation frameworks/libraries (e.g., Selenium, Requests, Scrapy, Django, etc.). Experience with VPS setup, server monitoring, and configuration. Good understanding of Git and working with version control systems. Familiarity with REST APIs and webhooks. Preferred/Bonus Skills: Knowledge of Docker, Azure, Kubernetes, or other cloud technologies. Experience with CI/CD pipelines. Basic Linux server management and shell scripting Show more Show less

Job Description : Should have 4+ years hands-on experience in algorithms and implementation of analytics solutions in predictive analytics, text analytics and image analytics Should have handson experience in leading a team of data scientists, works closely with client’s technical team to plan, develop and execute on client requirements providing technical expertise and project leadership. Leads efforts to foster innovative ideas for developing high impact solutions. Evaluates and leads broad range of forward looking analytics initiatives, track emerging data science trends, and knowledge sharing Engaging key stakeholders to source, mine and validate data and findings and to confirm business logic and assumptions in order to draw conclusions. Helps in design and develop advanced analytic solutions across functional areas as per requirement/opportunities. Technical Role and Responsibilities Demonstrated strong capability in statistical/Mathematical modelling or Machine Learning or Artificial Intelligence Demonstrated skills in programming for implementation and deployment of algorithms preferably in Statistical/ML based programming languages in Python Sound Experience with traditional as well as modern statistical techniques, including Regression, Support Vector Machines, Regularization, Boosting, Random Forests, and other Ensemble Methods; Visualization tool experience - preferably with Tableau or Power BI Sound knowledge of ETL practices preferably spark in Data Bricks cloud big data technologies like AWS, Google, Microsoft, or Cloudera. Communicate complex quantitative analysis in a lucid, precise, clear and actionable insight. Developing new practices and methodologies using statistical methods, machine learning and predictive models under mentorship. Carrying out statistical and mathematical modelling, solving complex business problems and delivering innovative solutions using state of the art tools and cutting-edge technologies for big data & beyond. Preferred to have Bachelors/Masters in Statistics/Machine Learning/Data Science/Analytics Should be a Data Science Professional with a knack for solving problems using cutting-edge ML/DL techniques and implementing solutions leveraging cloud-based infrastructure. Should be strong in GCP, TensorFlow, Numpy, Pandas, Python, Auto ML, Big Query, Machine learning, Artificial intelligence, Deep Learning Exposure to below skills: Preferred Tech Skills : Python, Computer Vision,Machine Learning,RNN,Data Visualization,Natural Language Processing,Voice Modulation,Speech to text,Spicy,Lstm,Object Detection,Sklearn,Numpy, NLTk,Matplotlib,Cuinks, seaborn,Imageprocessing, NeuralNetwork,Yolo, DarkFlow,DarkNet,Pytorch, CNN,Tensorflow,Keras,Unet, ImageSegmentation,ModeNet OCR,OpenCV,Pandas,Scrapy, BeautifulSoup,LabelImg ,GIT. Machine Learning, Deep Learning, Computer Vision, Natural Language Processing,Statistics Programming Languages-Python Libraries & Software Packages- Tensorflow, Keras, OpenCV, Pillow, Scikit-Learn, Flask, Numpy, Pandas, Matplotlib,Docker Cloud Services- Compute Engine, GCP AI Platform, Cloud Storage, GCP AI & MLAPIs Job Types: Full-time, Permanent, Fresher Pay: ₹30,000.00 - ₹80,000.00 per month Education: Bachelor's (Preferred) Experience: Machine learning: 4 years (Preferred) Work Location: In person

Hi, With reference to your profile on job portal we would like to share an opportunity with you for one of our Gurgaon Based client for Gurgaon location. Please find below the details regarding same: Location: Remote/WFH Experience: 6-10 Years Title: Manager-Data Engineer (Web Scraping) Notice Period: Only Immediate Joiner - 30 Days Max Job Responsibilities Technical Skills Required: Proficiency in Python and SQL/Database skills is required. Must have strong expertise in using Pandas library (Python). Experience with web technologies (HTML/JS, APIs, etc.) is essential. Should have a good understanding of tools such as Scrapy, BeautifulSoup, and Selenium. Responsible for reviewing and approving pull requests to ensure clean, maintainable, and efficient code. Experience building scalable scraping solutions for large-scale data collection Familiarity with AWS technologies like S3, RDS, SNS, SQS, Lambda, and others is necessary. Qualifications Bachelors/masters degree in computer science or in any related field. Role Summary Leading and mentoring a team of seasoned Data engineers performing Web Scraping using various scraping techniques and then utilizing Pythons Pandas library for data cleaning and manipulation. Then ingesting the data into a Database/Warehouse, and scheduling the scrapers using Airflow or other tools Role Overview The Web Scraping Team is seeking a creative and detail-oriented Leaders to contribute to client projects and lead by examples. This team develops essential applications, datasets, and alerts that directly support client investment decisions. Our focus is to maintain operational excellence by providing high-quality proprietary datasets, timely notifications, and exceptional service. The ideal candidate will be self-motivated, self-sufficient, and possess a passion for tinkering and a love for automation. If in case you are interested to avail this opportunity then please revert with your updated profile asap to sachin@vanassociates.com Note: Do not change the subject line while reverting. 1. Total Exp: 2. Relevant experience in Python, Pandas, Data Cleansing, Data Transformation, Team Management: 3. Current CTC: 4. Expected CTC: 5. Official Notice Period: 6. Ready to work in Gurgaon: 7. Availability for MS Teams Interviews in Weekdays:

Be a part of India’s largest and most admired news network! Network18 is India's most diversified Media Company in the fast growing Media market. The Company has a strong Heritage and we possess a strong presence in Magazines, Television and Internet domains. Our brands like CNBC, Forbes and Moneycontrol are market leaders in their respective segments. The Company has over 7,000 employees across all major cities in India and has been consistently managed to stay ahead of the growth curve of the industry. Network 18 brings together employees from varied backgrounds under one roof united by the hunger to create immersive content and ideas. We take pride in our people, who we believe are the key to realizing the organization’s potential. We continually strive to enable our employees to realize their own goals, by providing opportunities to learn, share and grow. Role Overview: We are seeking a passionate and skilled Data Scientist with over a year of experience to join our dynamic team. You will be instrumental in developing and deploying machine learning models, building robust data pipelines, and translating complex data into actionable insights. This role offers the opportunity to work on cutting-edge projects involving NLP, Generative AI, data automation, and cloud technologies to drive business value. Key Responsibilities: Design, develop, and deploy machine learning models, with a strong focus on NLP (including advanced techniques and Generative AI) and other AI applications. Build, maintain, and optimize ETL pipelines for automated data ingestion, transformation, and standardization from various sources Work extensively with SQL for data extraction, manipulation, and analysis in environments like BigQuery. Develop solutions using Python and relevant data science/ML libraries (Pandas, NumPy, Hugging Face Transformers, etc.). Utilize Google Cloud Platform (GCP) services for data storage, processing, and model deployment. Create and maintain interactive dashboards and reporting tools (e.g., Power BI) to present insights to stakeholders. Apply basic Docker concepts for containerization and deployment of applications. Collaborate with cross-functional teams to understand business requirements and deliver data-driven solutions. Stay abreast of the latest advancements in AI/ML and NLP best practices. Required Qualifications & Skills: 2 to 5 years of hands-on experience as a Data Scientist or in a similar role. Solid understanding of machine learning fundamentals, algorithms, and best practices. Proficiency in Python and relevant data science libraries. Good SQL skills for complex querying and data manipulation. Demonstrable experience with Natural Language Processing (NLP) techniques, including advanced models (e.g., transformers) and familiarity with Generative AI concepts and applications. Excellent problem-solving and analytical skills. Strong communication and collaboration skills. Preferred Qualifications & Skills: Familiarity and hands-on experience with Google Cloud Platform (GCP) services, especially BigQuery, Cloud Functions, and Vertex AI. Basic understanding of Docker and containerization for deploying applications. Experience with dashboarding tools like Power BI and building web applications with Streamlit. Experience with web scraping tools and techniques (e.g., BeautifulSoup, Scrapy, Selenium). Knowledge of data warehousing concepts and schema design. Experience in designing and building ETL pipelines. Disclaimer: Please note Network18 and related group companies do not use the services of vendors or agents for recruitment. Please beware of such agents or vendors providing assistance. Network18 will not be responsible for any losses incurred. “We correspond only from our official email address” Show more Show less

Hi, With reference to your profile on job portal we would like to share an opportunity with you for one of our Gurgaon Based client for Gurgaon location. Please find below the details regarding same: Location: Remote/WFH Experience: 6-10 Years Title: Manager-Data Engineer (Web Scraping) Notice Period: Only Immediate Joiner - 30 Days Max Job Responsibilities Technical Skills Required: Proficiency in Python and SQL/Database skills is required. Must have strong expertise in using Pandas library (Python). Experience with web technologies (HTML/JS, APIs, etc.) is essential. Should have a good understanding of tools such as Scrapy, BeautifulSoup, and Selenium. Responsible for reviewing and approving pull requests to ensure clean, maintainable, and efficient code. Experience building scalable scraping solutions for large-scale data collection Familiarity with AWS technologies like S3, RDS, SNS, SQS, Lambda, and others is necessary. Qualifications Bachelors/masters degree in computer science or in any related field. Role Summary Leading and mentoring a team of seasoned Data engineers performing Web Scraping using various scraping techniques and then utilizing Pythons Pandas library for data cleaning and manipulation. Then ingesting the data into a Database/Warehouse, and scheduling the scrapers using Airflow or other tools Role Overview The Web Scraping Team is seeking a creative and detail-oriented Leaders to contribute to client projects and lead by examples. This team develops essential applications, datasets, and alerts that directly support client investment decisions. Our focus is to maintain operational excellence by providing high-quality proprietary datasets, timely notifications, and exceptional service. The ideal candidate will be self-motivated, self-sufficient, and possess a passion for tinkering and a love for automation. If in case you are interested to avail this opportunity then please revert with your updated profile asap to dbetal@vanassociates.com Note: Do not change the subject line while reverting. 1. Total Exp: 2. Relevant experience in Python, Pandas, Data Cleansing, Data Transformation, Team Management: 3. Current CTC: 4. Expected CTC: 5. Official Notice Period: 6. Ready to work in Gurgaon: 7. Availability for MS Teams Interviews in Weekdays: Thanks & Regards, Devanshu Betal Vanguard HR Associates Pvt. Ltd. Phone No- +918851081580 E-Mail:- dbetal@vanassociates.com

Role : Senior Python Locations : Teynampet, Chennai/Kk Nagar, from Office : (1pm to 10pm) Monday of Interview : : 3+ Skills : 3+ years of Strong experience in Python & 2 years in Django Web framework Experience or Knowledge in implementing various Design Patterns Good Understanding of MVC framework & Object-Oriented Programming Experience in PGSQL / MySQL and MongoDB Good knowledge in different frameworks, packages & libraries Django/Flask, Django ORM, Unit Test, NumPy, Pandas, Scrapy etc. Experience developing in a Linux environment, GIT & Agile methodology Good to have knowledge in any one of the JavaScript frameworks: jQuery, Angular, ReactJS Good to have experience in implementing charts, graphs using various libraries Good to have experience in Multi-Threading, REST API management (ref:hirist.tech) Show more Show less

Roles and Responsibilities Design, develop, test, and deploy scalable web applications using Python frameworks such as Django or Flask. Collaborate with cross-functional teams to identify requirements and implement solutions that meet business needs. Ensure high-quality code by writing unit tests, integrating with databases like MongoDB, and good to have implementing data science concepts using Numpy, Pandas, Scrapy etc. Participate in Agile development methodologies to deliver projects on time and within budget. Troubleshoot issues related to application performance optimization and scalability. Desired Candidate Profile 3-7 years of experience in software development with expertise in Python programming language. Added advantage if have good understanding of full-stack development principles including front-end technologies like HTML/CSS/Javascript. Proficiency in at least one Python framework (Django or Flask) along with knowledge of RESTful API design principles. Experience working with databases like MySQL and MongoDB; familiarity with Data Science libraries like Numpy/Pandas/Scrapy is an added advantage. We are looking for immediate joinee and Interest are requested to send their resume to vishal.d@techraq.com.

Job Designation - Data Scraping Location - Ahmedabad Years of Experience - 3+ years We are seeking a skilled and motivated Web Crawler to join our team. The ideal candidate will have 3 to 5 years of experience in developing and maintaining robust web scraping solutions. You will be responsible for designing, implementing, and optimizing web crawlers to extract valuable data from diverse online sources. This role requires a strong understanding of web technologies, data handling, and problem-solving skills. Responsibilities: •Design, develop, and maintain efficient and scalable web crawlers using Python, Mozenda, etc. •Utilize web scraping frameworks such as Scrapy, Beautiful Soup, or Selenium to extract data from websites. •Implement and optimize data extraction logic using XPath, CSS selectors, and JSONPath. •Understand and effectively navigate website structures and implement strategies to bypass anti-scraping measures. •Test, maintain, and troubleshoot web scraping processes to identify and resolve any issues or errors. •Ensure data integrity and quality through rigorous testing and validation. •Monitor and troubleshoot crawler performance, identifying and resolving complex technical issues. •Work with SQL and NoSQL databases to store and manage extracted data. •Collaborate with cross-functional teams to define data requirements and deliver actionable insights. •Maintain comprehensive documentation for all crawler development and maintenance activities. •Demonstrate a strong understanding of the HTTP protocol and web technologies. Show more Show less

We are accepting applications for experienced Data Engineer with a strong background in data scraping, cleaning, transformation, and automation. The ideal candidate will be responsible for building robust data pipelines, maintaining data integrity, and generating actionable dashboards and reports to support business decision-making. Key Responsibilities: Develop and maintain scripts for scraping data from various sources including APIs, websites, and databases. Perform data cleaning, transformation, and normalization to ensure consistency and usability across all data sets. Design and implement relational and non-relational data tables and frames for scalable data storage and analysis. Build automated data pipelines to ensure timely and accurate data availability. Create and manage interactive dashboards and reports using tools such as Power BI, Tableau, or similar platforms. Write and maintain data automation scripts to streamline ETL (Extract, Transform, Load) processes. Ensure data quality, governance, and compliance with internal and external regulations. Monitor and optimize the performance of data workflows and pipelines. Qualifications & Skills: Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Systems, or a related field. Minimum of 5 years of experience in a data engineering or similar role. Proficient in Python (especially for data scraping and automation), and strong hands-on experience with Pandas, NumPy , and other data manipulation libraries. Experience with web scraping tools and techniques (e.g., BeautifulSoup, Scrapy, Selenium). Strong SQL skills and experience working with relational databases (e.g., PostgreSQL, MySQL) and data warehouses (e.g., Redshift, Snowflake, BigQuery). Familiarity with data visualization tools like Power BI, Tableau, or Looker. Knowledge of ETL tools and orchestration frameworks such as Apache Airflow, Luigi, or Prefect . Experience with version control systems like Git and collaborative platforms like Jira or Confluence . Strong understanding of data security, privacy , and governance best practices. Excellent problem-solving skills and attention to detail. Preferred Qualifications: Experience with cloud platforms such as AWS, GCP, or Azure. Familiarity with NoSQL databases like MongoDB, Cassandra, or Elasticsearch. Understanding of CI/CD pipelines and DevOps practices related to data engineering. Job Type: Full-Time (In-Office) Work Days: Monday to Saturday Job Types: Full-time, Permanent Pay: ₹700,000.00 - ₹1,500,000.00 per year Schedule: Day shift Work Location: In person

About the Role: We are looking for a skilled Python Developer with strong expertise in web scraping and data extraction. You will be responsible for designing and maintaining scalable scraping systems, handling large volumes of data, and ensuring the accuracy and integrity of data from various online sources. Responsibilities: Develop and maintain Python scripts for scraping structured and unstructured data from websites and APIs. Build robust, scalable, and efficient scraping solutions using libraries such as BeautifulSoup, Scrapy, Selenium, or Playwright. Monitor and optimize scraping performance and manage data pipelines. Handle website structure changes, anti-bot protections, and CAPTCHA bypassing when necessary. Store, clean, and normalize scraped data using databases (e.g., PostgreSQL, MongoDB) or cloud solutions. Collaborate with data analysts, engineers, and product managers to define data needs and deliver insights. Ensure compliance with legal and ethical standards of data collection. Required Skills: Strong proficiency in Python, especially in web scraping. Solid understanding of HTML, CSS, JavaScript, HTTP protocols, and browser behavior. Familiarity with RESTful APIs, JSON, and XML. Experience working with databases (SQL or NoSQL). Basic knowledge of cloud platforms (AWS, GCP, or Azure) and containerization (Docker) is a plus. Preferred Qualifications: Bachelor's degree in Computer Science, Information Technology, or related field. Experience handling large-scale scraping projects. Background in using version control systems (e.g., Git). Understanding of data privacy laws (e.g., GDPR, CCPA). Job Type: Full-time Pay: ₹20,000.00 - ₹30,000.00 per month Benefits: Health insurance Leave encashment Paid sick time Paid time off Provident Fund Schedule: Day shift Work Location: In person

We are looking for a skilled Python Developer with strong expertise in web scraping and data extraction. You will be responsible for designing and maintaining scalable scraping systems, handling large volumes of data, and ensuring data accuracy and integrity from various online sources. Responsibilities: Develop and maintain Python scripts for scraping structured and unstructured data from websites and APIs. Build robust, scalable, and efficient scraping solutions using libraries such as BeautifulSoup, Scrapy, Selenium, or Playwright. Monitor and optimize scraping performance and manage data pipelines. Handle website structure changes, anti-bot protections, and CAPTCHA bypassing when necessary. Store, clean, and normalize scraped data using databases (e.g., PostgreSQL, MongoDB) or cloud solutions. Collaborate with data analysts, engineers, and product managers to define data needs and deliver insights. Ensure compliance with legal and ethical standards of data collection. Required Skills: Strong proficiency in Python, especially in web scraping. Solid understanding of HTML, CSS, JavaScript, HTTP protocols, and browser behavior. Familiarity with RESTful APIs, JSON, and XML. Experience working with databases (SQL or NoSQL). Basic knowledge of cloud platforms (AWS, GCP, or Azure) and containerization (Docker) is a plus. Preferred Qualifications: Bachelor's degree in Computer Science, Information Technology, or related field. Experience handling large-scale scraping projects. Background in using version control systems (e.g., Git). Understanding of data privacy laws (e.g., GDPR, CCPA). Job Type: Full-time Pay: ₹20,000.00 - ₹30,000.00 per month Benefits: Health insurance Leave encashment Paid sick time Paid time off Provident Fund Schedule: Day shift Work Location: In person