19 Beautifulsoup Jobs

Spherex is seeking an Artificial Intelligence (AI) Machine Learning (ML) Engineer to contribute to the development, enhancement, and expansion of our product platform catering to the Media and Entertainment sector. As the AI/ML Engineer, your duties will involve the creation of machine learning models and system retraining. The position is based in Navi Mumbai, India. The ideal candidate should hold a degree in computer science or software development. Proficiency in .Net, Azure, Project management, Team and Client management is essential. Additionally, familiarity with Python, Tensorflow, Pytorch, MySQL, Artificial Intelligence, and Machine Learning is desired. Key requirements for this role include expertise in Python with OOPS concepts, a solid foundation in Natural Language Understanding, Machine Learning, and Artificial Intelligence. Knowledge of ML/DL libraries such as Numpy, Pandas, Tensorflow, Pytorch, Keras, scikit-learn, Jupyter, and spaCy/NLTK is crucial. Hands-on experience with MySQL and NoSQL databases, along with proficiency in scraping tools like BeautifulSoup and Scrapy, is also required. The successful candidate should have experience in web development frameworks like Django and Flask, as well as working with RESTful APIs using Django. Familiarity with end-to-end data science pipelines, strong unit testing and debugging abilities, and applied statistical skills are necessary. Proficiency in Git, Linux OS, ML architectures, and approaches including object detection, semantic segmentation, classification, regression, RNNs, and data fusion is expected. Knowledge of OpenCV, OCR, Yolo, Docker, Kubernetes, ETLPentaho is considered a plus. Candidates must possess a minimum of 4+ years of experience in advanced AI/ML projects within commercial environments. Experience in utilizing AI/ML for video and audio content analysis is advantageous. Education-wise, a college degree in computer science or software development is required, along with excellent documentation and effective communication skills in both technical and non-technical contexts.,

You will be responsible for developing cutting-edge solutions by designing, developing, and maintaining robust web scraping solutions to extract large datasets from various websites, supporting our data-driven initiatives. Your role will involve mastering Python programming to implement and optimize sophisticated scraping scripts and tools. You will leverage industry-leading tools such as BeautifulSoup, Scrapy, Selenium, and other scraping frameworks to efficiently collect and process data. Additionally, you will innovate with AI technologies like ChatGPT to automate and enhance data extraction processes, pushing the boundaries of what is possible. It will be crucial to optimize data management by cleaning, organizing, and storing extracted data in structured formats for seamless analysis and usage. Ensuring peak performance by optimizing scraping scripts for efficiency, scalability, and reliability will also be part of your responsibilities. You will troubleshoot data scraping issues with precision to maintain data accuracy and completeness, along with maintaining clear and comprehensive documentation of scraping processes, scripts, and tools for transparency and knowledge sharing. As a qualified candidate, you should have a minimum of 5 years of experience in web data scraping, with a strong emphasis on handling large datasets. Advanced skills in Python programming, especially in the context of web scraping, are essential for this role. You are expected to have in-depth knowledge and experience with tools such as BeautifulSoup, Scrapy, Selenium, and other relevant scraping tools. Strong expertise in data cleaning, organization, and storage, along with excellent problem-solving and analytical skills to address complex scraping challenges, will be required. Meticulous attention to detail is crucial to ensure data accuracy and completeness, and the ability to work independently, manage multiple tasks, and meet deadlines effectively is essential. Preferred skills for this role include experience with API integration for data extraction, familiarity with cloud platforms like AWS, Azure, or Google Cloud for data storage and processing, understanding of database management systems and SQL for data storage and retrieval, and proficiency in using version control systems like Git.,

TransForm Solutions is a trailblazer in the business process management and IT-enabled services industry, known for delivering top-notch solutions that drive business efficiency and growth. With a focus on innovation and excellence, the company empowers businesses to transform their operations and achieve their full potential. As the company continues to expand, they are looking for a dynamic Senior Web Data Scraping Engineer to join their team and help harness the power of data. Your mission in this role will involve developing cutting-edge solutions by designing, developing, and maintaining robust web scraping solutions that extract large datasets from various websites to fuel data-driven initiatives. You will need to master Python programming skills to implement and optimize sophisticated scraping scripts and tools. Utilizing industry-leading tools such as BeautifulSoup, Scrapy, Selenium, and other scraping frameworks will be essential for collecting and processing data efficiently. Additionally, you will be required to innovate with AI, using ChatGPT prompt skills to automate and enhance data extraction processes. Data management will be a key aspect of your responsibilities, involving cleaning, organizing, and storing extracted data in structured formats for seamless analysis and usage. Ensuring peak performance by optimizing scraping scripts for efficiency, scalability, and reliability will be crucial. You will also need to work independently, managing tasks and deadlines with minimal supervision, while demonstrating the ability to collaborate effectively with team members to understand data requirements and deliver actionable insights. Troubleshooting data scraping issues with precision to ensure data accuracy and completeness, as well as maintaining clear and comprehensive documentation of scraping processes, scripts, and tools used for transparency and knowledge sharing, will be part of your daily tasks. In terms of qualifications, the ideal candidate should have a minimum of 3 years of experience in web data scraping with a strong focus on handling large datasets. Advanced skills in Python programming, proficiency in relevant scraping tools such as BeautifulSoup, Scrapy, Selenium, and ChatGPT prompts, as well as strong data management and analytical skills, are required. Attention to detail, effective communication, and the ability to work independently are also essential qualities. Preferred skills include experience with API integration for data extraction, familiarity with cloud platforms like AWS, Azure, or Google Cloud for data storage and processing, understanding of database management systems and SQL, and proficiency in using version control systems like Git. In terms of compensation, the company offers a competitive base salary based on experience and skills, along with potential performance-based bonuses tied to successful project outcomes and contributions. Joining TransForm Solutions means being part of a forward-thinking team that values innovation, collaboration, and excellence. You will have the opportunity to work on groundbreaking projects, leveraging the latest technologies to transform data into actionable insights. The company is committed to professional growth and provides an environment where skills and expertise are recognized and rewarded. If you are a top-tier web data scraping engineer passionate about pushing the envelope and delivering impactful results, TransForm Solutions invites you to apply and be a key player in their journey to harness the power of data to transform businesses.,

Get ready to be a part of fast-growing team making next generation marketing platform to make it really simple to launch marketing campaigns with AI Agents. We have been recognized among the Tech50 companies of 2021 by Yourstory and got Startup Maharathi award at Startup Mahakumbh 2025. Tired of being a small cog in a big machine At Intellemo, you won&apost just be writing codeyou&aposll be a core member of our engineering team, building the brain behind our AI marketing agents that are changing the game for thousands of businesses. We are a funded, profitable, fast-growing startup on a mission to make sophisticated marketing and sales automation accessible to everyone. The Opportunity This isn&apost just another backend role. As our core backend hire, you will have unparalleled ownership, a direct impact on our product&aposs success, and the opportunity to work directly alongside the CEO/CTO to shape our entire technical foundation. You will be instrumental in solving our biggest bottleneck and paving the way for us to scale 10x. If you thrive on challenges and want your work to matter, this is the role for you. Key Responsibilities: Develop and maintain our backend services , including building and consuming RESTful APIs and working with our GraphQL API gateway. Integrate with external platforms such as Google Ads, Meta Ads, Linkedin Ads, Pinterest, etc. to automate campaign management and reporting. Connect with AI/ML APIs and contribute to the development of our agentic AI capabilities. Orchestrate agentic behavior using internal Agentic framework to create intelligent, automated marketing workflows. Build and optimize our creative rendering engine for generating images and videos using libraries like wand, ImageMagick, and FFmpeg. Develop web scraping and data extraction capabilities for website/landing page analysis using tools like BeautifulSoup. Contribute to our microservices-oriented architecture , ensuring our services are scalable, maintainable, and resilient. Collaborate with frontend developers to ensure seamless integration of our backend services with the user interface. Write clean, efficient, and well-documented code , and participate in code reviews to maintain our high standards of quality. Required Skills and Qualifications: 1-2 years of professional experience as a Python Backend Engineer. Strong proficiency in Python FastAPI. Demonstrable experience with building and integrating with third-party APIs, particularly for platforms like Google Ads and Meta Ads. A solid understanding of AI/ML concepts and experience working with AI/ML APIs. Familiarity with Langchain, MCP or similar frameworks for building applications with large language models (LLMs). Experience with image or video processing libraries such as CV2, ImageMagick, or FFmpeg. Proficiency with web scraping libraries like BeautifulSoup. A good understanding of microservices architecture and its principles. Familiarity with GraphQL and experience working with API gateways. Solid knowledge of relational and NoSQL databases (e.g., PostgreSQL, MongoDB). A Bachelor&aposs degree in Computer Science is a must. What We Offer Direct Mentorship: You will be mentored directly by the CEO/CTO, offering a unique learning opportunity you won&apost find anywhere else. Unmatched Impact & Ownership: See the code you write immediately affect our clients and our company&aposs bottom line. No bureaucracy, just pure building. A Problem-Solver&aposs Paradise: We offer a constant stream of complex and fascinating challenges at the intersection of AI, marketing, and creative automation. Rapid Growth Trajectory: As a critical early hire, you&aposll be on the fast track for technical leadership as the company scales. Competitive Salary: A salary in the range of ?7-10 LPA + option of ESOPs post probation. Location: This is a full-time, in-office position at our Gurgaon, Haryana office. We don&apost have Work from home role or remote work to iterate and build faster. We believe in the power of in-person collaboration and are excited to build a strong, cohesive team. How to Apply Just click on &aposApply' button and you&aposll be directed to our Whatsapp where you can interact and submit answers and ask any queries you have. Once shortlisted, we&aposll call you for in-person interview (there are no virtual interviews) Show more Show less

You are someone who thrives in a full-stack Python + QA + Database + AI environment and is excited about technical growth. If you are a passionate Python engineer with hands-on experience in building web services, supporting QA/automation workflows, and working confidently across SQL/NoSQL databases, this opportunity is for you. Responsibilities: - Develop, test, and maintain Python-based backend services using Django, FastAPI, or Flask. - Execute efficient web scraping tasks using BeautifulSoup or Scrapy. - Automate browser workflows using Selenium. - Write scalable database queries and schema designs for SQL (PostgreSQL/MySQL) and MongoDB. - Design and implement QA automation scripts and frameworks. - (Optional but desirable) Integrate and utilize AI/ML tools, packages, or APIs. Must-Have Skills: - Proficiency in Python (3.x) and modern frameworks (Django/FastAPI/Flask). - Web scraping experience with BeautifulSoup, Scrapy, or equivalent. - Hands-on automation using Selenium. - Strong database expertise in SQL and MongoDB. - Solid understanding of testing principles, automation techniques, and version control (Git). Nice-to-Have: - Exposure to AI or machine learning libraries (e.g. TensorFlow, PyTorch, OpenAI APIs). - Familiarity with CI/CD pipelines (Jenkins, GitHub Actions) and containerization (Docker). Why This Role is Great: - Balanced challenge across backend development, QA automation, and data work. - Opportunity to explore AI/ML tools in real-world projects. - Engage in end-to-end ownership: backend QA deployment pipelines. Job Location: Gurugram Work Model: On-site Budget: 15-18 LPA,

As a Web Scraping & Data Automation Intern at D-Vivid Consultant, you will play a crucial role in developing, maintaining, and optimizing scripts to collect data from various platforms such as Instagram, Facebook, Reddit, and more. Your primary responsibilities will include utilizing automation tools and libraries like Make.com (Integromat), Python (BeautifulSoup, Scrapy, Selenium), and browser automation frameworks to ensure efficient data collection. You will be responsible for cleaning and organizing the extracted data to facilitate marketing intelligence and lead generation campaigns. Additionally, you will conduct research to enhance scraping practices and handle dynamic website content effectively. Collaboration with the marketing and tech teams will be essential to identify data requirements and enhance output quality. Furthermore, you will be expected to maintain meticulous documentation of workflows, scripts, and scraping logs to ensure transparency and compliance with legal, ethical, and privacy standards in data scraping activities. The ideal candidate for this role should possess proficiency in at least one scraping library/tool, a basic understanding of Python or JavaScript, and hands-on experience with automation platforms like Make.com. A strong grasp of HTML, CSS, and JavaScript for DOM navigation is crucial, along with a passion for data-driven marketing and social media platforms. Problem-solving skills, attention to detail, and the ability to work independently while managing timelines effectively are key attributes for success in this role. Desirable skills include experience in scraping or analyzing data from social media platforms, familiarity with proxy management, headless browsers, and anti-bot detection strategies, as well as knowledge of data handling libraries like Pandas or NumPy. Prior internship or project experience in automation or data scraping would be advantageous. As part of the internship at D-Vivid Consultant, you will benefit from flexible working hours, a remote-friendly experience, an Internship Certificate upon successful completion, a Letter of Recommendation (LOR) from leadership, and the potential for a Pre-Placement Offer (PPO) based on performance. Additionally, you will receive mentorship from industry leaders and gain exposure to real-world automation projects, enhancing your professional growth and development.,

About Taxmann At Taxmann, our vision is to achieve perfection, skill, and accuracy in everything we do. From its humble beginnings as a small family business in 1972, Taxmann has evolved into a technology-oriented Publishing/Product company with independent Research & Editorial, Production, Sales & Marketing, and Technology divisions, which have now become the backbone of Taxmann. Our mission at Taxmann is to spearhead the pursuit of expertise and authenticity in providing the most authentic and fastest information reporting. With over 60 years of domain knowledge and the trust of more than 500k legal professionals nationwide, Taxmann proudly stands as the #1 source for everything related to Tax & Corporate Laws of India. Taxmann owes its success to its core strength, the Editorial and Research Division, which consists of over 200 highly motivated associates from the legal community, including advocates, lawyers, chartered accountants, and company secretaries. These professionals monitor all developments in the judicial, administrative, and legislative fields with impeccable perfection, skill, and accuracy. Job Responsibilities We are currently looking for a talented Data Scientist with expertise in GenAI, Python, Power BI, and SQL to join our team. The selected candidate will be responsible for collecting, analyzing, and interpreting complex data sets to support business decision-making. Additionally, they will be involved in creating APIs and web applications in Python, designing Python-based Rest APIs, developing Power BI dashboards, generating reports, and implementing data-driven strategies to enhance organizational efficiency. Key Responsibilities - Collect, process, and analyze large datasets to derive actionable insights. - Develop and enhance GenAI, Machine learning, and Natural Language Processing models for predictive modeling. - Create and optimize complex RAG projects for Chatbots. - Build and maintain RESTful APIs and web applications using Python frameworks such as Flask and FastAPI. - Utilize pandas and beautifulsoup for data extraction and manipulation. - Design and deploy Power BI dashboards and visualizations for real-time and ad hoc reporting needs. - Write efficient and scalable SQL queries for data extraction and analysis. - Collaborate with cross-functional teams to understand data requirements and deliver analytical solutions. - Present findings and recommendations to stakeholders in a clear and concise manner. - Ensure data accuracy, integrity, and governance throughout all analytics processes. - Automate data collection and reporting pipelines to enhance efficiency. Requirements - Bachelor's or Master's degree in Data Science, Computer Science, Statistics, or a related field. - Minimum of 2 years of experience as a Data Scientist or in a similar analytical role. - Proficiency in Python for data manipulation, API development, and GenAI chatbot app creation. - Strong expertise in Power BI, including DAX and custom visual creation. - Advanced knowledge of SQL and relational databases. - Familiarity with tools like Pandas, NumPy, Scikit-learn, and other data analysis libraries. - Strong problem-solving and communication skills. - Experience with version control (e.g., Git) and deployment tools is advantageous. Preferred Qualifications - Experience with cloud platforms such as Azure, AWS, or GCP. - Knowledge of machine learning techniques and model deployment. - Background in business intelligence, KPIs, and performance tracking. (ref:hirist.tech),

As a Data Engineer at our company, you will be an integral part of a skilled Data Engineering team focused on developing reusable capabilities and tools to automate various data processing pipelines. Your responsibilities will include contributing to data acquisition, ingestion, processing, monitoring pipelines, and validating data. Your role is pivotal in maintaining the smooth operation of data ingestion and processing pipelines, ensuring that data in the data lake is up-to-date, valid, and usable at all times. With a minimum of 3 years of experience in data engineering, you should be proficient in Python programming and have a strong background in working with both RDBMS and NoSQL systems. Experience in the AWS ecosystem, including components like Airflow, EMR, Redshift, S3, Athena, and PySpark, is essential. Additionally, you should have expertise in developing REST APIs using Python frameworks such as flask and fastapi. Familiarity with crawling libraries like BeautifulSoup in Python would be advantageous. Your skill in writing complex SQL queries to retrieve key metrics and working with various data lake storage formats will be key to your success in this role. Key Responsibilities: - Design and implement scalable data pipelines capable of handling large data volumes. - Develop ETL/ELT pipelines to extract data from upstream sources and synchronize it with data lakes in formats like parquet, iceberg, and delta. - Optimize and maintain data pipelines to ensure smooth operation and business continuity. - Collaborate with cross-functional teams to source data for various business use cases. - Stay informed about emerging data technologies and trends to enhance our data infrastructure and architecture continuously. - Adhere to best practices in data querying and manipulation to uphold data integrity. If you are a motivated Data Engineer with a passion for building robust data pipelines and ensuring data quality, we invite you to join our dynamic team and contribute to the success of our data engineering initiatives.,

As a skilled Web Scraping Data Analyst, you will be responsible for collecting, cleaning, and analyzing data from various online sources. Your expertise in Python-based scraping frameworks, data transformation, and experience with proxy/VPN rotation and IP management will be crucial in building data pipelines that support our analytics and business intelligence initiatives. Your key responsibilities will include designing, developing, and maintaining robust web scraping scripts using tools like Python, BeautifulSoup, Scrapy, Selenium, etc. You will also implement IP rotation, proxy management, and anti-bot evasion techniques, deploy scraping tools on cloud-based or edge servers, and monitor scraping jobs for uptime and efficiency. Additionally, you will parse and structure unstructured or semi-structured web data into clean, usable datasets, collaborate with data analysts and data engineers to integrate web-sourced data into internal databases and reporting systems, conduct exploratory data analysis (EDA), and ensure compliance with website scraping policies, robots.txt, and relevant data privacy regulations. To excel in this role, you should have proficiency in Python and experience with libraries like Requests, BeautifulSoup, Scrapy, Pandas. Knowledge of proxy/VPN usage, IP rotation, and web traffic routing tools (e.g., Smartproxy, BrightData, Tor, etc.), familiarity with cloud platforms (AWS, Azure, or GCP) and Linux-based environments, experience deploying scraping scripts on edge servers or containerized environments (e.g., Docker), solid understanding of HTML, CSS, JSON, and browser dev tools for DOM inspection, strong analytical mindset with experience in data cleansing, transformation, and visualization, good knowledge of SQL and basic data querying, and ability to handle large volumes of data and build efficient data pipelines. Preferred qualifications for this role include experience with headless browsers like Puppeteer or Playwright, familiarity with scheduling tools like Airflow or Cron, background in data analytics or reporting using tools like Tableau, Power BI, or Jupyter Notebooks, and knowledge of anti-captcha solutions and browser automation challenges. This is a full-time position with the work location being in person.,

As a skilled Web Scraping Data Analyst, your primary responsibility will involve collecting, cleaning, and analyzing data from various online sources. You will leverage your expertise in Python-based scraping frameworks to design, develop, and maintain robust web scraping scripts using tools such as Python, BeautifulSoup, Scrapy, Selenium, and more. Additionally, you will be tasked with implementing IP rotation, proxy management, and anti-bot evasion techniques to ensure efficient data collection. Your role will be instrumental in constructing data pipelines that drive our analytics and business intelligence initiatives. Collaboration will be a key aspect of your work as you engage with data analysts and data engineers to integrate web-sourced data into internal databases and reporting systems. Furthermore, you will be involved in conducting exploratory data analysis (EDA) to derive valuable insights from the scraped data. It will be essential to adhere to website scraping policies, robots.txt guidelines, and relevant data privacy regulations to ensure compliance. To excel in this role, you should possess proficiency in Python and have experience with libraries like Requests, BeautifulSoup, Scrapy, and Pandas. Knowledge of proxy/VPN usage, IP rotation, and web traffic routing tools will be crucial for effective data collection. Familiarity with cloud platforms such as AWS, Azure, or GCP, as well as Linux-based environments, will be advantageous. Experience in deploying scraping scripts on edge servers or containerized environments and a solid understanding of HTML, CSS, JSON, and browser dev tools are also desirable skills. A strong analytical mindset coupled with experience in data cleansing, transformation, and visualization will be beneficial in handling large volumes of data and building efficient data pipelines. Proficiency in SQL and basic data querying will be necessary for data manipulation tasks. Preferred qualifications include experience with headless browsers like Puppeteer or Playwright, familiarity with scheduling tools like Airflow or Cron, and a background in data analytics or reporting using tools like Tableau, Power BI, or Jupyter Notebooks. This full-time role requires an in-person work location.,

We are looking for a highly skilled and experienced Senior Full Stack Developer to join our team. You should have expertise in React and Next.js on the frontend, and Express on the backend, along with hands-on experience in AI and machine learning integration. Your responsibilities will include developing high-quality web applications, implementing AI models for chart pattern detection, and market sentiment analysis through social network scraping. If you are passionate about developing scalable solutions and leveraging AI for cutting-edge financial technology, this opportunity is for you! Your responsibilities will involve developing and maintaining web applications using React, Next.js, and Express, designing and implementing RESTful APIs, integrating with external data sources, and services, owning AI/ML integration tasks, optimizing applications for speed, security, and scalability, collaborating with designers, product managers, and engineers, writing clean and efficient code, staying updated with industry trends, providing technical guidance to junior team members, and more. Qualifications for this role include at least 5 years of full stack development experience, proficiency in React, Next.js, and Express, strong understanding of JavaScript, TypeScript, and modern frameworks, experience in AI/ML integration, database technologies, version control, CI/CD processes, problem-solving skills, knowledge of HTML, CSS, preprocessors like SASS, and familiarity with Docker, cloud services, and microservices architecture. Bonus skills include experience with financial market data, trading algorithms, scraping frameworks, common chart patterns, market structures, and NLP for sentiment analysis. Our work environment offers flexible options including on-site, hybrid, or fully remote arrangements, competitive salary, bonuses, potential project profits, a challenging yet supportive environment, opportunities for growth, professional development, and learning new technologies. If you are excited about this opportunity and meet the criteria, please send your resume and cover letter. Join us in shaping our technology strategy, developing our team, and delivering innovative solutions.,

The position of Data Scrapper + QA Tester in Malad, Mumbai requires a skilled and proactive individual to join the team. The primary responsibilities include designing, managing, and implementing data-scraping tools to meet project requirements and performing Quality Assurance (QA) testing to ensure data accuracy and system reliability. As the Data Scrapper + QA Tester, you will be responsible for developing customized data-scraping tools based on short notice project requirements, scraping and compiling datasets from various global sources, and staying updated with the latest scraping tools and technologies to enhance efficiency. You will also need to identify and resolve challenges in data generation, optimize scraping processes, and conduct thorough QA testing to ensure data accuracy, consistency, and completeness. Collaboration with cross-functional teams to understand project goals, refine scraping and QA processes, and provide detailed documentation of tools developed, challenges encountered, and solutions implemented is essential. The ideal candidate should have proven experience in designing and implementing data-scraping tools, proficiency in programming languages commonly used for web scraping, ability to handle large datasets efficiently, and strong problem-solving skills. Preferred qualifications include experience with database management systems, familiarity with APIs and web scraping using API integrations, knowledge of data protection regulations and ethical scraping practices, and exposure to machine learning techniques for data refinement. If you are a problem-solver with expertise in data scraping and QA testing, and thrive in a fast-paced environment, we encourage you to apply for this position.,

Responsibilities: * Collaborate with cross-functional teams on project delivery. * Develop backend solutions using Python, FastAPI & AWS. * Optimize performance through Redis DB & Nginx.

Hi, With reference to your profile on job portal we would like to share an opportunity with you for one of our Gurgaon Based client for Gurgaon location. Please find below the details regarding same: Location: Remote/WFH Experience: 6-10 Years Title: Manager-Data Engineer (Web Scraping) Notice Period: Only Immediate Joiner - 30 Days Max Job Responsibilities Technical Skills Required: Proficiency in Python and SQL/Database skills is required. Must have strong expertise in using Pandas library (Python). Experience with web technologies (HTML/JS, APIs, etc.) is essential. Should have a good understanding of tools such as Scrapy, BeautifulSoup, and Selenium. Responsible for reviewing and approving pull requests to ensure clean, maintainable, and efficient code. Experience building scalable scraping solutions for large-scale data collection Familiarity with AWS technologies like S3, RDS, SNS, SQS, Lambda, and others is necessary. Qualifications Bachelors/masters degree in computer science or in any related field. Role Summary Leading and mentoring a team of seasoned Data engineers performing Web Scraping using various scraping techniques and then utilizing Pythons Pandas library for data cleaning and manipulation. Then ingesting the data into a Database/Warehouse, and scheduling the scrapers using Airflow or other tools Role Overview The Web Scraping Team is seeking a creative and detail-oriented Leaders to contribute to client projects and lead by examples. This team develops essential applications, datasets, and alerts that directly support client investment decisions. Our focus is to maintain operational excellence by providing high-quality proprietary datasets, timely notifications, and exceptional service. The ideal candidate will be self-motivated, self-sufficient, and possess a passion for tinkering and a love for automation. If in case you are interested to avail this opportunity then please revert with your updated profile asap to sachin@vanassociates.com Note: Do not change the subject line while reverting. 1. Total Exp: 2. Relevant experience in Python, Pandas, Data Cleansing, Data Transformation, Team Management: 3. Current CTC: 4. Expected CTC: 5. Official Notice Period: 6. Ready to work in Gurgaon: 7. Availability for MS Teams Interviews in Weekdays:

Hi, With reference to your profile on job portal we would like to share an opportunity with you for one of our Gurgaon Based client for Gurgaon location. Please find below the details regarding same: Location: Remote/WFH Experience: 6-10 Years Title: Manager-Data Engineer (Web Scraping) Notice Period: Only Immediate Joiner - 30 Days Max Job Responsibilities Technical Skills Required: Proficiency in Python and SQL/Database skills is required. Must have strong expertise in using Pandas library (Python). Experience with web technologies (HTML/JS, APIs, etc.) is essential. Should have a good understanding of tools such as Scrapy, BeautifulSoup, and Selenium. Responsible for reviewing and approving pull requests to ensure clean, maintainable, and efficient code. Experience building scalable scraping solutions for large-scale data collection Familiarity with AWS technologies like S3, RDS, SNS, SQS, Lambda, and others is necessary. Qualifications Bachelors/masters degree in computer science or in any related field. Role Summary Leading and mentoring a team of seasoned Data engineers performing Web Scraping using various scraping techniques and then utilizing Pythons Pandas library for data cleaning and manipulation. Then ingesting the data into a Database/Warehouse, and scheduling the scrapers using Airflow or other tools Role Overview The Web Scraping Team is seeking a creative and detail-oriented Leaders to contribute to client projects and lead by examples. This team develops essential applications, datasets, and alerts that directly support client investment decisions. Our focus is to maintain operational excellence by providing high-quality proprietary datasets, timely notifications, and exceptional service. The ideal candidate will be self-motivated, self-sufficient, and possess a passion for tinkering and a love for automation. If in case you are interested to avail this opportunity then please revert with your updated profile asap to dbetal@vanassociates.com Note: Do not change the subject line while reverting. 1. Total Exp: 2. Relevant experience in Python, Pandas, Data Cleansing, Data Transformation, Team Management: 3. Current CTC: 4. Expected CTC: 5. Official Notice Period: 6. Ready to work in Gurgaon: 7. Availability for MS Teams Interviews in Weekdays: Thanks & Regards, Devanshu Betal Vanguard HR Associates Pvt. Ltd. Phone No- +918851081580 E-Mail:- dbetal@vanassociates.com

Summary To enhance user profiling and risk assessment, we are building web crawlers to collect relevant user data from third-party sources, forums, and the dark web. We are seeking a Senior Web Crawler & Data Extraction Engineer to design and implement these data collection solutions. Job Responsibilities Design, develop, and maintain web crawlers and scrapers to extract data from open web sources, forums, marketplaces, and the dark web. Implement data extraction pipelines that aggregate, clean, and structure data for fraud detection and risk profiling. Use Tor, VPNs, and other anonymization techniques to safely crawl the dark web while avoiding detection. Develop real-time monitoring solutions for tracking fraudulent activities, data breaches, and cybercrime discussions. Optimize crawling speed and ensure compliance with website terms of service, ethical standards, and legal frameworks. Integrate extracted data with fraud detection models, risk scoring algorithms, and cybersecurity intelligence tools. Work with data scientists and security analysts to develop threat intelligence dashboards from collected data. Implement anti-bot detection evasion techniques and handle CAPTCHAs using AI-driven solvers where necessary. Stay updated on OSINT (Open-Source Intelligence) techniques, web scraping best practices, and cybersecurity trends. Requirements 5+ years of experience in web crawling, data scraping, or cybersecurity data extraction. Strong proficiency in Python, Scrapy, Selenium, BeautifulSoup, Puppeteer, or similar frameworks. Experience working with Tor, proxies, and VPNs for anonymous web scraping. Deep understanding of HTTP protocols, web security, and bot detection mechanisms. Experience parsing structured and unstructured data from JSON, XML, and web pages. Strong knowledge of database management (SQL, NoSQL) for storing large-scale crawled data. Familiarity with AI/ML-based fraud detection techniques and data classification methods. Experience working with cybersecurity intelligence sources, dark web monitoring, and OSINT tools. Ability to implement scalable, distributed web crawling architectures. Knowledge of data privacy regulations (GDPR, CCPA) and ethical data collection practices. Nice to Have Experience in fintech, fraud detection, or threat intelligence. Knowledge of natural language processing (NLP) for analyzing cybercrime discussions. Familiarity with machine learning-driven anomaly detection for fraud prevention. Hands-on experience with cloud-based big data solutions (AWS, GCP, Azure, Elasticsearch, Kafka).

We are looking for a Python Developer who has expertise in web scraping and backend development. The ideal candidate should be proficient in Python frameworks, data extraction techniques, and API integration.

Job Overview: We are seeking a highly skilled Python Developer to join our dynamic team. The ideal candidate should have strong expertise in Python and its associated libraries, with experience in web scraping, data handling, and automation. You should be an excellent problem solver with great communication skills and a solid understanding of object-oriented programming and data structures. Key Responsibilities: . Develop, test, and maintain efficient Python-based desktop applications. . Work with pandas for data manipulation and analysis. . Write optimized SQL queries for database interactions. . Utilize BeautifulSoup and Selenium for web scraping and automation. . Handle JSON data efficiently for API integrations and data exchange. . Apply object-oriented programming (OOP) principles to software development. . Implement data structures and algorithms to optimize performance. . Troubleshoot and debug code for functionality and efficiency. . Collaborate with cross-functional teams to deliver high-quality solutions. . Document processes and write clean, maintainable code. Must-Have Skills: Python - Strong proficiency in Python programming. Pandas - Experience with data manipulation and analysis. SQL - Ability to write and optimize queries. BeautifulSoup - Web scraping and parsing HTML/XML data. JSON - Handling structured data for APIs and storage. Selenium - Automation and web testing. OOP Concepts - Strong understanding of object-oriented principles. Data Structures & Algorithms - Efficient problem-solving abilities. Problem-Solving Skills - Ability to tackle complex technical challenges. Communication Skills - Strong verbal and written communication.

Proficient in Python, web scraping (BeautifulSoup, Scrapy, Selenium), HTML/CSS/JS, HTTP, DevTools, IP rotation, proxies, automation, bypassing protections (e.g. Cloudflare), GCP, GraphQL, NoSQL, and Selenium Wire. Skilled in scalable data handling. Provident fund Health insurance