110 Web Scraping Jobs

You will be joining our team as a Junior Data Scraping Specialist at one of the leading IT product companies. In this role, you will receive training and mentorship to support in collecting, cleaning, and transforming data from various online sources. This position offers an excellent opportunity to initiate your career in data engineering and web scraping. Your responsibilities will include assisting in data collection by scraping information from websites using basic scraping tools and techniques. You will learn to utilize popular web scraping libraries and tools, clean and organize raw data into structured formats like CSV, JSON, or Excel files, and support in automating data scraping tasks through simple scripts and tools guided by senior team members. Additionally, you will conduct research to identify potential data sources, test scraping scripts, and collaborate closely with data analysts and developers to gather data for reports or analysis. To excel in this role, you should have basic knowledge of HTML, CSS, and web structures, problem-solving abilities to troubleshoot data extraction issues, attention to detail to ensure data accuracy, eagerness to learn new skills related to web scraping and data processing, and excellent communication skills for effective collaboration within the team. This position requires 0-1 year of experience, making it an ideal opportunity for individuals looking to kick-start their career in the field of data engineering and web scraping.,

As an individual contributing to the team, you will assist in the development of web scraping and automation scripts under supervision. You will have the opportunity to learn and implement basic scraping techniques for both static and dynamic websites. Collaboration with the team will be crucial as you work together to build and test data ingestion pipelines. Your responsibility will include writing clean and maintainable Python code for small to medium-sized tasks. Additionally, you will be expected to debug and resolve basic issues in scraping workflows and scripts. Working with APIs such as REST and GraphQL to fetch and ingest data will be a part of your regular tasks. Documenting your code and contributing to the maintenance of internal knowledge bases will also be an essential aspect of your role. In terms of your basic programming skills, you should have a good understanding of Python fundamentals, including modules, functions, loops, and error handling. Familiarity with libraries like `requests`, `BeautifulSoup`, and `XML` will be beneficial for your tasks. Having a grasp of web development basics, including HTML, CSS, JavaScript, as well as concepts like XPath and DOM, will be advantageous. Basic knowledge of JSON and CSV file formats for data handling is expected, along with the ability to perform simple operations on databases like MySQL or MongoDB. Familiarity with Git for basic version control tasks like commits, branches, and pulls is also desirable. Join our team at an IT Consulting and Services company that specializes in cutting-edge solutions in Data Analytics, AI, and Web Scraping technologies.,

About Us Thoucentric provides end-to-end consulting solutions to tackle diverse business challenges across industries. With a focus on leveraging deep domain expertise, cutting-edge technology, and a results-driven approach, we assist organizations in optimizing operations, improving decision-making, and fostering growth. Headquartered in Bangalore, we have a global presence in India, US, UK, Singapore, and Australia. Our services span Business Consulting, Program & Project Management, Digital Transformation, Product Management, Process & Technology Solutioning, and Execution, encompassing areas such as Analytics & Emerging Tech in functional domains like Supply Chain, Finance & HR, Sales & Distribution. We pride ourselves on executing solutions rather than just offering advice, collaborating with leading names in the CPG industry, tech sector, and start-up ecosystem. Recognized as a "Great Place to Work" by AIM and ranked among the "50 Best Firms for Data Scientists to Work For", we boast a seasoned consulting team of over 500 professionals across six global locations. Job Description About the Role We are in search of a BI Architect to support the BI Lead of a global CPG organization by designing an intelligent and scalable Business Intelligence ecosystem. The role involves crafting an enterprise-wide KPI dashboard suite enhanced by a GenAI-powered natural language interface for insightful exploration. Responsibilities - Architect BI Stack: Develop and supervise a scalable and efficient BI platform serving as the central source for critical business metrics across functions. - Advise BI Lead: Collaborate as a technical advisor to the BI Lead, ensuring alignment of architecture decisions with long-term strategies and business priorities. - Design GenAI Layer: Create a GenAI-driven natural language interface for BI dashboards to enable conversational querying of KPIs, trends, and anomalies by business users. - RAG/Graph Approach: Implement suitable architectures like RAG with vector stores or Knowledge Graphs to deliver intelligent, context-rich insights. - External Data Integration: Establish mechanisms for organizing and integrating data from external sources (e.g., competitor websites, industry reports) to enhance internal insights. - Security & Governance: Maintain adherence to enterprise data governance, security, and compliance standards across all layers (BI + GenAI). - Cross-functional Collaboration: Engage closely with Data Engineering, Analytics, and Product teams to ensure seamless integration and operationalization of the BI ecosystem. Requirements Qualifications - 9 years of BI architecture and analytics platform experience, with at least 2 years focused on GenAI, RAG, or LLM-based solutions. - Profound expertise in BI tools (e.g., Power BI, Tableau, Looker) and data modeling. - Familiarity with GenAI frameworks (e.g., LangChain, LlamaIndex, Semantic Kernel) and vector databases (e.g., Pinecone, FAISS, Weaviate). - Knowledge of graph-based data models and tools (e.g., Neo4j, RDF, SPARQL) is advantageous. - Proficiency in Python or relevant scripting language for pipeline orchestration and AI integration. - Experience in web scraping and structuring external/third-party datasets. - Previous exposure to CPG domain or large-scale KPI dashboarding is preferred. Benefits Joining Thoucentric as a Consultant offers: - Opportunity to shape your career path independently. - Engaging consulting environment working with Fortune 500 companies and startups. - Supportive and dynamic workplace fostering personal growth. - Inclusive culture with opportunities for bonding beyond work. - Participation in Thoucentric's growth journey. Skills Required: BI architecture, Analytics, Data Visualization Practice Name: Data Visualization Date Opened: 07/15/2025 Work Mode: Hybrid Job Type: Full time Industry: Consulting Corporate Office: Thoucentric, The Hive, Mahadevapura, Bengaluru, Karnataka, India, 560048,

YipitData is a leading market research and analytics firm in the disruptive economy that has recently secured up to $475M from The Carlyle Group, valuing over $1B. For three consecutive years, YipitData has been recognized as one of Inc's Best Workplaces, fostering a people-centric culture centered around mastery, ownership, and transparency. As a fast-growing technology company with offices in various locations worldwide, including NYC, Austin, Miami, and others, we offer exciting opportunities for individuals looking to make a significant impact. As a Web Scraping Specialist at YipitData, you will be part of the Data Solutions team, reporting directly to the Data Solutions Engineering Manager. Your role will involve designing, refactoring, and maintaining web scrapers crucial for generating key reports across the organization. By ensuring the efficiency, reliability, and scalability of our data ingestion processes, you will directly support multiple business units and products. Key Responsibilities: - Refactor and maintain web scrapers to enhance reliability, maintainability, and efficiency. - Implement advanced scraping techniques using fingerprinting methods to evade detection and blocking. - Collaborate with cross-functional teams to gather requirements, align on targets, and ensure data quality. - Monitor, troubleshoot, and optimize scraper performance to address failures and bottlenecks. - Drive continuous improvement by proposing new methodologies and technologies to enhance scraping capabilities. To succeed in this role, effective communication skills in English, a minimum of 4 years of experience with web scraping frameworks, and a strong understanding of HTTP, RESTful APIs, and HTML parsing are essential. Additionally, expertise in advanced fingerprinting and evasion strategies, managing cookies, headers, and proxies, as well as troubleshooting skills for optimizing scraper performance, are required. This is a fully-remote position based in India with flexible working hours from 11am to 8pm IST. At YipitData, we offer a competitive compensation package, comprehensive benefits, and a supportive work environment focused on personal growth and skill mastery. Your impact and contributions are what drive your growth at YipitData, where learning, self-improvement, and skill development are encouraged and valued.,

As a global leader in assurance, tax, transaction, and advisory services, EY is dedicated to hiring and developing passionate professionals to contribute to building a better working world. At EY, you are offered a culture that values training, opportunities, and creative freedom, focusing not just on your current abilities but also on your potential for growth. Your career at EY is yours to shape, with limitless possibilities, motivating experiences, and continuous support to help you reach your professional best. EY has an exciting opportunity for the position of Senior Consultant in the National-Assurance-ASU - Audit - Standards and Methodologies team based in Kolkata. The Assurance team at EY aims to inspire confidence and trust in a complex world by protecting the public interest, promoting transparency, supporting investor confidence, and fostering talent for future business leaders. You will be involved in ensuring compliance with audit standards, providing clear perspectives to audit committees, and delivering critical information for stakeholders. Your responsibilities will include demonstrating technical excellence in audit analytics, foundational analytics, visualization, data extraction, risk-based analytics, and business understanding. Proficiency in databases, ETL, SQL, visualization tools like Tableau, Spotfire, or Qlikview, and experience in Machine Learning using R or Python is required. You should also have expertise in MS Office Suite, NLP, Web Scraping, Log Analytics, TensorFlow, AI, and Beautiful Soup. To qualify for this role, you should hold a qualification in BE/B.Tech, MSC in Computer Science/Statistics, or M.C.A, along with 5-7 years of relevant experience. EY is looking for individuals who can work collaboratively across various client departments, offer practical solutions to complex problems, and possess agility, curiosity, mindfulness, and positive energy. EY offers a dynamic work environment with a focus on skills development, learning, and career progression. As a part of EY, you will have access to personalized career journeys and resources to enhance your roles, skills, and opportunities. EY is dedicated to being an inclusive employer, ensuring the well-being and career growth of its employees while delivering excellent client service. If you meet the criteria mentioned above and are ready to contribute to building a better working world, apply now to join EY and be a part of a team committed to making a positive impact.,

Hiring Senior Data Engineer for an AI-native startup. Work on scalable data pipelines, LLM workflows, web scraping (Scrapy, lxml), Pandas, APIs, and Django. Strong in Python, data quality, mentoring, and large-scale systems. Health insurance

As a Data Analyst with Market Research and Web Scraping skills at our company located in Udyog Vihar Phase-1, Gurgaon, you will be expected to leverage your 2-5 years of experience in data analysis, particularly in competitive analysis and market research within the Fashion/garment/apparel industry. A Bachelor's degree in Data Science, Computer Science, Statistics, Business Analytics, or related field is required, while advanced degrees or certifications in data analytics or market research are considered a plus. Your main responsibility will be to analyze large datasets to identify trends, patterns, and insights related to market trends and competitor performance. You will conduct quantitative and qualitative analyses to support decision-making in product development and strategy. Additionally, you will be involved in performing in-depth market research to track competitor performance, emerging trends, and customer preferences. Furthermore, you will design and implement data scraping solutions to gather competitor data from websites, ensuring compliance with legal standards and respect of website terms of service. Creating and maintaining organized databases with market and competitor data for easy access and retrieval will be part of your routine, along with collaborating closely with cross-functional teams to align data insights with company objectives. To excel in this role, you should have proven experience with data scraping tools such as BeautifulSoup, Scrapy, or Selenium, proficiency in SQL, Python, or R for data analysis and data manipulation, and experience with data visualization tools like Tableau, Power BI, or D3.js. Strong analytical skills and the ability to interpret data to draw insights and make strategic recommendations are essential. If you are passionate about data analysis, market research, and web scraping and possess the technical skills and analytical mindset required, we encourage you to apply by sending your updated resume with current salary details to jobs@glansolutions.com. For any inquiries, please contact Satish at 8802749743 or visit our website at www.glansolutions.com. Join us on this exciting journey of leveraging data to drive strategic decisions and make a meaningful impact in the Fashion/garment/apparel industry.,

The Alien Technology Transfer Group leads top-class innovators to success by converting visionary ideas into tangible realities. They support companies with concrete growth ambitions by funding their product development through the Innovation Consulting Line. Additionally, they design, fund, and launch innovative companies through the timely transformation of high-potential concepts into profitable ventures with the support of the Venture Studio. They also deliver unparalleled talent hunting solutions for businesses of all sizes, startups, and SMEs particularly through the Talent Hunting Line. As an Innovation Scout, your role involves sourcing and analyzing the highest quality prospect clients for the business lines among innovative high-tech high-impact start-ups and Small and Medium Enterprises (SMEs) in Europe and the USA. To excel in this position, you must possess a genuine interest in technology and business, an aptitude for quickly understanding complex engineering or medical innovations, be highly analytical and articulate, and have a strong command of English. Being a team player, well-organized, and keen to take on responsibility are key qualities. Demonstrating an entrepreneurial mindset, high self-motivation, and a high degree of stamina to work within an ambitious and goal-driven environment are essential. Your responsibilities will include continuously staying up to date in the technological and business field to recognize business opportunities and industry/market emerging trends effectively. Identifying top-level sources/hubs of potential clients among web platforms, online databases, and events, monitoring current trends in private and public innovative project financing and fund-raising, handling and analyzing databases, preparing regular reports on funding trends, and maintaining internal databases for prospects, clients, and public grant awardees. Job Requirements: - Masters degree in Life Science (Bioengineering, Biomedical, Biotechnology, Neurosciences, Biochemistry, Microbiology, etc.) - Advanced use of Excel and aptitude to master the use of professional industry database - Proficiency in using digital resources (e.g. search engines) to uncover new leads - Self-starting, inquisitive, and pragmatic attitude - Fluent English comprehension, oral and written - Knowledge of Python programming for web scraping/extraction mechanisms is a plus - An added advantage if you have an idea for the creation of web scrapers for extracting startup data from different websites supporting the development of startup Lead Generation. Please note: Due to the high volume of candidates, individual feedback regarding the outcome of every single application cannot be provided.,

This is a 6-month paid internship designed as a path to full-time employment. You will work on complex, high-visibility systems from day one, solving problems where AI and engineering meet real-world data. Top performers will be offered a full-time role with compensation and equity. Responsibilities - Build intelligent scraping pipelines with fallback logic (GET Playwright LLM). - Parse messy HTML using DOM-aware logic to extract emails, names, and keywords. - Use lexical similarity and subsequence scoring to associate entities with goals. - Integrate and optimize LLM APIs (OpenAI, Claude, Sonar) for reasoning and enrichment. - Create structured reports in JSON/HTML from semi-structured sources. - Optimize backend performance for multi-threaded, concurrent web crawling. - Experiment with goal-classification models, AI-driven contact curation, and relevance ranking. Requirements - A strong academic background in CS, engineering, AI, or related fields. - Solid programming skills in Python (and optionally Node.js). - Direct experience with LLMs, including usage of OpenAI/Claude APIs in real projects. - Projects or work demonstrating hands-on use of AI for reasoning, enrichment, or extraction. - Deep curiosity and creativity in solving open-ended, data-heavy problems. - The ability to move fast, think clearly, and work hard. - A willingness to iterate, debug, and own real production features. Bonus If You Also Have - Experience scraping the web at scale using Playwright, asyncio, or similar. - Understanding of DOM traversal and proximity-based entity matching. - Built tools or algorithms using subsequences and string similarity measures. - Strong opinions on when and how to use AI to supplement human reasoning.,

Target international markets (US, UK, EU, APAC). Responsible for generating qualified leads using cold outreach (email, LinkedIn), web scraping, AI-based personalisation, CRM tracking, with a target of delivering 40–50 high-quality meetings/month. Required Candidate profile -2+ years of experience in B2B lead generation for software/marketing companies. -Hands-on with scraping tools (Browse.ai, Webscraper.io, or Python-based), Calendly or Zoho Booking for meeting setup. Perks and benefits Performance-based incentives Flexible hours

As a skilled and motivated Python Developer with 3 to 6 years of experience, you will be joining our dynamic development team. Your primary responsibility will be to build high-quality, scalable back-end systems, integrate APIs, and contribute to all phases of the software development lifecycle. Additionally, having knowledge of Robotic Process Automation (RPA) and web scraping tools will be beneficial for this role. Your key responsibilities will include designing, developing, testing, and deploying scalable and secure Python applications. You will work on server-side logic, RESTful APIs, microservices, and database models while adhering to best practices in code writing. Collaboration with front-end developers, DevOps engineers, and product teams will be essential for successful project outcomes. Troubleshooting, debugging, and upgrading existing applications, as well as participating in code reviews and maintaining code documentation, will also be part of your routine tasks. Moreover, you will be expected to optimize applications for performance and scalability, ensure the security and data protection of applications, and design and implement user interfaces using WinForms or WPF. To excel in this role, you should possess 36 years of hands-on experience in Python development. Strong knowledge of Python frameworks like Django, Flask, or FastAPI, experience with RESTful APIs and integrating third-party services, and a good understanding of databases such as PostgreSQL, MySQL, or MongoDB are essential requirements. Familiarity with version control tools like Git and experience with cloud platforms like AWS, Azure, or GCP will be advantageous. This is a full-time position based in Barakhamba Road, New Delhi, where you will be expected to work in person. If you are excited about this opportunity, feel free to contact the employer at +91 9899129159. The expected start date for this position is 01/08/2025.,

Alien Technology Transfer empowers top-class innovators to transform visionary product concepts into commercial realities. The company helps companies with concrete growth ambitions to secure funding for their product development through their innovation consulting expertise. With a track record of raising more than $500 million for Small and Medium Enterprises (SMEs) across various domains such as agri-tech, artificial intelligence, biotechnology, clean-tech, cyber-security, quantum computing, space, and transport, Alien Technology Transfer is now seeking an Innovation Scout. As an Innovation Scout, you will be responsible for sourcing and analyzing the highest quality prospect clients for the business lines among innovative high-tech high-impact start-ups and SMEs in Europe and the USA. To excel in this role, you must possess a genuine interest in technology and business, quickly understand complex engineering or medical innovations, be highly analytical and articulate, and have a strong command of English. Additionally, being a team player, well-organized, and eager to take on responsibility are key qualities. Demonstrating an entrepreneurial mindset, high self-motivation, and the ability to work in an ambitious and goal-driven environment are also essential. Your duties and responsibilities will include: - Keeping yourself updated in the technological and business field to identify business opportunities and industry/market trends effectively. - Identifying potential clients from web platforms, online databases, and events. - Monitoring innovative project financing and fund-raising trends. - Handling and analyzing databases to deliver appropriate results. - Evaluating information related to innovative technologies and businesses. - Maintaining and enriching internal databases for prospects, clients, and public grant awardees. - Preparing reports on funding trends to define yearly targets. Job requirements include: - A Masters degree in Life Science (Bioengineering, Biomedical, Biotechnology, Neurosciences, Biochemistry, Microbiology, etc.) - Advanced Excel skills and ability to master professional industry databases. - Proficiency in using digital resources to uncover leads. - Self-starting, inquisitive, and pragmatic attitude. - Fluent English communication skills. - Knowledge of Python programming for web scraping/extraction mechanisms is a plus. - Added advantage if you have ideas for creating web scrapers for startup data extraction. Please note that due to the high volume of applications, individual feedback on application outcomes cannot be provided.,

Be a part of a team that harnesses advanced AI, ML, and big data technologies to develop cutting-edge healthcare technology platform, delivering innovative business solutions. Job Title : Data Engineer II / Senior Data Engineer Job Location : Bengaluru, Pune - India Job Summary: We are a leading Software as a Service (SaaS) company that specializes in the transformation of data in the US healthcare industry through cutting-edge Artificial Intelligence (AI) solutions. We are looking for Software Developers, who should continually strive to advance engineering excellence and technology innovation. The mission is to power the next generation of digital products and services through innovation, collaboration, and transparency. You will be a technology leader and doer who enjoys working in a dynamic, fast-paced environment. Responsibilities: Design, develop, and maintain robust and scalable ETL/ELT pipelines to ingest and transform large datasets from various sources. Optimize and manage databases (SQL/NoSQL) to ensure efficient data storage, retrieval, and manipulation for both structured and unstructured data. Collaborate with data scientists, analysts, and engineers to integrate data from disparate sources and ensure smooth data flow between systems. Implement and maintain data validation and monitoring processes to ensure data accuracy, consistency, and availability. Automate repetitive data engineering tasks and optimize data workflows for performance and scalability. Work closely with cross-functional teams to understand their data needs and provide solutions that help scale operations. Ensure proper documentation of data engineering processes, workflows, and infrastructure for easy maintenance and scalability Roles and Responsibilities Design, develop, test, deploy, and maintain large-scale data pipelines using AWS services such as S3, Glue, Lambda, Step Functions. Collaborate with cross-functional teams to gather requirements and design solutions for complex data engineering projects. Develop ETL/ELT pipelines using Python scripts and SQL queries to extract insights from structured and unstructured data sources. Implement web scraping techniques to collect relevant data from various websites and APIs. Ensure high availability of the system by implementing monitoring tools like CloudWatch. Desired Profile: Bachelors or Masters degree in Computer Science, Information Technology, or a related field. 3-5 years of hands-on experience as a Data Engineer or in a related data-driven role. Strong experience with ETL tools like Apache Airflow, Talend, or Informatica. Expertise in SQL and NoSQL databases (e.g., MySQL, PostgreSQL, MongoDB, Cassandra). Strong proficiency in Python, Scala, or Java for data manipulation and pipeline development. Experience with cloud-based platforms (AWS, Google Cloud, Azure) and their data services (e.g., S3, Redshift, BigQuery). Familiarity with big data processing frameworks such as Hadoop, Spark, or Flink. Experience in data warehousing concepts and building data models (e.g., Snowflake, Redshift). Understanding of data governance, data security best practices, and data privacy regulations (e.g., GDPR, HIPAA). Familiarity with version control systems like Git.. HiLabs is an equal opportunity employer (EOE). No job applicant or employee shall receive less favorable treatment or be disadvantaged because of their gender, marital or family status, color, race, ethnic origin, religion, disability, or age; nor be subject to less favorable treatment or be disadvantaged on any other basis prohibited by applicable law. HiLabs is proud to be an equal opportunity workplace dedicated to pursuing and hiring a diverse and inclusive workforce to support individual growth and superior business results. Thank you for reviewing this opportunity with HiLabs! If this position appears to be a good fit for your skillset, we welcome your application.

You should have knowledge in web (PHP) development with PHP, Cake PHP, MySQL, jQuery, JavaScript, AJAX, Linux, JSON, and XML. In the back-end, you should be proficient in MVC/Object-Oriented PHP (v.5+), web scraping using Regular Expressions and XPATH, developing complex data-driven systems without a framework, secure e-commerce crawler development, adaptive problem-solving skills, performance optimization techniques, and debugging for issue diagnosis. Unit testing your code with assertions is essential along with hands-on experience with Linux/UNIX, RESTful paradigms, Apache, and MySQL database efficiency analysis. For front-end development, expertise in jQuery/JavaScript, scripting languages like Ajax and DOM manipulation, and a strong grasp of HTML5 and CSS3 is required. Regarding the database, you should have a good command of MySQL (v.5+) supported by phpMyAdmin, understanding of design patterns, PHP best practices, PHP Frameworks like Cake PHP, scalability architecture, server performance considerations, and push notification system implementation. The ideal candidate should hold a BS/MS degree in Computer Science or Engineering, possess development skills in PHP, MySQL, jQuery, JavaScript, AJAX, Linux, JSON, and XML, have knowledge of relational databases, version control tools, web services development, API development using PHP, a passion for sound design and coding practices, strong verbal and written communication skills, problem-solving attitude, and consistency in work ethics. Must-have skills include PHP, Core PHP, MySQL, Curl, Linux commands, API development using PHP, scripting for process automation, web crawling, and being consistent and dependable. Interested candidates can share their resumes at vaishali@mastheadtechnologies.com. This is a full-time position with a day shift schedule, requiring in-person work at the designated location. The application deadline is 13/07/2025, and the expected start date is 11/07/2025.,

As a Python Developer specializing in Web Scraping and Data Processing, you will play a crucial role in a core team dedicated to aggregating biomedical content from a variety of sources. With over 3 years of experience, you will be responsible for developing scalable Python scripts to extract and parse biomedical data from diverse web platforms, including grant repositories, scientific journals, conference abstracts, treatment guidelines, and clinical trial databases. Your key responsibilities will include building robust modules to split multi-record documents into manageable content units, implementing NLP-based field extraction pipelines using libraries like spaCy and NLTK, and designing automated data acquisition workflows utilizing tools like cron, Celery, or Apache Airflow. You will ensure efficient storage of parsed data in both relational (PostgreSQL) and NoSQL (MongoDB) databases, with a focus on optimal schema design for performance and scalability. Additionally, you will be responsible for maintaining robust logging, thorough exception handling, and comprehensive content quality validation throughout all data processing and scraping workflows. To excel in this role, you should possess a strong command over web scraping libraries such as BeautifulSoup, Scrapy, Selenium, and Playwright. Proficiency in PDF parsing libraries like PyMuPDF, pdfminer.six, and PDFPlumber, as well as experience with HTML/XML parsers (lxml, XPath, html5lib) are essential. Familiarity with regular expressions, NLP concepts, advanced field extraction techniques, SQL, NoSQL databases, and API integration (RESTful APIs) is also required. Experience with task schedulers, workflow orchestrators, version control using Git/GitHub, and collaborative development environments will be beneficial. While not mandatory, exposure to biomedical or healthcare data parsing, cloud environments like AWS, data validation frameworks, and understanding of ontologies and taxonomies will be advantageous in this role. Join us in this exciting opportunity to contribute to cutting-edge data processing and scraping projects in the field of biomedicine and healthcare.,

Primary Skills - Data Engineering Secondary Skills - SQL & Python Education - Bachelors or Masters degree in Computer Science, IT, or related Experience Range - 2 TO 4 YEARS exp on data engineering proficient in PuSpark or similar data-focused Role, experience in data/web scraping, experience with ETL tools, expertise in SQL, python, Scala, Java, experience in Cloud based platform, AWS, Google Cloud, Azure, experience in Data warehousing concepts, Familiar with version control systems like Git. Domain - IT Start Date - Immediate Duration of the Project - 6 months contract (extendable) Shift Timing - Regular Shift CTC - As per Industry Number of Interviews - L1 & 2 Client Interview+HR Location - Bangalore No. of Positions - 4 Job description - Design, develop and maintain scalable ETL/ELT pipelines, maintain web crawlers, manage data base SQL/NoSQL, implement and maintain data engineering, provide solutions that help scale operations. Documents Mandatory - Form 16, Salary slip, Aadahar, Pancard, Academic Documents, offer letter, Experience Letter all to be submitted after selection Note: Immediate joiners are welcome

As a Data Analyst with expertise in Market Research and Web Scraping, you will be responsible for analyzing large datasets to uncover trends and insights related to market dynamics and competitor performance. Your role will involve conducting thorough market research to track competitor activities, identify emerging trends, and understand customer preferences. Additionally, you will design and implement data scraping solutions to extract competitor data from various online sources while ensuring compliance with legal standards and website terms of service. Your key responsibilities will include developing dashboards, reports, and visualizations to communicate key insights effectively to stakeholders. You will collaborate with cross-functional teams to align data-driven insights with company objectives and support strategic decision-making in product development and marketing strategies. Furthermore, you will be involved in database management, data cleaning, and maintaining organized databases with accurate and consistent information for easy access and retrieval. To excel in this role, you should have a Bachelor's degree in Data Science, Computer Science, Statistics, Business Analytics, or a related field. Advanced degrees or certifications in data analytics or market research will be advantageous. Proficiency in SQL, Python, or R for data analysis, along with experience in data visualization tools like Tableau, Power BI, or D3.js, is essential. Strong analytical skills, the ability to interpret data effectively, and knowledge of statistical analysis techniques are key requirements for this position. Experience with data scraping tools such as BeautifulSoup, Scrapy, or Selenium, as well as familiarity with web analytics and SEO tools like Google Analytics or SEMrush, will be beneficial. Preferred skills include experience with e-commerce data analysis, knowledge of retail or consumer behavior analytics, and an understanding of machine learning techniques for data classification and prediction. Ethical data scraping practices and adherence to data privacy laws are essential considerations for this role. If you meet these qualifications and are excited about the opportunity to work in a dynamic environment where your analytical skills and market research expertise will be valued, we encourage you to apply by sending your updated resume along with your current salary details to jobs@glansolutions.com. For any inquiries, feel free to contact Satish at 8802749743 or visit our website at www.glansolutions.com to explore more job opportunities. Join us at Glan Solutions and leverage your data analysis skills to drive strategic decisions and contribute to our success in the fashion/garment/apparel industry! Note: This job was posted on 14th November 2024.,

Were seeking a talented and driven Python Web Scraping & Automation Engineer to develop scalable solutions for data extraction and automation. The ideal candidate will have hands-on experience with Python-based scraping tools and frameworks as well as a solid background in working with SQL databases. In this role, you’ll be responsible for building efficient web crawlers, automating data pipelines, creating RESTful APIs, and supporting backend development efforts as needed. Key Responsibilities: Build and maintain efficient web scrapers using tools like Scrapy, BeautifulSoup, Selenium, and Requests. Automate data collection, processing, and storage with clean, reusable, and well-documented code. Develop backend services and RESTful APIs using frameworks such as Flask, FastAPI, or Django. Design and manage SQL databases (MySQL, PostgreSQL) to support data storage, access, and analytics. Implement proxy rotation, session handling, and CAPTCHA bypass mechanisms for reliable data scraping. Monitor and optimize scraper and API performance to ensure scalability, speed, and reliability. Collaborate closely with data analysts and developers to build and deliver end-to-end data solutions. Follow ethical and legal best practices for web scraping and data handling. Diagnose and fix issues across web crawlers, backend services, and data pipelines. Preferred candidate profile Must have excellent experience in Python we scraping and Sql Server. This is a work from home job

Were seeking a talented and driven Python Web Scraping expert to develop scalable solutions for data extraction and automation. The ideal candidate will have hands-on experience with Python-based scraping tools and frameworks as well as a solid background in working with SQL databases. In this role, youll be responsible for building efficient web crawlers, automating data pipelines, creating RESTful APIs, and supporting backend development efforts as needed. Key Responsibilities: Build and maintain efficient web scrapers using tools like Scrapy, BeautifulSoup, Selenium, and Requests. Automate data collection, processing, and storage with clean, reusable, and well-documented code. Develop backend services and RESTful APIs using frameworks such as Flask, FastAPI, or Django. Design and manage SQL databases (MySQL, PostgreSQL) to support data storage, access, and analytics. Implement proxy rotation, session handling, and CAPTCHA bypass mechanisms for reliable data scraping. Monitor and optimize scraper and API performance to ensure scalability, speed, and reliability. Collaborate closely with data analysts and developers to build and deliver end-to-end data solutions. Follow ethical and legal best practices for web scraping and data handling. Diagnose and fix issues across web crawlers, backend services, and data pipelines. Preferred candidate profile Must have excellent experience in Python we scraping and Sql Server. This is a work from home job

You are an experienced Senior Automation Tester - Network Security with 5+ years of industry experience. You have hands-on experience in End-to-End Solution testing in the Security SDWAN area. Your strong understanding of network security concepts, protocols, and technologies is a key asset for this role. Your QA experience includes working with VPN technologies such as IKev2, IKev1, IPSec, SSL/TLS, as well as SD-WAN technologies and solutions. You are proficient in handling network devices, L2/L3 protocols, and traffic generation tools like Ixia and Spirent. Knowledge of next-generation network security standards, including Post-Quantum Cryptography, and best practices is essential. Proficiency in Python and standard libraries is required. You should have experience in developing APIs using Python to handle scaling, which involves infra automation work experience. Additionally, experience with automation tools and frameworks like Selenium and Rest API is necessary. A solid understanding of RESTful APIs, web scraping, and automation of web-based systems is expected. Familiarity with version control systems like Git and experience with CI/CD tools such as Jenkins and GitHub Actions will be beneficial for this role. You should have experience working with different key stakeholders throughout the software life cycle of the project. As a motivated self-starter with excellent communication skills, you have demonstrated the ability to deliver superior products in a cross-functional team environment under aggressive schedules. Your soft skills include strong communication and organizational skills, proven ability to deliver superior products in cross-functional team settings under tight schedules, and excellent leadership, problem-solving, and communication skills. You are experienced in system design and debugging, capable of designing, building, and debugging large-scale distributed systems.,

We are seeking a skilled Python Developer to join our team. The ideal candidate will be responsible for working with existing APIs or developing new APIs based on our requirements. You should have a strong foundation in Python and experience with RESTful services and cloud infrastructure. Requirements: Strong understanding of Python Experience with RESTful services and cloud infrastructure Ability to develop microservices/functions Familiarity with libraries such as Pandas, NumPy, Matplotlib & Seaborn, Scikit-learn, Flask , Django, Requests, FastAPI and TensorFlow & PyTorch. Basic understanding of SQL and databases Ability to write clean, maintainable code Experience deploying applications at scale in production environments Experience with web scraping using tools like BeautifulSoup, Scrapy, or Selenium Knowledge of equities, futures, or options microstructures is a plus Experience with data visualization and dashboard building is a plus Why Join Us? Opportunity to work on high-impact real-world projects Exposure to cutting-edge technologies and financial datasets A collaborative, supportive, and learning-focused team culture 5-day work week (Monday to Friday)

About Calfus: Calfus is a Silicon Valley headquartered software engineering and platforms company with a vision deeply rooted in the Olympic motto "Citius, Altius, Fortius Communiter". At Calfus, we aim to inspire our team to rise faster, higher, and stronger while fostering a collaborative environment to build software at speed and scale. Our primary focus is on creating engineered digital solutions that drive positive impact on business outcomes. Upholding principles of #Equity and #Diversity, we strive to create a diverse ecosystem that extends to the broader society. Join us at #Calfus and embark on an extraordinary journey with us! Position Overview: As a Data Engineer specializing in BI Analytics & DWH, you will be instrumental in crafting and implementing robust business intelligence solutions that empower our organization to make informed, data-driven decisions. Leveraging your expertise in Power BI, Tableau, and ETL processes, you will be responsible for developing scalable architectures and interactive visualizations. This role necessitates a strategic mindset, strong technical acumen, and effective collaboration with stakeholders across all levels. Key Responsibilities: - BI Architecture & DWH Solution Design: Develop and design scalable BI Analytical & DWH Solution aligning with business requirements, utilizing tools like Power BI and Tableau. - Data Integration: Supervise ETL processes through SSIS to ensure efficient data extraction, transformation, and loading into data warehouses. - Data Modelling: Establish and maintain data models that support analytical reporting and data visualization initiatives. - Database Management: Employ SQL for crafting intricate queries, stored procedures, and managing data transformations via joins and cursors. - Visualization Development: Spearhead the design of interactive dashboards and reports in Power BI and Tableau while adhering to best practices in data visualization. - Collaboration: Engage closely with stakeholders to gather requirements and translate them into technical specifications and architecture designs. - Performance Optimization: Analyze and optimize BI solutions for enhanced performance, scalability, and reliability. - Data Governance: Implement data quality and governance best practices to ensure accurate reporting and compliance. - Team Leadership: Mentor and guide junior BI developers and analysts to cultivate a culture of continuous learning and improvement. - Azure Databricks: Utilize Azure Databricks for data processing and analytics to seamlessly integrate with existing BI solutions. Qualifications: - Bachelor's degree in computer science, Information Systems, Data Science, or a related field. - 6-12 years of experience in BI architecture and development, with a strong emphasis on Power BI and Tableau. - Proficiency in ETL processes and tools, particularly SSIS. Strong command over SQL Server, encompassing advanced query writing and database management. - Proficient in exploratory data analysis using Python. - Familiarity with the CRISP-DM model. - Ability to work with various data models and databases like Snowflake, Postgres, Redshift, and MongoDB. - Experience with visualization tools such as Power BI, QuickSight, Plotly, and Dash. - Strong programming foundation in Python for data manipulation, analysis, serialization, database interaction, data pipeline and ETL tools, cloud services, and more. - Familiarity with Azure SDK is a plus. - Experience with code quality management, version control, collaboration in data engineering projects, and interaction with REST APIs and web scraping tasks is advantageous. Calfus Inc. is an Equal Opportunity Employer.,

Role & responsibilities As a Data Engineer focused on web crawling and platform data acquisition, you will design, develop, and maintain large-scale web scraping pipelines to extract valuable platform data. You will be responsible for implementing scalable and resilient data extraction solutions, ensuring seamless data retrieval while working with proxy management, anti-bot bypass techniques, and data parsing. Optimizing scraping workflows for performance, reliability, and efficiency will be a key part of your role. Additionally, you will ensure that all extracted data maintains high quality and integrity. Preferred candidate profile We are seeking candidates with: Strong experience in Python and web scraping frameworks such as Scrapy, Selenium, Playwright, or BeautifulSoup. Knowledge of distributed web crawling architectures and job scheduling. Familiarity with headless browsers, CAPTCHA-solving techniques, and proxy management to handle dynamic web challenges. Experience with data storage solutions, including SQL, and cloud storage. Understanding of big data technologies like Spark and Kafka (a plus). Strong debugging skills to adapt to website structure changes and blockers. A proactive, problem-solving mindset and ability to work effectively in a team-driven environment.

As a Python Developer at F33 Baseline IT Development Pvt. Ltd. in Mohali, you will be responsible for developing and maintaining web applications using Python, Django, and Odoo. Your key responsibilities will include designing and implementing RESTful APIs, performing data extraction and automation through web scraping tools, debugging and optimizing code for performance and scalability, collaborating with front-end developers and project managers, and writing clean, well-documented, and testable code. To excel in this role, you should have a minimum of 2 years of experience in Python development, proficiency in the Django framework, hands-on experience with Odoo ERP, expertise in web scraping using libraries like BeautifulSoup, Scrapy, or Selenium, good understanding of databases such as PostgreSQL and MySQL, familiarity with Git version control, and excellent problem-solving and communication skills. Preferred qualifications for this position include a Bachelor's degree in Computer Science, IT, or a related field, experience with API integrations, and knowledge of Linux server environments. At F33 Baseline IT Development Pvt. Ltd., you will enjoy a friendly and collaborative work environment, career growth opportunities, and a 5-day working culture. This is a full-time position requiring in-person work during morning shifts. If you are interested in this opportunity, please speak with the employer at +91 9888122266.,

The position of Software Engineer - Automation at Wildnet Technologies requires a smart and enthusiastic individual with 02 years of experience in automation, web scraping, or bot development. You will be involved in AI-driven projects, utilizing tools such as Puppeteer or Selenium, and primarily coding in JavaScript/Node.js. This role offers the opportunity to work on modern web protocols, browser automation, and the development of scalable systems within a fast-paced, tech-focused environment. Your responsibilities will include designing, developing, and maintaining automation scripts and web scraping tools. You will be responsible for building intelligent bots using frameworks like Puppeteer or Selenium, working with HTTP protocols, cookies, headers, and session management to mimic user behavior effectively. Collaboration with the AI/ML team to integrate automation into smart workflows and writing clean, scalable code in Node.js and JavaScript will also be key aspects of the role. Additionally, you will optimize automation systems for performance, reliability, and scalability. To excel in this role, you should possess 02 years of relevant experience in automation, scraping, or bot development. A strong understanding of HTTP protocols, cookies, headers, and web session mechanics is essential, along with hands-on experience with Puppeteer or Selenium automation frameworks. Proficiency in JavaScript and Node.js is required, as well as exposure to AI frameworks or tools like ChatGPT, Copilot, or other LLM-based systems. The ability to adapt quickly and contribute to fast-paced, evolving projects is also crucial. Experience with browser emulation, proxies, and anti-bot bypass strategies would be advantageous, as well as familiarity with cloud services and deployment tools. Wildnet Technologies offers a dynamic work environment with ongoing training, career advancement, and leadership development opportunities. As an established industry leader in digital marketing and IT services, you will have the opportunity to work on diverse projects with leading global brands across industries. The company is recognized for fostering a flexible, positive, and people-first work culture, with comprehensive insurance and wellness support for employees and their families. Flexible working hours, a 5-day work week, and a generous leave policy ensure a healthy work-life balance for employees.,