28509 Pyspark Jobs - Page 4

Company Description DigitalXForce provides a Unified Enterprise Security Risk Posture Management (ESRPM) SaaS platform that enables real-time, continuous, and automated GRC through Cybersecurity Mesh Architecture. By leveraging data-driven insights, security blueprints, and regulatory control mapping, DigitalXForce optimizes and automates organizations' digital risk posture. The platform offers solutions in attack surface management, risk quantification, automated audit & compliance, and third-party risk management, among others. Our innovative approach empowers organizations to enhance their security posture while maximizing their investments in digital transformation. Role Overview : Team ...

Dear Professionals Greetings from TCS!!! Job Title : Senior Data Engineer (Snowflake, DBT, Azure) Experience : 5-8 Years Location : Only Bangalore/Chennai/Hyderabad/Pune Data Engineer: Required Skills and Qualifications • 5+ years of hands-on experience as a Data Engineer, with a strong portfolio of successful projects delivering robust, scalable, and reliable data solutions. • Strong proficiency in Microsoft SQL Server and Structured Query Language (SQL), including query optimization and advanced data querying. • Experience with scripting and programming languages, including PowerShell, Python, Scala, and working with Json, XML. • Solid experience leveraging technologies such as Snowflake, ...

Description At RBHU, we are committed to delivering data-driven solutions that empower decision-making. We work at the intersection of Data, AI, and Analytics, helping clients unlock the value of their data. We are now looking for an enthusiastic Data Engineer with hands-on Databricks experience to join our growing team. Key Responsibilities Develop, maintain, and optimize ETL/ELT pipelines using Databricks. Integrate data from multiple sources into scalable and reliable data solutions. Implement data transformations, cleansing, and enrichment processes. Work closely with data analysts, data scientists, and business teams to deliver analytics-ready datasets. Ensure data quality, performance,...

As a Data Scientist in the Pharma domain, you will be involved in analyzing large datasets such as Claims, Sales, and Promotions to generate valuable insights. Your role will also include building and deploying scalable ML/AI models, including Deep Learning, LLMs, and Generative AI. Collaborating with various teams across data, product, and commercial departments will be essential, along with mentoring junior analysts and contributing to SOPs/QC processes. Driving predictive modeling for Healthcare Professional (HCP) behavior and patient journey outcomes will be a significant part of your responsibilities. Key Responsibilities: - Analyze large datasets (Claims, Sales, Promotions, etc.) to ge...

Role Overview: As a Senior Data Scientist with experience in Google Cloud Platform (GCP), you will be responsible for leading complex data science projects, building and deploying advanced machine learning (ML) models, and leveraging GCP's ecosystem to drive strategic business decisions. Your role will require expert technical skills, strong business acumen, and the ability to collaborate with and mentor other team members. Key Responsibilities: - Lead and manage the entire data science project lifecycle, from problem definition to model deployment and monitoring. - Design, develop, and implement advanced statistical and machine learning models to solve complex business problems. - Architect...

As a Senior Azure Data Engineer cum Team Lead based in Pune, you will be responsible for the following: - Experience in Pyspark, Azure Databricks, and Azure Data Factory. - Proficiency in integrating, transforming, and consolidating data from various structured and unstructured data sources. - Good experience in SQL or native SQL query languages. - Strong experience in implementing Databricks notebooks using Python. - Good experience in Azure Data Factory, ADLS, Storage Services, Synapse, Serverless architecture, and Azure functions. Required Technical Skill Set: - Microsoft Azure Platform - Azure Data Factory - Azure Databricks - PySpark - Python - SQL If you are interested in this opportun...

As an MLOps Engineer, your role will involve researching and implementing MLOps tools, frameworks, and platforms for Data Science projects. You will work on activities to enhance MLOps maturity in the organization and introduce a modern, agile, and automated approach to Data Science. Conducting internal training and presentations on the benefits and usage of MLOps tools will also be part of your responsibilities. Key Responsibilities: - Model Deployment, Model Monitoring, Model Retraining - Deployment pipeline, Inference pipeline, Monitoring pipeline, Retraining pipeline - Drift Detection, Data Drift, Model Drift - Experiment Tracking - MLOps Architecture - REST API publishing Qualifications...

As part of Team Amex at American Express, you will be part of a culture that values innovation, shared values, and Leadership Behaviors. Your holistic well-being will be comprehensively supported, and you will have numerous opportunities to learn new skills, develop as a leader, and advance in your career. Your voice and ideas will be valued, your work will have a meaningful impact, and together, we will shape the future of American Express. In the role of a Lead Developer within the ASI (Analytics & Strategic Insights) Marketing Analytics team, you will have the opportunity to play a key role in developing data capabilities for commercial sales prospecting in lead markets within ICS. You wi...

As the Manager, Data Engineer at Mastercard, you will be responsible for managing multiple scrum teams of software developers and testers to develop quality software solutions in a timely and cost-effective manner. You will lead the definition, development, and delivery of major cross-department initiatives with broad scope and long-term business implications. Your role will involve providing technical leadership and direction to software development teams in the Big Data space, enforcing software development standards, and collaborating closely with product and architecture teams. Key Responsibilities: - Manage multiple scrum teams to develop quality software solutions - Lead major cross-de...

As a Lead Software & Solution Architect at Nokia, you will play a crucial role in developing and implementing data-lake solutions for Nokia's Hardware Services. Your responsibilities will include designing cloud-native architectures, integrating big data platforms, and ensuring robust software engineering practices at every stage of the lifecycle. Collaborating closely with solution architects, business stakeholders, and agile delivery teams, you will bring digital solutions from concept to production, focusing on performance, scalability, and maintainability. - Design and implement end-to-end software and data solutions utilizing Azure, Databricks, Synapse, and other cloud-native tools. - T...

As a Lead II - Software Engineering at UST Global, you will have the opportunity to lead enterprise data projects around Business and IT operations, utilizing your expertise in developing and testing data pipelines, data analytics efforts, and issue resolution mechanisms using various technologies. Your role will include: Role Overview: You will be responsible for assembling large and complex sets of data to meet business requirements, designing and implementing internal process improvements, optimizing data delivery, and automating manual processes. Key Responsibilities: - Building infrastructure for data extraction, transformation, and loading from various sources using GCP/Azure and SQL t...

As an Engineer specializing in Artificial Intelligence and Machine Learning, you will be responsible for utilizing your technical skills to design, develop, and deploy AI solutions on the Azure platform. Your experience of 5 to 10 years in AI/ML, Azure ML Studio, AI/ML On Databricks, Python & CICD Devops will be essential in executing the following key responsibilities: - Implementing supervised and unsupervised ML techniques and Predictive Analytics using Python - Generating features through data exploration and SME requirements - Querying relational databases - Applying computational algorithms and statistical methods to both structured and unstructured data - Communicating results effecti...

You are a detail-oriented and technically proficient professional with strong expertise in Python and SQL. Additionally, you have working knowledge of Power BI or Tableau for data visualization and exposure to PySpark. Key Responsibilities: - Develop and maintain scalable data pipelines using Python and SQL. - Perform data extraction, transformation, and loading (ETL) from various sources. - Conduct exploratory data analysis and statistical modeling to support business decisions. - Create dashboards and reports using Power BI or Tableau to visualize key metrics. - Collaborate with cross-functional teams to understand data requirements and deliver solutions. - Optimize query performance and e...

As a Data/Information Management AVP at our company, you will be a seasoned professional responsible for contributing to the development of new techniques and processes within the area or function. Your role will involve driving enhancements and projects to create efficiencies through automation and standardization. Here are the key responsibilities and qualifications for this position: Responsibilities: - Project management and governance experience with a track record of achieving critical milestones - Independently managing end-to-end business partner communication and collaborating with different departments to understand reporting needs - Ability to manage competing priorities in a comp...

As an ML Platform Specialist, your role involves designing, implementing, and maintaining robust machine learning infrastructure and workflows using Databricks Lakehouse Platform. Your responsibilities are critical in ensuring the smooth deployment, monitoring, and scaling of machine learning models across the organization. Key Responsibilities: - Design and implement scalable ML infrastructure on Databricks Lakehouse Platform - Develop and maintain continuous integration and continuous deployment (CI/CD) pipelines for machine learning models using Databricks workflows - Create automated testing and validation processes for machine learning models with Databricks MLflow - Implement and manag...

Role** Big Data Engineer Required Technical Skill Set** Hadoop, HIVE, PySpark, Spark SQL Must-Have** Strong hands-on experience of Hadoop, HIVE, Spark, PySpark, Big Data Eco System Tools. Strong SQL experience. Should be able to develop, tweak queries and work on performance enhancement. Solid understanding of object-oriented programming and HDFS concepts.

Role: Senior Data Engineer Location: Mumbai & Pune Experience: 3yrs to 8yrs Technologies / Skills: Advanced SQL, Python and associated libraries like Pandas, NumPy etc., Pyspark , Shell scripting, Data-Modelling, Big data, Hadoop, Hive, ETL pipelines. Responsibilities: • Proven success in communicating with users, other technical teams, and senior management to collect requirements, describe data modeling decisions and develop data engineering strategy. • Ability to work with business owners to define key business requirements and convert to user stories with required technical specifications. • Communicate results and business impacts of insight initiatives to key stakeholders to collaborat...

Job Title: Lead Data Engineer Location: Mumbai / Pune Experience: 8+ yrs Job Summary: We are seeking a technically strong and delivery-focused Lead Engineer to support and enhance enterprise-grade data and application products under the Durables model. The ideal candidate will act as the primary technical interface for the client, ensuring high system availability, performance, and continuous improvement. This role requires a hands-on technologist with strong team management experience, cloud (AWS) expertise, and excellent communication skills to handle client interactions and drive technical decisions. Key Responsibilities: Support & Enhancement Leadership Act as the primary technical lead ...

Job description Hiring for Spark Developer Mandatory Skills: Spark, Python, SQL Responsibilities A day in the life of an Infoscion As part of the Infosys consulting team, your primary role would be to actively aid the consulting team in different phases of the project including problem definition, effort estimation, diagnosis, solution generation and design and deployment You will explore the alternatives to the recommended solutions based on research that includes literature surveys, information available in public domains, vendor evaluation information, etc. and build POCs You will create requirement specifications from the business needs, define the to-be-processes and detailed functional...

Job description Hiring for Pyspark Develper-Female Diversity Mandatory Skills: Pyspark, Python, Big data, ETL Responsibilities A day in the life of an Infoscion As part of the Infosys consulting team, your primary role would be to actively aid the consulting team in different phases of the project including problem definition, effort estimation, diagnosis, solution generation and design and deployment You will explore the alternatives to the recommended solutions based on research that includes literature surveys, information available in public domains, vendor evaluation information, etc. and build POCs You will create requirement specifications from the business needs, define the to-be-pro...

Job description Hiring for Pyspark Develper-Female Candidates Mandatory Skills: Pyspark, Python, Big data, ETL Responsibilities A day in the life of an Infoscion As part of the Infosys consulting team, your primary role would be to actively aid the consulting team in different phases of the project including problem definition, effort estimation, diagnosis, solution generation and design and deployment You will explore the alternatives to the recommended solutions based on research that includes literature surveys, information available in public domains, vendor evaluation information, etc. and build POCs You will create requirement specifications from the business needs, define the to-be-pr...

Job description Hiring for Databrick Engineer-Female candidate Mandatory Skills: Databrick, Data lake, Azure Responsibilities A day in the life of an Infoscion As part of the Infosys consulting team, your primary role would be to actively aid the consulting team in different phases of the project including problem definition, effort estimation, diagnosis, solution generation and design and deployment You will explore the alternatives to the recommended solutions based on research that includes literature surveys, information available in public domains, vendor evaluation information, etc. and build POCs You will create requirement specifications from the business needs, define the to-be-proc...

Primary skills:Domain->Big Data - Data Processing->Spark,Scala, Pyspark->Azure A day in the life of an Infoscion As part of the Infosys delivery team, your primary role would be to interface with the client for quality assurance, issue resolution and ensuring high customer satisfaction. You will understand requirements, create and review designs, validate the architecture and ensure high levels of service offerings to clients in the technology domain. You will participate in project estimation, provide inputs for solution delivery, conduct technical risk planning, perform code reviews and unit test plan reviews. You will lead and guide your teams towards developing optimized high quality cod...

Primary skills:Technology->Cloud Platform->Amazon Webservices DevOps,Technology->OpenSystem->Python - OpenSystem, Pyspark A day in the life of an Infoscion As part of the Infosys delivery team, your primary role would be to interface with the client for quality assurance, issue resolution and ensuring high customer satisfaction. You will understand requirements, create and review designs, validate the architecture and ensure high levels of service offerings to clients in the technology domain. You will participate in project estimation, provide inputs for solution delivery, conduct technical risk planning, perform code reviews and unit test plan reviews. You will lead and guide your teams to...

TCS is inviting applications!!! Role: AWS Data Engineer EXP: 6 - 8 YEARS LOCATION: Chennai/ Pune / Bangalore/ Hyderabad / Kolkata/ Delhi/ Kochi/ Indore Job Description Professional experience in data integration and management 3+ years of hands-on development experience in AWS Glue, Lambda functions, AWS Athena, Redshift, Pyspark is a Must Hands on working experience in AWS GLUE ETL tool and having good knowledge on any of the ETL tools (Informatica cloud, Talend etc.) Must have working experience with SQL, and databases (e.g.: Postgres, Redshift) implementing Data Models. Expertise with Python Language and Apache Spark. Have working experience in projects involving these. Strong hands-on ex...