23086 Pyspark Jobs - Page 25

You will play a crucial role in the technology sector, focusing on AI and machine learning. Your responsibilities will include: - Demonstrating proficiency in Gen AI - Designing, implementing, and optimizing end-to-end AI/ML use cases at scale - Expertise in Python for backend code development - Establishing and enforcing engineering excellence standards for the team - Hands-on development of Machine Learning code and collaborating with Data Scientists to enhance ML models for high throughput and low-latency scenarios - Integrating ML model training and inferencing into Data pipelines To excel in this role, you should have the following qualifications: - Strong coding skills in Python and fa...

Role Overview: As a senior Data Engineer with 8+ years of experience, you will be responsible for building scalable data pipelines, integrating LLM/GenAI components, and supporting analytics workloads on Azure. Your expertise in Python and hands-on experience in Azure data services will be crucial for this role. Key Responsibilities: - Utilize Python for advanced data engineering tasks - Work with Azure Data Factory, Data Lake, Databricks/Spark, and Synapse - Perform SQL queries and data modeling - Implement GenAI/LLM workflows including RAG, embeddings, vector DBs, LangChain/Semantic Kernel Qualifications Required: - Strong expertise in Python - Hands-on experience with Azure Data Factory, ...

Role Overview: You will be responsible for leveraging your 3+ years of experience in Data Science, machine learning, or related fields to tackle structured and unstructured data problems. Your expertise in Machine Learning and Statistics will be crucial as you develop, implement, and deploy machine learning models on cloud platforms such as Azure, AWS, and GCP. Your role will involve integrating machine learning models into existing systems, developing MLOps pipelines, monitoring model performance, and providing insights to stakeholders. Key Responsibilities: - Apply practical experience in time series forecasting, clustering, classification techniques, regression, boosting algorithms, optim...

Data Platform - Design and build an in house data warehouse for near real time data sharing multiple products, including partnerships, multi-tenancy and RBAC Preferably someone with experience using these technologies like: Flink, Airflow, Hudi, Delta

At EY, we’re all in to shape your future with confidence. We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go. Join EY and help to build a better working world. EY GDS – Data and Analytics (D And A) – AWS DBX - Manager As part of our EY-GDS D&A (Data and Analytics) team, we help our clients solve complex business challenges with the help of data and technology. We dive deep into data to extract the greatest value and discover opportunities in key business and functions like Banking, Insurance, Manufacturing, Healthcare, Retail, Manufacturing and Auto, Supply Chain, and Finance. The opportunity We’re looking for Managers (...

At EY, we’re all in to shape your future with confidence. We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go. Join EY and help to build a better working world. EY GDS – Data and Analytics (D And A) – AWS DBX - Manager As part of our EY-GDS D&A (Data and Analytics) team, we help our clients solve complex business challenges with the help of data and technology. We dive deep into data to extract the greatest value and discover opportunities in key business and functions like Banking, Insurance, Manufacturing, Healthcare, Retail, Manufacturing and Auto, Supply Chain, and Finance. The opportunity We’re looking for Managers (...

Must have 3-5 years of hands-on experience on Data Engineering, Cloud, DevOps and AI/Model Engineering across the following technology stack o Advanced SQL o PySpark o Cloud services o Airflow or an equivalent cloud native orchestration tool o Kubernetes or an equivalent cloud native container service o Microservices o DevOps o Mlflow an equivalent cloud native model deployment and monitoring service o Basics of Generative AI --Large Language Models --Prompt Engineering --LangChain/LangGrapho Basics of Responsible AI Knowledge of design principles and fundamentals, Understanding of performance optimization/automations, Basic understanding of project domain, Good understanding of AI/ML deploy...

At EY, we’re all in to shape your future with confidence. We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go. Join EY and help to build a better working world. EY GDS – Data and Analytics (D And A) – AWS DBX - Manager As part of our EY-GDS D&A (Data and Analytics) team, we help our clients solve complex business challenges with the help of data and technology. We dive deep into data to extract the greatest value and discover opportunities in key business and functions like Banking, Insurance, Manufacturing, Healthcare, Retail, Manufacturing and Auto, Supply Chain, and Finance. The opportunity We’re looking for Managers (...

At EY, we’re all in to shape your future with confidence. We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go. Join EY and help to build a better working world. EY GDS – Data and Analytics (D And A) – AWS DBX - Manager As part of our EY-GDS D&A (Data and Analytics) team, we help our clients solve complex business challenges with the help of data and technology. We dive deep into data to extract the greatest value and discover opportunities in key business and functions like Banking, Insurance, Manufacturing, Healthcare, Retail, Manufacturing and Auto, Supply Chain, and Finance. The opportunity We’re looking for Managers (...

As a Data Engineer at our company, you will be responsible for designing, building, and maintaining scalable data pipelines and modern data lakehouse architectures using Microsoft Fabric. Your role will focus on enabling data ingestion, transformation, and governance across OneLake, Synapse, and Power BI to support downstream AI and analytics workloads. Key Responsibilities: - Design and implement end-to-end data pipelines across Bronze, Silver, and Gold layers within Microsoft Fabric. - Develop Dataflows Gen2, Spark notebooks, and Synapse pipelines to ingest data from databases, APIs, and files (Excel, CSV, etc.). - Manage and optimize data storage within OneLake, including partitioning, sc...

Job description Hiring for Azure Databrick Engineer Mandatory Skills: Databrick, Data lake, Azure Responsibilities A day in the life of an Infoscion As part of the Infosys consulting team, your primary role would be to actively aid the consulting team in different phases of the project including problem definition, effort estimation, diagnosis, solution generation and design and deployment You will explore the alternatives to the recommended solutions based on research that includes literature surveys, information available in public domains, vendor evaluation information, etc. and build POCs You will create requirement specifications from the business needs, define the to-be-processes and d...

Job description Hiring for Hadoop Developer Mandatory Skills: Hadoop, bigdata, spark Responsibilities A day in the life of an Infoscion As part of the Infosys consulting team, your primary role would be to actively aid the consulting team in different phases of the project including problem definition, effort estimation, diagnosis, solution generation and design and deployment You will explore the alternatives to the recommended solutions based on research that includes literature surveys, information available in public domains, vendor evaluation information, etc. and build POCs You will create requirement specifications from the business needs, define the to-be-processes and detailed funct...

Job description Hiring for Hadoop Developer Mandatory Skills: Hadoop, bigdata, spark Responsibilities A day in the life of an Infoscion As part of the Infosys consulting team, your primary role would be to actively aid the consulting team in different phases of the project including problem definition, effort estimation, diagnosis, solution generation and design and deployment You will explore the alternatives to the recommended solutions based on research that includes literature surveys, information available in public domains, vendor evaluation information, etc. and build POCs You will create requirement specifications from the business needs, define the to-be-processes and detailed funct...

EY GDS Data and Analytics (D And A) – AWS DBX - Manager As part of our EY-GDS D&A (Data and Analytics) team, we help our clients solve complex business challenges with the help of data and technology. We dive deep into data to extract the greatest value and discover opportunities in key business and functions like Banking, Insurance, Manufacturing, Healthcare, Retail, Manufacturing and Auto, Supply Chain, and Finance The opportunity We’re looking for Managers (GTM +Cloud/ Big Data Architects) with strong technology and data understanding having proven delivery capability in delivery and presales. This is a fantastic opportunity to be part of a leading firm as well as a part of a growing Data...

You will be joining a well-established cloud and data analytics company that focuses on empowering businesses to unlock insights and drive innovation through modern data solutions. **Role Overview:** As an AWS Data Engineer, you will utilize your 5 to 10 years of experience in Data Engineering or a related field to contribute to the development and maintenance of data solutions. Your role will be crucial in ensuring the efficient processing and analysis of large datasets. **Key Responsibilities:** - Strong proficiency in Python, PySpark, SQL, and AWS is mandatory for this role. - Demonstrate a good understanding of ETL processes, distributed systems, and microservices architecture. - Utilize...

As an experienced candidate with 7-12 years of expertise, you should have the following skills: - Proficiency in ML, AWS, Python, and MI Ops - Familiarity with Pyspark, Airflow, Jenkins, cloud formation, and Deep learning If you are interested in the position, please share your resume at deepika.eaga@quesscorp.com. As an experienced candidate with 7-12 years of expertise, you should have the following skills: - Proficiency in ML, AWS, Python, and MI Ops - Familiarity with Pyspark, Airflow, Jenkins, cloud formation, and Deep learning If you are interested in the position, please share your resume at deepika.eaga@quesscorp.com.

As a Data Engineer Senior Consultant at EY, you will be responsible for designing, building, and optimizing data solutions to enable advanced analytics and AI-driven business transformation. Your role will involve working with modern data engineering practices and cloud platforms to deliver robust and scalable data pipelines for various business domains such as finance, supply chain, energy, and commercial operations. Key Responsibilities: - Design, develop, and deploy end-to-end data pipelines for complex business problems, supporting analytics and modernizing data infrastructure. - Design and implement data models, ETL/ELT workflows, and data integration solutions across structured and uns...

Role Overview: As a highly energetic and collaborative Senior Data Engineer at UST, you will be responsible for leading enterprise data projects around Business and IT operations. Your expertise in developing and testing data pipelines, data analytics efforts, proactive issue identification, and resolution using traditional, new, and emerging technologies will play a key role in co-creating lasting change with our clients. Your excellent written and verbal communication skills are essential for liaising with technologists to executives and ensuring success in this role. Key Responsibilities: - Assemble large to complex sets of data meeting non-functional and functional business requirements ...

As a Lead Data Engineer at BrandIdea Analytics, you will have the exciting opportunity to shape the core of the analytics ecosystem by building data platforms from the ground up and leading with Python. **Key Responsibilities:** - Lead the design and development of scalable data pipelines using Python, Airflow, dbt, and PySpark. - Integrate multiple data sources including APIs, structured data, and geospatial data. - Build and optimize ETL/ELT workflows and data APIs using FastAPI/Flask. - Drive data quality, governance, and automation across different environments. - Mentor engineers and collaborate closely with Data Science & Product teams. - Contribute to cloud and architecture decisions,...

As a Machine Learning Scientist/Engineer at Uber, your role will involve designing, building, and deploying machine learning, statistical, and optimization models into Uber's production systems for various applications. You will collaborate with cross-functional teams including product, engineering, operations, and design to drive system development from conceptualization to production. Your responsibilities will include understanding product performance and identifying opportunities within data. **Key Responsibilities:** - Design, build, and deploy machine learning, statistical, and optimization models in Uber's production systems. - Collaborate with cross-functional teams to drive system d...

You have the opportunity to join as an Azure Data Factory with Python in Chennai. This is a hybrid job type with 3 days working from the office. **Role Overview:** As an Azure Data Factory with Python, you will be responsible for handling various tasks related to Azure Databricks, Azure Data Factory, Azure Synapse Analytics, Azure Analysis Services, Blob Storage, Databricks/Spark, Azure SQL DW/Synapse, and Azure functions. Your role will require in-depth knowledge of relational database design, data warehousing, and dimensional data modeling concepts. Additionally, you will be working with Python, Spark, and Pyspark. **Key Responsibilities:** - Minimum 6 to 13 years of experience - Hands-on ...

As a candidate for the position at Standard Chartered, you will be responsible for leading the design and development of advanced quantitative and AI-driven models for market abuse detection across multiple asset classes and trading venues. You will drive the solutioning and delivery of large-scale surveillance systems in a global investment banking environment, ensuring alignment of technical solutions with business objectives and regulatory requirements. Your key responsibilities will include: - Collaborating with consultants, MAR monitoring teams, and technology stakeholders to gather requirements and co-create innovative solutions - Translating regulatory and business requirements into a...

Warm Greetings from SP Staffing!! Role :AWS Data Engineer Experience Required :5 to 10 yrs Work Location: Hyderabad/Bangalore/Pune/Chennai/Kolkata/Hyderad Required Skills, Pyspark Interested candidates can send resumes to nandhini.spstaffing@gmail.com

As a highly skilled Technical Lead with 12 to 16 years of experience, your role will involve leading the design and implementation of scalable data solutions using Databricks Workflows to enhance data processing capabilities. Your responsibilities will include: - Overseeing the integration of Databricks Unity Catalog to ensure seamless data governance and security across platforms. - Providing technical expertise in Databricks to optimize performance and drive innovation in data analytics. - Collaborating with cross-functional teams to deliver high-quality solutions that meet business requirements and objectives. - Developing and maintaining comprehensive documentation for data workflows and...

Role Overview: As a Lead Data Engineer at Infosys, you will play a crucial role in providing high-quality consulting solutions to customers. Your responsibilities will include leading engagement efforts, reviewing proposals, identifying change management requirements, coaching team members, and contributing to organizational initiatives. Your expertise in AWS, Databricks, Python, and Pyspark will be instrumental in delivering value to clients throughout their digital transformation journey. Key Responsibilities: - Lead the engagement effort to deliver consulting solutions to customers at different stages of problem solving - Review proposals, provide guidance, and analyze solutions to identi...