23086 Pyspark Jobs - Page 40

Design and build data pipelines and workflows in Microsoft Fabric using Lakehouse, Dataflow Gen2, and Warehouse. Develop and optimize ETL/ELT processes for structured, semi-structured, and unstructured data. Integrate data from multiple sources into OneLake ensuring quality, lineage, and consistency. Implement data models, partitions, and performance tuning in Fabric and Synapse.

As a Data Scientist/Senior Data Scientist – Price Optimization, Demand Modelling & Forecasting, you will play a critical role in developing and deploying advanced machine learning models for price optimization, cannibalization impact analysis, and demand forecasting. In addition to your technical expertise, you'll be a key point of contact for client management—leading client-facing projects and delivering actionable insights to address complex business problems. You will collaborate closely with cross-functional teams, translating business requirements into scalable data solutions while ensuring alignment with client objectives. You’ll also mentor and guide junior data scientists, fostering...

Job Title: Data Engineer (Scala, Spark, Pyspark, Databricks) Remark - final round will be the F2F Experience: 10+ Years (Lead Role) Responsibilities Design, develop, and maintain robust and scalable data pipelines using Apache Spark and Scala on the Databricks platform. Implement ETL (Extract, Transform, Load) processes for various data sources, ensuring data quality, integrity, and efficiency. Optimize Spark applications for performance and cost-efficiency within the Databricks environment. Work with Delta Lake for building reliable data lakes and data warehouses, ensuring ACID transactions and data versioning. Collaborate with data scientists, analysts, and other engineering teams to under...

Role - AWS Data Engineer Required Technical Skill Set* - AWS,Snowflake,ETL,Python,Pyspark, DBT experience with Sowflake development and integration experience AWS cloud and AWS services such as S3 Buckets, Lambda, Glue, API Gateway, SQS queues, RDS, Redshift; Experience in developing and supporting web applications including familiarity with web technologies and frameworks (Angular js, React.js). experience with automation of DevOps build using GitLab/Bitbucket/Jenkins/Maven; Experience in integration technologies like Fivetran, Dbt Experience working directly with technical and business teams. Familiar with software DevOps CI/CD tools, such Git, Jenkins, Linux, and Shell Script. Ability to ...

Role & responsibilities Required Technical Skill Set - Aws data engineer having strong experience of Python 1. Proficient in Python, with experience in deploying Python packages and OOP, Experience in ingesting data from different data sources (APIs, Web scraping, Flat-files, Databases) 2. hands-on experience in designing ETL processes, utilizing various architectures, orchestration tools (e.g. airflow), and data quality testing. 3. experience with Snowflake, dbt and data warehousing, 1 year of experience with AWS Cloud Services, including Lambda, S3, DynamoDB and Glue. 4. Python for data engineering and strong SQL development skills. 5. Proven track record of software development. Ability t...

Salary: 15 to 25 LPA Exp: 4 to 7 years Location: Gurgaon/Pune/Bengalore Notice: Immediate to 30 days..!! Job Profile: Experienced Data Engineer with a strong foundation in designing, building, and maintaining scalable data pipelines and architectures. Skilled in transforming raw data into clean, structured formats for analytics and business intelligence. Proficient in modern data tools and technologies such as SQL, T-SQL, Python, Databricks, and cloud platforms (Azure). Adept at data wrangling, modeling, ETL/ELT development, and ensuring data quality, integrity, and security. Collaborative team player with a track record of enabling data-driven decision-making across business units. As a Dat...

Job Title: Data Engineer (Scala, Spark, Pyspark, Databricks) Remark - final round will be the F2F Experience: 10+ Years (Lead Role) Responsibilities Design, develop, and maintain robust and scalable data pipelines using Apache Spark and Scala on the Databricks platform. Implement ETL (Extract, Transform, Load) processes for various data sources, ensuring data quality, integrity, and efficiency. Optimize Spark applications for performance and cost-efficiency within the Databricks environment. Work with Delta Lake for building reliable data lakes and data warehouses, ensuring ACID transactions and data versioning. Collaborate with data scientists, analysts, and other engineering teams to under...

Thorough working knowledge of Azure Databricks, Workflows, Pipelines. Proficient working knowledge of PySpark. Experience in Azure Cloud Services – Azure Data Factory, EventHub, SQL Server Good Hands-on experience in Kafka

GyanSys is looking for Data Lead/Technical Manager - Databricks for one of our internal customer. Title: Data Lead/Technical Manager - Databricks Location: Bangalore, India. 8-10 years of experience in data engineering, analytics, or data management with a strong focus on Azure cloud data solutions. Hands-on expertise in Azure Databricks (PySpark, Delta Lake, Workflows, Notebooks) for large-scale data processing and pipeline orchestration. Advanced proficiency in SQL for complex data modeling, transformation, and performance optimization in Azure environments. Proven experience designing and implementing end-to-end data pipelines using Azure Data Factory (ADF), Synapse Analytics, and Data La...

SNOWFLAKE DATA ENGINEER-(Immediate-45 Days Max) We are looking for an experienced Data Engineer with strong expertise in Snowflake and Azure to design and implement scalable data solutions. The ideal candidate will have hands-on experience in building data pipelines, managing data lakes, and delivering robust data warehousing solutions. Key Responsibilities Design, develop, and maintain data engineering solutions using Snowflake and Azure . Build and optimize ETL pipelines leveraging Azure Data Factory (ADF) ; experience with other ETL tools such as Glue, Qlik Replicate, or DBT is an added advantage. Work extensively with Snowflake data lake and ensure efficient data ingestion and transforma...

Experience : 3.00 + years Salary : INR 1800000-2200000 / year (based on experience) Expected Notice Period : 15 Days Shift : (GMT+05:30) Asia/Kolkata (IST) Opportunity Type : Remote Placement Type : Full Time Permanent position(Payroll and Compliance to be managed by: Aria Intelligent Solutions) (*Note: This is a requirement for one of Uplers' client - Aria Intelligent Solutions) What do you need for this opportunity? Must have skills required: Databricks, Develop and deploy production-ready solutions on AWS, PySpark, hands-on experience in applied ML, or forecasting., Time Series Aria Intelligent Solutions is Looking for: Key Skills  4–6 years of hands-on experience in applied ML, time ser...

Overview Data Analyst will be responsible to partner closely with business and S&T teams in preparing final analysis reports for the stakeholders enabling them to make important decisions based on various facts and trends and lead data requirement, source analysis, data analysis, data transformation and reconciliation activities. This role will be interacting with DG, DPM, EA, DE, EDF, PO and D &Ai teams for historical data requirement and sourcing the data for Mosaic AI program to scale solution to new markets. Responsibilities Lead data requirement, source analysis, data analysis, data transformation and reconciliation activities. Partners with FP&A Product Owner and associated business SM...

Warm Greetings from SP Staffing!! Role: Azure Data Architect Experience Required :12 to 17 yrs Work Location :PAN India Required Skills, Azure Databricks Interested candidates can send resumes to nandhini.spstaffing@gmail.com

Warm Greetings from SP Staffing!! Role: Azure Data Engineer Experience Required :5 to 8 yrs Work Location :Bangalore/Mumbai/Chennai/Bhubaneswar Required Skills, Azure Databricks Pyspark Interested candidates can send resumes to nandhini.spstaffing@gmail.com

Experience : 3.00 + years Salary : INR 1800000-2200000 / year (based on experience) Expected Notice Period : 15 Days Shift : (GMT+05:30) Asia/Kolkata (IST) Opportunity Type : Remote Placement Type : Full Time Permanent position(Payroll and Compliance to be managed by: Aria Intelligent Solutions) (*Note: This is a requirement for one of Uplers' client - Aria Intelligent Solutions) What do you need for this opportunity? Must have skills required: Databricks, Develop and deploy production-ready solutions on AWS, PySpark, hands-on experience in applied ML, or forecasting., Time Series Aria Intelligent Solutions is Looking for: Key Skills  4–6 years of hands-on experience in applied ML, time ser...

Experience : 3.00 + years Salary : INR 1800000-2200000 / year (based on experience) Expected Notice Period : 15 Days Shift : (GMT+05:30) Asia/Kolkata (IST) Opportunity Type : Remote Placement Type : Full Time Permanent position(Payroll and Compliance to be managed by: Aria Intelligent Solutions) (*Note: This is a requirement for one of Uplers' client - Aria Intelligent Solutions) What do you need for this opportunity? Must have skills required: Databricks, Develop and deploy production-ready solutions on AWS, PySpark, hands-on experience in applied ML, or forecasting., Time Series Aria Intelligent Solutions is Looking for: Key Skills  4–6 years of hands-on experience in applied ML, time ser...

Experience : 3.00 + years Salary : INR 1800000-2200000 / year (based on experience) Expected Notice Period : 15 Days Shift : (GMT+05:30) Asia/Kolkata (IST) Opportunity Type : Remote Placement Type : Full Time Permanent position(Payroll and Compliance to be managed by: Aria Intelligent Solutions) (*Note: This is a requirement for one of Uplers' client - Aria Intelligent Solutions) What do you need for this opportunity? Must have skills required: Databricks, Develop and deploy production-ready solutions on AWS, PySpark, hands-on experience in applied ML, or forecasting., Time Series Aria Intelligent Solutions is Looking for: Key Skills  4–6 years of hands-on experience in applied ML, time ser...

Job Title: Data Engineer (Scala, Spark, Pyspark, Databricks) Remark - final round will be the F2F Experience: 10+ Years (Lead Role) Responsibilities Design, develop, and maintain robust and scalable data pipelines using Apache Spark and Scala on the Databricks platform. Implement ETL (Extract, Transform, Load) processes for various data sources, ensuring data quality, integrity, and efficiency. Optimize Spark applications for performance and cost-efficiency within the Databricks environment. Work with Delta Lake for building reliable data lakes and data warehouses, ensuring ACID transactions and data versioning. Collaborate with data scientists, analysts, and other engineering teams to under...

Experience : 3.00 + years Salary : INR 1800000-2200000 / year (based on experience) Expected Notice Period : 15 Days Shift : (GMT+05:30) Asia/Kolkata (IST) Opportunity Type : Remote Placement Type : Full Time Permanent position(Payroll and Compliance to be managed by: Aria Intelligent Solutions) (*Note: This is a requirement for one of Uplers' client - Aria Intelligent Solutions) What do you need for this opportunity? Must have skills required: Databricks, Develop and deploy production-ready solutions on AWS, PySpark, hands-on experience in applied ML, or forecasting., Time Series Aria Intelligent Solutions is Looking for: Key Skills  4–6 years of hands-on experience in applied ML, time ser...

Join Amgen’s Mission of Serving Patients At Amgen, if you feel like you’re part of something bigger, it’s because you are. Our shared mission—to serve patients living with serious illnesses—drives all that we do. Since 1980, we’ve helped pioneer the world of biotech in our fight against the world’s toughest diseases. With our focus on four therapeutic areas –Oncology, Inflammation, General Medicine, and Rare Disease– we reach millions of patients each year. As a member of the Amgen team, you’ll help make a lasting impact on the lives of patients as we research, manufacture, and deliver innovative medicines to help people live longer, fuller happier lives. Our award-winning culture is collabo...

About the Role We are looking for a Machine Learning Scientist to join our Payments Machine Learning team and make meaningful contributions to our mission to streamline and optimize Uber's global payment experiences. In this role, you will be able to use your strong quantitative skills in the fields of machine learning, statistics, economics, operations research to improve the Uber Payments experience. Example projects we work on include optimizing transaction routing and automated retry strategies to improve payment authorization success rates and rider/eater conversion. We are looking for experienced candidates with a passion for solving new and difficult problems with data. In this role, ...

Management Level: - Ind & Func AI Decision Science Consultant – Level 9 Location: Gurgaon / Bangalore / Mumbai / Hyderabad Functional: Commercial Analytics (Route to market), Revenue growth management - pricing , promotion, assortment, Descriptive, Diagnostic, Predictive, prescriptive analytics Analytics Models knowledge: Econometric Modeling, Statistical Timeseries Models, Store Clustering Algorithms, Causal models, State Space Modeling, Mixed Effect Regression, NLP Techniques, Large Language Models, non-parametric models, AI/ML model development, Supervised and unsupervised learning models, Agentic frameworks, Generative AI models. Technical: Azure ML Tech Stack, SQL, PySpark, Python, Clou...

Seeking Senior Product & Platform Specialist (9+ yrs) with expertise in GCP, BigQuery, PySpark, Python, SQL, and Agile. Lead data platform initiatives, drive innovation, ensure quality, and deliver scalable cloud solutions. Certifications preferred. Required Candidate profile Senior Product & Platform Specialist with 9+ yrs in GCP, BigQuery, PySpark & Python. Expert in data platforms, SQL, and Agile delivery. Strong leader driving innovation and cloud transformation.

Job Title: Data Engineer (Scala, Spark, Pyspark, Databricks) Remark - final round will be the F2F Experience: 10+ Years (Lead Role) Responsibilities Design, develop, and maintain robust and scalable data pipelines using Apache Spark and Scala on the Databricks platform. Implement ETL (Extract, Transform, Load) processes for various data sources, ensuring data quality, integrity, and efficiency. Optimize Spark applications for performance and cost-efficiency within the Databricks environment. Work with Delta Lake for building reliable data lakes and data warehouses, ensuring ACID transactions and data versioning. Collaborate with data scientists, analysts, and other engineering teams to under...

We are looking for a QA Engineer Data Testing with experience in validating data pipelines, warehouses, and modern data platforms. The role focuses on ensuring data quality, accuracy, and reliability across data systems. Key Responsibilities: Perform testing on ETL/ELT pipelines and data warehouses. Validate data quality, schema, reconciliation, and transformations. Work on cloud data platforms (Snowflake, BigQuery, AWS/GCP). Use SQL and Python for test automation and validation. Hand on experience with Pyspark Hand on experience with Automation Support performance and data quality checks on large datasets. Required Skills: 3+ years of QA testing experience, with at least 1–2 years in data t...