23220 Pyspark Jobs - Page 45

At EY, we’re all in to shape your future with confidence. We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go. Join EY and help to build a better working world. EY GDS – Data and Analytics (D&A) – Manager – Azure Data Architect As part of our EY-GDS D&A (Data and Analytics) team, we help our clients solve complex business challenges with the help of data and technology. We dive deep into data to extract the greatest value and discover opportunities in key business and functions like Banking, Insurance, Manufacturing, Healthcare, Retail, Manufacturing and Auto, Supply Chain, and Finance. The opportunity We’re looking for M...

Join us as a Software Engineer, PySpark This is an opportunity for a driven Software Engineer to take on an exciting new career challenge Day-to-day, you'll be engineering and maintaining innovative, customer centric, high performance, secure and robust solutions It’s a chance to hone your existing technical skills and advance your career while building a wide network of stakeholders We're offering this role at associate level What you'll do In your new role, you’ll be working within a feature team to engineer software, scripts and tools, as well as liaising with other engineers, architects and business analysts across the platform. You’ll Also Be Producing complex and critical software rapi...

Line of Service Advisory Industry/Sector Not Applicable Specialism Data, Analytics & AI Management Level Associate Job Description & Summary At PwC, our people in data and analytics engineering focus on leveraging advanced technologies and techniques to design and develop robust data solutions for clients. They play a crucial role in transforming raw data into actionable insights, enabling informed decision-making and driving business growth. In data engineering at PwC, you will focus on designing and building data infrastructure and systems to enable efficient data processing and analysis. You will be responsible for developing and implementing data pipelines, data integration, and data tra...

Greetings from Tata Consultancy Services (TCS) TCS has always been in the spotlight for being adept in the next big technologies. What we can offer you is a space to explore varied technologies and quench your techie soul. What we are looking for : AWS Data Engineer Location : Hyderabad Interview Mode : Virtual mode (Microsoft Teams) Design, develop and maintain scalable and robust data pipelines using Apache Spark and Java on AWS Implement ETL/ELT processes to ingest, transform, and load large datasets from various sources into data warehouse on AWS(e.g S3, Redshift) Develop and Optimize Spark applications using Java to process and analyze large Java to process and analyze large volume of d...

Required Qualifications & Skills: Bachelors or Master’s degree in Computer Science / Information Technology / Information Science or equivalent. 5–8 years of experience in Data Engineering, Data Analytics, or Data Warehousing. Minimum 2+ years of experience with AWS Cloud Data Services (S3, Glue, DMS, Redshift, RDS, Aurora). Strong hands-on experience in SQL, Python, PowerShell, and PySpark. Deep knowledge of ETL/ELT frameworks and BI tools such as SSIS, SSRS, Power BI, Athena, QuickSight. Proficiency in data migration (homogeneous & heterogeneous) using AWS DMS, SCT, and related tools. Sound understanding of AI/ML concepts and implementation on real-world datasets (minimum 1 year). AWS Asso...

Job Summary We are seeking a highly skilled Sr. Developer with 6 to 10 years of experience to join our team in a hybrid work model. The ideal candidate will have expertise in Databricks SQL Databricks Delta Lake Databricks Workflows and PySpark. This role involves developing and optimizing data solutions to drive business insights and efficiencies. The position is a day shift role with no travel requirements. Responsibilities Develop and implement data solutions using Databricks SQL to enhance business intelligence capabilities. Optimize data processing workflows with Databricks Delta Lake to ensure data integrity and performance. Design and manage Databricks Workflows to automate data pipel...

About the Role: We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. The ideal candidate will have a strong background in Python, Azure, CI/CD, deployment, and writing efficient data pipelines. You will be responsible for designing, developing, and maintaining robust data infrastructure and pipelines that support our data-driven decision-making processes. Key Responsibilities: Data Pipeline Development: Design, develop, and maintain efficient and scalable data pipelines using Python and Azure tech stack. Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet b...

SUMMARY Job Description: As a Associate SRE, you’ll support the design, deployment, and monitoring of cloud and data systems to ensure reliability, scalability, and uptime. You’ll collaborate with developers and infrastructure teams to maintain production environments and automate routine operations. Responsibilities: Support CI/CD pipeline setup and basic monitoring systems for applications and data pipelines. Assist in automating deployments using tools like Jenkins, Git, and Terraform. Monitor performance metrics and escalate incidents to senior engineers. Maintain documentation and participate in root cause analysis for minor incidents. Work under supervision to implement application and...

SUMMARY Job Summary You will own and manage the reliability of large-scale data pipelines and storage platforms. Your goal is to ensure availability, data accuracy, and performance of analytics systems while driving automation and observability. Responsibilities Design and maintain highly available data pipelines in Databricks , Azure Data Factory , or AWS Glue . Automate reliability checks, error handling, and notification systems. Implement Infrastructure as Code (IaC) for data resources using Terraform or CloudFormation . Optimize ETL performance and improve recovery mechanisms. Configure and manage data observability tools (Monte Carlo, Bigeye, or custom monitoring). Participate in incid...

Design and build data pipelines and workflows in Microsoft Fabric using Lakehouse, Dataflow Gen2, and Warehouse. Develop and optimize ETL/ELT processes for structured, semi-structured, and unstructured data. Integrate data from multiple sources into OneLake ensuring quality, lineage, and consistency. Implement data models, partitions, and performance tuning in Fabric and Synapse.

As a Data Scientist/Senior Data Scientist – Price Optimization, Demand Modelling & Forecasting, you will play a critical role in developing and deploying advanced machine learning models for price optimization, cannibalization impact analysis, and demand forecasting. In addition to your technical expertise, you'll be a key point of contact for client management—leading client-facing projects and delivering actionable insights to address complex business problems. You will collaborate closely with cross-functional teams, translating business requirements into scalable data solutions while ensuring alignment with client objectives. You’ll also mentor and guide junior data scientists, fostering...

Job Title: Data Engineer (Scala, Spark, Pyspark, Databricks) Remark - final round will be the F2F Experience: 10+ Years (Lead Role) Responsibilities Design, develop, and maintain robust and scalable data pipelines using Apache Spark and Scala on the Databricks platform. Implement ETL (Extract, Transform, Load) processes for various data sources, ensuring data quality, integrity, and efficiency. Optimize Spark applications for performance and cost-efficiency within the Databricks environment. Work with Delta Lake for building reliable data lakes and data warehouses, ensuring ACID transactions and data versioning. Collaborate with data scientists, analysts, and other engineering teams to under...

Role - AWS Data Engineer Required Technical Skill Set* - AWS,Snowflake,ETL,Python,Pyspark, DBT experience with Sowflake development and integration experience AWS cloud and AWS services such as S3 Buckets, Lambda, Glue, API Gateway, SQS queues, RDS, Redshift; Experience in developing and supporting web applications including familiarity with web technologies and frameworks (Angular js, React.js). experience with automation of DevOps build using GitLab/Bitbucket/Jenkins/Maven; Experience in integration technologies like Fivetran, Dbt Experience working directly with technical and business teams. Familiar with software DevOps CI/CD tools, such Git, Jenkins, Linux, and Shell Script. Ability to ...

Role & responsibilities Required Technical Skill Set - Aws data engineer having strong experience of Python 1. Proficient in Python, with experience in deploying Python packages and OOP, Experience in ingesting data from different data sources (APIs, Web scraping, Flat-files, Databases) 2. hands-on experience in designing ETL processes, utilizing various architectures, orchestration tools (e.g. airflow), and data quality testing. 3. experience with Snowflake, dbt and data warehousing, 1 year of experience with AWS Cloud Services, including Lambda, S3, DynamoDB and Glue. 4. Python for data engineering and strong SQL development skills. 5. Proven track record of software development. Ability t...

Salary: 15 to 25 LPA Exp: 4 to 7 years Location: Gurgaon/Pune/Bengalore Notice: Immediate to 30 days..!! Job Profile: Experienced Data Engineer with a strong foundation in designing, building, and maintaining scalable data pipelines and architectures. Skilled in transforming raw data into clean, structured formats for analytics and business intelligence. Proficient in modern data tools and technologies such as SQL, T-SQL, Python, Databricks, and cloud platforms (Azure). Adept at data wrangling, modeling, ETL/ELT development, and ensuring data quality, integrity, and security. Collaborative team player with a track record of enabling data-driven decision-making across business units. As a Dat...

Job Title: Data Engineer (Scala, Spark, Pyspark, Databricks) Remark - final round will be the F2F Experience: 10+ Years (Lead Role) Responsibilities Design, develop, and maintain robust and scalable data pipelines using Apache Spark and Scala on the Databricks platform. Implement ETL (Extract, Transform, Load) processes for various data sources, ensuring data quality, integrity, and efficiency. Optimize Spark applications for performance and cost-efficiency within the Databricks environment. Work with Delta Lake for building reliable data lakes and data warehouses, ensuring ACID transactions and data versioning. Collaborate with data scientists, analysts, and other engineering teams to under...

Thorough working knowledge of Azure Databricks, Workflows, Pipelines. Proficient working knowledge of PySpark. Experience in Azure Cloud Services – Azure Data Factory, EventHub, SQL Server Good Hands-on experience in Kafka

GyanSys is looking for Data Lead/Technical Manager - Databricks for one of our internal customer. Title: Data Lead/Technical Manager - Databricks Location: Bangalore, India. 8-10 years of experience in data engineering, analytics, or data management with a strong focus on Azure cloud data solutions. Hands-on expertise in Azure Databricks (PySpark, Delta Lake, Workflows, Notebooks) for large-scale data processing and pipeline orchestration. Advanced proficiency in SQL for complex data modeling, transformation, and performance optimization in Azure environments. Proven experience designing and implementing end-to-end data pipelines using Azure Data Factory (ADF), Synapse Analytics, and Data La...

SNOWFLAKE DATA ENGINEER-(Immediate-45 Days Max) We are looking for an experienced Data Engineer with strong expertise in Snowflake and Azure to design and implement scalable data solutions. The ideal candidate will have hands-on experience in building data pipelines, managing data lakes, and delivering robust data warehousing solutions. Key Responsibilities Design, develop, and maintain data engineering solutions using Snowflake and Azure . Build and optimize ETL pipelines leveraging Azure Data Factory (ADF) ; experience with other ETL tools such as Glue, Qlik Replicate, or DBT is an added advantage. Work extensively with Snowflake data lake and ensure efficient data ingestion and transforma...

Experience : 3.00 + years Salary : INR 1800000-2200000 / year (based on experience) Expected Notice Period : 15 Days Shift : (GMT+05:30) Asia/Kolkata (IST) Opportunity Type : Remote Placement Type : Full Time Permanent position(Payroll and Compliance to be managed by: Aria Intelligent Solutions) (*Note: This is a requirement for one of Uplers' client - Aria Intelligent Solutions) What do you need for this opportunity? Must have skills required: Databricks, Develop and deploy production-ready solutions on AWS, PySpark, hands-on experience in applied ML, or forecasting., Time Series Aria Intelligent Solutions is Looking for: Key Skills  4–6 years of hands-on experience in applied ML, time ser...

Overview Data Analyst will be responsible to partner closely with business and S&T teams in preparing final analysis reports for the stakeholders enabling them to make important decisions based on various facts and trends and lead data requirement, source analysis, data analysis, data transformation and reconciliation activities. This role will be interacting with DG, DPM, EA, DE, EDF, PO and D &Ai teams for historical data requirement and sourcing the data for Mosaic AI program to scale solution to new markets. Responsibilities Lead data requirement, source analysis, data analysis, data transformation and reconciliation activities. Partners with FP&A Product Owner and associated business SM...

Warm Greetings from SP Staffing!! Role: Azure Data Architect Experience Required :12 to 17 yrs Work Location :PAN India Required Skills, Azure Databricks Interested candidates can send resumes to nandhini.spstaffing@gmail.com

Warm Greetings from SP Staffing!! Role: Azure Data Engineer Experience Required :5 to 8 yrs Work Location :Bangalore/Mumbai/Chennai/Bhubaneswar Required Skills, Azure Databricks Pyspark Interested candidates can send resumes to nandhini.spstaffing@gmail.com

Experience : 3.00 + years Salary : INR 1800000-2200000 / year (based on experience) Expected Notice Period : 15 Days Shift : (GMT+05:30) Asia/Kolkata (IST) Opportunity Type : Remote Placement Type : Full Time Permanent position(Payroll and Compliance to be managed by: Aria Intelligent Solutions) (*Note: This is a requirement for one of Uplers' client - Aria Intelligent Solutions) What do you need for this opportunity? Must have skills required: Databricks, Develop and deploy production-ready solutions on AWS, PySpark, hands-on experience in applied ML, or forecasting., Time Series Aria Intelligent Solutions is Looking for: Key Skills  4–6 years of hands-on experience in applied ML, time ser...

Experience : 3.00 + years Salary : INR 1800000-2200000 / year (based on experience) Expected Notice Period : 15 Days Shift : (GMT+05:30) Asia/Kolkata (IST) Opportunity Type : Remote Placement Type : Full Time Permanent position(Payroll and Compliance to be managed by: Aria Intelligent Solutions) (*Note: This is a requirement for one of Uplers' client - Aria Intelligent Solutions) What do you need for this opportunity? Must have skills required: Databricks, Develop and deploy production-ready solutions on AWS, PySpark, hands-on experience in applied ML, or forecasting., Time Series Aria Intelligent Solutions is Looking for: Key Skills  4–6 years of hands-on experience in applied ML, time ser...