74 Sparksql Jobs

Role Overview: As a PySpark Developer, you will be responsible for developing and maintaining PySpark applications. You will work with various technologies within the Big Data ecosystem, such as Apache Spark, Hadoop, and SparkSQL. Additionally, you will be expected to have a good understanding of AWS and Snowflake, as well as experience with CICD and system design. Candidates with prior experience in technologies related to Fund transfer AML will have an added advantage. Strong communication skills, the ability to work under pressure, and a proactive attitude towards learning are essential for this role. Key Responsibilities: - Develop and maintain PySpark applications using Apache Spark and...

Role Overview: As a Scala Spark Developer, you will be responsible for leveraging your expertise in Apache Spark, Big Data Hadoop Ecosystem, SparkSQL, and Scala to design and develop big data platforms. Your deep understanding of modern data processing technology stacks and streaming data architectures will be pivotal in enabling real-time and low-latency data processing. Key Responsibilities: - Possessing 5+ years of Scala and Spark experience, with a focus on Data Engineering - Demonstrating hands-on experience in the design and development of big data platforms - Utilizing expertise in Spark, HBase, Hive, and other Hadoop ecosystem technologies for development using Scala - Understanding ...

In this role, you will ensure effective Design Development Validation and Support activities to meet client needs and architectural requirements. Your responsibilities will include gathering requirements, translating them into system requirements, and contributing to project estimations. Additionally, you will play a key role in building efficient programs and systems to support clients in their digital transformation journey. **Key Responsibilities:** - Ensure effective Design Development Validation and Support activities - Gather and translate client requirements into system requirements - Contribute to project estimations - Build efficient programs and systems to support clients" digital ...

As a Developer at our company, you will have the opportunity to leverage your expertise in Spark, Scala, Python, and SparkSQL to drive innovative solutions and enhance data processing capabilities. Your contributions will play a direct role in our company's growth and societal advancements through data-driven insights. **Key Responsibilities:** - Develop and implement scalable data processing solutions using Spark and Scala. - Collaborate with data scientists and analysts to optimize data workflows. - Design and maintain data pipelines using SparkSQL and Python. - Troubleshoot and resolve technical issues related to Spark applications. - Conduct code reviews and provide feedback to maintain ...

As an Associate Manager - Data Engineering at EY, your main objective will be to support cloud and on-prem platform analytics and data engineering projects initiated across engagement teams. You will be responsible for conceptualizing, designing, developing, deploying, and maintaining complex technology solutions to help EY solve business problems for clients. Your role will involve working closely with technical architects, product and business subject matter experts, back-end developers, and other solution architects, and you will also have on-shore facing interactions. **Key Responsibilities:** - Evaluating and selecting data warehousing tools for business intelligence, data population, m...

As a Data Engineer - Pyspark at Barclays, you will be spearheading the evolution of the digital landscape, driving innovation and excellence. You will be utilizing cutting-edge technology to revolutionize digital offerings, ensuring unparalleled customer experiences. As part of a team of developers, you will be responsible for delivering the technology stack, using strong analytical and problem-solving skills to understand business requirements and deliver quality solutions. **Key Responsibilities:** - Hands-on experience in Pyspark and strong knowledge of Dataframes, RDD, and SparkSQL. - Hands-on experience in Pyspark performance optimization techniques. - Hands-on experience in developing,...

Will be responsible for the design development and administration of data management solutions that align with the business needs of the organization. Will also respond to support Operations specially on production tickets and business requests for access and reporting. Overall 5+ years of experience and has at least 2-3 years of experience in Data Engineering especially ADF Python PySpark SparkSql Sql and Databricks programming and cloud data ingestion data pipelines handling structured and semi-structured data formats like JSON XML. Collaborate with internal and external personnel including senior management system architects software developers database administrators design analysts and ...

Description: Technical i. Pyspark SparkSQL SQL and Glue. ii. AWS cloud experience iii. Good understanding of dimensional modelling iv. Good understanding DevOps CloudOps DataOps CI/CD & with a SRE mindset v. Understanding of Lakehouse and DW architecture vi. strong analysis and analytical skills vii. understanding of version control systems specifically Git viii. strong in software engineering APIs Microservices etc. Soft skills i. written and oral communication skills ii. ability to translate business needs to system and s Named Job Posting? (if Yes - needs to be approved by SCSC) Additional Details Global Grade : C Level : To Be Defined Named Job Posting? (if Yes - needs to be approved by ...

As a candidate for this role, you would be responsible for the following: - Understanding of Spark core concepts like RDDs, DataFrames, DataSets, SparkSQL, and Spark Streaming. - Experience with Spark optimization techniques. - Deep knowledge of Delta Lake features like time travel, schema evolution, data partitioning. - Ability to design and implement data pipelines using Spark and Delta Lake as the data storage layer. - Proficiency in Python/Scala/Java for Spark development and integrate with ETL process. - Knowledge of data ingestion techniques from various sources (flat files, CSV, API, database). - Understanding of data quality best practices and data validation techniques. Qualificatio...

As a Lead Consultant- Databricks Developer at Genpact, your role involves solving real-world cutting-edge problems to meet both functional and non-functional requirements. Your primary responsibilities will include: - Maintaining close awareness of new and emerging technologies and their potential application for service offerings and products. - Collaborating with architects and lead engineers to develop solutions that meet functional and non-functional requirements. - Demonstrating knowledge of relevant industry trends and standards. - Showcasing strong analytical and technical problem-solving skills. - Having experience in the Data Engineering domain. Qualifications we seek in you: - Bach...

As an experienced candidate with over 8 years of overall experience and 5+ years of relevant experience, your role will involve the following key responsibilities: - Strong understanding of Spark core including RDDs, DataFrames, DataSets, SparkSQL, and Spark Streaming - Proficiency in Delta Lake features such as time travel, schema evolution, and data partitioning - Designing and building data pipelines using Spark and Delta Lake - Solid experience in Python/Scala/Java for Spark development - Knowledge of data ingestion from files, APIs, and databases - Familiarity with data validation and quality best practices - Working knowledge of data warehouse concepts and data modeling - Hands-on expe...

As a Data Engineer specializing in Spark & Delta Lake, your role will involve understanding key Spark concepts such as RDDs, DataFrames, DataSets, SparkSQL, and Spark Streaming. You will also be expected to have experience in optimizing Spark performance and a deep understanding of Delta Lake features like time travel, schema evolution, and data partitioning. Your responsibilities will include designing and implementing data pipelines using Spark and Delta Lake as the data storage layer. Proficiency in Python, Scala, or Java for Spark development and integration with ETL processes is essential. Additionally, you should possess knowledge of data ingestion techniques from various sources such ...

As a Software Engineer III at JPMorgan Chase within the Corporate Technology, your role involves serving as a seasoned member of an agile team to design and deliver trusted market-leading technology products in a secure, stable, and scalable manner. Your responsibilities include executing software solutions, design, development, and technical troubleshooting with the ability to think innovatively, creating secure and high-quality production code, producing architecture and design artifacts for complex applications, and contributing to software engineering communities of practice and events exploring new technologies. You are also expected to proactively identify hidden problems and patterns ...

As a Senior Analyst - Data Engineering at EY-Strategy and Transactions - SaT DnA, you will play a crucial role in supporting cloud and on-prem platform analytics and data engineering projects across engagement teams. Your primary responsibilities will involve conceptualizing, designing, developing, deploying, and maintaining complex technology solutions to solve business problems for clients. You will work closely with technical architects, product and business subject matter experts, back-end developers, and solution architects, contributing to the design, development, and evolution of modern data warehousing solutions and data integration build-outs using cutting-edge tools and platforms f...

Role Overview: As a Software Engineer III at JPMorgan Chase within the Corporate Technology, you will be a seasoned member of an agile team responsible for designing and delivering trusted market-leading technology products in a secure, stable, and scalable manner. Your role involves executing software solutions, development, and technical troubleshooting while maintaining high-quality production code and algorithms. You will contribute to the architecture and design of complex applications, ensuring design constraints are met by software code development. Additionally, you will analyze and synthesize data sets to drive continuous improvement of software applications and systems. Your proact...

You have 5 to 8 years of experience in Databricks Delta Lake and have hands-on experience with Azure. Your expertise includes Python scripting and relevant experience with ETL methods, retrieving data from dimensional data models, and data warehouses. You also possess strong experience with relational databases, data access methods, especially SQL, and knowledge of Azure architecture and design. Key Responsibilities: - Utilize Databricks, Java, Python, Scala, and SparkSQL for efficient data processing - Implement ETL methods for data extraction, transformation, and loading - Retrieve data from dimensional data models and data warehouses - Work with relational databases and SQL for data manip...

You are ready to gain the skills and experience needed to grow within your role and advance your career, and there is a perfect software engineering opportunity for you at JPMorgan Chase within the Corporate Technology team. As a Software Engineer II, you will be part of an agile team that enhances, designs, and delivers the software components of state-of-the-art technology products in a secure, stable, and scalable manner. Your role involves executing software solutions through design, development, and technical troubleshooting of multiple components within a technical product, application, or system while acquiring the necessary skills and experience to advance in your role. **Key Respons...

As a Data Engineering Consultant with GCP experience at LTIMINDTREE, your role will involve the following responsibilities: - Good experience in SQL Queries - Understanding of databases data flows and data architecture - Experience on Python and GCP platform - Willingness to Collaborate with Cross-Functional teams to drive validation and Project Execution - Review and evaluate policies and procedures - To develop, implement, monitor, and manage the compliance needs as per the AEMP70 policy Qualifications Required: - Mandatory Skills: Apache Spark, Java, Python, Scala, SparkSQL, Databricks (Note: No additional details about the company were provided in the job description),

Job title: Senior Software Engineer Experience: 5- 8 years Primary skills: Python, Spark or Pyspark, DWH ETL. Database: SparkSQL or PostgreSQL Secondary skills: Databricks ( Delta Lake, Delta tables, Unity Catalog) Work Model: Hybrid (Weekly Twice) Cab Facility: Yes Work Timings: Rotational Shift- 8am-5pm / 1pm- 10 pm Interview Process: 3 rounds (1st & 2nd F2F mandatory) Work Location: Karle Town Tech Park Nagawara, Hebbal Bengaluru 560045 About Business Unit: The Architecture Team plays a pivotal role in the end-to-end design, governance, and strategic direction of product development within Epsilon People Cloud (EPC). As a centre of technical excellence, the team ensures that every product...

Location: Remote / Pan India- Delhi / NCR,Bangalore/Bengaluru,Hyderabad/Secunderabad,Chennai,Pune,Kolkata,Ahmedabad,Mumbai Job Responsibilities: To transition legacy Rules from Python using the Polars Library to SparkSQL To create new Rules using SparkSQL based on written requirements. Must Have Skills: Understanding of Polars library Understanding of SparkSQL (this is more important than Polars) Good English communication Ability to work in a collaborative environment Experience with US healthcare data preferred Understanding of SparkSQL (this is more important than Polars)

ISource Services is hiring for one the of their client for the position of Data Engineer. Job Responsibilities: To transition legacy Rules from Python using the Polars Library to SparkSQL To create new Rules using SparkSQL based on written requirements. Must Have Skills: Understanding of Polars library Understanding of SparkSQL (this is more important than Polars) Good English communication Ability to work in a collaborative environment Experience with US healthcare data preferred Location: Remote / Pan India,Delhi NCR,Bangalore,Chennai,Pune,Kolkata,Ahmedabad,Mumbai,Hyderabad

Looking For a Better Opportunity? Join Us!! About DMI DMI, the worlds first end-to-end mobility company, combines all the skills and services necessary to deliver mobile enterprise solutions. Built to reinvent business through mobility, DMI has expertise in mobile strategy, UX, web, and app development, omni-channel commerce, brand and marketing, big data management and analytics, and secure device and app management. The companys unique, integrated approach to mobility has resulted in dramatic growth as well as an expanding client base, which includes hundreds of Fortune 1000 commercial clients and all fifteen U.S. Federal Departments. DMI is headquartered in Washington DC area, with satell...

As a Data Engineer specializing in Pyspark and SQL at Barclays, your main role will involve spearheading the evolution of the digital landscape, driving innovation and excellence within the company. You will be responsible for harnessing cutting-edge technology to revolutionize digital offerings, ensuring unparalleled customer experiences. Working as part of a team of developers, your primary focus will be delivering a technology stack using your strong analytical and problem-solving skills to understand business requirements and deliver quality solutions. Key Responsibilities: - Hands-on experience in Pyspark with a strong knowledge of Dataframes, RDD, and SparkSQL. - Proficiency in Pyspark...

What this job involves: JLL, an international real estate management company, is seeking an Data Engineer to join our JLL Technologies Team. We are seeking candidates that are self-starters to work in a diverse and fast-paced environment that can join our Enterprise Data team. We are looking for a candidate that is responsible for designing and developing of data solutions that are strategic for the business using the latest technologies Azure Databricks, Python, PySpark, SparkSQL, Azure functions, Delta Lake, Azure DevOps CI/CD. Responsibilities Design, Architect, and Develop solutions leveraging cloud big data technology to ingest, process and analyze large, disparate data sets to exceed b...

What this job involves: JLL, an international real estate management company, is seeking an Data Engineer to join our JLL Technologies Team. We are seeking candidates that are self-starters to work in a diverse and fast-paced environment that can join our Enterprise Data team. We are looking for a candidate that is responsible for designing and developing of data solutions that are strategic for the business using the latest technologies Azure Databricks, Python, PySpark, SparkSQL, Azure functions, Delta Lake, Azure DevOps CI/CD. Responsibilities Design, Architect, and Develop solutions leveraging cloud big data technology to ingest, process and analyze large, disparate data sets to exceed b...