16 Dataframes Jobs

As a Data Engineer specializing in Pyspark and SQL at Barclays, your main role will involve spearheading the evolution of the digital landscape, driving innovation and excellence within the company. You will be responsible for harnessing cutting-edge technology to revolutionize digital offerings, ensuring unparalleled customer experiences. Working as part of a team of developers, your primary focus will be delivering a technology stack using your strong analytical and problem-solving skills to understand business requirements and deliver quality solutions. Key Responsibilities: - Hands-on experience in Pyspark with a strong knowledge of Dataframes, RDD, and SparkSQL. - Proficiency in Pyspark...

Join us as a Data Engineer responsible for supporting the successful delivery of Location Strategy projects to plan, budget, agreed quality, and governance standards. You'll spearhead the evolution of our digital landscape, driving innovation and excellence. You will harness cutting-edge technology to revolutionize our digital offerings, ensuring unparalleled customer experiences. To be successful as a Data Engineer, you should have experience with: - Hands-on experience in PySpark and strong knowledge of Dataframes, RDD, and SparkSQL. - Hands-on experience in developing, testing, and maintaining applications on AWS Cloud. - Strong hold on AWS Data Analytics Technology Stack (Glue, S3, Lambd...

We are searching for a proficient Python Developer to become a valuable member of our product development team. Your primary focus will involve automating data ingestion, processing, and validation workflows utilizing PySpark to create robust, scalable, and efficient data pipelines. In this role, you will collaborate closely with data engineers, analysts, and stakeholders to provide impactful data solutions within a dynamic work environment. Your responsibilities will include collaborating with data engineers and stakeholders to outline requirements and implement automated solutions. You will design, develop, and upkeep scalable and efficient PySpark-based automation for data ingestion, proc...

The ideal candidate will be responsible for designing and implementing streaming data pipelines that integrate Kafka with Databricks using Structured Streaming. You will also be tasked with architecting and maintaining the Medallion Architecture, which consists of well-defined Bronze, Silver, and Gold layers. Additionally, you will need to implement efficient data ingestion processes using Databricks Autoloader for high-throughput data loads. You will work with large volumes of structured and unstructured data to ensure high availability and performance, applying performance tuning techniques like partitioning, caching, and cluster resource optimization. Collaboration with cross-functional t...

You have an exciting opportunity to join a dynamic and high-impact firm as a Python, PySpark, and SQL Developer with 8-12 years of relevant experience. In this role, you will be responsible for working on development activities, collaborating with cross-functional teams, and designing scalable data pipelines using Python and PySpark. You will also be involved in implementing ETL processes, developing Power BI reports and dashboards, and optimizing data pipelines for performance and reliability. The ideal candidate should have 8+ years of experience in Spark, Scala, and PySpark for big data processing. Proficiency in Python programming for data manipulation and analysis is essential, along wi...

NTT DATA is looking for an Azure Databricks Developer to join their team in Bengaluru, Karnataka, India. As an Azure Databricks Developer, you will be responsible for advanced experience with Databricks, including Databricks SQL, DataFrames, and Spark using PySpark or Scala. You should have a deep understanding of Spark architecture and optimization strategies, such as tuning Spark SQL configurations, managing data partitions, handling data skew, and leveraging broadcast joins. Your role will also involve proficiency in building and optimizing large-scale data pipelines for ETL/ELT processes using Databricks and familiarity with Delta Lake and data lake architectures. Strong programming skil...

You are a PySpark Developer with over 7 years of experience and expertise in Reltio MDM. As a part of the Data Engineering team, your primary responsibility will be to design, build, and optimize scalable data pipelines, ensuring seamless integration with Reltio's cloud-native MDM platform. This is an immediate requirement, and we are looking for candidates who can join us promptly. Your key responsibilities will include designing, developing, and maintaining scalable data pipelines using PySpark in distributed computing environments like AWS EMR and Databricks. You will also be responsible for integrating and synchronizing data between enterprise systems and the Reltio MDM platform, impleme...

About the Company At Tide, we are dedicated to creating a business management platform that aims to streamline operations for small businesses, enabling them to save valuable time and resources. Our services include offering business accounts, banking solutions, as well as a range of integrated administrative tools spanning from invoicing to accounting. Established in 2017, Tide has garnered a user base of over 1 million small businesses globally, catering to SMEs in the UK, India, and Germany. Headquartered in central London, we also have offices in Sofia, Hyderabad, Delhi, Berlin, and Belgrade, with a team of more than 2,000 employees. Tide is on a trajectory of rapid growth, continuously ...

Join us as a Data Engineer responsible for supporting the successful delivery of Location Strategy projects to plan, budget, agreed quality and governance standards. You'll spearhead the evolution of our digital landscape, driving innovation and excellence. You will harness cutting-edge technology to revolutionise our digital offerings, ensuring unparalleled customer experiences. To be successful as a Data Engineer, you should have experience with hands-on experience in pyspark and strong knowledge of Dataframes, RDD, and SparkSQL. Additionally, experience in developing, testing, and maintaining applications on AWS Cloud is crucial. A strong hold on the AWS Data Analytics Technology Stack (G...

Wissen Technology is hiring for Python+ Pyspark About Wissen Technology: At Wissen Technology, we deliver niche, custom-built products that solve complex business challenges across industries worldwide. Founded in 2015, our core philosophy is built around a strong product engineering mindsetensuring every solution is architected and delivered right the first time. Today, Wissen Technology has a global footprint with 2000+ employees across offices in the US, UK, UAE, India, and Australia. Our commitment to excellence translates into delivering 2X impact compared to traditional service providers. How do we achieve this Through a combination of deep domain knowledge, cutting-edge technology exp...

Requirements: Develop,maintain, andoptimizescalable data processing applications using Python andPySpark. Design and implement data solutions that meet performance and reliability requirements. Collaborate with data engineers, data scientists, and other stakeholders to gather requirements and deliver high-quality solutions. Write clean, efficient, and maintainable code following best practices and coding standards. Perform data analysis and ensure data quality and integrity. Monitorand troubleshoot performance issues in the data processing pipelines. Implement andmaintainCI/CD pipelines for automated testing and deployment. Stayup-to-datewith the latest industry trends and technologies in Py...

The FDR/CVR Data Analyst will play a critical role in ensuring the safety and efficiency of aircraft operations by performing in-depth analysis of flight data. You will work with large datasets from flight recorders to monitor aircraft health and support return-to-service processes. Collaborating with various teams, you will interpret data, develop reports, and ensure compliance with industry standards. Additionally, you will contribute to the development of new data analysis methodologies and tools, provide technical expertise, and maintain thorough documentation. Your key responsibilities will include performing detailed readouts and analysis of FDR and CVR data to support aircraft health ...

Join us as a Data Engineer at Barclays, where you will spearhead the evolution of our infrastructure and deployment pipelines, driving innovation and operational excellence. You will harness cutting-edge technology to build and manage robust, scalable and secure infrastructure, ensuring seamless delivery of our digital solutions. To be successful as a Data Engineer, you should have experience with hands-on experience in Pyspark and a strong knowledge of Dataframes, RDD, and SparkSQL. You should also have hands-on experience in developing, testing, and maintaining applications on AWS Cloud. A strong hold on AWS Data Analytics Technology Stack (Glue, S3, Lambda, Lake formation, Athena) is esse...

The FDR/CVR Data Analyst will play a critical role in ensuring the safety and efficiency of aircraft operations by performing in-depth analysis of flight data. You will work with large datasets from flight recorders to monitor aircraft health and support return-to-service processes. Collaboration with various teams to interpret data, develop reports, and ensure compliance with industry standards is an essential part of your responsibilities. In addition, you will contribute to the development of new data analysis methodologies and tools, provide technical expertise, and maintain thorough documentation. Your key responsibilities will include performing detailed readouts and analysis of FDR an...

Ready to build the future with AI At Genpact, we don&rsquot just keep up with technology&mdashwe set the pace. AI and digital innovation are redefining industries, and we&rsquore leading the charge. Genpact&rsquos AI Gigafactory, our industry-first accelerator, is an example of how we&rsquore scaling advanced technology solutions to help global enterprises work smarter, grow faster, and transform at scale. From large-scale models to agentic AI, our breakthrough solutions tackle companies most complex challenges. If you thrive in a fast-moving, innovation-driven environment, love building and deploying cutting-edge AI solutions, and want to push the boundaries of what&rsquos possible, this is...

Educational Requirements MCA,MSc,Bachelor of Engineering,BBA,BCom,BSc Service Line Data & Analytics Unit Responsibilities Spark Expertise Expert proficiency in Spark Ability to design and implement efficient data processing workflows Experience with Spark SQL and DataFrames Good exposure to Big Data architectures and good understanding of Big Data eco system Experience with some framework building experience on Hadoop Good with DB knowledge with SQL tuning experience. Good to have experience with Python, APIs and exposure to Kafka. Additional Responsibilities: Ability to work with clients to identify business challenges and contribute to client deliverables by refining, analyzing, and struct...