1597 Spark Jobs - Page 4

Employment Type: Contract Overview We are seeking a Palantir Architect to design and lead large-scale data platforms using Palantir Foundry/Gotham, enabling advanced analytics and enterprise-wide data transformation. Primary Skills Palantir | Foundry | Gotham | DataArchitecture | DataPipelines | Python | SQL | Spark | Cloud | Security | Governance Key Responsibilities Architect end-to-end solutions on Palantir platforms Design scalable data ingestion, transformation, and analytics pipelines Establish governance, security, and access controls Work with stakeholders to translate business needs into technical solutions Guide engineering teams and ensure architectural best practices How to Apply...

Required Skills & Experience 5–7 years of experience in data engineering, with at least 2–3 years of Databricks hands-on experience. Strong expertise in Apache Spark (PySpark/Scala/SQL) and distributed data processing. Solid experience with Delta Lake, Lakehouse architecture, and data modeling. Hands-on experience with at least one cloud platform: Azure Data Lake, AWS S3, or GCP BigQuery/Storage. Strong proficiency in SQL for data manipulation and performance tuning. Experience with ETL frameworks, workflow orchestration tools (Airflow, ADF, DBX Workflows). Good understanding of CI/CD, Git-based workflows, and DevOps practices. Exposure to MLOps and MLflow is a strong plus. Knowledge of data...

5 - 9 years of professional experience as a Data Analyst with good decision-making, analytical and problem-solving skills. Working knowledge / experience of Big Data frameworks like Hadoop, Hive and Spark. Hands-on experience in query languages like HQL or SQL (Spark SQL) for Data exploration. Data mapping: Determine the data mapping required to join multiple data sets together across multiple sources. Documentation - Data Mapping, Subsystem Design, Technical Design, Business Requirements. Exposure to Logical to Physical Mapping, Data Processing Flow to measure the consistency, etc. Data Asset design / build: Working with the data model / asset generation team to identify critical data eleme...

Your role As part of the team, you will contribute to creating one of UBS’s most ambitious and critical technology projects. You will play an important role to understand the data platform needs from other crews, work with architects and peers to design, build and deploy robust data engineering solution on Azure cloud data platform. We are looking for a hands-on Data Engineer in the Group Compliance & Regulatory Governance (GCRG) Technology team to: engineer reliable data pipelines for sourcing, processing, distributing, and storing data in different ways, using Azure cloud data platform infrastructure effectively. transform data into valuable insights that inform business decisions, making ...

Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Databricks Unified Data Analytics Platform Good to have skills : NA Minimum 3 Year(s) Of Experience Is Required Educational Qualification : 15 years full time education Summary: As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quali...

Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Databricks Unified Data Analytics Platform Good to have skills : NA Minimum 3 Year(s) Of Experience Is Required Educational Qualification : 15 years full time education Summary: As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quali...

Position Summary... What you'll do... About Team: Pulse is Walmart’s user telemetry system, designed to facilitate analytics processing of large-scale data across Walmart’s global online properties, including Walmart US, Sam’s Club and other international markets. Pulse simplifies the capture of online customer behaviour, enables secure data sharing with partners, and supports standardized data processing. The platform consists of tools, libraries, and services that help teams onboard data, validate payloads, and access analytics data through persistent storage or live streams. Our mission is to empower teams with reliable, scalable, and compliant telemetry infrastructure to drive business i...

Project Role : Custom Software Engineer Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills : AWS BigData Good to have skills : NA Minimum 3 Year(s) Of Experience Is Required Educational Qualification : 15 years full time education Summary: As a Custom Software Engineer, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. Your typical day will involve collaborating with cross-functional teams to gather requirements, developing innovative solutions, and ensuring the successful implementation of applications that meet client needs. You will als...

Your role Are you keen to help engineering teams materially improve the way they work? You have the opportunity to join the team working on the new DevLens initiative, an ambitious data initiative to help our 40,000 IT employees organization accelerate value delivery to customers, improve organization efficiency and developer experience. Together with your team you will contribute to building and maintaining innovative data products spanning all engineering activities across UBS IT lines of business. We’re looking for a passionate Software Engineer to: Create generic scalable data collectors for a variety of data sources and data types Apply a broad range of software engineering practices, f...

Your role Do you want to design and build attractive digital products and services? Do you want to play a key role in transforming our firm into an agile organization? At UBS, we re-imagine the way we work, the way we connect with each other – our colleagues, clients and partners – and the way we deliver value. Being agile will make us more responsive, more adaptable, and ultimately more innovative. We’re looking for a Software Engineer who can design, build, and maintain scalable and secure data pipelines on Azure (preferred) or other cloud platforms. develop and optimize data ingestion, transformation, and storage solutions using Python, Java, and Apache Kafka. work with structured, semi-s...

Your role Do you have a curious mind, want to be involved in the latest technology trends and like to solve problems that have a meaningful benefit to hundreds of users across the bank? Join our Tech Services- Group Chief Technology Office team and become a core contributor for the execution of the banks global AI Strategy, particularly to help the bank deploy AI models quickly and efficiently! We are looking for an experienced Data Engineer or ML Engineer to drive the delivery of an innovative ecosystem of tools and services. In this AI focused role, you will contribute to the development of a state-of-the-art Model Development, Deployment and Governance environment for classic, generative ...

At EY, we’re all in to shape your future with confidence. We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go. Join EY and help to build a better working world. EY GDS – Data and Analytics (D&A) – GCP As part of our EY-GDS D&A (Data and Analytics) team, we help our clients solve complex business challenges with the help of data and technology. We dive deep into data to extract the greatest value and discover opportunities in key business and functions like Banking, Insurance, Manufacturing, Healthcare, Retail, Manufacturing and Auto, Supply Chain, and Finance. The opportunity We’re looking for candidates with strong techn...

Job Title: Senior Data Solution Engineer Experience 8+ Years Educational Qualification B.Tech / BE / M.Tech / MCA / M.Sc. (Computer Science, IT or related) Job Summary We are looking for an experienced Senior Data Solution Engineer to join our Data Solutions and Data Engineering team. The ideal candidate will be responsible for designing, building, and maintaining scalable data pipelines and data platforms using Azure data services. This role involves working closely with business and technical stakeholders to deliver high-quality data solutions that enable analytics and insights across the organization. Must-Have Skills Strong Data Warehousing concepts (Facts, Dimensions, Star/Snowflake sch...

Role Description Job Responsibilities Design Test suites, QA, develop automations, and maintain automation scripts Java or Scala, and Python. Test RESTful and SOAP-based APIs. Test ETL solutions based on Spark, Glue, EMR. Collaborate with cross-functional teams to troubleshoot issues and optimize application behavior. Independently analyze and resolve technical challenges with minimal supervision. Contribute to a culture of collaboration, innovation, and continuous improvement. Work with Agile methodologies and DevOps practices. Required Skills Strong hands-on experience in test automation usingjava or python and SQL, contribution to test automation frameworks. Experience on ETL Manual and a...

At EY, we’re all in to shape your future with confidence. We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go. Join EY and help to build a better working world. EY GDS – Data and Analytics (D&A) – GCP As part of our EY-GDS D&A (Data and Analytics) team, we help our clients solve complex business challenges with the help of data and technology. We dive deep into data to extract the greatest value and discover opportunities in key business and functions like Banking, Insurance, Manufacturing, Healthcare, Retail, Manufacturing and Auto, Supply Chain, and Finance. The opportunity We’re looking for candidates with strong techn...

At EY, we’re all in to shape your future with confidence. We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go. Join EY and help to build a better working world. EY GDS – Data and Analytics (D&A) – GCP As part of our EY-GDS D&A (Data and Analytics) team, we help our clients solve complex business challenges with the help of data and technology. We dive deep into data to extract the greatest value and discover opportunities in key business and functions like Banking, Insurance, Manufacturing, Healthcare, Retail, Manufacturing and Auto, Supply Chain, and Finance. The opportunity We’re looking for candidates with strong techn...

ZS is a place where passion changes lives. As a management consulting and technology firm focused on improving life and how we live it, we transform ideas into impact by bringing together data, science, technology and human ingenuity to deliver better outcomes for all. Here you’ll work side-by-side with a powerful collective of thinkers and experts shaping life-changing solutions for patients, caregivers and consumers, worldwide. ZSers drive impact by bringing a client-first mentality to each and every engagement. We partner collaboratively with our clients to develop custom solutions and technology products that create value and deliver company results across critical areas of their busines...

Introduction A career in IBM Consulting is built on long-term client relationships and close collaboration worldwide. You’ll work with leading companies across industries, helping them shape their hybrid cloud and AI journeys. With support from our strategic partners, robust IBM technology, and Red Hat, you’ll have the tools to drive meaningful change and accelerate client impact. At IBM Consulting, curiosity fuels success. You’ll be encouraged to challenge the norm, explore new ideas, and create innovative solutions that deliver real results. Our culture of growth and empathy focuses on your long-term career development while valuing your unique skills and experiences. Your Role And Respons...

Role : Azure Data Engineer Experience : 7 to 10 years Location : Hyderabad Work Mode : Hybrid Interview mode : F2F Job type : C2H Or Contract Required Skills: Core Azure Services: Azure Data Factory (ADF) – pipeline orchestration, triggers, linked services, integration runtime. Azure Databricks – PySpark, Delta Lake, notebooks, job orchestration. Azure Synapse Analytics – dedicated and serverless SQL pools, data modeling, query optimization. Azure Data Lake Storage (ADLS Gen2) – hierarchical namespace, lifecycle management. Azure Functions / Logic Apps – event-driven data workflows. Azure Event Hubs / Kafka / Stream Analytics – real-time data ingestion. Programming & Tools: Strong in Python ...

About Client: Our Client is a global IT services company headquartered in Southborough, Massachusetts, USA. Founded in 1996, with a revenue of $1.8B, with 35,000+ associates worldwide, specializes in digital engineering, and IT services company helping clients modernize their technology infrastructure, adopt cloud and AI solutions, and accelerate innovation. It partners with major firms in banking, healthcare, telecom, and media. Our Client is known for combining deep industry expertise with agile development practices, enabling scalable and cost-effective digital transformation. The company operates in over 50 locations across more than 25 countries, has delivery centers in Asia, Europe, an...

Work Schedule Standard (Mon-Fri) Environmental Conditions Office As part of the Thermo Fisher Scientific team, you’ll discover meaningful work that makes a positive impact on a global scale. Join our colleagues in bringing our Mission to life every single day to enable our customers to make the world healthier, cleaner and safer. We provide our global teams with the resources needed to achieve individual career goals while helping to take science a step beyond by developing solutions for some of the world’s toughest challenges, like protecting the environment, making sure our food is safe or helping find cures for cancer. Job Title: Data Engineer – Pharmaceutical CDMO Location: [Insert Locat...

Job Title: AWS Data EngineerLocation: Hyderabad Experience: 5+ Years Employment Type: Permanent Notice Period: Immediate Joiners / About the CompanyOur client is a global digital transformation leader with a presence in over 50 countries. Specializing in Oracle Cloud, automation, data intelligence, and enterprise consulting, they help businesses modernize their systems and accelerate innovation at scale.Job Summary:We are looking for a highly skilled Senior AWS Data Engineer with 5+ years of experience in designing, building, and maintaining scalable data pipelines on AWS. The ideal candidate will have strong expertise in AWS data services, big data processing, and modern data engineering be...

About Company: Our client is a global technology consulting and digital solutions company that enables enterprises to reimagine business models and accelerate innovation through digital technologies. Powered by more than 84,000 entrepreneurial professionals across more than 30 countries, it caters to over 700 clients with its extensive domain and technology expertise to help drive superior competitive differentiation, customer experiences, and business outcomes. Job Title: Data Engineer Location: (Pan India)-Bangalore (Global village Tech Park) / Hyderabad (Rai Durg) / Mumbai (Powai / Mahape) / Chennai (DLF IT Park) / Pune (Shivajinagar) / Noida (Candor Techspace, Industrial Area) / Gurgaon ...

About The Team Meesho is on a mission to enable 100 million businesses to succeed online. With your help, we can go beyond! We’re making e-commerce accessible for tier-2+ markets who are new to e-commerce and have almost never transacted online before🤩 Our Design team is as diverse as our users. Today, the team comprises of 30 Designers – including Product and Visual Designers🙌 As a Visual Designer, you’ll be part of a team solving a unique problem, one that’s mostly untouched by other e-commerce players. But wait, there’s more! We’re individuals who are obsessed with our users. We empathize. We solve at grassroots levels. We’re breaking all the barriers -- be it language, literacy, culture,...

Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together. As a Director of Technology - AI Transformation, you will be both a strategic leader and a hands-on engineer, guidi...