35346 Spark Jobs - Page 22

1 Designing and implementing data ingestion pipelines from multiple sources using Azure Databricks 2 Ensure data pipelines run smoothly and efficiently 3 Developing scalable and re-usable frameworks for ingesting of data sets 4 Integrating the end-to-end data pipeline - to take data from source systems to target data repositories ensuring the quality and consistency of data is maintained at all times 5 Working with event based / streaming technologies to ingest and process data 6 Working with other members of the project team to support delivery of additional project components (API interfaces, Search) 7 Evaluating the performance and applicability of multiple tools against customer requirem...

Who We Are Boston Consulting Group partners with leaders in business and society to tackle their most important challenges and capture their greatest opportunities. BCG was the pioneer in business strategy when it was founded in 1963. Today, we help clients with total transformation-inspiring complex change, enabling organizations to grow, building competitive advantage, and driving bottom-line impact. To succeed, organizations must blend digital and human capabilities. Our diverse, global teams bring deep industry and functional expertise and a range of perspectives to spark change. BCG delivers solutions through leading-edge management consulting along with technology and design, corporate...

Responsibilities An Azure Data Engineer is responsible for designing, building, and maintaining the data infrastructure for an organization using Azure cloud services. This includes creating data pipelines, integrating data from various sources, and implementing data security and privacy measures. The Azure Data Engineer will also be responsible for monitoring and troubleshooting data flows and optimizing data storage and processing for performance and cost efficiency. Mandatory Skills Bachelor's or higher degree in Computer Science, Data Engineering, or a related discipline, or equivalent (minimum 10+ years overall work experience). Expertise in designing and implementing high-performance d...

Your responsibilities as a Media Performance Associate Manager (Programmatic)–  Manage multichannel programmatic campaigns (desktop, mobile, and video) while troubleshooting campaign delivery and performance issues  Conduct full analysis of campaign performance, optimizations, issues, etc.  Work with internal teams to provide clients with programmatic media recommendations and optimization strategies that align with objectives  Traffic all creatives into platform and set up campaigns prior to launch date  Maintain knowledge of media technology buying platforms and analytic tools  Navigate through variety of third-party systems to complete monthly billing documentation  Analyze campaig...

Job Title: Data Analyst (Python +Pyspark) About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 by Avtar & Seramount . With our presence across 32 cities across globe, we support 100+ clients across banking, financial and Energy sectors. We are recognized for our deep transformation execution and delivery. WHY JOIN CAPCO? You will work on engaging projects with the largest international and local banks, insurance companies, payment service providers and other key players in the industry. The projects that will transform the finan...

Job Description 5+ years of overall IT experience, which includes hands on experience in Big Data technologies. Mandatory - Hands on experience in Python and PySpark. Build pySpark applications using Spark Dataframes in Python. Worked on optimizing spark jobs that processes huge volumes of data. Hands on experience in version control tools like Git. Worked on Amazon’s Analytics services like Amazon EMR, Amazon Athena, AWS Glue. Worked on Amazon’s Compute services like Amazon Lambda, Amazon EC2 and Amazon’s Storage service like S3 and few other services like SNS. Good to have knowledge of datawarehousing concepts – dimensions, facts, schemas- snowflake, star etc. Have worked with columnar sto...

Key Responsibilities: Design, develop, and manage scalable, secure, and efficient data pipelines on Google Cloud Platform (GCP) to process and transform large datasets from multiple sources. Implement and optimize GCP data services such as BigQuery, Cloud Storage, Cloud SQL, Dataflow, Dataproc, Pub/Sub, and Cloud Functions. Architect and maintain ETL/ELT processes to ensure efficient data ingestion, transformation, and storage using Cloud Data Fusion, Dataflow (Apache Beam), or Composer (Airflow). Architect and implement data models on GCP to support efficient data storage and retrieval in BigQuery, Cloud Spanner, or Firestore. Collaborate with data architects to design and implement data la...

Key Responsibilities: Data Architecture & Management: Design and implement scalable, cloud-agnostic Data Lake, Data LakeHouse, Data Mesh and Data Fabric architectures to efficiently store, process, and manage structured and unstructured data from various sources. Data Pipeline Development : Design, develop, and maintain robust data pipelines to ingest, process, and transform data from multiple sources into usable formats for analytics and reporting using services like AWS Glue, Azure Data Factory, GCP Dataflow, Apache Spark, or Apache Airflow. Data Integration and ETL: Develop and optimize Extract, Transform, Load (ETL) and ELT processes to integrate disparate data sources into the data lake...

Greetings from Getronics! We have permanent opportunities for GCP Data Engineers in Chennai Location . Hope you are doing well! This is Abirami from Getronics Talent Acquisition team. We have multiple opportunities for GCP Data Engineers for our automotive client in Chennai Sholinganallur location. Please find below the company profile and Job Description. If interested, please share your updated resume, recent professional photograph and Aadhaar proof at the earliest to abirami.rsk@getronics.com. Company : Getronics (Permanent role) Client : Automobile Industry Experience Required : 5+ Years in IT and minimum 3+ years in GCP Data Engineering Location : Chennai (Elcot - Sholinganallur) Work ...

Key Responsibilities: Develop and maintain scalable full-stack applications using Java, Spring Boot, and Angular for building rich UI screens and custom/reusable components. Design and implement cloud-based solutions leveraging Google Cloud Platform (GCP) services such as BigQuery, Google Cloud Storage, Cloud Run, and PubSub. Manage and optimize CI/CD pipelines using Tekton to ensure smooth and efficient development workflows. Deploy and manage Google Cloud services using Terraform, ensuring infrastructure as code principles. Mentor and guide junior software engineers, fostering professional development and promoting systemic change across the development team. Collaborate with cross-functio...

Requirements • Proficient in Java or similar object-oriented programming languages • Familiarity with developing and deploying enterprise applications in cloud platforms (e.g., AWS, GCP, Azure) • Understanding of distributed systems or microservices architecture • Problem-solving skills with the ability to debug and troubleshoot distributed systems issues in a cloud environment • 5+ years of experience in full time agile software development roles • Experience with owning features end to end from design and development through deployment and production support • Strong communication skills and a collaborative mindset • Enthusiasm for learning new technologies and solving complex problems • O...

Job Description Responsibilities: Lead and mentor a team of data engineers, providing technical guidance and fostering a culture of innovation and continuous improvement. Collaborate closely with clients, data scientists, software engineers, and analysts to gather requirements and architect scalable, robust data solutions. Drive solution design and technical delivery end-to-end, managing project timelines, risks, and quality to ensure successful outcomes. Serve as the primary technical point of contact for clients, managing expectations, providing regular updates, and addressing technical concerns promptly. Lead presales and solutioning activities, including responding to RFPs, preparing tec...

DESCRIPTION AWS Sales, Marketing, and Global Services (SMGS) is responsible for driving revenue, adoption, and growth from the largest and fastest growing small- and mid-market accounts to enterprise-level customers including public sector. AWS Global Services includes experts from across AWS who help our customers design, build, operate, and secure their cloud environments. Customers innovate with AWS Professional Services, upskill with AWS Training and Certification, optimize with AWS Support and Managed Services, and meet objectives with AWS Security Assurance Services. Our expertise and emerging technologies include AWS Partners, AWS Sovereign Cloud, AWS International Product, and the Ge...

Job Summary: Are you an outcome-oriented problem solver? Do you enjoy working on transformation strategies for global clients? Have you been part of large Business transformation programs for global clients? Does working in an inclusive and collaborative environment spark your interest? Then, Accenture Strategy and Consulting is the right place for you to explore limitless possibilities. As a part of our Operations and Process Transformation Center of Excellence within Accenture Strategy and Consulting, you will help organizations reimagine and transform their processes for tomorrow—with a positive impact on the business, society and the planet. While we are housed within Supply Chain and Op...

Senior Data Engineer Hyderabad, Telangana, India Date posted Oct 09, 2025 Job number 1890854 Work site Fully on-site Travel 0-25 % Role type Individual Contributor Profession Software Engineering Discipline Data Engineering Employment type Full-Time Overview Are you passionate about shaping the future applications of AI and empowering millions of users to unlock their full potential? The OneNote team is at the forefront of an exciting transformation with Copilot Notebooks: intelligent, dynamic notebooks infused with powerful AI that act as a true "second brain." Imagine effortlessly capturing ideas, intuitively understanding complex information, and seamlessly taking informed action. This is...

About Gap Inc. Our past is full of iconic moments — but our future is going to spark many more. Our brands — Gap, Banana Republic, Old Navy and Athleta — have dressed people from all walks of life and all kinds of families, all over the world, for every occasion for more than 50 years. But we’re more than the clothes that we make. We know that business can and should be a force for good, and it’s why we work hard to make product that makes people feel good, inside and out. It’s why we’re committed to giving back to the communities where we live and work. If you're one of the super-talented who thrive on change, aren't afraid to take risks and love to make a difference, come grow with us. Abo...

About Gap Inc. Our past is full of iconic moments — but our future is going to spark many more. Our brands — Gap, Banana Republic, Old Navy and Athleta — have dressed people from all walks of life and all kinds of families, all over the world, for every occasion for more than 50 years. But we’re more than the clothes that we make. We know that business can and should be a force for good, and it’s why we work hard to make product that makes people feel good, inside and out. It’s why we’re committed to giving back to the communities where we live and work. If you're one of the super-talented who thrive on change, aren't afraid to take risks and love to make a difference, come grow with us. Abo...

About Providence Providence, one of the US’s largest not-for-profit healthcare systems, is committed to high quality, compassionate healthcare for all. Driven by the belief that health is a human right and the vision, ‘Health for a better world’, Providence and its 121,000 caregivers strive to provide everyone access to affordable quality care and services. Providence has a network of 51 hospitals, 1,000+ care clinics, senior services, supportive housing, and other health and educational services in the US. Providence India is bringing to fruition the transformational shift of the healthcare ecosystem to Health 2.0. The India center will have focused efforts around healthcare technology and ...

About Gap Inc. Our past is full of iconic moments — but our future is going to spark many more. Our brands — Gap, Banana Republic, Old Navy and Athleta — have dressed people from all walks of life and all kinds of families, all over the world, for every occasion for more than 50 years. But we’re more than the clothes that we make. We know that business can and should be a force for good, and it’s why we work hard to make product that makes people feel good, inside and out. It’s why we’re committed to giving back to the communities where we live and work. If you're one of the super-talented who thrive on change, aren't afraid to take risks and love to make a difference, come grow with us. Abo...

Role Description: This role acts as technical architect and hands-on lead for Data Engineering practices across the Smart Supply Chain initiative within Amgen. Additionally, responsible for designing, building, maintaining, analyzing, and interpreting data to provide actionable insights that drive business decisions. This role involves working with large datasets, developing reports, supporting and executing data governance initiatives and, visualizing data to ensure data is accessible, reliable, and efficiently managed. The ideal candidate has strong technical skills, experience with big data technologies, and a deep understanding of data architecture and ETL processes and will architect, b...

JLL supports the Whole You, personally and professionally. Our people at JLL are shaping the future of real estate for a better world by combining world class services, advisory and technology to our clients. We are committed to hiring the best, most talented people in our industry; and we support them through professional growth, flexibility, and personalized benefits to manage life in and outside of work. Whether you’ve got deep experience in commercial real estate, skilled trades, and technology, or you’re looking to apply your relevant experience to a new industry, we empower you to shape a brighter way forward so you can thrive professionally and personally. Project and Development Serv...

POSITION SUMMARY Zoetis has created a Data and Digital Platform (DDP) team that is driving a step change in the company’s ability to harness data, including data generated by the expanding portfolio of Digital Products, to generate disruptive data insights for our customers and colleagues. As the world leader in animal health, Zoetis seeks to integrate Data and Analytics with its diverse portfolio of animal health products to create additional value to customers and enhance overall customer experience. Our view of analytics spans from the sensor data collection and analysis on the edge, to the delivery of data insights that help optimize and, in some cases, automate the decision-making proce...

JLL supports the Whole You, personally and professionally. Our people at JLL are shaping the future of real estate for a better world by combining world class services, advisory and technology to our clients. We are committed to hiring the best, most talented people in our industry; and we support them through professional growth, flexibility, and personalized benefits to manage life in and outside of work. Whether you’ve got deep experience in commercial real estate, skilled trades, and technology, or you’re looking to apply your relevant experience to a new industry, we empower you to shape a brighter way forward so you can thrive professionally and personally. Facilities Executive Integra...

Data Engineer Data Engineer Location Hyderabad, Telangana IN Department Product & Engineering Job Role ‍ We are seeking a skilled Data Engineer with 5–10 years of experience to join our dynamic Data Engineering team. The ideal candidate will design, build, and optimize large-scale data pipelines and warehouses that enable real-time analytics and business intelligence across Tanla’s CPaaS ecosystem. You’ll work on diverse datasets, including Call Detail Records (CDR), and collaborate closely with cross-functional teams to ensure data reliability, scalability, and performance. ‍ What you'll be responsible for? ‍ Design and implement ETL/ELT pipelines for large-scale data processing and analyti...

JLL supports the Whole You, personally and professionally. Our people at JLL are shaping the future of real estate for a better world by combining world class services, advisory and technology to our clients. We are committed to hiring the best, most talented people in our industry; and we support them through professional growth, flexibility, and personalized benefits to manage life in and outside of work. Whether you’ve got deep experience in commercial real estate, skilled trades, and technology, or you’re looking to apply your relevant experience to a new industry, we empower you to shape a brighter way forward so you can thrive professionally and personally. City Chief Engineer, Work Dy...