35104 Spark Jobs - Page 12

Job Summary: Data Engineer will be responsible to Develop set processes for data mining, data modeling, and data production. Develop and maintain scalable data pipelines and builds out new integrations to support increases in data volume and complexity. Employ an array of technological languages and tools to connect data sets. Collaborate with analytics and business teams to improve data models that feed business intelligence tools, increasing data accessibility and fostering data-driven decision making across the organization. Role & responsibilities Data Engineer, Key Deliverables Work in collaboration of IT infrastructure, Business teams and team members to create/develop and maintains op...

Job Description We are seeking a passionate and experienced Full Stack AI/ML Engineer with a strong background in machine learning and a drive for building intelligent systems. As a Full-Stack AI/ML Engineer on the Ford Pro Charging team, you will design, build, and ship intelligent services that power our global EV-charging platform. If you love turning data into real-world impact and thrive on end-to-end ownership—from research notebooks to production APIs—this is your playground. Responsibilities Design & Develop AI Solutions: Lead the design, development, training, and evaluation of machine learning models and AI solutions across various domains to enhance our products and services. Iden...

The Mission: Build the operating system for the future of AI-powered entertainment. About Us Our journey at Dashverse began with a simple question: what if AI could power the next generation of entertainment? As a Series A company backed by leading investors like Peak XV, Matrix, and Stellaris, we first redefined storytelling with Dashtoon, our revolutionary AI comics creation and distribution platform. Now, with Dashreels, we are creating a new universe of short-form video, pairing addictive short dramas with our own game-changing AI-generated originals. We're on a mission to become the world's go-to destination for compelling, short-form narrative content. About the Role We are looking for...

As a Technical Lead - Big Data Platform at Incedo, you will be responsible for designing and implementing big data platforms to support large-scale data integration projects. You will work with data architects and data engineers to define the platform architecture and build the necessary infrastructure. You will be skilled in big data technologies such as Hadoop, Spark, and Kafka and have experience in cloud computing platforms such as AWS or Azure. You will be responsible for ensuring the performance, scalability, and security of the big data platform and troubleshooting any issues that arise. Roles & Responsibilities: Designing, developing and maintaining large-scale big data platforms usi...

Data Engineer - Quant Trading About Deeter Investments Deeter Investments is a founder‑led proprietary trading firm built around real‑time, data‑driven decision‑making. We prize curiosity, collaboration, and a bias for action. After years of discretionary success, we’re launching a dedicated algorithmic division. Role Description As our first Data Engineer, you’ll own critical datasets end-to-end from ingestion and system architecture to reliability and access. You'll be designing, building, and running the data backbone for our algorithmic team. You’ll work directly with traders and researchers to turn messy external feeds into high-performance, well-structured datasets that guide decisions...

At EY, we’re all in to shape your future with confidence. We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go. Join EY and help to build a better working world. EY GDS – Data and Analytics (D&A) – Manager – Azure Data Architect As part of our EY-GDS D&A (Data and Analytics) team, we help our clients solve complex business challenges with the help of data and technology. We dive deep into data to extract the greatest value and discover opportunities in key business and functions like Banking, Insurance, Manufacturing, Healthcare, Retail, Manufacturing and Auto, Supply Chain, and Finance. The opportunity We’re looking for M...

At EY, we’re all in to shape your future with confidence. We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go. Join EY and help to build a better working world. EY GDS – Data and Analytics (D&A) – Manager – Azure Data Architect As part of our EY-GDS D&A (Data and Analytics) team, we help our clients solve complex business challenges with the help of data and technology. We dive deep into data to extract the greatest value and discover opportunities in key business and functions like Banking, Insurance, Manufacturing, Healthcare, Retail, Manufacturing and Auto, Supply Chain, and Finance. The opportunity We’re looking for M...

We are currently looking for an experienced Lead Data Engineer with demonstrated technical experience within pySpark, Phyton, and Java technologies to join the team in India, within our global Technology department. If this sounds interesting to you and you are skilled in Query engines Postgres/SQL/AWS/Java/Kotlin, we would love to speak with you. About You experience, education, skills, and accomplishments: Bachelor/master's degree in computer science, computer engineering, information technology, or relevant field At least 8+ years experience in working in software development At least 8+ years experience in building massively scalable distributed data processing solutions At least 6+ year...

We are currently looking for an experienced Lead Data Engineer with demonstrated technical experience within pySpark, Phyton, and Java technologies to join the team in India, within our global Technology department. If this sounds interesting to you and you are skilled in Query engines Postgres/SQL/AWS/Java/Kotlin, we would love to speak with you. About You experience, education, skills, and accomplishments: At least 6+ years experience in working in software development At least 6+ years experience in building massively scalable distributed data processing solutions At least 6+ years experience of database design & development. Alt least 4 years of experience in Apache Spark, ElasticSearch ...

Are you passionate about building robust Python applications and leveraging big data technologies to solve complex business challenges? Do you thrive in environments where your technical expertise drives meaningful impact? We are seeking an experienced Python Lead Developer to join our dynamic LSH Data Platform Team. You will play a pivotal role in designing, developing, and maintaining Python-based data processing applications and services that support scalable big data solutions. If you're excited about combining deep Python skills with cutting-edge technologies like Apache Spark, PySpark , GraphDB , SQL, and relational databases, we want to hear from you. About You experience, education, ...

We are looking for a data engineer with 2-5 years of expertise. Design and develop the database schemas as well as scripts for importing and exporting data utilizing AWS Glue tasks. In an agile environment, the data engineer will work closely with the development team. About You experience, education, skills, and accomplishments Minimum of 2+ years industry experience in data engineering. Bachelors or masters degree in computer science (or similar) Extensive experience in Pyspark/SQL, postgres Experience with AWS services like Glue, s3, Lambda Experience with data modelling and data warehouse. Experience of working within an Agile delivery environment. Good knowledge and understanding of Ver...

We are looking for a Manager Software Engineering to join our IMS Team in Bangalore . This is an amazing opportunity to work on Big Data technologies involved in content ingestion. The team consists of 10-12 engineers and is reporting to the Sr Manager. We have a great skill set in Spark, Java, Scala, Hive, Sql, XSLT, AWS EMR, S3, etc and we would love to speak with you if you have skills in the same. About You experience, education, skills, and accomplishments: Work Experience: Minimum 7 years experience in Big Data projects involved in content ingestion, curation, transformation Technical Skill: Databricks, Spark, Python/Java , Scala, AWS EMR, S3, SQS, Hive, XSLT, Education (bachelors degr...

Established in 2004, OLIVER is the world’s first and only specialist in designing, building, and running bespoke in-house agencies and marketing ecosystems for brands. We partner with over 300 clients in 40+ countries and counting. Our unique model drives creativity and efficiency, allowing us to deliver tailored solutions that resonate deeply with audiences. As a part of The Brandtech Group , we're at the forefront of leveraging cutting-edge AI technology to revolutionise how we create and deliver work. Our AI solutions enhance efficiency, spark creativity, and drive insightful decision-making, empowering our teams to produce innovative and impactful results. Job Title: Copy Supervisor Loca...

Years of experience - 4 to 10 years Job Description: We are seeking a highly skilled and motivated Lead/Senior Data Engineer to join our Data Integration and Analytics team. The ideal candidate will bring deep expertise in AWS Redshift, with strong knowledge of Redshift architecture, query performance tuning, and workload optimization. You will be responsible for leading the design, development, and optimization of scalable ETL/ELT pipelines that enable efficient data integration and power critical business intelligence and analytics initiatives. This role requires a proven ability to troubleshoot complex data challenges, drive performance improvements, and ensure our data ecosystem is both ...

Job Title: GCP Data Engineer Location: Pune, India Experience: 4 to 7 Years Job Type: Full-Time Job Summary: We are looking for a highly skilled GCP Data Engineer with 4 to 7 years of experience to join our data engineering team in Pune . The ideal candidate should have strong experience working with Google Cloud Platform (GCP) , including Dataproc , Cloud Composer (Apache Airflow) , and must be proficient in Python , SQL , and Apache Spark . The role involves designing, building, and optimizing data pipelines and workflows to support enterprise-grade analytics and data science initiatives. Key Responsibilities: Design and implement scalable and efficient data pipelines on GCP, leveraging Da...

SPARK EDM OPERATOR TO WORK ON OSCAR MAX ZNC MACHINE WORK PIECE DIALLING, ELECTRODE DIALLING, CENTRALIZING, PROGRAMMING AND INSPECTION SHOULD BE ABLE TO READ AND INTERPRET 2D DRAWINGS Job Type: Full-time Pay: ₹28,000.00 - ₹30,000.00 per month Benefits: Health insurance Ability to commute/relocate: Kamakshipalya, Bengaluru - 560079, Karnataka: Reliably commute or planning to relocate before starting work (Preferred) Education: Secondary(10th Pass) (Preferred) Experience: total work: 1 year (Preferred) License/Certification: ITI (Preferred) Work Location: In person

Line of Service Advisory Industry/Sector Not Applicable Specialism Data, Analytics & AI Management Level Senior Manager Job Description & Summary At PwC, our people in data and analytics focus on leveraging data to drive insights and make informed business decisions. They utilise advanced analytics techniques to help clients optimise their operations and achieve their strategic goals. In data analysis at PwC, you will focus on utilising advanced analytical techniques to extract insights from large datasets and drive data-driven decision-making. You will leverage skills in data manipulation, visualisation, and statistical modelling to support clients in solving complex business problems. Why ...

1 Designing and implementing data ingestion pipelines from multiple sources using Azure Databricks 2 Ensure data pipelines run smoothly and efficiently 3 Developing scalable and re-usable frameworks for ingesting of data sets 4 Integrating the end-to-end data pipeline - to take data from source systems to target data repositories ensuring the quality and consistency of data is maintained at all times 5 Working with event based / streaming technologies to ingest and process data 6 Working with other members of the project team to support delivery of additional project components (API interfaces, Search) 7 Evaluating the performance and applicability of multiple tools against customer requirem...

Who We Are Boston Consulting Group partners with leaders in business and society to tackle their most important challenges and capture their greatest opportunities. BCG was the pioneer in business strategy when it was founded in 1963. Today, we help clients with total transformation-inspiring complex change, enabling organizations to grow, building competitive advantage, and driving bottom-line impact. To succeed, organizations must blend digital and human capabilities. Our diverse, global teams bring deep industry and functional expertise and a range of perspectives to spark change. BCG delivers solutions through leading-edge management consulting along with technology and design, corporate...

Responsibilities An Azure Data Engineer is responsible for designing, building, and maintaining the data infrastructure for an organization using Azure cloud services. This includes creating data pipelines, integrating data from various sources, and implementing data security and privacy measures. The Azure Data Engineer will also be responsible for monitoring and troubleshooting data flows and optimizing data storage and processing for performance and cost efficiency. Mandatory Skills Bachelor's or higher degree in Computer Science, Data Engineering, or a related discipline, or equivalent (minimum 10+ years overall work experience). Expertise in designing and implementing high-performance d...

Your responsibilities as a Media Performance Associate Manager (Programmatic)–  Manage multichannel programmatic campaigns (desktop, mobile, and video) while troubleshooting campaign delivery and performance issues  Conduct full analysis of campaign performance, optimizations, issues, etc.  Work with internal teams to provide clients with programmatic media recommendations and optimization strategies that align with objectives  Traffic all creatives into platform and set up campaigns prior to launch date  Maintain knowledge of media technology buying platforms and analytic tools  Navigate through variety of third-party systems to complete monthly billing documentation  Analyze campaig...

Job Title: Data Analyst (Python +Pyspark) About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 by Avtar & Seramount . With our presence across 32 cities across globe, we support 100+ clients across banking, financial and Energy sectors. We are recognized for our deep transformation execution and delivery. WHY JOIN CAPCO? You will work on engaging projects with the largest international and local banks, insurance companies, payment service providers and other key players in the industry. The projects that will transform the finan...

Job Description 5+ years of overall IT experience, which includes hands on experience in Big Data technologies. Mandatory - Hands on experience in Python and PySpark. Build pySpark applications using Spark Dataframes in Python. Worked on optimizing spark jobs that processes huge volumes of data. Hands on experience in version control tools like Git. Worked on Amazon’s Analytics services like Amazon EMR, Amazon Athena, AWS Glue. Worked on Amazon’s Compute services like Amazon Lambda, Amazon EC2 and Amazon’s Storage service like S3 and few other services like SNS. Good to have knowledge of datawarehousing concepts – dimensions, facts, schemas- snowflake, star etc. Have worked with columnar sto...

Key Responsibilities: Design, develop, and manage scalable, secure, and efficient data pipelines on Google Cloud Platform (GCP) to process and transform large datasets from multiple sources. Implement and optimize GCP data services such as BigQuery, Cloud Storage, Cloud SQL, Dataflow, Dataproc, Pub/Sub, and Cloud Functions. Architect and maintain ETL/ELT processes to ensure efficient data ingestion, transformation, and storage using Cloud Data Fusion, Dataflow (Apache Beam), or Composer (Airflow). Architect and implement data models on GCP to support efficient data storage and retrieval in BigQuery, Cloud Spanner, or Firestore. Collaborate with data architects to design and implement data la...

Key Responsibilities: Data Architecture & Management: Design and implement scalable, cloud-agnostic Data Lake, Data LakeHouse, Data Mesh and Data Fabric architectures to efficiently store, process, and manage structured and unstructured data from various sources. Data Pipeline Development : Design, develop, and maintain robust data pipelines to ingest, process, and transform data from multiple sources into usable formats for analytics and reporting using services like AWS Glue, Azure Data Factory, GCP Dataflow, Apache Spark, or Apache Airflow. Data Integration and ETL: Develop and optimize Extract, Transform, Load (ETL) and ELT processes to integrate disparate data sources into the data lake...