1597 Spark Jobs

As an entry level Application Developer at IBM, you''ll work with clients to co-create solutions to major real-world challenges by using best practice technologies, tools, techniques, and products to translate system requirements into the design and development of customized systems. In your role, you may be responsible for: Working across the entire system architectureto design, develop, and support high quality, scalable products and interfaces for our clients Collaborate with cross-functional teams to understand requirements and define technical specifications for generative AI projects Employing IBM''s Design Thinking to create products that provide a great user experience along with hig...

* As a consultant at IBM, you''ll play a vital role in the development, design of application, provide regular support/guidance to project teams on complex coding, issue resolution and execution. Your primary responsibilities include * Lead the design and construction of new solutions using the latest technologies, always looking to add business value and meet user requirements. * Strive for continuous improvements by testing the build solution and working under an agile framework. * Discover and implement the latest technologies trends to maximize and build creative solutions Required education Bachelor''s Degree Preferred education Master''s Degree Required technical and professional exper...

* As a Data Engineer at IBM, you''ll play a vital role in the development, design of application, provide regular support/guidance to project teams on complex coding, issue resolution and execution. * Your primary responsibilities include: * Lead the design and construction of new solutions using the latest technologies, always looking to add business value and meet user requirements. * Strive for continuous improvements by testing the build solution and working under an agile framework. * Discover and implement the latest technologies trends to maximize and build creative solutions Required education Bachelor''s Degree Preferred education Master''s Degree Required technical and professional...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark and Hive, Hbase or other NoSQL databases on Azure Cloud Data Platform or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developing strea...

As a consultant at IBM, you''ll play a vital role in the development, design of application, provide regular support/guidance to project teams on complex coding, issue resolution and execution. Your primary responsibilities include Lead the design and construction of new solutions using the latest technologies, always looking to add business value and meet user requirements. Strive for continuous improvements by testing the build solution and working under an agile framework. Discover and implement the latest technologies trends to maximize and build creative solutions Required education Bachelor''s Degree Preferred education Master''s Degree Required technical and professional expertise 8-1...

Position Summary The candidate will work along with other data scientists and data engineers to support activities related to development and maintenance of products that generate key insights on industry trend by applying various machine learning and statistical techniques on proprietary as well as open data sources. The person will work alongside Product managers, Industry Analysts and Technology experts to develop and deliver the product in accordance with customer requirements and agreed timeline. The candidate should have 10+ years work experience in developing products/services using Data Science and Analytics. Duties and responsibilities Work as part of analytics team based in Bangalo...

Overview We are hiring a Senior Data Engineer with strong expertise in Palantir Foundry to build and maintain high-performance data pipelines that support enterprise analytics and reporting. Primary Skills PalantirFoundry | DataEngineering | Python | SQL | Spark | ETL | DataPipelines | Cloud | Git | CI/CD Key Responsibilities Build and optimize data pipelines using Palantir Foundry. Ensure data quality, reliability, and performance. Collaborate with architects, analysts, and business teams. Troubleshoot production issues and support continuous improvements. Follow best practices for version control, testing, and deployment.

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark and Hive, Hbase or other NoSQL databases on Azure Cloud Data Platform or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developing strea...

As a Big Data Engineer, you will develop, maintain, evaluate, and test big data solutions. You will be involved in data engineering activities like creating pipelines/workflows for Source to Target and implementing solutions that tackle the clients needs. Your primary responsibilities include: Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements. Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization. Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need too. Required education Bachel...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

As a Data Engineer specializing in Google's data platforms, you will design, build, and maintain data engineering solutions on Google's Cloud ecosystem. This role requires expertise in utilizing various Google services for batch and real-time data pipelines, data migration, and data layer design. Your primary responsibilities will include: Design Data Pipelines: Design and develop batch and real-time data pipelines for Data Warehouse and Datalake using Google services such as DataProc, DataFlow, PubSub, BigQuery, and Big Table. Develop Data Engineering Solutions: Utilize Google Cloud Storage, BigTable, BigQuery DataProc with Spark and Hadoop, and Google DataFlow with Apache Beam or Python to...

Primary Skills Azure Data Factory, Azure Synapse Analytics, Azure Databricks : Extensive experience in building and managing scalable data pipelines for data integration, transformation, and analysis. Azure Data Lake Storage, Azure SQL Database : Proven ability to manage and optimize data storage solutions, ensuring data accessibility and reliability. Azure Machine Learning : Expertise in developing, training, and deploying machine learning models using Azure''s machine learning tools and services. Data Warehousing Data Modeling : Strong experience with designing and maintaining data warehouses, with deep understanding of ETL processes and data models. Programming in Python : Skilled in writ...

JD for Data Scientist Hands-on experience with data analysis tools: Proficient in using tools such as Python and R for data manipulation, querying, and analysis. Skilled in utilizing libraries like Pandas, NumPy, and Scikit-Learn to perform in-depth data analysis and modeling. Skilled in machine learning and predictive analytics: Expertise in building, training, and deploying machine learning models using frameworks such as TensorFlow and PyTorch. Capable of performing tasks like regression, classification, clustering, and recommendation, leading to data-driven predictions and insights. Expertise in big data technologies: Proficient in handling large datasets using big data tools such as Spa...

Essential Bachelors degree or equivalent in Computer Engineering Computer Science or a related field 5 years of experience in data software engineering role 3 years of strong hands on experience with Databricks with AWS as a cloud platform which including Spark based processing data pipelines and the platforms tools for analytics and machine learning 2 years of experience with building AWS cloud based data pipelines and AI solutions 2 years of Hands on experience with Python and PySpark The role holder will possess a blend of design skills needed for Agile data development projects Proficiency or demonstratable passion for learning in data engineer techniques and testing methodologies Strong...

Essential Bachelors degree or equivalent in Computer Engineering Computer Science or a related field 5 years of experience in data software engineering role 3 years of strong hands on experience with Databricks with AWS as a cloud platform which including Spark based processing data pipelines and the platforms tools for analytics and machine learning 2 years of experience with building AWS cloud based data pipelines and AI solutions 2 years of Hands on experience with Python and PySpark The role holder will possess a blend of design skills needed for Agile data development projects Proficiency or demonstratable passion for learning in data engineer techniques and testing methodologies Strong...

Job Requirements At Quest Global, it’s not just what we do but how and why we do it that makes us different. With over 25 years as an engineering services provider, we believe in the power of doing things differently to make the impossible possible. Our people are driven by the desire to make the world a better place—to make a positive difference that contributes to a brighter future. We bring together technologies and industries, alongside the contributions of diverse individuals who are empowered by an intentional workplace culture, to solve problems better and faster. Summary We are seeking an innovative and results-driven Data Scientist with a proven track record and portfolio. In this r...

Req ID: 350367 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Databricks Engineer to join our team in Delhi, Delhi (IN-DL), India (IN). Responsibilities Proven, hands-on experience with Azure Data Factory and end-to-end pipeline development. Strong experience with Azure Databricks , building notebooks/jobs, and optimizing workloads. Advanced Python skills for data processing, unit testing, and scripting automation. Strong SQL skills: designing and writing mid-to-complex queries, stored procedures, functions, and opt...

5 - 8 Years 1 Opening Trivandrum Role description We are seeking a Data Engineer with Spark & Streaming skills builds real-time, scalable data pipelines using tools like Spark, Kafka, and cloud services (GCP) to ingest, transform, and deliver data for analytics and ML. Responsibilities: As a Senior Data Engineer, you will Design, develop, and maintain ETL/ELT data pipelines for batch and real-time data ingestion, transformation, and loading using Spark (PySpark/Scala) and streaming technologies (Kafka, Flink). Build and optimize scalable data architectures, including data lakes, data warehouses (BigQuery), and streaming platforms. Performance Tuning: Optimize Spark jobs, SQL queries, and dat...

Job Description: Data Engineer with Palantir Expertise We are seeking a seasoned Data Engineer with 6 years of experience in data engineering, AWS, microservices architecture, and AI/ML frameworks, along with hands-on expertise or certification in Palantir Foundry. The ideal candidate will work on designing scalable data solutions, building robust pipelines, and integrating machine learning models into data workflows. Location - Gurugram Key Requirements 3-6-years of hands-on experience in data engineering with a focus on ETL workflows, data pipelines, and cloud computing. Proven expertise or certification in Palantir Foundry is highly preferred. Strong experience with AWS services for data ...

Position Title Senior Executive Programmer Analyst Location Chennai, India Band A2 Designation Lead Programmer Analyst Overview We are seeking a highly skilled and experienced Lead Programmer Analyst specializing in Microsoft Technologies to join our dynamic team. As a Lead Programmer Analyst, you will play a critical role in shaping the success of our technology projects. Your responsibilities will include architecture design, implementation, and overseeing the development and deployment of Microsoft-based solutions that meet our internal needs. You’ll collaborate closely with architects, business analysts, project managers, developers, testers and other stakeholders to ensure the successfu...

Position Title Manager Location Chennai, India Band C1 Designation Technical Manger (Full Stack) Overview We are seeking a highly skilled and experienced Technical Manager specializing in Microsoft Technologies to join our dynamic team. As a Technical Manager, you will play a critical role in shaping the success of our technology projects. Your responsibilities will include architecture design, implementation, and overseeing the development and deployment of Microsoft-based solutions that meet our internal needs. You’ll collaborate closely with management, business analysts, project managers, developers, testers and other stakeholders to ensure the successful delivery of projects within scop...

Position Title Manager Location Chennai, India Band C1 Designation Technical Manger (Full Stack) Overview We are seeking a highly skilled and experienced Technical Manager specializing in Microsoft Technologies to join our dynamic team. As a Technical Manager, you will play a critical role in shaping the success of our technology projects. Your responsibilities will include architecture design, implementation, and overseeing the development and deployment of Microsoft-based solutions that meet our internal needs. You’ll collaborate closely with management, business analysts, project managers, developers, testers and other stakeholders to ensure the successful delivery of projects within scop...

Position Title Technical Architect Location Chennai, India Designation Technical Architect (Microsoft Solutions) Overview We are seeking a highly skilled and experienced Technical Architect specializing in Microsoft Technologies to join our dynamic team. As a Technical Architect, you will play a critical role in shaping the success of our technology projects. Your responsibilities will include architecture design, implementation, and overseeing the deployment of Microsoft-based solutions that meet our internal needs. You’ll collaborate closely with management, business analysts, project managers, developers, and other stakeholders to ensure the successful delivery of projects within scope, b...

Position Title Technical Architect Location Chennai, India Designation Technical Architect (Microsoft Solutions) Overview We are seeking a highly skilled and experienced Technical Architect specializing in Microsoft Technologies to join our dynamic team. As a Technical Architect, you will play a critical role in shaping the success of our technology projects. Your responsibilities will include architecture design, implementation, and overseeing the deployment of Microsoft-based solutions that meet our internal needs. You’ll collaborate closely with management, business analysts, project managers, developers, and other stakeholders to ensure the successful delivery of projects within scope, b...

Role: Data Engineer Location: Gurugram Key Role And Responsibilities Understand and translate business needs into data models supporting long-term solutions. Perform reverse engineering of physical data models from databases and SQL scripts. Analyse data-related system integration challenges and propose appropriate solutions. Assist with and support setting the data architecture direction (including data movement approach, architecture/technology strategy, and any other data related considerations to ensure business value). Requirements 5+ Years of experience as a Data Engineer Strong technical expertise in SQL Advanced SQL querying skills (joins, subqueries, CTEs, aggregation) Strong knowle...