210 Spark Streaming Jobs - Page 4

As a Senior Data Engineer at JLL Technologies, you will: Design, Architect, and Develop solutions leveraging cloud big data technology to ingest, process and analyze large, disparate data sets to exceed business requirements Develop systems that ingest, cleanse and normalize diverse datasets, develop data pipelines from various internal and external sources and build structure for previously unstructured data Interact with internal colleagues and external professionals to determine requirements, anticipate future needs, and identify areas of opportunity to drive data development Develop good understanding of how data will flow & stored through an organization across multiple applications suc...

What this job involves: JLL, an international real estate management company, is seeking an Data Engineer to join our JLL Technologies Team. We are seeking candidates that are self-starters to work in a diverse and fast-paced environment that can join our Enterprise Data team. We are looking for a candidate that is responsible for designing and developing of data solutions that are strategic for the business using the latest technologies Azure Databricks, Python, PySpark, SparkSQL, Azure functions, Delta Lake, Azure DevOps CI/CD. Responsibilities Design, Architect, and Develop solutions leveraging cloud big data technology to ingest, process and analyze large, disparate data sets to exceed b...

What this job involves: JLL, an international real estate management company, is seeking an Data Engineer to join our JLL Technologies Team. We are seeking candidates that are self-starters to work in a diverse and fast-paced environment that can join our Enterprise Data team. We are looking for a candidate that is responsible for designing and developing of data solutions that are strategic for the business using the latest technologies Azure Databricks, Python, PySpark, SparkSQL, Azure functions, Delta Lake, Azure DevOps CI/CD. Responsibilities Design, Architect, and Develop solutions leveraging cloud big data technology to ingest, process and analyze large, disparate data sets to exceed b...

Responsibilities * Design, develop, test, and deploy Spark Streaming applications using Scala/Spark Core. * Collaborate with cross-functional teams to gather requirements and deliver high-quality solutions. * Troubleshoot issues in Spark Streaming applications and optimize performance for improved efficiency. * Stay up-to-date with the latest developments in Spark Streaming and contribute to its evolution. Job Requirements * 3-10 years of experience in developing Spark Streaming applications using Scala/Spark Core. * Strong understanding of Spark programming concepts, including RDDs, Dataframes, Datasets, etc. * Experience with big data processing technologies such as Hadoop/Hive/Pig is a pl...

As a Senior Python Developer specializing in Python and Spark programming, you will be responsible for leveraging your thorough and hands-on knowledge to write efficient, reusable, and reliable Python code. You should possess a minimum of 5+ years of experience in Python programming and have a strong proficiency in both Python and Spark programming. Your key responsibilities will include implementing Spark Core, Spark SQL, and Spark Streaming, working with Spark within the Hadoop Ecosystem, and designing and implementing low-latency, high-availability, and performance applications. Additionally, you will lead and guide a team of junior Python developers, collaborate with other team members a...

At PwC, the focus of individuals in managed services is on a variety of outsourced solutions and supporting clients across various functions. They play a crucial role in helping organizations streamline operations, reduce costs, and enhance efficiency by managing key processes and functions on behalf of clients. Skilled in project management, technology, and process optimization, they are adept at delivering high-quality services to clients. Those in managed service management and strategy at PwC concentrate on transitioning and running services, managing delivery teams, programs, commercials, performance, and delivery risk. The role involves continuous improvement processes and optimizing m...

This position falls under the ICG TTS Operations Technology (OpsTech) Group, focusing on assisting in the implementation of a next-generation Digital Automation Platform and Imaging Workflow Technologies. The ideal Candidate should have relevant experience in managing development teams within the distributed systems Eco-System and must exhibit strong teamwork skills. The candidate is expected to possess superior technical knowledge of current programming languages, technologies, and leading-edge development tools. The primary objective of this role is to contribute to applications, systems analysis, and programming activities. As a Lead Spark Scala Engineer, the candidate should have hands-o...

ZS is a place where passion changes lives. As a management consulting and technology firm focused on improving life and how we live it, we transform ideas into impact by bringing together data, science, technology and human ingenuity to deliver better outcomes for all. Here you'll work side-by-side with a powerful collective of thinkers and experts shaping life-changing solutions for patients, caregivers and consumers, worldwide. ZSers drive impact by bringing a client-first mentality to each and every engagement. We partner collaboratively with our clients to develop custom solutions and technology products that create value and deliver company results across critical areas of their busines...

As an exceptionally skilled individual, you will be part of a dedicated team at TNS, collaborating daily to contribute to the success of the organization. If you are driven by excellence in both professional and personal aspects, this is the place for you! The role entails being a Java and/or Scala developer with expertise in Big Data tools and frameworks. You should have 8 to 12 years of proven experience in Java and/or Scala development. Your responsibilities will include hands-on work with prominent Big Data tools like Hadoop, Spark, Map Reduce, Hive, and Impala. Additionally, you should possess a deep understanding of streaming technologies such as Kafka and/or Spark Streaming. Strong fa...

As a Databricks AWS/Azure/GCP Architect at Koantek based in Mumbai, you will play a crucial role in building secure and highly scalable big data solutions that drive tangible, data-driven outcomes while emphasizing simplicity and operational efficiency. Collaborating with teammates, product teams, and cross-functional project teams, you will lead the adoption and integration of the Databricks Lakehouse Platform into the enterprise ecosystem and AWS/Azure/GCP architecture. Your responsibilities will include implementing securely architected big data solutions that are operationally reliable, performant, and aligned with strategic initiatives. Your expertise should include an expert-level know...

Overview This position is for Lead Data Engineer in the Commercial Data as a Service group. In this position you will enjoy being responsible for helping define and maintain the data systems key to delivering successful outcomes for our customers. You will be hands on and work closely to guide a team of Data Engineers in the associated data maintenance, integrations, enhancements, loads and transformation processes for the organization. This key individual will work closely with Data Architects to design and implement solutions and insure successful implementations. Role Leads initiatives to build and maintain database technologies, environments, and applications, seeking opportunities for i...

Overview This position is for Manager Data Engineering in the Commercial Data as a Service group. In this position you will enjoy being responsible for helping define and maintain the data systems key to delivering successful outcomes for our customers. will lead a software development team focused on our core services someone who combines deep technical experience with great people management skills and can help elevate our teams and our approach to engineering. We are building a culture that fosters innovation, built on a foundation of inclusion and diversity. We are looking for a candidate that has proven experience in coding, architecture leadership, and leading engineers in a fast-paced...

Job Title: Big Data Developer Project Support & Mentorship Location: Mumbai Employment Type: Full-Time/Contract Work Arrangement: Onsite Mumbai COE Position Overview: We are seeking a skilled Big Data Developer to join our growing delivery team, with a dual focus on hands-on project support and mentoring junior engineers. This role is ideal for a developer who not only thrives in a technical, fast-paced environment but is also passionate about coaching and developing the next generation of talent. You will work on live client projects, provide technical support, contribute to solution delivery, and serve as a go-to technical mentor for less experienced team members. Key Responsibilities: Per...

Join a team recognized for leadership, innovation, and diversity. You must have a Ph.D. or Master's degree in Computer Science, Engineering, Applied Mathematics, or a related field. Exposure to the Finance domain and use cases in a larger global enterprise setting is required. Additionally, you should have a minimum of 8 to 10 years of Data Science prototyping experience using machine learning techniques and algorithms such as k-means, k-NN, Naive Bayes, SVM, and Decision Trees, with proficiency in Python and/or R tool-stack. Moreover, a minimum of 8 to 10 years of Machine Learning experience of physical systems is expected. You should also possess a minimum of 4 to 6 years of experience wit...

Required Skills & Experience 6-9 years of experience in Consulting, Data, and Analytics , with a strong background in Databricks solutions . Hands-on experience in designing and implementing big data solutions , including data pipelines for large and complex datasets. Expertise in Python, PySpark , and orchestration frameworks such as Airflow, Oozie, Luigi , etc. Strong understanding of Databricks workspace and its components (workspace, compute, clusters, jobs, Unity Catalog, UC permissions). Ability to configure, schedule, and manage Databricks clusters and jobs , including performance optimization and monitoring. Experience with data modeling , SQL queries, joins, stored procedures, relat...

Primary Responsibilities: Analyze business requirements & functional specifications Be able to determine the impact of changes in current functionality of the system Interaction with diverse Business Partners and Technical Workgroups Be flexible to collaborate with onshore business, during US business hours Be flexible to support project releases, during US business hours Required Qualifications: Undergraduate degree or equivalent experience 3+ years of working experience in Python, Pyspark, Scala 3+ years of experience working on MS Sql Server and NoSQL DBs like Cassandra, etc. Hands-on working experience in Azure Databricks Solid healthcare domain knowledge Exposure to following DevOps met...

Job Title: Developer Experience: 4-6 Years Work Location: Chennai, TN || Bangalore, KA || Hyderabad, TS Skill Required: Digital : Bigdata and Hadoop Ecosystems Digital : PySpark Job Description: "? Need to work as a developer in Bigdata, Hadoop or Data Warehousing Tools and Cloud Computing ? Work on Hadoop, Hive SQL?s, Spark, Bigdata Eco System Tools.? Experience in working with teams in a complex organization involving multiple reporting lines.? The candidate should have strong functional and technical knowledge to deliver what is required and he/she should be well acquainted with Banking terminologies. ? The candidate should have strong DevOps and Agile Development Framework knowledge.? Cr...

The Machine Learning Engineer position based in GGN requires a professional with 6-9 years of experience. The ideal candidate should possess expertise in Spark, SQL, Python/Scala, AWS EMR, AWS S3, ML Life Cycle Management, and Machine Learning Operations (ML Ops). Additionally, experience with Airflow or any other orchestrator is considered a good to have qualification. Experience with Kafka, Spark Streaming, Datadog, and Kubernetes are also valued assets for this role. If you meet these qualifications and are passionate about machine learning, this position could be an excellent fit for you.,

Contract Duration : 4 Months (Extendable based on Performance) Job Timings : India Evening Shift (till 11 : 30 PM IST) We are looking for an experienced Databricks Tech Lead to join our team on a 4-month extendable contract. The ideal candidate will bring deep expertise in data engineering, big data platforms, and cloud-based data warehouse solutions, with the ability to work in a fast-paced remote environment. Key Responsibilities Lead the design, optimization, and management of large-scale data pipelines using Databricks, Spark (PySpark), and AWS data services. Productionize and deploy Big Data platforms and applications across multi-cloud environments (AWS, Azure, GCP). Build and manage d...

As a skilled professional with over 7 years of experience, you will be responsible for reviewing and understanding business requirements to ensure timely completion of development tasks with rigorous testing to minimize defects. Collaborating with a software development team is crucial to implement best practices and enhance the performance of Data applications, meeting client needs effectively. In this role, you will collaborate with various teams within the company and engage with customers to comprehend, translate, define, and design innovative solutions for their business challenges. Your tasks will also involve researching new Big Data technologies to evaluate their maturity and alignme...

You are seeking a highly experienced Azure PySpark Solution Architect to lead the design and implementation of scalable data solutions on Microsoft Azure in Trivandrum. Your role will involve leveraging your expertise in Azure services, PySpark, and enterprise-grade solution architecture to drive efficient data processing and analytics workflows. Your responsibilities will include designing and implementing end-to-end data solutions using Azure Data Services and PySpark, developing high-performance ETL pipelines with tools such as Azure Databricks, Azure Data Factory, and Synapse Analytics, and architecting scalable, secure, and cost-efficient cloud solutions aligned with business goals. You...

Position: Spark-Scala Work location : Bangalore , Chennai, Hyderabad, Pune, Chandigarh. Type: C2H (Convert into Full-Time Employment (FTE) within 90 days) Experience: 8-10 yrs. Skillsets: Mandatory skills - Spark & Scala Desired skills - Spark Streaming, Hadoop, Hive, SQL, Sqoop, Impala Joining: Immediate joiner or 15 to 30 days

Roles and Responsibilities Collaborate with cross-functional teams to gather requirements and deliver high-quality solutions. Develop scalable and efficient ETL processes for large datasets on cloud platforms like AWS or GCP. Ensure compliance with security standards and best practices in the development of data systems. Design, develop, test, and deploy data pipelines using DataBricks, GCP, Microsoft Azure, Spark, SQL, and Python. Desired Candidate Profile 3.5 -7 years of experience as a Data Engineer with expertise in Big Data technologies such as pySpark (Data frame and SparkSQL), Hadoop, and Hive Good hands-on experience of python and Bash Scripts Strong analytical, problem-solving, data...

EPAM has presence across 40+ countries globally with 55,000 + professionals & numerous delivery centers, Key locations are North America, Eastern Europe, Central Europe, Western Europe, APAC, Mid East & Development Centers in India (Hyderabad, Pune & Bangalore). Location: Gurgaon/Pune/Hyderabad/Bengaluru/ChennaiWork Mode: Hybrid (2-3 days office in a week) Job Description: 5-14 Years of in Big Data & Data related technology experience Expert level understanding of distributed computing principles Expert level knowledge and experience in Apache Spark Hands on programming with Python Experience with building stream-processing systems, using technologies such as Apache Storm or Spark-Streaming ...

Hiring Sr Data Engineer in Bangalore with 8+ years exp in the below skills: Must Have: - Big Data technologies: Hadoop, MapReduce, Spark, Kafka, Flink - Programming languages: Java/ Scala/ Python - Cloud: Azure, AWS, Google Cloud - Docker/Kubernetes Required Candidate profile - Strong in Communication Skills - Experience with relational SQL/ NoSQL databases- Postgres & Cassandra - Experience with ELK stack - Immediate Join is plus - Must be ready to work from office