16 Structured Streaming Jobs

Creative Synergies Group Overview Creative is a leading Global Technology Services firm delivering innovative solutions that build upon synergies between our Deep Engineering Domain Expertise and Digital Engineering Services We work with clients across the Global Transportation, Energy/Process, Industrial Products, Hi-Tech, and EdTech verticals Creative's global team of 1,500+ Can-Do engineers collaborate with our 40+ Fortune 500 clients to help them achieve their strategic and operational objectives (Including Google, GM, Samsung, Tesla, Mercedes, Sony, Mitsubishi) 95% of our revenue comes from US, Europe, Japan based customers Service Offerings: Data, Analytics, and AI Engineering Services...

About the Role We are seeking a highly skilled Data Engineer with deep expertise in Microsoft Azure to design, build, and optimize scalable data pipelines and Lakehouse architectures. You will work closely with cross-functional teams to provide thought leadership deliver robust, secure, and high-performance data solutions that empower analytics and data science initiatives. Key Responsibilities Design, develop, and maintain scalable data pipelines using Azure Data Factory and Azure Synapse Analytics . Build and optimize lakehouse architectures on Azure using Delta Lake . Work closely with client stakeholders to understand their requirements and provide thought leadership to translate the sam...

Role Overview The Technical Architect - Databricks designs and implements scalable data architectures and solutions. The jobholder has expertise in Databricks Lakehouse, data modeling, and cloud integration, ensuring high performance, security, and reliability. Responsibilities Design and implement Databricks-based data architectures to meet business requirements. Develop and optimize data pipelines using PySpark, Scala, or SQL. Establish the Databricks Lakehouse architecture for batch and streaming data. Collaborate with cross-functional teams to integrate Databricks with cloud platforms (e.g., AWS, Azure, GCP). Ensure data security and compliance with best practices. Monitor and troublesho...

Responsibilities ETL Development: Build and maintain ETL/ELT pipelines using Databricks & Apache Spark (PySpark/Scala). Data Warehousing: Design and implement Star Schema, Snowflake Schema, and Data Vault models. Databricks Optimization: Optimize Delta Lake storage, caching, and job execution to improve query performance. Databricks Optimization: Optimize Delta Lake storage, caching, and job execution to improve query performance. Data Ingestion & Processing: Develop scalable ingestion pipelines using Structured Streaming, Auto Loader, and Databricks Workflows. Data Governance & Security: Implement Unity Catalog, role-based access controls (RBAC), and compliance standards. Performance Tuning...

Role Overview: At Derevo, we empower companies and individuals by unlocking the value of data within organizations. With over 15 years of experience, we design end-to-end data and AI solutions - from integration into modern architectures to implementing intelligent models in key business processes. We are currently seeking a talented Senior Data Engineer specialized in Databricks to join our team. Key Responsibilities: - Design and implement modern data architectures with high quality, driving analytical solutions based on Big Data technologies. - Maintain and optimize parallel processing systems, applying best practices for storage and management in data warehouses, data lakes, and lakehous...

As a Data Engineer, your role will involve collaborating with data scientists and business stakeholders to design, develop, and maintain efficient data pipelines feeding into the organization's data lake. It will be your responsibility to maintain the integrity and quality of the data lake, enabling accurate and actionable insights for data scientists and informed decision-making for business stakeholders. Leveraging your extensive knowledge of data engineering and cloud technologies, you will enhance the organization's data infrastructure to promote a culture of data-driven decision-making. Your expertise in data engineering will be crucial in defining and optimizing data pipelines using ad...

Job Title: Big Data Architect / Databricks Architect Company Overview: At Codvo, software and people transformations go hand-in-hand. We are a global empathy-led technology services company. Product innovation and mature software engineering are part of our core DNA. Respect, Fairness, Growth, Agility, and Inclusiveness are the core values that we aspire to live by each day. We continue to expand our digital strategy, design, architecture, and product management capabilities to offer expertise, outside-the-box thinking, and measurable results. The Role We are seeking an experienced Big Data Architect with a specialization in Databricks to design the sophisticated data solutions that drive bu...

Req ID: 341055 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Databricks Architect to join our team in Bangalore, Karn?taka (IN-KA), India (IN). Databricks Data Platform - Architect Job Description Role Summary As a Databricks Architect , you will lead the design and implementation of scalable, secure, and high-performance data solutions using the Databricks Lakehouse Platform. You will define architectural standards, guide engineering teams, and collaborate with stakeholders to align data strategies with business g...

At Medtronic, you can embark on a lifelong journey of exploration and innovation, while contributing to the advancement of healthcare access and equity for all. As an IT Director of Data Engineering, you will play a pivotal role in designing and implementing data solutions for the Diabetes operating unit. Your responsibilities will involve leading the development and maintenance of the enterprise data platform, overseeing data pipeline management, ensuring scalability and performance, integrating new technologies, collaborating with various teams, and managing projects and budgets effectively. Your role will require a master's degree in statistics, computer science, mathematics, data science...

As a Senior/Lead Data Engineer in our team based in Mumbai, IN, you will be responsible for leveraging your 6-8 years of IT experience, with at least 5+ years specifically in Data Engineering. Your expertise will be utilized in various areas including Kafka integration to Databricks, understanding of Structured Streaming concepts such as Processing modes, output modes, and checkpointing. Familiarity with Medallion Architecture (Bronze, Silver, Gold layers) and Databricks Autoloader will be key aspects of your role. Moreover, your experience in working with large volumes of data and implementing performance optimization techniques like partitioning, caching, and cluster tuning will be crucial...

Position Summary... Drives the execution of multiple business plans and projects by identifying customer and operational needs; developing and communicating business plans and priorities; removing barriers and obstacles that impact performance; providing resources; identifying performance standards; measuring progress and adjusting performance accordingly; developing contingency plans; and demonstrating adaptability and supporting continuous learning. Provides supervision and development opportunities for associates by selecting and training; mentoring; assigning duties; building a team-based work environment; establishing performance expectations and conducting regular performance evaluatio...

The Databricks Data Engineering Lead role requires a highly skilled individual who will architect and lead the implementation of scalable, high-performance data pipelines and platforms using the Databricks Lakehouse ecosystem. As a Data Engineering Lead, you will be responsible for managing a team of data engineers, establishing best practices, and collaborating with cross-functional stakeholders to unlock advanced analytics, AI/ML, and real-time decision-making capabilities. Your key responsibilities will include leading the design and development of modern data pipelines, data lakes, and lakehouse architectures using Databricks and Apache Spark. You will manage and mentor a team of data en...

You will be responsible for designing, developing, and maintaining scalable data pipelines using Azure Databricks. Your role will involve building and optimizing ETL/ELT processes for structured and unstructured data, collaborating with data scientists, analysts, and business stakeholders, integrating Databricks with Azure Data Lake, Synapse, Data Factory, and Blob Storage, developing real-time data streaming pipelines, and managing data models/data warehouses. Additionally, you will optimize performance, manage resources, ensure cost efficiency, implement best practices for data governance, security, and quality, troubleshoot and improve existing data workflows, contribute to architecture a...

As a Big Data Architect specializing in Databricks at Codvo, a global empathy-led technology services company, your role is critical in designing sophisticated data solutions that drive business value for enterprise clients and power internal AI products. Your expertise will be instrumental in architecting scalable, high-performance data lakehouse platforms and end-to-end data pipelines, making you the go-to expert for modern data architecture in a cloud-first world. Your key responsibilities will include designing and documenting robust, end-to-end big data solutions on cloud platforms (AWS, Azure, GCP) with a focus on the Databricks Lakehouse Platform. You will provide technical guidance a...

Key Skills: Spark, Scala, Flink, Big Data, Structured Streaming, Data Architecture, Data Modeling, NoSQL, AWS, Azure, GCP, JVM tuning, Performance Optimization. Roles & Responsibilities: Design and build robust data architectures for large-scale data processing. Develop and maintain data models and database designs. Work on stream processing engines like Spark Structured Streaming and Flink. Perform analytical processing on Big Data using Spark. Administer, configure, monitor, and tune performance of Spark workloads and distributed JVM-based systems. Lead and support cloud deployments across AWS, Azure, or Google Cloud Platform. Manage and deploy Big Data technologies such as Business Data L...

Genpact (NYSE: G) is a global professional services and solutions firm delivering outcomes that shape the future. Our 125,000+ people across 30+ countries are driven by our innate curiosity, entrepreneurial agility, and desire to create lasting value for clients. Powered by our purpose - the relentless pursuit of a world that works better for people - we serve and transform leading enterprises, including the Fortune Global 500, with our deep business and industry knowledge, digital operations services, and expertise in data, technology, and AI. Inviting applications for the role of Lead Consultant/ Data Engineer. In this role, you will collaborate closely with cross-functional teams, includi...