146 Spark Streaming Jobs

Data Architect is responsible to define and lead the Data Architecture, Data Quality, Data Governance, ingesting, processing, and storing millions of rows of data per day. This hands-on role helps solve real big data problems. You will be working with our product, business, engineering stakeholders, understanding our current eco-systems, and then building consensus to designing solutions, writing codes and automation, defining standards, establishing best practices across the company and building world-class data solutions and applications that power crucial business decisions throughout the organization. We are looking for an open-minded, structured thinker passionate about building systems at scale. Role Design, implement and lead Data Architecture, Data Quality, Data Governance Defining data modeling standards and foundational best practices Develop and evangelize data quality standards and practices Establish data governance processes, procedures, policies, and guidelines to maintain the integrity and security of the data Drive the successful adoption of organizational data utilization and self-serviced data platforms Create and maintain critical data standards and metadata that allows data to be understood and leveraged as a shared asset Develop standards and write template codes for sourcing, collecting, and transforming data for streaming or batch processing data Design data schemes, object models, and flow diagrams to structure, store, process, and integrate data Provide architectural assessments, strategies, and roadmaps for data management Apply hands-on subject matter expertise in the Architecture and administration of Big Data platforms, Data Lake Technologies (AWS S3/Hive), and experience with ML and Data Science platforms Implement and manage industry best practice tools and processes such as Data Lake, Databricks, Delta Lake, S3, Spark ETL, Airflow, Hive Catalog, Redshift, Kafka, Kubernetes, Docker, CI/CD Translate big data and analytics requirements into data models that will operate at a large scale and high performance and guide the data analytics engineers on these data models Define templates and processes for the design and analysis of data models, data flows, and integration Lead and mentor Data Analytics team members in best practices, processes, and technologies in Data platforms Qualifications B.S. or M.S. in Computer Science, or equivalent degree 10+ years of hands-on experience in Data Warehouse, ETL, Data Modeling & Reporting 7+ years of hands-on experience in productionizing and deploying Big Data platforms and applications, Hands-on experience working with: Relational/SQL, distributed columnar data stores/NoSQL databases, time-series databases, Spark streaming, Kafka, Hive, Delta Parquet, Avro, and more Extensive experience in understanding a variety of complex business use cases and modeling the data in the data warehouse Highly skilled in SQL, Python, Spark, AWS S3, Hive Data Catalog, Parquet, Redshift, Airflow, and Tableau or similar tools Proven experience in building a Custom Enterprise Data Warehouse or implementing tools like Data Catalogs, Spark, Tableau, Kubernetes, and Docker Knowledge of infrastructure requirements such as Networking, Storage, and Hardware Optimization with hands-on experience inAmazon Web Services (AWS) Strong verbal and written communications skills are a must and should work effectively across internal and external organizations and virtual teams Demonstrated industry leadership in the fields of Data Warehousing, Data Science, and Big Data related technologies Strong understanding of distributed systems and container-based development using Docker and Kubernetes ecosystem Deep knowledge of data structures and algorithms Experience working in large teams using CI/CD and agile methodologies.

About Koantek: Koantek is a Databricks Pure-Play Elite Partner, helping enterprises modernize faster and unlock the full power of Data and AI. Backed by Databricks Ventures and honored as a six- As time Databricks Partner of the Year, we enable global enterprises to modernize at speed, operationalize AI, and realize the full value of their data. Our deep expertise spans industries such as healthcare, financial services, retail, and SaaS, delivering end-to-end solutions from rapid prototyping to production-scale AI deployments. We deliver tailored solutions that enable businesses to leverage data for growth and innovation. Our team of experts utilizes deep industry knowledge combined with cutting-edge technologies, tools, and methodologies to drive impactful results. By partnering with clients across a diverse range of industries, from emerging startups to established enterprises we help them uncover new opportunities and achieve a competitive advantage in the digital age. About the Role: As a Solutions Architect at Koantek, you will collaborate with customers to design scalable data architectures utilizing Databricks technology and services. The RSA at Koantek builds secure, highly scalable big data solutions to achieve tangible, data-driven outcomes all the while keeping simplicity and operational effectiveness in mind. Leveraging your technical expertise and business acumen, you will navigate complex technology discussions, showcasing the value of the Databricks platform throughout the sales process. Working alongside Account Executives, you will engage with customers' technical leaders, including architects, engineers, and operations teams, aiming to become a trusted advisor who delivers concrete outcomes. This role collaborates with teammates, product teams, and cross-functional project teams to lead the adoption and integration of the Databricks Platform into the enterprise ecosystem and AWS/Azure/GCP architecture. The impact you will have: Develop Account Strategies: Work with Sales and other essential partners to develop strategies for your assigned accounts to grow their usage of Databricks platform. Establish Architecture Standards: Establish the Databricks Lakehouse architecture as the standard data architecture for customers through excellent. Technical account planning. Demonstrate Value: Build and present reference architectures and demos applications to help prospects understand how Databricks can be used to achieve their goals and land new use cases. Capture Technical Wins: Consult on big data architectures, data engineering pipelines, and data science/machine learning projects to prove out Databricks technology for strategic customer projects. Validate integrations with cloud services and other third-party applications. Promote Open-Source Projects: Become an expert in and promote Databricks- inspired open-source projects (Spark, Delta Lake, MLflow) acrothe ss developer communities through meetups, conferences, and webinars. Technical Expertise: Experience translating a customer's business needs to technology solutions, including establishing buy-in with essential customer stakeholders at all levels of the business. Experienced at designing, architecting, and presenting data systems for customers and managing the delivery of production solutions of those data architectures. Projects delivered with hands-on experience in development on databricks Expert-level knowledge of data frameworks, data lakes and open-source projects such as Apache Spark, MLflow, and Delta Lake Expert-level hands-on coding experience in Spark/Scala, Python or PySpark In-depth understanding of Spark Architecture, including Spark Core, Spark SQL, and Data Frames, Spark Streaming, RDD caching, Spark MLibT/event-driven/microservices in the cloud Deep experience with distributed computing with spark with knowledge of spark runtime Experience with private and public cloud architectures, pros/cons, and migration considerations. Extensive hands-on experience implementing data migration and data processing using AWS/Azure/GCP services Familiarity with CI/CD for production deployments Familiarity with optimization for performance and scalability Completed data engineering professional certification and required classes SQL Proficiency: Fluent in SQL and database technology Educational Background: Degree in a quantitative discipline (Computer Science, Applied Mathematics, Operations Researh). Relevant certifications (e.g., Databricks certifications, AWS/Azure/GCP AI/ML certificatins) are a plus. Workplace Flexibility This is a hybrid role with remote flexibility. On-site presence at customer locations MAY be required based on project and business needs. Candidates should be willing and able to travel for short or medium-term assignments when necessary.

About the role PubMatic is seeking Data Analytics focused Senior Software Engineers with expertise in building and optimizing AI agents, including strong skills in Hadoop, Spark, Scala, Kafka, Spark Streaming, and cloud-based solutions, with proficiency in programming languages such as Scala and Python. You will be responsible for developing advanced AI agents that enhance our data analytics capabilities, enabling our platform to handle complex information retrieval, contextual understanding, and adaptive interactions, ultimately improving our data-driven insights and analytical workflows What You’ll Do Build, design, and implement our highly scalable, fault-tolerant big data platform to process terabytes of data and provide customers with in-depth analytics. Develop backend services using Java, REST APIs, JDBC, and AWS. Build and maintain Big Data pipelines using technologies like Spark, Hadoop, Kafka, and Snowflake. Architect and implement real-time data processing workflows and automation frameworks. Design and develop GenAI-powered agents for analytics, operations, and data enrichment use cases using frameworks like LangChain, LlamaIndex, or custom orchestration systems. Integrate LLMs (e.g., OpenAI, Claude, Mistral) into existing services for query understanding, summarization, and decision support. Manage end-to-end GenAI workflows including prompt engineering, fine-tuning, vector embeddings, and retrieval-augmented generation (RAG). Working closely with cross-functional teams on improving the availability and scalability of large data platforms and the functionality of PubMatic software. Participate in Agile/Scrum processes such as sprint planning, sprint retrospective, backlog grooming, user story management, and work item prioritization. Frequently discuss with product managers about the software features to include in the PubMatic Data Analytics platform. Support customer issues over email or JIRA (bug tracking system), provide updates, and patches to customers to fix the issues. Perform code and design reviews for code implemented by peers or as per the code review process. We’d Love for You to Have 3-7 plus years of coding experience in Java and backend development. Solid computer science fundamentals, including data structure and algorithm design, and creation of architectural specifications. Expertise in developing the Implementation of professional software engineering best practices for the full software development life cycle, including coding standards, code reviews. Hands-on experience with Big Data tools and systems like Scala Spark, Kafka, Hadoop, Snowflake. Proven expertise in building GenAI applications, including: LLM integration (OpenAI, Anthropic, Cohere, etc.) LangChain, or similar agent orchestration libraries Prompt engineering, embeddings, and retrieval-based generation (RAG) Experience in developing and deploying scalable, production-grade AI or data systems. Ability to lead end-to-end feature development and debug distributed systems. Experience in developing and delivering large-scale big data pipelines, real-time systems & data warehouses would be preferred. Demonstrated ability to achieve stretch goals in a very innovative and fast-paced environment. Demonstrated ability to learn new technologies quickly and independently. Excellent verbal and written communication skills, especially in technical communications. Strong interpersonal skills and a desire to work collaboratively. Qualifications Should have a bachelor’s degree in engineering (CS / IT) or equivalent degree from a well-known Institutes / Universities. Additional Information Return to Office : PubMatic employees throughout the global have returned to our offices via a hybrid work schedule (3 days “in office” and 2 days “working remotely”) that is intended to maximize collaboration, innovation, and productivity among teams and across functions. Benefits : Our benefits package includes the best of what leading organizations provide, such as paternity/maternity leave, healthcare insurance, broadband reimbursement. As well, when we’re back in the office, we all benefit from a kitchen loaded with healthy snacks and drinks and catered lunches and much more! Diversity and Inclusion : PubMatic is proud to be an equal opportunity employer; we don’t just value diversity, we promote and celebrate it. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status. About PubMatic PubMatic is one of the world’s leading scaled digital advertising platforms, offering more transparent advertising solutions to publishers, media buyers, commerce companies and data owners, allowing them to harness the power and potential of the open internet to drive better business outcomes. Founded in 2006 with the vision that data-driven decisioning would be the future of digital advertising, we enable content creators to run a more profitable advertising business, which in turn allows them to invest back into the multi-screen and multi-format content that consumers demand. #LI-HYBRID

Role Overview: At PwC, you will be part of the managed services team focusing on outsourced solutions and providing support to clients across various functions. Your role will involve helping organizations streamline operations, reduce costs, and enhance efficiency by managing key processes and functions. You will leverage your skills in project management, technology, and process optimization to deliver high-quality services to clients. Specifically, as a Data Engineer Offshore, you will play a crucial role in designing, implementing, and maintaining scalable data pipelines and systems to support data-driven initiatives. Key Responsibilities: - Design, develop, and maintain scalable ETL pipelines using DataStage and other ETL tools. - Utilize AWS cloud services for data storage, processing, and analytics. - Implement and optimize Delta Live Tables and Delta Tables for efficient data storage and querying. - Collaborate with cross-functional teams to gather requirements and deliver data solutions that meet business needs. - Ensure data quality, integrity, and security across all data systems and pipelines. - Monitor and troubleshoot data workflows to ensure smooth operations. Qualifications: - Bachelor's degree in Computer Science, Information Technology, or a related field. - Proven experience as a Data Engineer or in a similar role. - Strong proficiency in SQL and experience with relational databases such as Teradata. - Hands-on experience with AWS services such as S3, EMR, Redshift, and Lambda. - Experience with Delta Live Tables and Delta Tables in a data engineering context. - Solid understanding of Apache Spark, Kafka, and Spark Streaming. - Strong problem-solving skills and attention to detail. - Excellent communication and collaboration skills. Additional Details: The company is seeking an experienced Data Engineer with a strong background in data engineering and proficiency in various data technologies such as Teradata, DataStage, AWS, Databricks, SQL, and more. The role will involve continuous improvement and optimization of managed services processes, tools, and services to deliver high-quality services to clients.,

What this job involves: JLL, an international real estate management company, is seeking an Data Engineer to join our JLL Technologies Team. We are seeking candidates that are self-starters to work in a diverse and fast-paced environment that can join our Enterprise Data team. We are looking for a candidate that is responsible for designing and developing of data solutions that are strategic for the business using the latest technologies Azure Databricks, Python, PySpark, SparkSQL, Azure functions, Delta Lake, Azure DevOps CI/CD. Responsibilities Design, Architect, and Develop solutions leveraging cloud big data technology to ingest, process and analyze large, disparate data sets to exceed business requirements. Design & develop data management and data persistence solutions for application use cases leveraging relational, non-relational databases and enhancing our data processing capabilities. Develop POCs to influence platform architects, product managers and software engineers to validate solution proposals and migrate. Develop data lake solution to store structured and unstructured data from internal and external sources and provide technical guidance to help migrate colleagues to modern technology platform. Contribute and adhere to CI/CD processes, development best practices and strengthen the discipline in Data Engineering Org. Develop systems that ingest, cleanse and normalize diverse datasets, develop data pipelines from various internal and external sources and build structure for previously unstructured data. Using PySpark and Spark SQL, extract, manipulate, and transform data from various sources, such as databases, data lakes, APIs, and files, to prepare it for analysis and modeling. Build and optimize ETL workflows using Azure Databricks and PySpark. This includes developing efficient data processing pipelines, data validation, error handling, and performance tuning. Perform the unit testing, system integration testing, regression testing and assist with user acceptance testing. Articulates business requirements in a technical solution that can be designed and engineered. Consults with the business to develop documentation and communication materials to ensure accurate usage and interpretation of JLL data. Implement data security best practices, including data encryption, access controls, and compliance with data protection regulations. Ensure data privacy, confidentiality, and integrity throughout the data engineering processes. Performs data analysis required to troubleshoot data related issues and assist in the resolution of data issues. Experience & Education Minimum of 4 years of experience as a data developer using Python, PySpark, Spark Sql, ETL knowledge, SQL Server, ETL Concepts. Bachelors degree in Information Science, Computer Science, Mathematics, Statistics or a quantitative discipline in science, business, or social science. Experience in Azure Cloud Platform, Databricks, Azure storage. Effective written and verbal communication skills, including technical writing. Excellent technical, analytical and organizational skills. Technical Skills & Competencies Experience handling un-structured, semi-structured data, working in a data lake environment, leveraging data streaming and developing data pipelines driven by events/queues Hands on Experience and knowledge on real time/near real time processing and ready to code Hands on Experience in PySpark, Databricks, and Spark Sql. Knowledge on json, Parquet and Other file format and work effectively with them No Sql Databases Knowledge like Hbase, Mongo, Cosmos etc. Preferred Cloud Experience on Azure or AWS Python-spark, Spark Streaming, Azure SQL Server, Cosmos DB/Mongo DB, Azure Event Hubs, Azure Data Lake Storage, Azure Search etc. Team player, Reliable, self-motivated, and self-disciplined individual capable of executing on multiple projects simultaneously within a fast-paced environment working with cross functional teams.

JLL is a leading professional services firm that specializes in real estate and investment management. Our vision is to reimagine the world of real estate, creating rewarding opportunities and amazing spaces where people can achieve their ambitions. In doing so, we will build a better tomorrow for our clients, our people and our communities. JLL Technologies is a specialized group within JLL. At JLL Technologies, our mission is to bring technology innovation to commercial real estate. We deliver unparalleled digital advisory, implementation, and services solutions to organizations globally. Our goal is to leverage technology to increase the value and liquidity of the world's buildings, while enhancing the productivity and the happiness of those that occupy them. What this job involves JLL Technologies Enterprise Data team is a newly established central organization that oversees JLLs data strategy. We are seeking data professionals to work with our colleagues at JLL around the globe in providing solutions, developing new products, building enterprise reporting & analytics capability to reshape the business of Commercial Real Estate using the power of data and we are just getting started on that journey! We are looking for a Staff Data Engineer who is self-starter to work in a diverse and fast-paced environment that can join our Enterprise Data team. This is an individual contributor role that is responsible for designing and developing of data solutions that are strategic for the business and built on the latest technologies and patterns. This a global role that requires partnering with the broader JLLT team at the country, regional and global level by utilizing in-depth knowledge of data, infrastructure, technologies and data engineering experience. As a Senior Data Engineer at JLL Technologies, you will: Design, Architect, and Develop solutions leveraging cloud big data technology to ingest, process and analyze large, disparate data sets to exceed business requirements Develop systems that ingest, cleanse and normalize diverse datasets, develop data pipelines from various internal and external sources and build structure for previously unstructured data Interact with internal colleagues and external professionals to determine requirements, anticipate future needs, and identify areas of opportunity to drive data development Develop good understanding of how data will flow & stored through an organization across multiple applications such as CRM, Broker & Sales tools, Finance, HR etc Unify, enrich, and analyze variety of data to derive insights and opportunities Design & develop data management and data persistence solutions for application use cases leveraging relational, non-relational databases and enhancing our data processing capabilities Develop POCs to influence platform architects, product managers and software engineers to validate solution proposals and migrate Develop data lake solution to store structured and unstructured data from internal and external sources and provide technical guidance to help migrate colleagues to modern technology platform Contribute and adhere to CI/CD processes, development best practices and strengthen the discipline in Data Engineering Org Mentor other members in the team and organization and contribute to organizations growth. What we are looking for: 6+ years work experience and bachelors degree in Information Science, Computer Science, Mathematics, Statistics or a quantitative discipline in science, business, or social science. Hands-on engineer who is curious about technology, should be able to quickly adopt to change and one who understands the technologies supporting areas such as Cloud Computing (AWS, Azure(preferred), etc.), Micro Services, Streaming Technologies, Network, Security, etc. 3 or more years of active development experience as a data developer using Python-spark, Spark Streaming, Azure SQL Server, Cosmos DB/Mongo DB, Azure Event Hubs, Azure Data Lake Storage, Azure Search etc. Design & develop data management and data persistence solutions for application use cases leveraging relational, non-relational databases and enhancing our data processing capabilities Build, test and enhance data curation pipelines integration data from wide variety of sources like DBMS, File systems, APIs and streaming systems for various KPIs and metrics development with high data quality and integrity Maintain the health and monitoring of assigned data engineering capabilities that span analytic functions by triaging maintenance issues; ensure high availability of the platform; monitor workload demands; work with Infrastructure Engineering teams to maintain the data platform; serve as an SME of one or more application Team player, Reliable, self-motivated, and self-disciplined individual capable of executing on multiple projects simultaneously within a fast-paced environment working with cross functional teams 3+ years of experience working with source code control systems and Continuous Integration/Continuous Deployment tools Independent and able to manage, prioritize & lead workload What you can expect from us: Youll join an entrepreneurial, inclusive culture. One where we succeed together across the desk and around the globe. Where like-minded people work naturally together to achieve great things. Our Total Rewards program reflects our commitment to helping you achieve your ambitions in career, recognition, well-being, benefits and pay. Join us to develop your strengths and enjoy a fulfilling career full of varied experiences. Keep those ambitions in sights and imagine where JLL can take you...

We are looking for a Senior Data Engineer who is self-starter to work in a diverse and fast-paced environment that can join our Enterprise Data team. This is an individual contributor role that is responsible for designing and developing of data solutions that are strategic for the business and built on the latest technologies and patterns. This a global role that requires partnering with the broader JLLT team at the country, regional and global level by utilizing in-depth knowledge of data, infrastructure, technologies and data engineering experience. As a Data Engineer 2 at JLL Technologies, you will: Contributes to the design of information infrastructure, and data management processes to move the organization to a more sophisticated, agile and robust target state data architecture Develop systems that ingest, cleanse and normalize diverse datasets, develop data pipelines from various internal and external sources and build structure for previously unstructured data Develop good understanding of how data will flow & stored through an organization across multiple applications such as CRM, Broker & Sales tools, Finance, HR etc Design & develop data management and data persistence solutions for application use cases leveraging relational, non-relational databases and enhancing our data processing capabilities Develop POCs to influence platform architects, product managers and software engineers to validate solution proposals and migrate What we are looking for: 4+ years overall work experience and bachelor's degree in Information Science, Computer Science, Mathematics, Statistics or a quantitative discipline in science, business, or social science. Minimum of 3 years of experience as a data developer using Python, Kafka, Spark Streaming, Azure SQL Server, Cosmos DB/Mongo DB, Azure Event Hubs, Azure Data Lake Storage, Azure Search etc. Excellent technical, analytical and organizational skills. Effective written and verbal communication skills, including technical writing. Hands-on engineering lead who is curious about technology, should be able to quickly adopt to change and one who understands the technologies supporting areas such as Cloud Computing (AWS, Azure(preferred), etc.), Micro Services, Streaming Technologies, Network, Security etc Hands-on Experience for building Data Pipelines in Cloud. Experience in working with databases especially SQL server databases. Design & develop data management and data persistence solutions for application use cases leveraging relational, non-relational databases and enhancing our data processing capabilities. Experience handling un-structured data, working in a data lake environment, leveraging data streaming and developing data pipelines driven by events/queues Team player, Reliable, self-motivated, and self-disciplined individual capable of executing on multiple projects simultaneously within a fast-paced environment working with cross functional teams

We are looking for a Senior Data Engineer who is self-starter to work in a diverse and fast-paced environment that can join our Enterprise Data team. This is an individual contributor role that is responsible for designing and developing of data solutions that are strategic for the business and built on the latest technologies and patterns. This a global role that requires partnering with the broader JLLT team at the country, regional and global level by utilizing in-depth knowledge of data, infrastructure, technologies and data engineering experience. As a Data Engineer 2 at JLL Technologies, you will: Contributes to the design of information infrastructure, and data management processes to move the organization to a more sophisticated, agile and robust target state data architecture Develop systems that ingest, cleanse and normalize diverse datasets, develop data pipelines from various internal and external sources and build structure for previously unstructured data Develop good understanding of how data will flow & stored through an organization across multiple applications such as CRM, Broker & Sales tools, Finance, HR etc Design & develop data management and data persistence solutions for application use cases leveraging relational, non-relational databases and enhancing our data processing capabilities Develop POCs to influence platform architects, product managers and software engineers to validate solution proposals and migrate What we are looking for: 4+ years overall work experience and bachelor's degree in Information Science, Computer Science, Mathematics, Statistics or a quantitative discipline in science, business, or social science. Minimum of 3 years of experience as a data developer using Python, Kafka, Spark Streaming, Azure SQL Server, Cosmos DB/Mongo DB, Azure Event Hubs, Azure Data Lake Storage, Azure Search etc. Excellent technical, analytical and organizational skills. Effective written and verbal communication skills, including technical writing. Hands-on engineering lead who is curious about technology, should be able to quickly adopt to change and one who understands the technologies supporting areas such as Cloud Computing (AWS, Azure(preferred), etc.), Micro Services, Streaming Technologies, Network, Security etc Hands-on Experience for building Data Pipelines in Cloud. Experience in working with databases especially SQL server databases. Design & develop data management and data persistence solutions for application use cases leveraging relational, non-relational databases and enhancing our data processing capabilities. Experience handling un-structured data, working in a data lake environment, leveraging data streaming and developing data pipelines driven by events/queues Team player, Reliable, self-motivated, and self-disciplined individual capable of executing on multiple projects simultaneously within a fast-paced environment working with cross functional teams

JLL is a leading professional services firm that specializes in real estate and investment management. Our vision is to reimagine the world of real estate, creating rewarding opportunities and amazing spaces where people can achieve their ambitions. In doing so, we will build a better tomorrow for our clients, our people and our communities. JLL Technologies is a specialized group within JLL. At JLL Technologies, our mission is to bring technology innovation to commercial real estate. We deliver unparalleled digital advisory, implementation, and services solutions to organizations globally. Our goal is to leverage technology to increase the value and liquidity of the world's buildings, while enhancing the productivity and the happiness of those that occupy them. What this job involves JLL Technologies Enterprise Data team is a newly established central organization that oversees JLLs data strategy. We are seeking data professionals to work with our colleagues at JLL around the globe in providing solutions, developing new products, building enterprise reporting & analytics capability to reshape the business of Commercial Real Estate using the power of data and we are just getting started on that journey! We are looking for a Staff Data Engineer who is self-starter to work in a diverse and fast-paced environment that can join our Enterprise Data team. This is an individual contributor role that is responsible for designing and developing of data solutions that are strategic for the business and built on the latest technologies and patterns. This a global role that requires partnering with the broader JLLT team at the country, regional and global level by utilizing in-depth knowledge of data, infrastructure, technologies and data engineering experience. As a Senior Data Engineer at JLL Technologies, you will: Design, Architect, and Develop solutions leveraging cloud big data technology to ingest, process and analyze large, disparate data sets to exceed business requirements Develop systems that ingest, cleanse and normalize diverse datasets, develop data pipelines from various internal and external sources and build structure for previously unstructured data Interact with internal colleagues and external professionals to determine requirements, anticipate future needs, and identify areas of opportunity to drive data development Develop good understanding of how data will flow & stored through an organization across multiple applications such as CRM, Broker & Sales tools, Finance, HR etc Unify, enrich, and analyze variety of data to derive insights and opportunities Design & develop data management and data persistence solutions for application use cases leveraging relational, non-relational databases and enhancing our data processing capabilities Develop POCs to influence platform architects, product managers and software engineers to validate solution proposals and migrate Develop data lake solution to store structured and unstructured data from internal and external sources and provide technical guidance to help migrate colleagues to modern technology platform Contribute and adhere to CI/CD processes, development best practices and strengthen the discipline in Data Engineering Org Mentor other members in the team and organization and contribute to organizations growth. What we are looking for: 6+ years work experience and bachelors degree in Information Science, Computer Science, Mathematics, Statistics or a quantitative discipline in science, business, or social science. Hands-on engineer who is curious about technology, should be able to quickly adopt to change and one who understands the technologies supporting areas such as Cloud Computing (AWS, Azure(preferred), etc.), Micro Services, Streaming Technologies, Network, Security, etc. 3 or more years of active development experience as a data developer using Python-spark, Spark Streaming, Azure SQL Server, Cosmos DB/Mongo DB, Azure Event Hubs, Azure Data Lake Storage, Azure Search etc. Design & develop data management and data persistence solutions for application use cases leveraging relational, non-relational databases and enhancing our data processing capabilities Build, test and enhance data curation pipelines integration data from wide variety of sources like DBMS, File systems, APIs and streaming systems for various KPIs and metrics development with high data quality and integrity Maintain the health and monitoring of assigned data engineering capabilities that span analytic functions by triaging maintenance issues; ensure high availability of the platform; monitor workload demands; work with Infrastructure Engineering teams to maintain the data platform; serve as an SME of one or more application Team player, Reliable, self-motivated, and self-disciplined individual capable of executing on multiple projects simultaneously within a fast-paced environment working with cross functional teams 3+ years of experience working with source code control systems and Continuous Integration/Continuous Deployment tools Independent and able to manage, prioritize & lead workload What you can expect from us: Youll join an entrepreneurial, inclusive culture. One where we succeed together across the desk and around the globe. Where like-minded people work naturally together to achieve great things. Our Total Rewards program reflects our commitment to helping you achieve your ambitions in career, recognition, well-being, benefits and pay. Join us to develop your strengths and enjoy a fulfilling career full of varied experiences. Keep those ambitions in sights and imagine where JLL can take you...

We are looking for a Senior Data Engineer who is self-starter to work in a diverse and fast-paced environment that can join our Enterprise Data team. This is an individual contributor role that is responsible for designing and developing of data solutions that are strategic for the business and built on the latest technologies and patterns. This a global role that requires partnering with the broader JLLT team at the country, regional and global level by utilizing in-depth knowledge of data, infrastructure, technologies and data engineering experience. As a Data Engineer 2 at JLL Technologies, you will: Contributes to the design of information infrastructure, and data management processes to move the organization to a more sophisticated, agile and robust target state data architecture Develop systems that ingest, cleanse and normalize diverse datasets, develop data pipelines from various internal and external sources and build structure for previously unstructured data Develop good understanding of how data will flow & stored through an organization across multiple applications such as CRM, Broker & Sales tools, Finance, HR etc Design & develop data management and data persistence solutions for application use cases leveraging relational, non-relational databases and enhancing our data processing capabilities Develop POCs to influence platform architects, product managers and software engineers to validate solution proposals and migrate What we are looking for: 4+ years overall work experience and bachelor's degree in Information Science, Computer Science, Mathematics, Statistics or a quantitative discipline in science, business, or social science. Minimum of 3 years of experience as a data developer using Python, Kafka, Spark Streaming, Azure SQL Server, Cosmos DB/Mongo DB, Azure Event Hubs, Azure Data Lake Storage, Azure Search etc. Excellent technical, analytical and organizational skills. Effective written and verbal communication skills, including technical writing. Hands-on engineering lead who is curious about technology, should be able to quickly adopt to change and one who understands the technologies supporting areas such as Cloud Computing (AWS, Azure(preferred), etc.), Micro Services, Streaming Technologies, Network, Security etc Hands-on Experience for building Data Pipelines in Cloud. Experience in working with databases especially SQL server databases. Design & develop data management and data persistence solutions for application use cases leveraging relational, non-relational databases and enhancing our data processing capabilities. Experience handling un-structured data, working in a data lake environment, leveraging data streaming and developing data pipelines driven by events/queues Team player, Reliable, self-motivated, and self-disciplined individual capable of executing on multiple projects simultaneously within a fast-paced environment working with cross functional teams

What this job involves: JLL, an international real estate management company, is seeking an Data Engineer to join our JLL Technologies Team. We are seeking candidates that are self-starters to work in a diverse and fast-paced environment that can join our Enterprise Data team. We are looking for a candidate that is responsible for designing and developing of data solutions that are strategic for the business using the latest technologies Azure Databricks, Python, PySpark, SparkSQL, Azure functions, Delta Lake, Azure DevOps CI/CD. Responsibilities Design, Architect, and Develop solutions leveraging cloud big data technology to ingest, process and analyze large, disparate data sets to exceed business requirements. Design & develop data management and data persistence solutions for application use cases leveraging relational, non-relational databases and enhancing our data processing capabilities. Develop POCs to influence platform architects, product managers and software engineers to validate solution proposals and migrate. Develop data lake solution to store structured and unstructured data from internal and external sources and provide technical guidance to help migrate colleagues to modern technology platform. Contribute and adhere to CI/CD processes, development best practices and strengthen the discipline in Data Engineering Org. Develop systems that ingest, cleanse and normalize diverse datasets, develop data pipelines from various internal and external sources and build structure for previously unstructured data. Using PySpark and Spark SQL, extract, manipulate, and transform data from various sources, such as databases, data lakes, APIs, and files, to prepare it for analysis and modeling. Build and optimize ETL workflows using Azure Databricks and PySpark. This includes developing efficient data processing pipelines, data validation, error handling, and performance tuning. Perform the unit testing, system integration testing, regression testing and assist with user acceptance testing. Articulates business requirements in a technical solution that can be designed and engineered. Consults with the business to develop documentation and communication materials to ensure accurate usage and interpretation of JLL data. Implement data security best practices, including data encryption, access controls, and compliance with data protection regulations. Ensure data privacy, confidentiality, and integrity throughout the data engineering processes. Performs data analysis required to troubleshoot data related issues and assist in the resolution of data issues. Experience & Education Minimum of 4 years of experience as a data developer using Python, PySpark, Spark Sql, ETL knowledge, SQL Server, ETL Concepts. Bachelors degree in Information Science, Computer Science, Mathematics, Statistics or a quantitative discipline in science, business, or social science. Experience in Azure Cloud Platform, Databricks, Azure storage. Effective written and verbal communication skills, including technical writing. Excellent technical, analytical and organizational skills. Technical Skills & Competencies Experience handling un-structured, semi-structured data, working in a data lake environment, leveraging data streaming and developing data pipelines driven by events/queues Hands on Experience and knowledge on real time/near real time processing and ready to code Hands on Experience in PySpark, Databricks, and Spark Sql. Knowledge on json, Parquet and Other file format and work effectively with them No Sql Databases Knowledge like Hbase, Mongo, Cosmos etc. Preferred Cloud Experience on Azure or AWS Python-spark, Spark Streaming, Azure SQL Server, Cosmos DB/Mongo DB, Azure Event Hubs, Azure Data Lake Storage, Azure Search etc. Team player, Reliable, self-motivated, and self-disciplined individual capable of executing on multiple projects simultaneously within a fast-paced environment working with cross functional teams.

As a Senior Data Engineer at JLL Technologies, you will: Design, Architect, and Develop solutions leveraging cloud big data technology to ingest, process and analyze large, disparate data sets to exceed business requirements Develop systems that ingest, cleanse and normalize diverse datasets, develop data pipelines from various internal and external sources and build structure for previously unstructured data Interact with internal colleagues and external professionals to determine requirements, anticipate future needs, and identify areas of opportunity to drive data development Develop good understanding of how data will flow & stored through an organization across multiple applications such as CRM, Broker & Sales tools, Finance, HR etc Unify, enrich, and analyze variety of data to derive insights and opportunities Design & develop data management and data persistence solutions for application use cases leveraging relational, non-relational databases and enhancing our data processing capabilities Develop POCs to influence platform architects, product managers and software engineers to validate solution proposals and migrate Develop data lake solution to store structured and unstructured data from internal and external sources and provide technical guidance to help migrate colleagues to modern technology platform Contribute and adhere to CI/CD processes, development best practices and strengthen the discipline in Data Engineering Org Mentor other members in the team and organization and contribute to organizations growth. What we are looking for: 6+ years work experience and bachelors degree in Information Science, Computer Science, Mathematics, Statistics or a quantitative discipline in science, business, or social science. Hands-on engineer who is curious about technology, should be able to quickly adopt to change and one who understands the technologies supporting areas such as Cloud Computing (AWS, Azure(preferred), etc.), Micro Services, Streaming Technologies, Network, Security, etc. 3 or more years of active development experience as a data developer using Python-spark, Spark Streaming, Azure SQL Server, Cosmos DB/Mongo DB, Azure Event Hubs, Azure Data Lake Storage, Azure Search etc. Design & develop data management and data persistence solutions for application use cases leveraging relational, non-relational databases and enhancing our data processing capabilities Build, test and enhance data curation pipelines integration data from wide variety of sources like DBMS, File systems, APIs and streaming systems for various KPIs and metrics development with high data quality and integrity Maintain the health and monitoring of assigned data engineering capabilities that span analytic functions by triaging maintenance issues; ensure high availability of the platform; monitor workload demands; work with Infrastructure Engineering teams to maintain the data platform; serve as an SME of one or more application Team player, Reliable, self-motivated, and self-disciplined individual capable of executing on multiple projects simultaneously within a fast-paced environment working with cross functional teams 3+ years of experience working with source code control systems and Continuous Integration/Continuous Deployment tools Independent and able to manage, prioritize & lead workload What you can expect from us: Our Total Rewards program reflects our commitment to helping you achieve your ambitions in career, recognition, well-being, benefits and pay. Join us to develop your strengths and enjoy a fulfilling career full of varied experiences. Keep those ambitions in sights and imagine where JLL can take you...

What this job involves: JLL, an international real estate management company, is seeking an Data Engineer to join our JLL Technologies Team. We are seeking candidates that are self-starters to work in a diverse and fast-paced environment that can join our Enterprise Data team. We are looking for a candidate that is responsible for designing and developing of data solutions that are strategic for the business using the latest technologies Azure Databricks, Python, PySpark, SparkSQL, Azure functions, Delta Lake, Azure DevOps CI/CD. Responsibilities Design, Architect, and Develop solutions leveraging cloud big data technology to ingest, process and analyze large, disparate data sets to exceed business requirements. Design & develop data management and data persistence solutions for application use cases leveraging relational, non-relational databases and enhancing our data processing capabilities. Develop POCs to influence platform architects, product managers and software engineers to validate solution proposals and migrate. Develop data lake solution to store structured and unstructured data from internal and external sources and provide technical guidance to help migrate colleagues to modern technology platform. Contribute and adhere to CI/CD processes, development best practices and strengthen the discipline in Data Engineering Org. Develop systems that ingest, cleanse and normalize diverse datasets, develop data pipelines from various internal and external sources and build structure for previously unstructured data. Using PySpark and Spark SQL, extract, manipulate, and transform data from various sources, such as databases, data lakes, APIs, and files, to prepare it for analysis and modeling. Build and optimize ETL workflows using Azure Databricks and PySpark. This includes developing efficient data processing pipelines, data validation, error handling, and performance tuning. Perform the unit testing, system integration testing, regression testing and assist with user acceptance testing. Articulates business requirements in a technical solution that can be designed and engineered. Consults with the business to develop documentation and communication materials to ensure accurate usage and interpretation of JLL data. Implement data security best practices, including data encryption, access controls, and compliance with data protection regulations. Ensure data privacy, confidentiality, and integrity throughout the data engineering processes. Performs data analysis required to troubleshoot data related issues and assist in the resolution of data issues. Experience & Education Minimum of 4 years of experience as a data developer using Python, PySpark, Spark Sql, ETL knowledge, SQL Server, ETL Concepts. Bachelors degree in Information Science, Computer Science, Mathematics, Statistics or a quantitative discipline in science, business, or social science. Experience in Azure Cloud Platform, Databricks, Azure storage. Effective written and verbal communication skills, including technical writing. Excellent technical, analytical and organizational skills. Technical Skills & Competencies Experience handling un-structured, semi-structured data, working in a data lake environment, leveraging data streaming and developing data pipelines driven by events/queues Hands on Experience and knowledge on real time/near real time processing and ready to code Hands on Experience in PySpark, Databricks, and Spark Sql. Knowledge on json, Parquet and Other file format and work effectively with them No Sql Databases Knowledge like Hbase, Mongo, Cosmos etc. Preferred Cloud Experience on Azure or AWS Python-spark, Spark Streaming, Azure SQL Server, Cosmos DB/Mongo DB, Azure Event Hubs, Azure Data Lake Storage, Azure Search etc. Team player, Reliable, self-motivated, and self-disciplined individual capable of executing on multiple projects simultaneously within a fast-paced environment working with cross functional teams. Youll join an entrepreneurial, inclusive culture. One where we succeed together across the desk and around the globe. Where like-minded people work naturally together to achieve great things. Our Total Rewards program reflects our commitment to helping you achieve your ambitions in career, recognition, well-being, benefits and pay. Join us to develop your strengths and enjoy a fulfilling career full of varied experiences. Keep those ambitions in sights and imagine where JLL can take you.

What this job involves: JLL, an international real estate management company, is seeking an Data Engineer to join our JLL Technologies Team. We are seeking candidates that are self-starters to work in a diverse and fast-paced environment that can join our Enterprise Data team. We are looking for a candidate that is responsible for designing and developing of data solutions that are strategic for the business using the latest technologies Azure Databricks, Python, PySpark, SparkSQL, Azure functions, Delta Lake, Azure DevOps CI/CD. Responsibilities Design, Architect, and Develop solutions leveraging cloud big data technology to ingest, process and analyze large, disparate data sets to exceed business requirements. Design & develop data management and data persistence solutions for application use cases leveraging relational, non-relational databases and enhancing our data processing capabilities. Develop POCs to influence platform architects, product managers and software engineers to validate solution proposals and migrate. Develop data lake solution to store structured and unstructured data from internal and external sources and provide technical guidance to help migrate colleagues to modern technology platform. Contribute and adhere to CI/CD processes, development best practices and strengthen the discipline in Data Engineering Org. Develop systems that ingest, cleanse and normalize diverse datasets, develop data pipelines from various internal and external sources and build structure for previously unstructured data. Using PySpark and Spark SQL, extract, manipulate, and transform data from various sources, such as databases, data lakes, APIs, and files, to prepare it for analysis and modeling. Build and optimize ETL workflows using Azure Databricks and PySpark. This includes developing efficient data processing pipelines, data validation, error handling, and performance tuning. Perform the unit testing, system integration testing, regression testing and assist with user acceptance testing. Articulates business requirements in a technical solution that can be designed and engineered. Consults with the business to develop documentation and communication materials to ensure accurate usage and interpretation of JLL data. Implement data security best practices, including data encryption, access controls, and compliance with data protection regulations. Ensure data privacy, confidentiality, and integrity throughout the data engineering processes. Performs data analysis required to troubleshoot data related issues and assist in the resolution of data issues. Experience & Education Minimum of 4 years of experience as a data developer using Python, PySpark, Spark Sql, ETL knowledge, SQL Server, ETL Concepts. Bachelors degree in Information Science, Computer Science, Mathematics, Statistics or a quantitative discipline in science, business, or social science. Experience in Azure Cloud Platform, Databricks, Azure storage. Effective written and verbal communication skills, including technical writing. Excellent technical, analytical and organizational skills. Technical Skills & Competencies Experience handling un-structured, semi-structured data, working in a data lake environment, leveraging data streaming and developing data pipelines driven by events/queues Hands on Experience and knowledge on real time/near real time processing and ready to code Hands on Experience in PySpark, Databricks, and Spark Sql. Knowledge on json, Parquet and Other file format and work effectively with them No Sql Databases Knowledge like Hbase, Mongo, Cosmos etc. Preferred Cloud Experience on Azure or AWS Python-spark, Spark Streaming, Azure SQL Server, Cosmos DB/Mongo DB, Azure Event Hubs, Azure Data Lake Storage, Azure Search etc. Team player, Reliable, self-motivated, and self-disciplined individual capable of executing on multiple projects simultaneously within a fast-paced environment working with cross functional teams. Youll join an entrepreneurial, inclusive culture. One where we succeed together across the desk and around the globe. Where like-minded people work naturally together to achieve great things. Our Total Rewards program reflects our commitment to helping you achieve your ambitions in career, recognition, well-being, benefits and pay. Join us to develop your strengths and enjoy a fulfilling career full of varied experiences. Keep those ambitions in sights and imagine where JLL can take you.

Responsibilities * Design, develop, test, and deploy Spark Streaming applications using Scala/Spark Core. * Collaborate with cross-functional teams to gather requirements and deliver high-quality solutions. * Troubleshoot issues in Spark Streaming applications and optimize performance for improved efficiency. * Stay up-to-date with the latest developments in Spark Streaming and contribute to its evolution. Job Requirements * 3-10 years of experience in developing Spark Streaming applications using Scala/Spark Core. * Strong understanding of Spark programming concepts, including RDDs, Dataframes, Datasets, etc. * Experience with big data processing technologies such as Hadoop/Hive/Pig is a plus.

As a Senior Python Developer specializing in Python and Spark programming, you will be responsible for leveraging your thorough and hands-on knowledge to write efficient, reusable, and reliable Python code. You should possess a minimum of 5+ years of experience in Python programming and have a strong proficiency in both Python and Spark programming. Your key responsibilities will include implementing Spark Core, Spark SQL, and Spark Streaming, working with Spark within the Hadoop Ecosystem, and designing and implementing low-latency, high-availability, and performance applications. Additionally, you will lead and guide a team of junior Python developers, collaborate with other team members and stakeholders, and contribute to performance tuning, improvement, balancing, usability, and automation throughout the application development process. To excel in this role, you must have experience with data manipulation and analysis using Pandas, as well as knowledge of Polars for efficient data processing. Strong problem-solving skills, attention to detail, and the ability to work collaboratively in a team environment are essential qualities for success in this position. This is a full-time position based in either Chennai or Bangalore. If you are passionate about Python and Spark programming and are eager to take on a challenging role that involves leading a team and driving the development of high-performance applications, we encourage you to apply.,

At PwC, the focus of individuals in managed services is on a variety of outsourced solutions and supporting clients across various functions. They play a crucial role in helping organizations streamline operations, reduce costs, and enhance efficiency by managing key processes and functions on behalf of clients. Skilled in project management, technology, and process optimization, they are adept at delivering high-quality services to clients. Those in managed service management and strategy at PwC concentrate on transitioning and running services, managing delivery teams, programs, commercials, performance, and delivery risk. The role involves continuous improvement processes and optimizing managed services through tools and services. We are currently looking for a Data Engineer Offshore with expertise in Tera Data, DataStage, AWS, Databricks, SQL, Delta Live tables, Delta tables, Spark - Kafka, Spark Streaming, MQ, ETL. As a Data Engineer, you will be a valuable addition to our dynamic team. The ideal candidate should have a solid background in data engineering and proficiency in various data technologies, including Teradata, DataStage, AWS, Databricks, SQL, among others. Your responsibilities will include designing, implementing, and maintaining scalable data pipelines and systems to support our data-driven initiatives. Minimum Qualifications: - Bachelor's degree in computer science/IT or a relevant field - 3 - 5 years of experience Key Responsibilities: - Design, develop, and maintain scalable ETL pipelines using DataStage and other ETL tools. - Utilize AWS cloud services for data storage, processing, and analytics. - Leverage Databricks for data analysis, processing, and transformation, ensuring high performance and reliability. - Implement and optimize Delta Live Tables and Delta Tables for efficient data storage and querying. - Work with Apache Spark for processing large datasets, ensuring optimal performance and scalability. - Integrate Kafka and Spark Streaming for building real-time data processing applications. - Collaborate with cross-functional teams to gather requirements and deliver data solutions meeting business needs. - Ensure data quality, integrity, and security across all data systems and pipelines. - Monitor and troubleshoot data workflows to ensure smooth operations. - Document data processes, architecture designs, and technical specifications. Preferred Qualifications: - Master's degree in computer science/IT or a relevant field - Certification in AWS or Databricks Qualifications: - Bachelor's degree in Computer Science, Information Technology, or a related field - Proven experience as a Data Engineer or in a similar role - Strong proficiency in SQL and experience with relational databases like Teradata - Hands-on experience with AWS services such as S3, EMR, Redshift, and Lambda - Proficiency in using Databricks for data engineering tasks - Experience with Delta Live Tables and Delta Tables in a data engineering context - Solid understanding of Apache Spark, Kafka, and Spark Streaming - Experience with messaging systems like MQ is a plus - Strong problem-solving skills and attention to detail - Excellent communication and collaboration skills Preferred Skills: - Experience with data warehousing and big data technologies - Familiarity with data governance and data security best practices,

This position falls under the ICG TTS Operations Technology (OpsTech) Group, focusing on assisting in the implementation of a next-generation Digital Automation Platform and Imaging Workflow Technologies. The ideal Candidate should have relevant experience in managing development teams within the distributed systems Eco-System and must exhibit strong teamwork skills. The candidate is expected to possess superior technical knowledge of current programming languages, technologies, and leading-edge development tools. The primary objective of this role is to contribute to applications, systems analysis, and programming activities. As a Lead Spark Scala Engineer, the candidate should have hands-on knowledge of SPARK, Py-Spark, Scala, Java, and RDBMS like MS-SQL/Oracle. Familiarity with CI/CD tools such as LightSpeed and uDeploy is also required. Key Responsibilities include: - Development & Optimization: Develop, test, and deploy production-grade Spark applications in Scala, ensuring optimal performance, scalability, and resource utilization. - Technical Leadership: Provide guidance to a team of data engineers, promoting a culture of technical excellence and collaboration. - Code Review & Best Practices: Conduct thorough code reviews, establish coding standards, and enforce best practices for Spark Scala development, data governance, and data quality. - Performance Tuning: Identify and resolve performance bottlenecks in Spark applications through advanced tuning techniques. - Deep Spark Expertise: Profound understanding of Spark's architecture, execution model, and optimization techniques. - Scala Proficiency: Expert-level proficiency in Scala programming, including functional programming paradigms and object-oriented design. - Big Data Ecosystem: Strong hands-on experience with the broader Hadoop ecosystem and related big data technologies. - Database Knowledge: Solid understanding of relational databases and NoSQL databases. - Communication: Excellent communication, interpersonal, and leadership skills to convey complex technical concepts effectively. - Problem-Solving: Exceptional analytical and problem-solving abilities with meticulous attention to detail. Education Requirement: - Bachelors degree/University degree or equivalent experience This position is a full-time role falling under the Technology Job Family Group and Applications Development Job Family. The most relevant skills include those mentioned in the requirements section, while additional complementary skills can be found above or by contacting the recruiter.,

ZS is a place where passion changes lives. As a management consulting and technology firm focused on improving life and how we live it, we transform ideas into impact by bringing together data, science, technology and human ingenuity to deliver better outcomes for all. Here you'll work side-by-side with a powerful collective of thinkers and experts shaping life-changing solutions for patients, caregivers and consumers, worldwide. ZSers drive impact by bringing a client-first mentality to each and every engagement. We partner collaboratively with our clients to develop custom solutions and technology products that create value and deliver company results across critical areas of their business. Bring your curiosity for learning, bold ideas, courage and passion to drive life-changing impact to ZS. ZS's Technology group focuses on scalable strategies, assets and accelerators that deliver enterprise-wide transformation via cutting-edge technology. We leverage digital and technology solutions to optimize business processes, enhance decision-making, and drive innovation. Our services include Digital and Technology advisory, Product and Platform development, and Data, Analytics and AI implementation. What you'll do: - Work with business stakeholders to understand their business needs. - Create data pipelines that extract, transform, and load (ETL) from various sources into a usable format in a Data warehouse. - Clean, filter, and validate data to ensure it meets quality and format standards. - Develop data model objects (tables, views) to transform the data into unified format for downstream consumption. - Expert in monitoring, controlling, configuring, and maintaining processes in cloud data platform. - Optimize data pipelines and data storage for performance and efficiency. - Participate in code reviews and provide meaningful feedback to other team members. - Provide technical support and troubleshoot issue(s). What you'll bring: - Bachelor's degree in Computer Science, Information Technology, or a related field, or equivalent work experience. - Experience working in the AWS cloud platform. - Data engineer with expertise in developing big data and data warehouse platforms. - Experience working with structured and semi-structured data. - Expertise in developing big data solutions, ETL/ELT pipelines for data ingestion, data transformation, and optimization techniques. - Experience working directly with technical and business teams. - Able to create technical documentation. - Excellent problem-solving and analytical skills. - Strong communication and collaboration abilities. - AWS (Big Data services) - S3, Glue, Athena, EMR - Programming - Python, Spark, SQL, Mulesoft, Talend, Dbt - Data warehouse - ETL, Redshift / Snowflake Additional Skills: - Experience in data modeling. - Certified in AWS platform for Data Engineer skills. - Experience with ITSM processes/tools such as ServiceNow, Jira - Understanding of Spark, Hive, Kafka, Kinesis, Spark Streaming, and Airflow Perks & Benefits: ZS offers a comprehensive total rewards package including health and well-being, financial planning, annual leave, personal growth and professional development. Our robust skills development programs, multiple career progression options and internal mobility paths and collaborative culture empowers you to thrive as an individual and global team member. Travel: Travel is a requirement at ZS for client-facing ZSers; business needs of your project and client are the priority. While some projects may be local, all client-facing ZSers should be prepared to travel as needed. Travel provides opportunities to strengthen client relationships, gain diverse experiences, and enhance professional growth by working in different environments and cultures. Considering applying At ZS, we honor the visible and invisible elements of our identities, personal experiences, and belief systemsthe ones that comprise us as individuals, shape who we are, and make us unique. We believe your personal interests, identities, and desire to learn are integral to your success here. We are committed to building a team that reflects a broad variety of backgrounds, perspectives, and experiences. Learn more about our inclusion and belonging efforts and the networks ZS supports to assist our ZSers in cultivating community spaces and obtaining the resources they need to thrive. If you're eager to grow, contribute, and bring your unique self to our work, we encourage you to apply. ZS is an equal opportunity employer and is committed to providing equal employment and advancement opportunities without regard to any class protected by applicable law. To complete your application: Candidates must possess or be able to obtain work authorization for their intended country of employment. An online application, including a full set of transcripts (official or unofficial), is required to be considered. NO AGENCY CALLS, PLEASE. Find Out More At: www.zs.com,

As an exceptionally skilled individual, you will be part of a dedicated team at TNS, collaborating daily to contribute to the success of the organization. If you are driven by excellence in both professional and personal aspects, this is the place for you! The role entails being a Java and/or Scala developer with expertise in Big Data tools and frameworks. You should have 8 to 12 years of proven experience in Java and/or Scala development. Your responsibilities will include hands-on work with prominent Big Data tools like Hadoop, Spark, Map Reduce, Hive, and Impala. Additionally, you should possess a deep understanding of streaming technologies such as Kafka and/or Spark Streaming. Strong familiarity with design, development, and utilization of NoSQL databases like HBase, Druid, and Solr is crucial. Experience in working with public cloud platforms like AWS and Azure is also expected. To be considered for this position, you should hold a BS/B.E./B.Tech degree in Computer Science or a related field. Desirable qualifications for this role include proficiency in object-oriented analysis, design patterns using Java/J2EE technologies, and expertise in Restful Web Services and data modeling. Familiarity with build and development tools like Maven, Gradle, and Jenkins, as well as experience with test frameworks such as JUnit and Mockito, are advantageous. Knowledge of Spring Framework, MVC architectures, and ORM frameworks like Hibernate would be a bonus. If you have a genuine passion for technology, a thirst for personal development, and a desire for growth opportunities, we invite you to discover the exciting world of TNS!,

As a Databricks AWS/Azure/GCP Architect at Koantek based in Mumbai, you will play a crucial role in building secure and highly scalable big data solutions that drive tangible, data-driven outcomes while emphasizing simplicity and operational efficiency. Collaborating with teammates, product teams, and cross-functional project teams, you will lead the adoption and integration of the Databricks Lakehouse Platform into the enterprise ecosystem and AWS/Azure/GCP architecture. Your responsibilities will include implementing securely architected big data solutions that are operationally reliable, performant, and aligned with strategic initiatives. Your expertise should include an expert-level knowledge of data frameworks, data lakes, and open-source projects like Apache Spark, MLflow, and Delta Lake. You should possess hands-on coding experience in Spark/Scala, Python, or Pyspark. An in-depth understanding of Spark Architecture, including Spark Core, Spark SQL, Data Frames, Spark Streaming, RDD caching, and Spark MLib, is essential for this role. Experience in IoT/event-driven/microservices in the cloud, familiarity with private and public cloud architectures, and extensive hands-on experience in implementing data migration and data processing using AWS/Azure/GCP services are key requirements. With over 9 years of consulting experience and a minimum of 7 years in data engineering, data platform, and analytics, you should have a proven track record of delivering projects with hands-on development experience on Databricks. Your knowledge of at least one cloud platform (AWS, Azure, or GCP) is mandatory, along with deep experience in distributed computing with Spark and familiarity with Spark runtime internals. Additionally, you should be familiar with CI/CD for production deployments, optimization for performance and scalability, and have completed data engineering professional certification and required classes. If you are a results-driven professional with a passion for architecting cutting-edge big data solutions and have the desired skill set, we encourage you to apply for this exciting opportunity.,

Overview This position is for Lead Data Engineer in the Commercial Data as a Service group. In this position you will enjoy being responsible for helping define and maintain the data systems key to delivering successful outcomes for our customers. You will be hands on and work closely to guide a team of Data Engineers in the associated data maintenance, integrations, enhancements, loads and transformation processes for the organization. This key individual will work closely with Data Architects to design and implement solutions and insure successful implementations. Role Leads initiatives to build and maintain database technologies, environments, and applications, seeking opportunities for improvements and efficiencies Architects internal data solutions as part of the full stack to include data modelling, integration with file based as well as event driven upstream systems Writes SQL statement procedures to optimize SQL execution and query development Effectively utilizes various tools such as Spark (Scala, Python), Nifi, Spark streaming, Informatica for data ETL, Manages the deployment of data solutions that are optimally standardized and database updates to meet project deliverables Leads database security posture, which includes proactively identifying security risks and implementing both risk mitigation plans and control functions Oversees the resolution of chronic complex problems to prevent future data performance issues Supports process improvement efforts to identify and test opportunities for automation and/or reduction in time to deployment Responsible for complex design (in conjunction with Data Architects), development, and performance and system testing, and provides functional guidance, advice to experienced engineers Mentors junior staff by providing training to develop technical skills and capabilities across the team All about you Experience developing a specialization in a particular functional area (e.g., modeling, data loads, transformations, replication, performance tuning, logical and physical database design, performance and troubleshooting, data replication, backup and recovery, and data security) leveraging Apache Spark, Nifi, Databricks, Snowflake, Informatica, streaming solutions. Experience leading a major work stream or multiple smaller work streams for a large domain initiative, often providing technical guidance and advice to project team members Experience creating deliverables within the global database technology domains and sub-domains, supporting cross-functional leaders in the technical community to derive new solutions Experience supporting automation and/or cloud delivery effort; may perform financial and cost analysis Experience in database architecture or other relevant IT experience Experience in leading business system application and database architecture design, influencing technology direction in range of breadth of IT areas.

Overview This position is for Manager Data Engineering in the Commercial Data as a Service group. In this position you will enjoy being responsible for helping define and maintain the data systems key to delivering successful outcomes for our customers. will lead a software development team focused on our core services someone who combines deep technical experience with great people management skills and can help elevate our teams and our approach to engineering. We are building a culture that fosters innovation, built on a foundation of inclusion and diversity. We are looking for a candidate that has proven experience in coding, architecture leadership, and leading engineers in a fast-paced environment. This position is a mix of hands-on-keyboard development, process engineering, and management responsibilities with a strong emphasis on technical leadership. Role Leads initiatives to build and maintain database technologies, environments, and applications, seeking opportunities for improvements and efficiencies Architects internal data solutions as part of the full stack to include data modelling, integration with file based as well as event driven upstream systems Writes SQL statement procedures to optimize SQL execution and query development Effectively utilizes various tools such as Spark (Scala, Python), Spark Streaming, Nifi, Informatica for data ETL Grow and inspire a team of engineers through a combination of coaching, mentoring, feedback and career development Cultivate a healthy, collaborative engineering culture thats in line with our values and DevOps principles. Make recommendations on future technical architecture for software, platform, integrations and tools Work collaboratively to help define, execute and implement product roadmaps All about you Experience developing a specialization in a particular functional area (e.g., modeling, data loads, transformations, replication, performance tuning, logical and physical database design, performance and troubleshooting, data replication, backup and recovery, and data security) leveraging Apache Spark, Nifi, Databricks, Snowflake, Informatica, streaming solutions. Experience leading a major work stream or multiple smaller work streams for a large domain initiative, often providing technical guidance and advice to project team members Experience creating deliverables within the global database technology domains and sub-domains, supporting cross-functional leaders in the technical community to derive new solutions Experience supporting automation and/or cloud delivery effort; may perform financial and cost analysis Experience in database architecture or other relevant IT experience Experience in leading business system application and database architecture design, influencing technology direction in range of breadth of IT areas

Job Title: Big Data Developer Project Support & Mentorship Location: Mumbai Employment Type: Full-Time/Contract Work Arrangement: Onsite Mumbai COE Position Overview: We are seeking a skilled Big Data Developer to join our growing delivery team, with a dual focus on hands-on project support and mentoring junior engineers. This role is ideal for a developer who not only thrives in a technical, fast-paced environment but is also passionate about coaching and developing the next generation of talent. You will work on live client projects, provide technical support, contribute to solution delivery, and serve as a go-to technical mentor for less experienced team members. Key Responsibilities: Perform hands-on Big Data development work, including coding, testing, troubleshooting, and deploying solutions. Support ongoing client projects, addressing technical challenges and ensuring smooth delivery. Collaborate with junior engineers to guide them on coding standards, best practices, debugging, and project execution. Review code and provide feedback to junior engineers to maintain high quality and scalable solutions. Assist in designing and implementing solutions using Hadoop, Spark, Hive, HDFS, and Kafka. Lead by example in object-oriented development, particularly using Scala and Java. Translate complex requirements into clear, actionable technical tasks for the team. Contribute to the development of ETL processes for integrating data from various sources. Document technical approaches, best practices, and workflows for knowledge sharing within the team. Required Skills and Qualifications: 8+ years of professional experience in Big Data development and engineering. Strong hands-on expertise with Hadoop, Hive, HDFS, Apache Spark, and Kafka. Solid object-oriented development experience with Scala and Java. Strong SQL skills with experience working with large data sets. Practical experience designing, installing, configuring, and supporting Big Data clusters. Deep understanding of ETL processes and data integration strategies. Proven experience mentoring or supporting junior engineers in a team setting. Strong problem-solving, troubleshooting, and analytical skills. Excellent communication and interpersonal skills. Preferred Qualifications: Professional certifications in Big Data technologies (Cloudera, Databricks, AWS Big Data Specialty, etc.). Experience with cloud Big Data platforms (AWS EMR, Azure HDInsight, or GCP Dataproc). Exposure to Agile or DevOps practices in Big Data project environments. What We Offer: Opportunity to work on challenging, high-impact Big Data projects. Leadership role in shaping and mentoring the next generation of engineers. Supportive and collaborative team culture. Flexible working environment Competitive compensation and professional growth opportunities.

Join a team recognized for leadership, innovation, and diversity. You must have a Ph.D. or Master's degree in Computer Science, Engineering, Applied Mathematics, or a related field. Exposure to the Finance domain and use cases in a larger global enterprise setting is required. Additionally, you should have a minimum of 8 to 10 years of Data Science prototyping experience using machine learning techniques and algorithms such as k-means, k-NN, Naive Bayes, SVM, and Decision Trees, with proficiency in Python and/or R tool-stack. Moreover, a minimum of 8 to 10 years of Machine Learning experience of physical systems is expected. You should also possess a minimum of 4 to 6 years of experience with distributed storage and compute tools like Hive and Spark. Furthermore, a minimum of 8 to 10 years of experience in deep learning frameworks like PyTorch and Keras is essential. Experience in designing, building models, and deploying models to Cloud Platforms such as Azure and Databricks is a plus. Having a working knowledge and experience of implementing Generative AI in the industry and keeping up with the latest developments in the field of Artificial Intelligence is highly valued. A research mindset with a problem-solving attitude is a MUST. Experience with Natural Language Processing models, Streaming Analytics (i.e., Spark Streaming), Recurrent Neural Network architectures, Image Analytics, SQL, and working with remote and global teams is beneficial. Knowledge of Corporate Finance or Financial Analytics is advantageous. The ideal candidate should be results-driven with a positive can-do attitude. If you meet these requirements, this role could be a great fit for you. Please note the following additional information: JOB ID: HRD251841 Category: Engineering Location: Devarabisanahalli Village, KR Varturhobli, East Taluk - Phase I, Bangalore, KARNATAKA, 560103, India Exempt Early Career (ALL),