43 Azure Hdinsight Jobs - Page 2

Project Role : Application Lead Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills : Microsoft BOT Framework Good to have skills : NA Minimum 7.5 year(s) of experience is required Educational Qualification : BE/B-Tech/M-Tech Project Role :Application Lead Project Role Description :Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have Skills :Microsoft BOT FrameworkGood to Have Skills :No Technology SpecializationJob Requirements :Key Responsibilities :aWork closely with the client teams to define and architect the solution for our clients Estimating the components required to provide a comprehensive AI solution that meets and exceeds the clients expectations delivering tangible business value to our clientsbCandidate is expected to deliver cognitive applications that solve/augment business issues using leading AI technology frameworkscDirect and influence client Bot/Virtual Agent architecture, solutioning and development Technical Experience :a7-8 years of experience and a thorough understanding of Azure PaaS landscapea2-3 years of experience in Microsoft AI/Bot FrameworkbGood to have - Knowledge experience in Azure Machine Learning, AI, Azure HDInsight Professional Attributes :AShould have strong problem-solving and Analytical skills BGood Communication skills CAbility to work independently with little supervision or as a team DShould be able to work and deliver under tight timelines ECandidate should have good analytical skills Educational Qualification:BE/B-Tech/M-TechAdditional Info : Qualifications BE/B-Tech/M-Tech

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark and Hive, Hbase or other NoSQL databases on Azure Cloud Data Platform or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developing streaming pipelines Experience to work with Hadoop / Azure eco system components to implement scalable solutions to meet the ever-increasing data volumes, using big data/cloud technologies Apache Spark, Kafka, any Cloud computing etc Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Minimum 4+ years of experience in Big Data technologies with extensive data engineering experience in Spark / Python or Scala; Minimum 3 years of experience on Cloud Data Platforms on Azure; Experience in DataBricks / Azure HDInsight / Azure Data Factory, Synapse, SQL Server DB Good to excellent SQL skills Exposure to streaming solutions and message brokers like Kafka technologies Preferred technical and professional experience Certification in Azure and Data Bricks or Cloudera Spark Certified developers

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark and Hive, Hbase or other NoSQL databases on Azure Cloud Data Platform or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developing streaming pipelines Experience to work with Hadoop / Azure eco system components to implement scalable solutions to meet the ever-increasing data volumes, using big data/cloud technologies Apache Spark, Kafka, any Cloud computing etc Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Minimum 4+ years of experience in Big Data technologies with extensive data engineering experience in Spark / Python or Scala; Minimum 3 years of experience on Cloud Data Platforms on Azure; Experience in DataBricks / Azure HDInsight / Azure Data Factory, Synapse, SQL Server DB Good to excellent SQL skills Exposure to streaming solutions and message brokers like Kafka technologies Preferred technical and professional experience Certification in Azure and Data Bricks or Cloudera Spark Certified developers

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark and Hive, Hbase or other NoSQL databases on Azure Cloud Data Platform or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developing streaming pipelines Experience to work with Hadoop / Azure eco system components to implement scalable solutions to meet the ever-increasing data volumes, using big data/cloud technologies Apache Spark, Kafka, any Cloud computing etc Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Minimum 4+ years of experience in Big Data technologies with extensive data engineering experience in Spark / Python or Scala; Minimum 3 years of experience on Cloud Data Platforms on Azure; Experience in DataBricks / Azure HDInsight / Azure Data Factory, Synapse, SQL Server DB Good to excellent SQL skills Exposure to streaming solutions and message brokers like Kafka technologies Preferred technical and professional experience Certification in Azure and Data Bricks or Cloudera Spark Certified developers

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark and Hive, Hbase or other NoSQL databases on Azure Cloud Data Platform or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developing streaming pipelines Experience to work with Hadoop / Azure eco system components to implement scalable solutions to meet the ever-increasing data volumes, using big data/cloud technologies Apache Spark, Kafka, any Cloud computing etc Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Minimum 4+ years of experience in Big Data technologies with extensive data engineering experience in Spark / Python or Scala; Minimum 3 years of experience on Cloud Data Platforms on Azure; Experience in DataBricks / Azure HDInsight / Azure Data Factory, Synapse, SQL Server DB Good to excellent SQL skills Exposure to streaming solutions and message brokers like Kafka technologies Preferred technical and professional experience Certification in Azure and Data Bricks or Cloudera Spark Certified developers

Description Primary Skills: C#, Azure API development Deep Knowledge and proven hands-on experience with Azure cloud technologies Minimum of 4+ years experience with Microsoft technologies; Experience on C3 is a must. Minimum of 2+ years experience with cloud technologies Strong understanding of cloud architecture and development Experience on cloud native development including event-driven architecture with microservices and REST API. Experience with objectoriented languages, including .Net, .Net Core and C# Experience using development tools, including Visual Studio, Azure DevOps, Git Experience on Azure cloud development using the following Azure native resources App Service Logic Apps Service Bus Blob Storage Cosmos DB Azure SQL Key Vault Application Insights Experience with XSLT Experience with branching/merging of code, testing code thoroughly Good communication and reporting skills both written and verbal with technical and business groups Self-motivated with the ability to work as part of a team including vendors Named Job Posting? (if Yes - needs to be approved by SCSC) Additional Details Global Grade C Level To Be Defined Named Job Posting? (if Yes - needs to be approved by SCSC) No Remote work possibility No Global Role Family To be defined Local Role Name To be defined Local Skills .NET C#;Azure API Apps Languages RequiredENGLISH Role Rarity To Be Defined

Description Primary Skills: C#, Azure API development Deep Knowledge and proven hands-on experience with Azure cloud technologies Minimum of 4+ years experience with Microsoft technologies; Experience on C3 is a must. Minimum of 2+ years experience with cloud technologies Strong understanding of cloud architecture and development Experience on cloud native development including event-driven architecture with microservices and REST API. Experience with objectoriented languages, including .Net, .Net Core and C# Experience using development tools, including Visual Studio, Azure DevOps, Git Experience on Azure cloud development using the following Azure native resources App Service Logic Apps Service Bus Blob Storage Cosmos DB Azure SQL Key Vault Application Insights Experience with XSLT Experience with branching/merging of code, testing code thoroughly Good communication and reporting skills both written and verbal with technical and business groups Self-motivated with the ability to work as part of a team including vendors Named Job Posting? (if Yes - needs to be approved by SCSC) Additional Details Global Grade C Level To Be Defined Named Job Posting? (if Yes - needs to be approved by SCSC) No Remote work possibility No Global Role Family To be defined Local Role Name To be defined Local Skills .NET C#;Azure API Apps Languages RequiredENGLISH Role Rarity To Be Defined

Description Primary Skills: C#, Azure API development Deep Knowledge and proven hands-on experience with Azure cloud technologies Minimum of 4+ years experience with Microsoft technologies; Experience on C3 is a must. Minimum of 2+ years experience with cloud technologies Strong understanding of cloud architecture and development Experience on cloud native development including event-driven architecture with microservices and REST API. Experience with objectoriented languages, including .Net, .Net Core and C# Experience using development tools, including Visual Studio, Azure DevOps, Git Experience on Azure cloud development using the following Azure native resources App Service Logic Apps Service Bus Blob Storage Cosmos DB Azure SQL Key Vault Application Insights Experience with XSLT Experience with branching/merging of code, testing code thoroughly Good communication and reporting skills both written and verbal with technical and business groups Self-motivated with the ability to work as part of a team including vendors Named Job Posting? (if Yes - needs to be approved by SCSC) Additional Details Global Grade B Level To Be Defined Named Job Posting? (if Yes - needs to be approved by SCSC) No Remote work possibility No Global Role Family To be defined Local Role Name To be defined Local Skills .NET C#;Azure API Apps Languages RequiredENGLISH Role Rarity To Be Defined

Project Role :Application Lead Project Role Description :Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills :Microsoft BOT Framework Good to have skills :NA Minimum 10+ year(s) of experience is required Educational Qualification :aMust have:BE/Btech/MCAbGood to have:ME/Mtech Project Role :Application Lead Project Role Description :Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have Skills :Microsoft BOT Framework, SSINON SSI:Good to Have Skills :SSI:No Technology Specialization NON SSI :Job Requirements :Key Responsibilities :aWork closely with the client teams to define and architect the solution for our clients Estimating the components required to provide a comprehensive AI solution that meets and exceeds the clients expectations delivering tangible business value to our clientsbCandidate is expected to deliver cognitive applications that solve/augment business issues using leading AI technology frameworkscDirect and influence client Bot/Virtual Agent architecture, solutioning and development Technical Experience :a7-8 years of experience and a thorough understanding of Azure PaaS landscapea2-3 years of experience in Microsoft AI/Bot FrameworkbGood to have - Knowledge experience in Azure Machine Learning, AI, Azure HDInsight Professional Attributes :AShould have strong problem-solving and Analytical skills BGood Communication skills CAbility to work independently with little supervision or as a team DShould be able to work and deliver under tight timelines ECandidate should have good analytical skills Educational Qualification:aMust have:BE/Btech/MCAbGood to have:ME/MtechAdditional Info : Qualifications aMust haveBE/Btech/MCAbGood to haveME/Mtech

Overview of the Company: Jio Platforms Ltd. is a revolutionary Indian multinational tech company, often referred to as India's biggest startup, headquartered in Mumbai. Launched in 2019, it's the powerhouse behind Jio, India's largest mobile network with over 400 million users. But Jio Platforms is more than just telecom. It's a comprehensive digital ecosystem, developing cutting-edge solutions across media, entertainment, and enterprise services through popular brands like JioMart, JioFiber, and JioSaavn. Join us at Jio Platforms and be part of a fast-paced, dynamic environment at the forefront of India's digital transformation. Collaborate with brilliant minds to develop next-gen solutions that empower millions and revolutionize industries. Team Overview: The Data Platforms Team is the launchpad for a data-driven future, empowering the Reliance Group of Companies. We're a passionate group of experts architecting an enterprise-scale data mesh to unlock the power of big data, generative AI, and ML modelling across various domains. We don't just manage data we transform it into intelligent actions that fuel strategic decision-making. Imagine crafting a platform that automates data flow, fuels intelligent insights, and empowers the organization that's what we do. Join our collaborative and innovative team, and be a part of shaping the future of data for India's biggest digital revolution! About the role. Title : Lead Data Engineer Location: Mumbai Responsibilities: End-to-End Data Pipeline Development: Design, build, optimize, and maintain robust data pipelines across cloud, on-premises, or hybrid environments, ensuring performance, scalability, and seamless data flow. Reusable Components & Frameworks: Develop reusable data pipeline components and contribute to the team's data pipeline framework evolution. Data Architecture & Solutions: Contribute to data architecture design, applying data modelling, storage, and retrieval expertise. Data Governance & Automation: Champion data integrity, security, and efficiency through metadata management, automation, and data governance best practices. Collaborative Problem Solving: Partner with stakeholders, data teams, and engineers to define requirements, troubleshoot, optimize, and deliver data-driven insights. Mentorship & Knowledge Transfer: Guide and mentor junior data engineers, fostering knowledge sharing and professional growth. Qualification Details: Education: Bachelor's degree or higher in Computer Science, Data Science, Engineering, or a related technical field. Core Programming: Excellent command of a primary data engineering language (Scala, Python, or Java) with a strong foundation in OOPS and functional programming concepts. Big Data Technologies: Hands-on experience with data processing frameworks (e.g., Hadoop, Spark, Apache Hive, NiFi, Ozone, Kudu), ideally including streaming technologies (Kafka, Spark Streaming, Flink, etc.). Database Expertise: Excellent querying skills (SQL) and strong understanding of relational databases (e.g., MySQL, PostgreSQL). Experience with NoSQL databases (e.g., MongoDB, Cassandra) is a plus. End-to-End Pipelines: Demonstrated experience in implementing, optimizing, and maintaining complete data pipelines, integrating varied sources and sinks including streaming real-time data. Cloud Expertise: Knowledge of Cloud Technologies like Azure HDInsights, Synapse, EventHub and GCP DataProc, Dataflow, BigQuery. CI/CD Expertise: Experience with CI/CD methodologies and tools, including strong Linux and shell scripting skills for automation. Desired Skills & Attributes: Problem-Solving & Troubleshooting: Proven ability to analyze and solve complex data problems, troubleshoot data pipeline issues effectively. Communication & Collaboration: Excellent communication skills, both written and verbal, with the ability to collaborate across teams (data scientists, engineers, stakeholders). Continuous Learning & Adaptability: A demonstrated passion for staying up-to-date with emerging data technologies and a willingness to adapt to new tools.

Project Role : Data Platform Engineer Project Role Description : Assists with the data platform blueprint and design, encompassing the relevant data platform components. Collaborates with the Integration Architects and Data Architects to ensure cohesive integration between systems and data models. Must have skills : Microsoft Azure Data Services Good to have skills : Microsoft Azure Analytics Services, Synapse Minimum 7.5 year(s) of experience is required Educational Qualification : Min 15 years of education Summary :As a Data Platform Engineer, you will be responsible for assisting with the data platform blueprint and design, encompassing the relevant data platform components. Your typical day will involve collaborating with Integration Architects and Data Architects to ensure cohesive integration between systems and data models using Microsoft Azure Data Services. Roles & Responsibilities: Design and develop data platform components using Microsoft Azure Data Services. Collaborate with Integration Architects and Data Architects to ensure cohesive integration between systems and data models. Assist with the development of data platform blueprints and designs. Ensure data platform components are scalable, secure, and performant. Provide technical guidance and mentorship to junior team members. Professional & Technical Skills: Must To Have Skills:Experience with Microsoft Azure Data Services. Good To Have Skills:Experience with Microsoft Azure Analytics Services and Synapse. Strong understanding of data platform components and architecture. Experience with designing and developing scalable, secure, and performant data platforms. Experience with data modeling and database design. Experience with ETL/ELT processes and tools. Additional Information: The candidate should have a minimum of 7.5 years of experience in Microsoft Azure Data Services. The ideal candidate will possess a strong educational background in computer science or a related field, along with a proven track record of delivering impactful data-driven solutions. This position is based at our Hyderabad office. Qualification Min 15 years of education

Project Role : Application Lead Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills : Microsoft Azure Analytics Services Good to have skills : Microsoft Azure Data Services Minimum 5 year(s) of experience is required Educational Qualification : 1.Must have:BE/BTech/MCA2.Good to have:ME/MTech Project Role :Application Lead Project Role Description :Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have Skills :Microsoft Azure Analytics ServicesGood to Have Skills :Microsoft Azure Data ServicesJob Requirements :Key Responsibilities :A.Function as the Lead Data Architect for a small, simple project/proposal or as a team lead for medium/large sized project or proposalB.Discuss specific Big data architecture and related issues with client architect/team (in area of expertise)C.Analyze and assess the impact of the requirements on the data and its lifecycleD.Lead Big data architecture and design medium-big Cloud based, Big Data and Analytical Solutions using Lambda architecture.E.Breadth of experience in various client scenarios and situations Technical Experience :A.Strong experience in Azure is preferred with hands-on experience in two or more of these skills :Azure Synapse Analytics, Azure HDInsight, Azure Databricks with PySpark / Scala / SparkSQL, Azure Analysis ServicesB.Experience in one or more Real-time/Streaming technologies including:Azure Stream Analytics, Azure Data Explorer, Azure Time Series Insights, etc.C.Experience in handling medium to large Big Data implementationsD.Candidate must have 5-6 years of IT experience and around 3 years of Big data experience (design + build) Professional Attributes :1.Should be able to drive the technology design meetings, propose technology design and architecture 2.Should have excellent client communication skills3.Should have good analytical and problem-solving skills Educational Qualification:1.Must have:BE/BTech/MCA2.Good to have:ME/MTechAdditional Info : Qualifications 1.Must have:BE/BTech/MCA2.Good to have:ME/MTech

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark and Hive, Hbase or other NoSQL databases on Azure Cloud Data Platform or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developing streaming pipelines Experience to work with Hadoop / Azure eco system components to implement scalable solutions to meet the ever-increasing data volumes, using big data/cloud technologies Apache Spark, Kafka, any Cloud computing etc Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Minimum 4+ years of experience in Big Data technologies with extensive data engineering experience in Spark / Python or Scala; Minimum 3 years of experience on Cloud Data Platforms on Azure; Experience in DataBricks / Azure HDInsight / Azure Data Factory, Synapse, SQL Server DB Good to excellent SQL skills Preferred technical and professional experience Certification in Azure and Data Bricks or Cloudera Spark Certified developers Experience in DataBricks / Azure HDInsight / Azure Data Factory, Synapse, SQL Server DB Knowledge or experience of Snowflake will be an added advantage

D: Locations: Any Location (Preferred must be Hyderabad or Bangalore) Exp: 6-12 Yrs CTC: 33 LPA Skills: Azure secure landing zone, Azure API, Logic apps, Azure SQL Job Code: TechM110415 Work Mode: Hybrid No.of Positions : 1 JD1: Senior Azure Engineer Expert in Azure Secure Landing Zone, Design area ,Architecture ? Experience with Azure Secure Landing Zones (SLZ). ? Know how on Azure lending zone Architecture ,design area. ? Knowledge of Azure API ? Knowledge of Logic Apps ? Knowledge of Azure SQL ? Good PowerShell command (preferrable but not a must) ? Strong experience in Migration of application to Azure Secure Landing Zone Good communication, documentation skills and working with team

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark and Hive, Hbase or other NoSQL databases on Azure Cloud Data Platform or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developing streaming pipelines Experience to work with Hadoop / Azure eco system components to implement scalable solutions to meet the ever-increasing data volumes, using big data/cloud technologies Apache Spark, Kafka, any Cloud computing etc Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Total 5-8 years of experience in Data Management (DW, DL, Data Platform, Lakehouse) and Data Engineering skills Minimum 4+ years of experience in Big Data technologies with extensive data engineering experience in Spark / Python or Scala; Minimum 3 years of experience on Cloud Data Platforms on Azure; Experience in DataBricks / Azure HDInsight / Azure Data Factory, Synapse, SQL Server DB Good to excellent SQL skills Preferred technical and professional experience Certification in Azure and Data Bricks or Cloudera Spark Certified developers Experience in DataBricks / Azure HDInsight / Azure Data Factory, Synapse, SQL Server DB Knowledge or experience of Snowflake will be an added advantage

Responsibilities As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark and Hive, Hbase or other NoSQL databases on Azure Cloud Data Platform or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developing streaming pipelines Experience to work with Hadoop / Azure eco system components to implement scalable solutions to meet the ever-increasing data volumes, using big data/cloud technologies Apache Spark, Kafka, any Cloud computing etc Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Total 6 - 7+ years of experience in Data Management (DW, DL, Data Platform, Lakehouse) and Data Engineering skills Minimum 4+ years of experience in Big Data technologies with extensive data engineering experience in Spark / Python or Scala; Minimum 3 years of experience on Cloud Data Platforms on Azure; Experience in DataBricks / Azure HDInsight / Azure Data Factory, Synapse, SQL Server DB Good to excellent SQL skills Exposure to streaming solutions and message brokers like Kafka technologies Experience Unix / Linux Commands and basic work experience in Shell Scripting Demonstrated ability in designing and building for data ingestion, data cleansing, ETL, loading data layers and exposing data for consumers. Experience of using DevOps and working in Agile environments Collaborative environments that use agile methodologies to encourage creative design thinking and find innovative ways to develop with cutting edge technologies. Proven interpersonal skills while contributing to team effort by accomplishing related results as required Preferred technical and professional experience Certification in Azure and Data Bricks or Cloudera Spark Certified developers

Responsibilities As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark and Hive, Hbase or other NoSQL databases on Azure Cloud Data Platform or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developing streaming pipelines Experience to work with Hadoop / Azure eco system components to implement scalable solutions to meet the ever-increasing data volumes, using big data/cloud technologies Apache Spark, Kafka, any Cloud computing etc Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Total 5 - 7+ years of experience in Data Management (DW, DL, Data Platform, Lakehouse) and Data Engineering skills Minimum 4+ years of experience in Big Data technologies with extensive data engineering experience in Spark / Python or Scala. Minimum 3 years of experience on Cloud Data Platforms on Azure; Experience in DataBricks / Azure HDInsight / Azure Data Factory, Synapse, SQL Server DB Exposure to streaming solutions and message brokers like Kafka technologies Experience Unix / Linux Commands and basic work experience in Shell Scripting Preferred technical and professional experience Certification in Azure and Data Bricks or Cloudera Spark Certified developers

Responsibilities As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark and Hive, Hbase or other NoSQL databases on Azure Cloud Data Platform or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developing streaming pipelines Experience to work with Hadoop / Azure eco system components to implement scalable solutions to meet the ever-increasing data volumes, using big data/cloud technologies Apache Spark, Kafka, any Cloud computing etc Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Minimum 4+ years of experience in Big Data technologies with extensive data engineering experience in Spark / Python or Scala; Minimum 3 years of experience on Cloud Data Platforms on Azure; Experience in DataBricks / Azure HDInsight / Azure Data Factory, Synapse, SQL Server DB Good to excellent SQL skills Exposure to streaming solutions and message brokers like Kafka technologies Preferred technical and professional experience Certification in Azure and Data Bricks or Cloudera Spark Certified developers