2404 Azure Databricks Jobs - Page 48

Data engineers: Help optimize the workflows handling millions of images with a minimum cost. •Every bit of optimization when scaled for million images we have lot of money to save. •You must be extremely good at programming for cloud workflows with a strong eye on optimization considering the scale Comptencies : First thing, belief that “right cost is everything for you” = optimized but fast on cloud •An attitude to make data flow through the workflows on cloud most efficiently, •Right cost can even mean – find the best of storage solution to retrieve cheap and fast •Extremely good at software development to not brag about language proficiency but rather I can learn ‘assembly language’ also ...

As a Microsoft Fabric Professional at YASH Technologies, you will be responsible for working with cutting-edge technologies to bring about real positive changes in an increasingly virtual world. You will have the opportunity to contribute to business transformation by leveraging your experience in Azure Fabric, Azure Data factory, Azure Databricks, Azure Synapse, Azure Storage Services, Azure SQL, ETL, Azure Cosmos DB, Event HUB, Azure Data Catalog, Azure Functions, and Azure Purview. With 5-8 years of experience in Microsoft Cloud solutions, you will be involved in creating pipelines, datasets, dataflows, Integration runtimes, and monitoring Pipelines. Your role will also entail extracting,...

You are a skilled Architect specializing in AIOps & MLOps Operations, responsible for supporting and enhancing the automation, scalability, and reliability of AI/ML operations across the enterprise. Your role involves deploying AI/ML models, ensuring continuous monitoring, and implementing self-healing automation to enhance system performance, minimize downtime, and improve decision-making with real-time AI-driven insights. Supporting and maintaining AIOps and MLOps programs is a key responsibility, ensuring alignment with business objectives, data governance standards, and enterprise data strategy. You will assist in implementing real-time data observability, monitoring, and automation fram...

As an Engineer, IT Data at American Airlines, you will be part of a diverse, high-performing team dedicated to technical excellence. Your primary focus will be on delivering unrivaled digital products that drive a more reliable and profitable airline. The Data Domain you will work in encompasses managing and leveraging data as a strategic asset, including data management, storage, integration, and governance. This domain also involves Machine Learning, AI, Data Science, and Business Intelligence. In this role, you will collaborate closely with source data application teams and product owners to design, implement, and support analytics solutions that provide insights to make better decisions....

At PwC, our audit and assurance team focuses on providing independent and objective assessments of financial statements, internal controls, and other assurable information to enhance credibility and reliability with stakeholders. We evaluate compliance with regulations, assess governance and risk management processes, and related controls. Those in data, analytics, and technology solutions assist clients in developing solutions that build trust, drive improvement, and detect, monitor, and predict risk. Your work involves using advanced analytics, data wrangling technology, and automation tools to leverage data and establish processes for clients to make efficient decisions based on accurate ...

YASH Technologies is a leading technology integrator specializing in helping clients reimagine operating models, enhance competitiveness, optimize costs, foster exceptional stakeholder experiences, and drive business transformation. At YASH, you will be part of a team of highly skilled professionals working with cutting-edge technologies. Our mission is centered around making real positive changes in an increasingly virtual world, transcending generational gaps and future disruptions. We are currently seeking Azure Databricks Professionals with 6-8 years of experience. The ideal candidate should possess hands-on experience with Azure services and Databricks, along with a strong understanding...

As a Data Engineer specializing in supply chain applications at NovintiX in Coimbatore, India, you will play a crucial role in enhancing our Supply Chain Analytics team. Your primary focus will be on developing intelligent data solutions that drive real-world logistics, procurement, and demand planning. Your responsibilities will include: - Creating and optimizing scalable data pipelines for inventory, shipping, and procurement data - Integrating data from ERP, PLM, and external sources through the development of APIs - Designing, building, and maintaining enterprise-grade data warehouses and data lakes while ensuring data quality, integrity, and security - Collaborating with stakeholders to...

As a Data Engineer at Gyansys with 6 to 8 years of experience, your primary focus will be on utilizing Azure Databricks, PySpark, and SQL to design and implement Hadoop big data solutions that align with business requirements and project timelines. Your responsibilities will include coding, testing, and documenting data systems to develop scalable applications for data analytics. Collaborating with other Big Data developers is essential to ensure the consistency of all data solutions. You will partner with the business community to gather requirements, identify training needs, and conduct user training sessions. Researching technologies and product enhancements will be part of your role to d...

We are seeking a skilled data engineering professional with over 5 years of experience in designing and implementing end-to-end ETL solutions using Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Azure Databricks, and SSIS. The ideal candidate should have a strong proficiency in SQL, REST API integration, and automation through CI/CD pipelines using Azure DevOps and Git. Additionally, a solid understanding of maintaining and optimizing data pipelines, warehouses, and reporting within the Microsoft SQL stack is required. As a Senior Data Engineer, you will be responsible for designing and implementing comprehensive data solutions using Microsoft Azure Data Factory (ADF) and Azure Da...

As a Data Engineer at Srijan, a Material company, you will play a crucial role in designing and developing scalable data pipelines within Microsoft Fabric. Your primary responsibilities will include optimizing data pipelines, collaborating with cross-functional teams, and ensuring documentation and knowledge sharing. You will work closely with the Data Architecture team to implement scalable and governed data architectures within OneLake and Microsoft Fabric's unified compute and storage platform. Your expertise in Microsoft Fabric will be utilized to build robust pipelines using both batch and real-time processing techniques, integrating with Azure Data Factory for seamless data movement. C...

As an Engineer, IT Data at American Airlines, you will be part of a diverse and high-performing team dedicated to technical excellence. Your main focus will be on delivering unrivaled digital products that drive a more reliable and profitable airline. The Data Domain you will be working in refers to the area within Information Technology that focuses on managing and leveraging data as a strategic asset. This includes data management, storage, integration, and governance, leaning into Machine Learning, AI, Data Science, and Business Intelligence. In this role, you will work closely with source data application teams and product owners to design, implement, and support analytics solutions that...

Hiring for Azure Developer with experience range 4 and above Mandatory Skills: Azure, auzre datafactory, azure datalake Education: BE/B.Tech/MCA/M.Tech/MSc./MS

You will be responsible for performing comprehensive testing of ETL pipelines to ensure data accuracy and completeness across different systems. This includes validating Data Warehouse objects such as fact and dimension tables, designing and executing test cases and test plans for data extraction, transformation, and loading processes, as well as conducting regression testing to validate enhancements with no breakage of existing data flows. You will also work with SQL to write complex queries for data verification and backend testing, and test data processing workflows in Azure Data Factory and Databricks environments. Collaboration with developers, data engineers, and business analysts to u...

Job Overview: We are looking for a Microsoft Fabric Data Engineer to join our dynamic team. In this role, you will be responsible for designing and delivering cutting-edge data solutions using Microsoft Fabric and related technologies. This role will be responsible for designing, developing, and optimizing data solutions using Microsoft Fabric. This role involves working with technical teams, ensuring best practices, and driving innovation in data engineering and hands on development of data pipelines. Key Responsibilities: Work with large datasets to solve complex analytical problems. Conduct end-to-end data analyses, including collection, processing, and visualization. Collaborate with cro...

Experience: - 5 to 15 years of hands-on experience in Azure data bricks, Pyspark and cloud. Knowledge and skills: App Dev Language: .Azure data bricks, Pyspark and cloud. Data Management: Database (SQL Server DB), APIs (Kong, APIC, APIM) Development & Delivery Methods: Agile (Scaled Agile Framework) DevOps and CI/CD: Containers (Azure Kubernetes), CI/CD (Azure DevOps, Git), Scheduling Tools (Azure Scheduler) Development Tools & Platforms: IDE (GitHub Copilot, VSCODE, IntelliJ), Cloud Platform (Microsoft Azure) Security and Monitoring: Secure Coding (Veracode), Writing and executing automated tests -- Using Java and Java script, Selenium, and using test automation framework"

Project Role : Data Platform Engineer Project Role Description : Assists with the data platform blueprint and design, encompassing the relevant data platform components. Collaborates with the Integration Architects and Data Architects to ensure cohesive integration between systems and data models. Must have skills : Databricks Unified Data Analytics Platform Good to have skills : Microsoft Power Business Intelligence (BI), Microsoft Azure DatabricksMinimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Platform Engineer, you will assist with the data platform blueprint and design, encompassing the relevant data platform compone...

Project Role : Data Platform Engineer Project Role Description : Assists with the data platform blueprint and design, encompassing the relevant data platform components. Collaborates with the Integration Architects and Data Architects to ensure cohesive integration between systems and data models. Must have skills : Databricks Unified Data Analytics Platform Good to have skills : Microsoft Power Business Intelligence (BI), Microsoft Azure DatabricksMinimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Platform Engineer, you will assist with the data platform blueprint and design, encompassing the relevant data platform compone...

Primary Responsibilities • Create and maintain data storage solutions including Azure SQL Database, Azure Data Lake, and Azure Blob Storage. • Design, implement, and maintain data pipelines for data ingestion, processing, and transformation in Azure Create data models for analytics purposes • Utilizing Azure Data Factory or comparable technologies, create and maintain ETL (Extract, Transform, Load) operations • Use Azure Data Factory and Databricks to assemble large, complex data sets • Implementing data validation and cleansing procedures will ensure the quality, integrity, and dependability of the data. • Ensure data security and compliance • Collaborate with data engineers, and other stak...

Azure Data Factory: - Develop Azure Data Factory Objects - ADF pipeline, configuration, parameters, variables, Integration services runtime - Hands-on knowledge of ADF activities(such as Copy, SP, lkp etc) and DataFlows - ADF data Ingestion and Integration with other services Azure Databricks: - Experience in Big Data components such as Kafka, Spark SQL, Dataframes, HIVE DB etc implemented using Azure Data Bricks would be preferred. - Azure Databricks integration with other services - Read and write data in Azure Databricks - Best practices in Azure Databricks Synapse Analytics: - Import data into Azure Synapse Analytics with and without using PolyBase - Implement a Data Warehouse with Azure...

About Company Founded in 2011, ReNew, is one of the largest renewable energy companies globally, with a leadership position in India. Listed on Nasdaq under the ticker RNW, ReNew develops, builds, owns, and operates utility-scale wind energy projects, utility-scale solar energy projects, utility-scale firm power projects, and distributed solar energy projects. In addition to being a major independent power producer in India, ReNew is evolving to become an end-to-end decarbonization partner providing solutions in a just and inclusive manner in the areas of clean energy, green hydrogen, value-added energy offerings through digitalisation, storage, and carbon markets that increasingly are integ...

Skills & ExperienceStrong hands on experience with Dynamics 365 understands data structures and business processes without relying on others.Experience working with Azure Data Lake and Synapse Link knows how D365 data is replicated and how to work with it.Skilled in Power BI can build datasets, dashboards, and reports independently.Experience in D365 development to pick up technical work beyond reporting.Comfortable working directly with business users to gather requirements and translate them into working solutions. Behaviours & ApproachSelf sufficient doesn t need to rely on other D365 consultants to understand tables, relationships, or data sources.Proactive and comfortable working with n...

Key Responsibilities: Team Leadership: Lead and mentor a team of Azure Data Engineers, providing technical guidance and support. Foster a collaborative and innovative team environment. Conduct regular performance reviews and set development goals for team members. Organize training sessions to enhance team skills and technical capabilities. Azure Data Platform: Design, implement, and optimize scalable data solutions using Azure data services such as Azure Databricks, Azure Data Factory, Azure SQL Database, and Azure Synapse Analytics. Ensure data engineering best practices and data governance are followed. Stay up-to-date with Azure data technologies and recommend improvements to enhance dat...

Job Title:Fullstack Architect/Application ArchitectExperience10-15 YearsLocation:Remote : A NASDAQ-listed company, known for its prominent position in the food and beverage sector, is seeking A Fullstack Architect with expertise in Java Spring Boot. In this role, you will understand problem statements and conduct architecture Assessments specifically in the Manufacturing and Warehousing industry. This is an exciting opportunity for an experienced architect eager to work with a leading multinational Corporation that has a significant impact on millions of lives worldwide. Required Skills: 10+ Years of Experience Minimum of 5 years of experience as an Architect. 5+ Years of experience in Java ...

Project Role : Application Designer Project Role Description : Assist in defining requirements and designing applications to meet business process and application requirements. Must have skills : Microsoft Azure Databricks Good to have skills : Microsoft Azure ArchitectureMinimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Designer, you will assist in defining requirements and designing applications to meet business process and application requirements. A typical day involves collaborating with cross-functional teams to gather insights, analyzing user needs, and translating them into functional specifications. You wi...

Project Role : Application Designer Project Role Description : Assist in defining requirements and designing applications to meet business process and application requirements. Must have skills : Microsoft Azure Databricks Good to have skills : Microsoft Azure ArchitectureMinimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Designer, you will assist in defining requirements and designing applications to meet business process and application requirements. Your typical day will involve collaborating with various stakeholders to gather and analyze requirements, creating application designs that align with business object...