2404 Azure Databricks Jobs - Page 38

Job Description: AWS/Azure/SAP ETL Data Modelling Data Integration & Ingestion Data Manipulation and Processing GITHUB, Action , Azure DevOps Data factory, Databricks, SQL DB, Synapse, Stream Analytics, Glue, Airflow, Kinesis, Redshift, SonarQube, PyTest

As a part of the data and analytics engineering team at PwC, your focus will be on utilizing advanced technologies and techniques to create robust data solutions for clients. Your role will involve transforming raw data into actionable insights, enabling informed decision-making, and contributing to business growth. Specifically in data engineering at PwC, you will be responsible for designing and constructing data infrastructure and systems that facilitate efficient data processing and analysis. This will include the development and implementation of data pipelines, data integration, and data transformation solutions. At PwC - AC, we are seeking an Azure Manager specializing in Data & AI, w...

Choosing Capgemini means choosing a company where you will be empowered to shape your career in the way you'd like, where you'll be supported and inspired by a collaborative community of colleagues around the world, and where you'll be able to reimagine what's possible. Join us and help the world's leading organizations unlock the value of technology and build a more sustainable, more inclusive world. Design, develop, and maintain data pipelines using Azure Data Factory, Azure Databricks, and Azure Synapse. Implement ETL solutions to integrate data from various sources into Azure Data Lake and Data Warehouse. Hands-on experience with SQL, Python, PySpark for data processing. Expertise in bui...

The role of a Databricks PySpark IICS professional at Infosys involves working as part of the consulting team to address customer issues, develop innovative solutions, and ensure successful deployment to achieve client satisfaction. Your responsibilities will include contributing to proposal development, solution design, product configuration, and conducting pilots and demonstrations. Additionally, you will lead small projects, provide high-quality solutions, and support organizational initiatives. The technical requirements for this role include expertise in Data On Cloud Platform, specifically Azure Data Lake (ADL). You should also possess the ability to develop strategies that drive innov...

As a technical leader at Advisor360, you will play a crucial role in the planning and implementation of mid- to large-scale projects. Your expertise will be instrumental in architecting and implementing cloud-based data solutions using your strong technical skills. You will be responsible for validating requirements, performing business and technical analysis, designing cloud-native applications, and writing optimized PySpark-based data pipelines. Ensuring compliance with coding standards and best practices for AI-assisted code generation will be a part of your responsibilities. You will utilize GenAI tools like Cursor, Claude, and other LLMs to decompose complex requirements and auto-genera...

As a Senior Associate in the Data, Analytics & Specialist Managed Service Tower at PwC, with 6 to 10 years of experience, you will be part of a team of problem solvers addressing complex business issues from strategy to execution. Your responsibilities at this management level include utilizing feedback and reflection for personal development, being flexible in stretch opportunities, and demonstrating critical thinking skills to solve unstructured problems. You will also be involved in ticket quality reviews, project status reporting, and ensuring adherence to SLAs and incident, change, and problem management processes. Seeking diverse opportunities, communicating effectively, upholding ethi...

As an Associate Data Engineer/Analyst at EY, you will leverage your expertise to create, develop, and maintain scalable big data processing pipelines in distributed computing environments. Your role will involve designing and implementing interactive Power BI reports and dashboards tailored to the specific needs of various business units. Collaborating with cross-functional teams, you will gather data requirements and design effective data solutions. Your responsibilities will also include executing data ingestion, processing, and transformation workflows to support analytical and machine learning applications. To excel in this role, you should have over 2 years of experience as a data analy...

You should have at least 5 years of experience working as a Data Engineer. Your expertise should include a strong background in Azure Cloud services and proficiency in tools such as Azure Databricks, PySpark, and Delta Lake. It is essential to have solid experience in Python and FastAPI for API development, as well as familiarity with Azure Functions for serverless API deployments. Experience in managing ETL pipelines using Apache Airflow is also required. Hands-on experience with databases like PostgreSQL and MongoDB is necessary. Strong SQL skills and the ability to work with large datasets are key for this role.,

As a member of the data engineering team at PepsiCo, you will play a crucial role in developing and overseeing data product build & operations. Your primary responsibility will be to drive a strong vision for how data engineering can proactively create a positive impact on the business. Working alongside a team of data engineers, you will build data pipelines, rest data on the PepsiCo Data Lake, and facilitate exploration and access for analytics, visualization, machine learning, and product development efforts across the company. Your contributions will directly impact the design, architecture, and implementation of PepsiCo's flagship data products in areas such as revenue management, suppl...

The role of Senior Data Engineer at GSPANN involves designing, developing, and optimizing scalable data solutions, utilizing expertise in Azure Data Factory, Azure Databricks, PySpark, Delta Tables, and advanced data modeling. The position also demands proficiency in performance optimization, API integrations, DevOps practices, and data governance. You will be responsible for designing, developing, and orchestrating scalable data pipelines using Azure Data Factory (ADF). Additionally, you will build and manage Apache Spark clusters, create notebooks, and execute jobs in Azure Databricks. Ingesting, organizing, and transforming data within the Microsoft Fabric ecosystem using OneLake will als...

As a Data Analytics Lead, you will be responsible for overseeing the design, development, and implementation of data analysis solutions to meet business needs. Working closely with business stakeholders and the Aviation Subject Matter Expert (SME), you will define data requirements, project scope, and deliverables. Your role will involve driving the design and development of analytics data models and data warehouse designs, as well as developing and maintaining data quality standards and procedures. In this position, you will manage and prioritize data analysis projects to ensure timely completion. You will also be expected to identify opportunities to improve data analysis processes and too...

You are the leading global provider of managed services, cybersecurity, and business transformation for mid-market financial services organizations across the globe. From your unmatched range of services, you provide stability, security, and improved business performance, freeing your clients from technology concerns and enabling them to focus on running their businesses. More than 1,000 customers worldwide with over $3 trillion of assets under management put their trust in you. You believe that success is driven by passion and purpose. Your passion for technology is only surpassed by your commitment to empowering your employees around the world. You have an exciting Opportunity for a Cloud ...

At EY, you'll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture, and technology to become the best version of you. And we're counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all. As a Staff - Data Engineer at EY, your responsibilities include designing and developing software components using various tools such as PySpark, Sqoop, Flume, Azure Databricks, and more. You will perform detailed analysis and effectively interact with onshore/offshore team members, ensuring all deliverables conform to the highest quality st...

As a Data Scientist at the Global Design Technology Studio within Gensler, you will play a crucial role in driving innovative solutions that connect data and outcomes from designs to practitioners, projects, and clients. You will be part of a dynamic team that fosters creativity and collaboration to identify trends, insights, and opportunities for improvement through the analysis, synthesis, and assessment of large datasets. Your responsibilities will include collaborating with the Data team to develop, train, and deploy ML classification algorithms using data from design authoring tools, exploring trends and defining insights aligned with business needs, developing data pipelines in MS Azur...

You have a great opportunity to join as a Data Software Engineer with 5-12 years of experience in Big Data & Data related technology. We are looking for candidates with an expert level understanding of distributed computing principles and hands-on experience in Apache Spark along with proficiency in Python. You should also have experience with technologies like Hadoop, Map Reduce, HDFS, Sqoop, Apache Storm, Spark-Streaming, Kafka, Hive, Impala, and integration of data from various sources such as RDBMS, ERP, and Files. Additionally, knowledge of NoSQL databases, ETL techniques, SQL queries, joins, stored procedures, relational schemas, and performance tuning of Spark Jobs is required. Moreov...

You should have 6 to 9 years of experience in Data Engineering with a focus on Azure Databricks, Azure Data Factory, Pyspark, and SQL. The work location for this role can be Bengaluru, Pune, Mumbai, Noida, or Gurugram with a notice period of 0 to 15 days. This job is in a Hybrid mode. As a Data Engineer in this role, your key responsibilities will include developing scalable data pipelines using Azure Data Factory (ADF), Databricks, PySpark, and Delta Lake to support ML and AI workloads. You will be optimizing and transforming large datasets for feature engineering, model training, and real-time AI inference. Building and maintaining lakehouse architecture using Azure Data Lake Storage (ADLS...

At EY, you'll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And we're counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all. Our Technology team builds innovative digital solutions rapidly and at scale to deliver the next generation of Financial and Non-Financial services across the globe. The Position is a senior technical, hands-on delivery role, requiring the knowledge of data engineering, cloud infrastructure and platform engineering, platform operations, and p...

As an Azure Databricks Developer with fluency in Japanese language, you will be responsible for leveraging your technical expertise to contribute to ongoing operations and development projects within our technology services client's team. With a focus on Azure Databricks and Pyspark, you will play a key role in enhancing data processing efficiency and optimizing solutions using various tools and technologies. Your role will involve working closely with Azure Databricks, PySpark Dataframes, RDD, DAG, Partitioning, SparkSQL, and Clustering to drive innovation and problem-solving. Your hands-on experience in Azure Databricks and Power BI tech stack will be beneficial in ensuring seamless operat...

At Capgemini Invent, we believe difference drives change. As inventive transformation consultants, we blend our strategic, creative, and scientific capabilities, collaborating closely with clients to deliver cutting-edge solutions. Join us to drive transformation tailored to our client's challenges of today and tomorrow, informed and validated by science and data, superpowered by creativity and design, all underpinned by technology created with purpose. Your role involves having IT experience with a minimum of 5+ years in creating data warehouses, data lakes, ETL/ELT, data pipelines on cloud. You should have experience in data pipeline implementation with cloud providers such as AWS, Azure, ...

As a Data Software Engineer at KG Invicta Services Pvt Ltd, you will leverage your 5-12 years of experience in Big Data & Data-related technologies to drive impactful solutions. Your expertise in distributed computing principles and Apache Spark, coupled with hands-on programming skills in Python, will be instrumental in designing and implementing efficient Big Data solutions. You will demonstrate proficiency in a variety of tools and technologies including Hadoop v2, Map Reduce, HDFS, Sqoop, Apache Storm, Spark-Streaming, Kafka, RabbitMQ, Hive, Impala, and NoSQL databases such as HBase, Cassandra, and MongoDB. Your ability to integrate data from diverse sources like RDBMS, ERP, and files, a...

As a Senior Data Engineer, you will be responsible for designing and building robust, scalable data pipelines with deep expertise in PySpark, SQL, and cloud data platforms such as Azure Databricks, with exposure to AWS or GCP environments. You will collaborate with data architects, analysts, and business stakeholders to deliver high-quality data solutions that drive insights and business value. Your key responsibilities will include designing, building, and optimizing scalable ETL/ELT pipelines using PySpark and SQL, developing data workflows on Azure Databricks, integrating across cloud platforms (AWS or GCP), implementing and maintaining data models using Kimball/star schema or similar dim...

As a Data Engineer, you will be responsible for designing and building efficient data pipelines using Azure Databricks (PySpark). Your role will involve implementing business logic for data transformation and enrichment at scale and managing and optimizing Delta Lake storage solutions. You will also be involved in developing REST APIs using FastAPI to expose processed data and deploying APIs on Azure Functions for scalable and serverless data access. Additionally, you will develop and manage Airflow DAGs to orchestrate ETL processes, ingest and process data from various internal and external sources on a scheduled basis. Database management will be a key part of your responsibilities, includ...

At PwC, as a member of the audit and assurance team, your focus will be on providing independent and objective assessments of financial statements, internal controls, and other assurable information. Your role will involve enhancing the credibility and reliability of information for various stakeholders. You will be responsible for evaluating compliance with regulations, assessing governance and risk management processes, and related controls. Additionally, as part of the data, analytics, and technology solutions team, you will assist clients in developing solutions that build trust, drive improvement, and detect, monitor, and predict risk. Your work at PwC will require utilizing advanced an...

You are seeking a skilled Azure Databricks Architect with 12 to 17 years of experience in Python, SQL. As an Azure Databricks Architect at our company, you will be responsible for data architecture, data engineering, and analytics. You should have at least 5 years of hands-on experience with Azure Databricks, Apache Spark, and Delta Lake. Your proficiency in Azure Data Lake, Azure Synapse, Azure Data Factory, and Azure SQL is essential. Expertise in Python, Scala, and SQL for data processing, along with a deep understanding of data modeling, ETL/ELT processes, and distributed computing, are key requirements. Experience with CI/CD pipelines and DevOps practices in data engineering is also exp...

As a skilled professional in data handling and processing, you will demonstrate expertise in SQL Server and query optimization, ensuring efficient application data design and process management. Your knowledge in data modeling will be extensive, supported by hands-on experience with Azure Data Factory, Azure Synapse Analytics, and Microsoft Fabric. Additionally, your familiarity with Azure Databricks will be utilized in your work. In the realm of data warehouse development, your proficiency in SSIS (SQL Server Integration Services) and SSAS (SQL Server Analysis Services) will be essential. You will excel in ETL processes, encompassing data extraction, transformation, and loading, including d...