Job
Description
Your Role
5+ yrs of total IT experience and 4+ yrs of relevant experience as Azure Data Engineer and having good hands-on experience on Azure Data Factory, Azure Databricks, Python, Azure Synapse, Azure Data Lake, PySpark, SQL Server and Power BI. Knowledge in design of Extract, Transform and Load environment using Data bricks, Spark Azure Data Factory (ADF). Setting Up Microsoft Azure with Data bricks, Data bricks Workspace for Business Analytics, Manage Clusters in Data bricks. Hands on experience Data extraction (extract, Schemas, corrupt record handling and parallelized code), transformations and loads (user defined functions, join optimizations) and Production (optimize and automate Extract, Transform and Load). Copy of data from on-premises data (SQL Server) to Azure Data Lake Store (ADLS) using Azure Data Factory (ADF V2). Data migrated using Azure Data Factory and created new pipelines, data flows. Hands on experience on Migrating SQL database to Azure Data Lake, Azure datalake Analytics, Azure SQL Database, Data Bricks and Azure SQL Data warehouse and controlling and granting database access. Experience in working with Data bricks notebooks for developing and executing transformation logic and data processing tasks. Extensive experience in designing ETL pipelines using Data bricks, Apache Spark, and Azure Data Factory (ADF). Works in the area of Software Engineering, which encompasses the development, maintenance and optimization of software solutions/applications.1. Applies scientific methods to analyse and solve software engineering problems.2. He/she is responsible for the development and application of software engineering practice and knowledge, in research, design, development and maintenance.3. His/her work requires the exercise of original thought and judgement and the ability to supervise the technical and administrative work of other software engineers.4. The software engineer builds skills and expertise of his/her software engineering discipline to reach standard software engineer skills expectations for the applicable role, as defined in Professional Communities.5. The software engineer collaborates and acts as team player with other software engineers and stakeholders.
Your Profile
Proficient in data extraction, schema management, and handling corrupt records with parallelized processing. Expertise in transformations and data loads, including optimization of joins and implementing user-defined functions (UDFs). Created Data bricks notebooks for data transformation, using various transformation processes such as join, merge, lookup, filter, remove duplicates, and aggregation. Designed ADF pipelines to execute Databricks notebooks and carry out complex transformations. Carryout transformations and creating notebooks using various transformation processes like Join, Merge, Lookup, remove Duplicates, Filter and Aggregator. Utilized spark transformations in Databricks for data cleaning, filter in and transformation tasks. Experience in troubleshooting SQL issues by using SQL Tools, execution plans, Trace, statistics, index tuning wizard.
What will you love working at Capgemini
You will have the opportunity to learn on one of the industry''s largest digital learning platforms, with access to 250,000+ courses and numerous certifications. We’re committed to ensure that people of all backgrounds feel encouraged and have a sense of belonging at Capgemini. You are valued for who you are, and you can bring your original self to work. At Capgemini, you can work on cutting-edge projects in tech and engineering with industry leaders or create solutions to overcome societal and environmental challenges. Capgemini office campuses in India are green and run on 100% renewable electricity. We have installed Solar plants across India locations and ‘Battery Energy Storage Solution’ (BESS) in the Noida and Mumbai campuses. You will have chance to make a difference everyday.
