Job
Description
Key Skills: Azure Synapse, Azure Databricks, Azure, Azure Devops, Azure AI, Azure Api, Azure AD, PLSQL Roles and Responsibilities: Design and Develop Data Pipelines: Build and maintain scalable data pipelines using Azure Data Factory, ensuring efficient and reliable data movement and transformation. File-Based Data Management: Handle data ingestion and management from various file sources, including CSV, JSON, and Parquet formats, ensuring data accuracy and consistency. ETL Implementation: Implement and optimize ETL (Extract, Transform, Load) processes using tools such as Azure Data Factory, Azure SQL Database, and Azure Synapse Analytics. Cloud Storage Management: Work with Azure Data Lake Storage to manage and utilize cloud storage solutions, ensuring data is securely stored and easily accessible. Automation with Data Factory: Leverage Azure Data Factory's automation capabilities to schedule and monitor data workflows, ensuring timely execution and error-free operations. Performance Monitoring: Continuously monitor and optimize data pipeline performance, troubleshoot issues, and implement best practices to enhance efficiency. Team Collaboration: Collaborate with Technical Architects, Business Analysts, and other engineers to build scalable and reliable end-to-end data solutions for reporting and analytics. DevOps Framework: Defining and implementing DevOps framework using CI/CD pipelines. SQL Development: Write efficient, clean, and well-documented SQL queries for data extraction, manipulation, and analysis. SQL Performance Optimization: Optimize performance of SQL-based queries, stored procedures, and jobs in Azure environments. Data Security & Compliance: Implement data security best practices and ensure compliance with data privacy regulations (HIPAA, etc.). Technical Leadership: Provide technical leadership and mentoring to junior engineers and team members. Technology Adoption: Stay current with emerging Azure technologies and trends, recommending improvements to existing systems and solutions. Skills Required: Strong expertise in Data Analytics for analyzing and interpreting large datasets. Proficiency in Azure Boards-GitHub for managing project tasks and source code version control. Extensive experience with Azure Data Factory for building and managing scalable data pipelines. In-depth knowledge of Azure Data Lake for managing cloud storage solutions and data access. Hands-on experience with Azure Synapse for data integration and analytics solutions. Proficient in Azure DevOps for implementing CI/CD pipelines and automating deployments. Education: Bachelor's or Master's degree in Computer Science, Information Systems, Data Engineering, or a related technical field
