Senior Data Engineer

Job Description

Company Name: Logicserve Digital Consultancy Services Pvt. Ltd. Location: Surat Experience: 4+ Years Working Days: 6 Days a Week Key Responsibilities: Requirement Gathering: Collaborate with stakeholders to gather and document data requirements. Translate business needs into technical specifications for data solutions. Data Pipeline Development: Design, implement, and maintain data pipelines using Microsoft Fabric. Create and manage ETL processes, ensuring efficient data flow. SSIS Integration: Develop and maintain ETL processes using SQL Server Integration Services (SSIS). Optimize SSIS packages for performance and reliability. CDC Pipelines: Implement Change Data Capture (CDC) pipelines to track and manage data changes. Ensure timely and accurate data updates across systems. PySpark Proficiency: Utilize PySpark for data transformation and processing tasks. Develop data processing scripts to handle large datasets efficiently. Collaboration: Work closely with data analysts, data scientists, and other teams to ensure data integrity. Participate in cross-functional projects to enhance data accessibility and usability. Required Skills: Technical Proficiency: Strong experience with Azure Data Factory, Azure Synapse, and Microsoft Fabric. Proficient in PySpark for data processing and transformation. Solid understanding of SQL Server and experience in writing complex queries. Data Modeling: Knowledge of data modeling concepts and best practices. Experience in designing data architectures that support business needs. Cloud Services: Familiarity with cloud computing concepts and services, particularly within the Azure ecosystem. Understanding of data storage solutions such as Azure Data Lake and Blob Storage. Problem-Solving: Strong analytical skills to troubleshoot data-related issues. Ability to optimize data workflows for performance and efficiency. Preferred Qualifications: Experience with data warehousing concepts and tools. Knowledge of data governance and compliance standards. Familiarity with version control systems and CI/CD practices. Show more Show less