Data Engineer

As a Data Engineer at our company, you will be responsible for designing, developing, and maintaining data pipelines and infrastructure to ensure accurate, timely, and accessible data for our organization's growth and data-driven decision-making. Key Responsibilities: - Design, develop, and implement data pipelines using Azure Data Factory and Databricks for data ingestion, transformation, and movement. - Develop and optimize ETL processes to facilitate efficient data flow and transformation. - Maintain Azure Data Lake solutions for efficient storage and retrieval of large datasets. - Collaborate with Azure Synapse Analytics to build scalable data warehousing solutions for advanced analytics and reporting. - Integrate various data sources into MS-Fabric, ensuring data consistency, quality, and accessibility. - Optimize data processing workflows and storage solutions to enhance performance and reduce costs. - Manage and optimize SQL and NoSQL databases for high-performance queries and data storage. - Implement data quality checks and monitoring processes to ensure data accuracy and consistency. - Work closely with data scientists, analysts, and stakeholders to understand data requirements and deliver actionable insights. - Create and maintain comprehensive documentation for data processes, pipelines, infrastructure, architecture, and best practices. - Identify and resolve issues in data pipelines, data lakes, and warehousing solutions, providing timely support and maintenance. Qualifications: - 2-4 years of experience in data engineering or a related field. Technical Skills: - Proficiency in Azure Data Factory, Azure Synapse Analytics, Databricks, and Azure Data Lake. - Experience with Microsoft Fabric is a plus. - Strong SQL skills and familiarity with data warehousing concepts (DWH). - Knowledge of data modeling, ETL processes, and data integration. - Hands-on experience with ETL tools and frameworks like Apache Airflow and Talend. - Familiarity with big data technologies such as Hadoop and Spark. - Experience with cloud platforms like AWS, Azure, Google Cloud, and associated data services. - Familiarity with data visualization tools like Power BI and programming languages such as Python, Java, or Scala. - Experience with schema design and dimensional data modeling. Analytical Skills: - Strong problem-solving abilities and attention to detail. Communication: - Excellent verbal and written communication skills to explain technical concepts to non-technical stakeholders. Education: - Bachelor's degree in computer science, engineering, mathematics, or a related field. Advanced degrees or certifications are a plus. Interested candidates can share their CV at sulabh.tailang@celebaltech.com.,