Senior Data Engineer with MS Fabric

Microsoft Fabric & Azure Expertise

• Design and implement comprehensive data solutions using Microsoft Fabric, including

  Data Pipelines

  ,

  Lakehouses

  , and

  Real-Time Analytics

  .
• Build and manage scalable

  OLAP architectures

  , including data layer design, dimensional modeling, and semantic models.
• Develop and optimize

  data pipelines

  within Fabric Workspaces for efficient ETL/ELT processing.
• Lead implementation of

  data governance practices

  , including

  IAM policies

  ,

  RBAC

  ,

  RLS

  ,

  data masking

  , and

  encryption

  .
• Manage and optimize

  Fabric resources

  , including Fabric SKUs, OneLake, and Workspace configurations.
• Set up and manage

  Power BI apps

  integrated with Fabric for reporting and dashboard distribution.
• Develop and maintain

  Fabric Notebooks

  using

  PySpark

  ,

  Python

  , and

  SQL

  for advanced data engineering and analytics use cases.
• Oversee Azure services integration such as

  App Registrations

  ,

  IAM

  , and other platform services to ensure secure and efficient operation.

Python Development

• Build robust, scalable Python scripts and applications with a strong focus on performance, reliability, and maintainability.
• Integrate external systems using

  RESTful APIs

  and automate workflows through Python-based orchestration.
• Utilize advanced Python libraries such as

  pandas

  ,

  numpy

  , and others for data manipulation and transformation.
• Develop database integration scripts to connect with

  relational databases

  (e.g., SQL Server, MySQL, PostgreSQL, Oracle) and execute

  complex SQL

  and

  PL/SQL

  queries.
• Handle and transform complex

  JSON

  data structures for ingestion and processing.

Collaboration & Leadership

• Work closely with cross-functional stakeholdersincluding data scientists, analysts, product managers, and business leaders—to understand requirements and deliver scalable solutions.
• Translate business needs into technical specifications and implement solutions that align with architectural best practices.
• Provide technical mentorship and guidance to junior team members and contribute to architectural decision-making.

Qualifications

• Experience in data engineering, analytics, or platform engineering roles.
• Proven expertise in Microsoft Fabric and Azure Data Services.
• Strong command of Python programming and experience working with large-scale data systems.
• Experience designing and implementing OLAP systems, including semantic models and multidimensional cubes.
• Solid understanding of data governance, security, and access control in enterprise environments.
• Exceptional problem-solving, communication, and collaboration skills.

Regular Tasks Required:

• Build and maintain MS Fabric data import, standardization, and governance procedures.
• Write effective Python code to tackle complex issues, but also use your business sense and analytical abilities to glean valuable insights from public databases
• Communicate clearly with team and help the organization in realizing its objectives
• Clearly express the reasoning and logic when writing code
• Fix bugs in the code and create thorough documentation
• Utilize your data analysis skills to develop and respond to important business queries using available datasets
• Effectively communicate with the team to comprehend the needs and provide the results
• Useful but non-mandatory skills:
• Hands-on experience with Generative AI tools, including LangChain, RAG, and vector databases.
• Experience with web scraping, browser automation, and workflow automation.
• Proficiency in traditional ML libraries (scikit-learn, TensorFlow, PyTorch, XGBoost).

Experience and Educational requirements:

• Bachelor’s in Engineering or Masters in Computer Science (or equivalent experience)
• At least 2+ years of relevant experience as a data scientist
• 2+ years of Data analysis experience and a desire to have a significant impact on the field of artificial intelligence
• 2+ years of experience working extensively with Python programming
• Experience with MS Fabric, OneLake, and the Microsoft data analytics stack
• Experience with PySpark and Spark SQL
• Extensive experience working with Data Science/Analysis
• Familiarity with SQL, Rest API, Tableau and related technologies is desirable
• Excellent communication abilities to work with stakeholders and researchers successfully
• Strong data analytic abilities and business sense are required to draw the appropriate conclusions from the dataset, respond to those conclusions, and clearly convey the key findings
• Excellent problem-solving and analytical skills
• Fluent in conversational and written English communication skills

Physical requirements:

• Normal, corrective vision range; ability to see color and to distinguish letters, numbers and symbols
• Frequently required to sit, stand, walk, talk, hear, bend and reach
• Ability to reach with hands and arms
• Occasionally lift and/or move up to 30lbs