Data Engineer

Job Title: Data Engineer

Data Engineer

Qualifications

Education

• Bachelor's degree in Computer Science, Data Engineering, Information Systems, or related field
• Master's degree is a plus

Experience

• 5+ years of professional experience in data engineering
• Hands-on experience with

  Databricks, Snowflake, and Microsoft Fabric

• Experience working with on-prem relational databases (SQL Server, Oracle, DB2, etc.)
• Strong knowledge of ETL/ELT, data warehousing, and data lakehouse concepts
• Experience working with both structured and unstructured data

Technical Skills

• Databricks:

  PySpark/Scala/Spark SQL, Delta Lake, MLflow, notebook orchestration, Delta Live Tables

• Snowflake:

  SnowSQL, Snowpipe, performance tuning, security/role management, data sharing

• Microsoft Fabric:

  Dataflows, Pipelines, OneLake, Power BI integration
• Advanced SQL
• Python, Scala, Java
• Git, CI/CD for data solutions
• Infrastructure as Code (Terraform, Bicep)
• Metadata management and data lineage documentation
• SSIS, ADF, and similar ETL/data pipeline tools
• Experience using Databricks notebooks to interact with REST APIs

Soft Skills

• Strong communication and collaboration skills
• Detail-oriented with a focus on data quality and reliability
• Ability to manage multiple priorities in a fast-paced environment

Preferred Qualifications

• Certifications in

  Databricks, Snowflake,

  and/or

  Microsoft Fabric

• Experience implementing CI/CD practices for data pipelines and ETL jobs

Key Responsibilities

1. Design, develop, and optimize data pipelines and ETL processes across cloud and on-prem systems
2. Integrate and manage data across Databricks, Snowflake, Fabric, and relational databases
3. Collaborate with business and analytics teams to deliver reusable, high-quality datasets
4. Ensure adherence to data governance, security, and privacy standards
5. Support batch, real-time, and streaming data processing
6. Work with architects to define scalable lakehouse and warehouse strategies
7. Monitor, automate, and optimize pipelines for performance and quality
8. Partner with data governance, analytics, and IT teams for various data initiatives
9. Document pipeline configurations, processes, and data models
10. Implement enhancements for ingestion, transformation, and automation
11. Stay updated with new features and best practices in Databricks, Snowflake, and Fabric