Senior Data Engineer

Be part of a transformative team that is shaping the way First American builds and delivers world-class technology products that fuel the real estate industry. We are looking for the best-of-the-best technology experts who will envision, design, build, and deliver innovative solutions that provide exceptional experience and lasting value to our customers.

We are looking for a Senior Data Engineer who is passionate about learning, experimenting, and building solutions in a fast-paced environment. This person will design, build, and maintain scalable data pipelines and cloud infrastructure, leveraging Databricks and PySpark to process large datasets efficiently. In this role, you will have the chance to make a big impact and drive efficiency across our organization.

What You’ll Do:

• Operationalize our Databricks Lakehouse:

  Develop and maintain a unified data platform using Azure Databricks, enabling efficient data storage, processing, and analysis across the organization.

• Build Data Pipelines:

  Design, develop, and maintain scalable data pipelines; establish API integrations for efficient data transfer. Implement ETL processes and ensure data integrity and quality.

• Deploy Data Technology:

  Utilize technologies such as Spark, Kafka, and Airflow to manage large-scale data processing. Test, monitor and troubleshoot systems to ensure smooth operation.

• Collaborate:

  Work closely with data engineers, analysts, and scientists to understand their needs and optimize the performance of data workflows

What You’ll Bring:

• Development Experience:

  5+ years with Python, Scala, Java, or C#. Experience with Python is preferred.

• Data Engineering Tools:

  5+ years with Databricks, Snowflake, BigQuery, Apache Spark, HIVE, Hadoop, Cloudera, or RedShift. Experience with PySpark on Databricks is preferred.[TN1]

• Data Architecture:

  Proficiency in high-performance data pipeline design and mastery of pipeline guarantees, such as Idempotency, At-Least-Once Processing, Exact-Once Processing, Fault Tolerance, Eventual Consistency, Streaming Consistency, Transactional Consistency, and Observability.

• Containerization:

  Experience developing in a containerized environment like Docker, Rancher, or Kubernetes.

• Data Orchestration:

  Experience with Apache Airflow (or similar tool) for orchestrating data processing jobs.

• Education:

  Bachelor's degree in computer science (or related field) or equivalent combination of education and experience.