Full Stack Data Engineer

Position Duties & Responsibilities

• Deliver complete data solutions across the SDLC, from design to deployment, based on senior guidance and best practices.
• Contribute quantifiably to business goals by ensuring technical work addresses key challenges or improves outcomes.
• Collaborate effectively with other team members to gain a clear understanding of business needs and translate them into technical tasks.
• Help the team meet deadlines and maintain high data quality standards across projects.
• Share knowledge and reuse existing code to accelerate development and maintain consistency.
• Evaluate and propose new tools or solutions that optimize time, effort, and support the best business choices.
• Contribute to code and design reviews, providing constructive feedback and identifying potential issues.
• Produce clear and comprehensive documentation, including technical design documents for moderately difficult features.
• Propose design improvements and apply patterns to solve new data problems effectively.
• Configure CI/CD pipelines, manage deployments, and resolve production incidents efficiently.
• Implement automated testing strategies and comprehensive logging in workflows.
• Create and maintain dashboards and alerts, and analyze performance metrics to ensure data system health.
• Proactively acquire new skills, stay current with industry trends, and apply learning to continuously improve systems.
• Lead small initiatives and identify process improvements, owning features end-to-end.
• Influence team technical decisions and contribute to coding standards.
  

Responsibilities

Required Skills & Qualifications

• 3+ years of experience working in a GCP environment and utilizing its relevant tools/services (e.g., Terraform, Tekton, Cloud Run, DataFlow, Cloud Storage, Cloud Scheduler, BigQuery, etc.).
• 7+ years of strong experience in Java development with strong SQL knowledge.
• 3+ years of experience with Big Data ecosystem tools like Hadoop, Hive, Sqoop, MySQL.
• Experience with RDBMS solutions (e.g., SQL, Oracle, MSSQL, Teradata).
• Proven experience with multiple successful data projects, pipeline deployments, and data products.
• Strong proficiency in Python or Scala, with working knowledge of SQL and Java.
• Experience developing Apache Beam pipelines using Dataflow or Spark jobs on Dataproc.
• Proficient in designing scalable and efficient data models for analytical workloads in BigQuery.
• Working experience and expertise with Google Cloud Platform (GCP) services including BigQuery, Dataflow, Dataproc, Cloud Run, Pub/Sub, IAM, VPC Service Controls.
• Understanding of data encryption at rest and in transit within GCP.
• Consistently deliver high-quality code with comprehensive test coverage.
• Ability to refactor code for maintainability and implement robust error handling.
• Experience configuring CI/CD and managing deployments.
• Ability to resolve production incidents and implement automated testing.
• Strong analytical and problem-solving skills, with the ability to translate business needs into technical tasks.
• Excellent written and oral communication skills, including producing clear documentation and contributing to technical design documents.
• Ability to work independently and collaborate effectively with diverse teams.
  

Qualifications

Desired Skills

• Experience advising management on project-level issues.
• Ability to guide and mentor junior engineers.
• A proactive approach to identifying process improvements.
• Familiarity with industry best practices and their application.