Scala & Databricks Developer

We are seeking an experienced Scala and Databricks Developer with a strong background in constructing data solutions on AWS. As the ideal candidate, you will possess hands-on expertise in crafting data pipelines, enhancing big data solutions, and enforcing best practices utilizing Scala, Databricks, and various AWS services. Your pivotal role will involve designing and delivering scalable, dependable, and high-performance data solutions to meet our business objectives effectively. Your responsibilities will encompass collaborating with data architects and analysts to devise and implement data models and processing frameworks. You will be tasked with optimizing the performance of data workflows, identifying and addressing bottlenecks within Databricks and AWS environments. Additionally, integrating Databricks seamlessly with AWS services like S3, Glue, Lambda, and Redshift will be a key component of your role. Ensuring data accuracy and reliability through the implementation of data validation and quality checks will also be crucial. Working closely with stakeholders to comprehend business requirements and translating them into technical solutions will be part of your routine tasks. Furthermore, monitoring, troubleshooting, and resolving issues within production data pipelines will be within your scope, along with documenting processes, configurations, and solutions for future reference. The required skills and qualifications for this position include 4-5 years of experience in developing data solutions using Scala, hands-on experience with Databricks for constructing and managing large-scale data workflows, proficiency in AWS services such as S3, Glue, Lambda, Redshift, and IAM. A solid grasp of distributed computing, big data concepts like Spark, Hadoop, or similar frameworks, experience with CI/CD pipelines, and version control systems like Git are necessary. Knowledge of data security, governance, and compliance practices, strong problem-solving skills, and the ability to thrive in a fast-paced environment are essential attributes. Preferred qualifications for this role consist of familiarity with AWS tools like Athena, EMR, or Step Functions, knowledge of data lakes and lakehouse architectures, experience in Python for scripting and auxiliary tasks, understanding of monitoring and logging tools such as CloudWatch, Datadog, or Prometheus, and exposure to Agile methodologies and tools like JIRA.,