Data Bricks

As a Data Engineer, you will be responsible for designing and developing data solutions using Data Bricks. Your primary focus will be on creating and maintaining data pipelines for processing large datasets. Collaborating with data scientists and analysts, you will optimize data workflows and implement ETL processes to streamline data management. Utilizing Apache Spark, you will enhance data processing capabilities and conduct data analysis to derive actionable insights. Ensuring data quality and integrity throughout the data lifecycle is crucial, and you will be expected to monitor and improve existing data systems. Integrate Data Bricks with cloud platforms such as Azure and AWS, and develop customized dashboards for data visualization. Documenting data architecture and development processes, troubleshooting data discrepancies, and resolving issues are also key responsibilities. Participating in Agile methodologies and sprint planning, you will train and support team members on data tools. It is essential to stay updated with the latest data technologies and trends to contribute effectively to the team. Qualifications: - Proven experience in data engineering or a similar role. - Strong proficiency in SQL and data query languages. - Experience with streaming data and batch processing. - Familiarity with Python or Scala for data manipulation. - Understanding of data modeling and warehousing concepts. - Knowledge of cloud services such as Azure or AWS. - Experience with data visualization tools like Tableau and Power BI. - Ability to work collaboratively in a team environment. - Strong analytical and problem-solving skills. - Excellent communication and interpersonal skills. - Experience with Agile/Scrum methodologies. - Familiarity with version control systems like Git. - Detail-oriented with a knack for troubleshooting. - Ability to handle multiple priorities in a fast-paced environment. - Willingness to learn and adapt to new technologies. Skills: Data Bricks, Power BI, SQL, Python.,