Sr. Big Data Engineer

As a Big Data Engineer, you will play a crucial role in developing and managing the Big Data solutions for our company. Your responsibilities will include designing and implementing Big Data tools and frameworks, implementing ELT processes, collaborating with development teams, building cloud platforms, and maintaining the production system. Key Responsibilities: - Meet with managers to determine the companys Big Data needs. - Develop Big Data solutions on AWS using tools such as Apache Spark, Databricks, Delta Tables, EMR, Athena, Glue, and Hadoop. - Load disparate data sets and conduct pre-processing services using Athena, Glue, Spark, etc. - Collaborate with software research and development teams. - Build cloud platforms for the development of company applications. - Maintain production systems. Qualifications Required: - 5+ years of experience as a Big Data Engineer. - Proficiency in Python & PySpark. - In-depth knowledge of Hadoop, Apache Spark, Databricks, Delta Tables, and AWS data analytics services. - Extensive experience with Delta Tables, JSON, Parquet file format. - Experience with AWS data analytics services like Athena, Glue, Redshift, EMR. - Familiarity with Data warehousing will be a plus. - Knowledge of NoSQL and RDBMS databases. - Good communication skills. - Ability to solve complex data processing and transformation related problems.,