Job
Description
Job Overview: We seek a highly skilled and motivated Data Engineer with strong expertise in Python, PySpark, AWS, Databricks, and Snowflake to join our dynamic team. The ideal candidate should have hands-on experience with Spark optimization, SQL data processing, and AWS tools and technologies . Exposure to Informatica and data streaming tools is a plus. Key Responsibilities: Design, develop, and optimize scalable data processing pipelines using Spark and PySpark. Implement Spark optimization techniques to enhance performance and efficiency. Work with complex SQL queries for data manipulation and transformation. Collaborate with data scientists, analysts, and engineering teams to meet data needs. Work on AWS Cloud services like Redshift, AWS Glue, SQL Server, and Databricks. Utilize Python for data processing tasks, ensuring modularization and packaging. Basic knowledge of Informatica. Engage in real-time data streaming and integration using tools such as NiFi, Kafka, and EventHub (optional but preferred). Hands-on experience with Snowflake for data warehousing and analytics. Must Have Experience _ Python, Pyspark, SQL, AWS, Spark Required Skills: Spark Expertise: Strong experience in Spark and PySpark, including optimization techniques. Complex SQL: Advanced SQL knowledge and data processing skills. Cloud Exposure: Familiarity with AWS services and Databricks. Python Proficiency: Experience with Python for data processing and scripting. Data Streaming: Experience with NiFi, Kafka, and EventHub is desirable. Snowflake: Hands-on experience with Snowflake for data warehousing and analytics. Qualifications: Bachelors or Masters degree in Computer Science, Information Technology, or related fields. 4-8 years of professional experience in Data Engineering, Data Analytics, or related roles.
