Data Engineer

Role & responsibilities

We are looking for an experienced Python Preferabbly on data Integration with SQL and ETL developer to join our team as a Data Lead who will work as part of a team to help design and develop data-driven solutions that deliver insights to the business. The ideal candidate is a data pipeline builder and data wrangler who enjoys building data-driven systems that drive analytical solutions and building them from the ground up. You will be responsible for building and optimizing our data as well as building automated processes for production jobs. You will support our software developers, database architects, data analysts and data scientists on data initiatives

Preferred candidate profile

• Use advanced complex Python or SQL to extract data from source systems for ingestion into a data pipeline (ETL).
• Design, develop and deploy scalable and efficient data pipelines.
• Analyze and assemble large, complex datasets that meet functional / non-functional business requirements.
• Identify, design, and implement internal process improvements. For example: automating manual processes, optimizing data delivery, re-designing data platform infrastructure for greater scalability.
• Build required infrastructure for optimal extraction, loading, and transformation (ELT) of data from various data sources using AWS and Snowflake leveraging Python or SQL technologies.
• Monitor cloud-based systems and components for availability, performance, reliability, security and efficiency
• Create and configure appropriate cloud resources to meet the needs of the end users.
• As needed, document topology, processes, and solution architecture.
• Share your passion for staying on top of tech trends, experimenting with and learning new technologies

• Bachelor's degree in computer science, computer engineering, or a related field.
• 5 + years of experience working with Python (Api) /SQL , ETL and DB (Snowflake Is Preferable)
• 3+ years of experience with the Database (snowflake) .
• Experience building, maintaining, and optimizing big data’ data pipelines, architectures, and data sets
• Experience cleaning, testing, and evaluating data quality from a wide variety of ingestible data sources
• Experience working with Linux and UNIX environments.
• Experience with profiling data, with and without data definition documentation
• Familiar with Git