Job
Description
Key Responsibilities: Design, develop, and maintain ETL/ELT pipelines to move and transform data from various sources. Build and optimize data warehouses, data lakes, and structured/unstructured storage solutions. Ensure data quality, integrity, and consistency across systems. Collaborate with data analysts and scientists to understand data needs and optimize performance. Integrate APIs, third-party data sources, and internal systems to build robust datasets. Monitor and troubleshoot pipeline performance and resolve data-related issues. Implement data security, governance, and compliance best practices. Required Skills & Qualifications: Bachelor's degree in Computer Science, Engineering, Information Systems, or a related field. 25 years of experience as a Data Engineer or in a related role. Strong programming skills in Python, SQL, and optionally Scala or Java. Experience with big data tools such as Spark, Hadoop, or Kafka. Hands-on experience with cloud platforms (e.g., AWS, GCP, or Azure) and services like S3, Redshift, BigQuery, or Snowflake. Proficiency with data modeling, schema design, and working with relational and NoSQL databases. Familiarity with workflow orchestration tools like Airflow, Luigi, or Prefect. Strong understanding of data warehousing concepts and performance optimization.
