Job
Description
Job Title: Data Engineer Experience: 6+ Years Location: Remote Employment Type: Full-Time Job Summary: We are seeking a highly skilled and experienced Data Engineer with a strong background in building and optimizing data pipelines, architectures, and data sets. The ideal candidate should have 6+ years of hands-on experience in data engineering, with mandatory expertise in both AWS and Azure cloud platforms . This is a remote position, offering flexibility and the opportunity to work with a global team. Key Responsibilities: Design, develop, and maintain robust, scalable, and efficient data pipelines and ETL/ELT processes. Integrate data from various structured and unstructured sources into data lakes and data warehouses. Develop and optimize data architectures on AWS (S3, Glue, Redshift, Lambda, Athena) and Azure (Data Lake, Data Factory, Synapse, Databricks) . Implement data quality, lineage, and observability frameworks. Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver reliable solutions. Monitor and troubleshoot production data pipelines and ensure data integrity. Apply best practices in data engineering, security, and governance. Mandatory Skills: Cloud Platforms: Strong hands-on experience with AWS services like S3, Glue, Redshift, Lambda, Athena, EMR. Deep knowledge of Azure services such as Azure Data Factory, Azure Data Lake Storage, Synapse Analytics, Azure Databricks. Programming Languages: Proficiency in Python , SQL , and PySpark for building ETL pipelines. Data Processing Frameworks: Experience with Apache Spark , Databricks , and Airflow or similar workflow orchestration tools. Data Modeling: Strong understanding of relational and dimensional data modeling, data warehousing concepts. DevOps/DataOps: Familiarity with CI/CD pipelines, Git, Docker, and Infrastructure as Code (Terraform/CloudFormation).
