Junior Data Engineer

As a Junior Data Engineer at NIQ in Chennai, you will be responsible for building reliable, secure, and scalable data pipelines using Python, Snowflake, and cloud-native services. Your role will involve supporting analytics, reporting, ML workflows, and data-driven products while adopting DevSecOps best practices. Key Responsibilities: - Develop, schedule, and monitor ELT/ETL pipelines in Python to ingest data from APIs, files, and databases into Snowflake. - Implement transformations (SQL & Python) following modular, reusable patterns. - Build tables, views, stages, and secure data shares in Snowflake. - Tune queries and warehouse configs; manage roles, RBAC, and resource usage. - Utilize cloud services (AWS/GCP/Azure) for storage, compute, and event triggers (e.g., S3/Blob/GCS; Lambda/Functions). - Orchestrate jobs (Airflow/Cloud Composer/Prefect/Azure Data Factory) with proper alerting & SLAs. - Apply CI/CD for data code (linting, unit/data tests, IaC review gates). - Embed security scanning, secrets management, and leastprivilege IAM. - Contribute to observability by setting up logging, metrics, lineage, and data quality checks. - Collaborate with Analytics/ML teams to productize models and enable selfservice. - Write clear documentation including readme, runbooks, and data dictionaries. Qualifications: Must-Have: - 25 years of experience in data engineering, analytics engineering, or backend engineering. - Strong programming skills in Python (pandas, SQLAlchemy/DB APIs, typing, testing with pytest). - Proficiency in SQL; experience with Snowflake (warehouses, stages, tasks, Snowpipe, RBAC). - Familiarity with one major cloud platform (AWS/Azure/GCP) and storage + compute + IAM fundamentals. - Experience with Git, CI/CD, code reviews, secret handling, and dependency scanning. - Exposure to unit tests, data validation, and monitoring principles. - Strong problem-solving skills, curiosity, and ability to learn quickly. Additional Information: Nice-to-Have: - Experience with orchestration tools like Airflow/Prefect/ADF/Composer. - Knowledge of IaC tools such as Terraform/Azure Bicep/CloudFormation and policy-as-code. - Exposure to streaming technologies like Kafka/Kinesis/PubSub and Change Data Capture. - Familiarity with dbt, SCD/CDC design patterns, and Dimensional modeling. - Understanding of security best practices and observability tools. - Exposure to MLOps is a plus. About NIQ: NIQ is the worlds leading consumer intelligence company, delivering insights into consumer buying behavior and growth pathways. With operations in 100+ markets, NIQ provides comprehensive consumer insights through advanced analytics. NIQ is committed to diversity, equity, and inclusion in the workplace. Our Benefits: - Flexible working environment - Volunteer time off - LinkedIn Learning - Employee-Assistance-Program (EAP) As a Junior Data Engineer at NIQ in Chennai, you will be responsible for building reliable, secure, and scalable data pipelines using Python, Snowflake, and cloud-native services. Your role will involve supporting analytics, reporting, ML workflows, and data-driven products while adopting DevSecOps best practices. Key Responsibilities: - Develop, schedule, and monitor ELT/ETL pipelines in Python to ingest data from APIs, files, and databases into Snowflake. - Implement transformations (SQL & Python) following modular, reusable patterns. - Build tables, views, stages, and secure data shares in Snowflake. - Tune queries and warehouse configs; manage roles, RBAC, and resource usage. - Utilize cloud services (AWS/GCP/Azure) for storage, compute, and event triggers (e.g., S3/Blob/GCS; Lambda/Functions). - Orchestrate jobs (Airflow/Cloud Composer/Prefect/Azure Data Factory) with proper alerting & SLAs. - Apply CI/CD for data code (linting, unit/data tests, IaC review gates). - Embed security scanning, secrets management, and leastprivilege IAM. - Contribute to observability by setting up logging, metrics, lineage, and data quality checks. - Collaborate with Analytics/ML teams to productize models and enable selfservice. - Write clear documentation including readme, runbooks, and data dictionaries. Qualifications: Must-Have: - 25 years of experience in data engineering, analytics engineering, or backend engineering. - Strong programming skills in Python (pandas, SQLAlchemy/DB APIs, typing, testing with pytest). - Proficiency in SQL; experience with Snowflake (warehouses, stages, tasks, Snowpipe, RBAC). - Familiarity with one major cloud platform (AWS/Azure/GCP) and storage + compute + IAM fundamentals. - Experience with Git, CI/CD, code reviews, secret handling, and dependency scanning. - Exposure to unit tests, data validation, and monitoring principles. - Strong problem-solving skills, curiosity, and ability to learn quickly. Additional Information: Nice-to-Have: - Experience with orchestration tools like Airflow/Prefect/ADF/Composer. - Knowledge of IaC tools such as Terraform/Azure Bicep/CloudFor