Posted:4 days ago|
Platform:
On-site
Full Time
Role - Data Engineer
Experience - 5+ years
Location - Vadodara
Days - Monday to Friday
Roles & responsibilities:
ETL & Data Integration:
● Design, develop, and manage scalable ETL pipelines using AWS Glue (PySpark).
● Extract data from multiple sources including RDS MySQL, MongoDB, internal/external APIs, and external files.
● Transform, clean, and normalize data into usable formats for various downstream consumers.
● Document data flows, ETL processes, and data transformations. Data Infrastructure & Warehousing:
● Load and manage structured data in data lakes (S3) and data warehouses (e.g., Amazon Redshift).
● Maintain and evolve AWS Glue Data Catalog, partitioning strategies, and schema versioning.
● Help set up BI/reporting platforms (e.g., QuickSight, Tableau) by exposing curated datasets.
Data Quality & Monitoring:
● Implement data validation, quality checks, and error handling in ETL pipelines.
● Monitor pipeline health, performance, and cost using CloudWatch, Glue job metrics, and alerts. Security & Governance:
● Enforce secure data handling practices, including PII redaction, encryption, and access control (IAM, Lake Formation).
● Ensure compliance with data governance policies and audit requirements. Collaboration:
● Collaborate with the AI development team to understand their data requirements and ensure the data provided is in the correct format and meets their needs.
● Assist in data modeling, business logic extraction, and ad-hoc data analysis as needed.
● Maintain clear documentation of pipeline logic, data contracts, and transformations.
Requirements:
● 3+ years of experience as a Data Engineer, preferably in a cloud-native environment.
● Strong expertise in AWS Glue, PySpark, and building distributed ETL workflows.
● Proficient in Python, SQL, and handling structured, semi-structured (JSON, CSV), and unstructured data. ● Solid experience with AWS services: S3, RDS (MySQL), IAM, CloudWatch, and Redshift.
● Familiarity with data warehouse concepts, dimensional modeling, and performance tuning.
● Comfortable working with Git, CI/CD pipelines, and infrastructure-as-code tools like Terraform or CloudFormation.
● Strong analytical and problem-solving skills; able to work independently and take ownership of data pipelines.
Nice to Have:
● Experience supporting SageMaker, Bedrock, AI/ML workloads (e.g., transforming data for structured knowledge bases).
● Exposure to event-driven architectures, Airflow, or Step Functions for orchestration.
● Experience working with streaming data (e.g., Kinesis, Kafka).
Aecor Digital
Upload Resume
Drag or click to upload
Your data is secure with us, protected by advanced encryption.
Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.
We have sent an OTP to your contact. Please enter it below to verify.
Instantly access job listings, apply easily, and track applications.
Practice Python coding challenges to boost your skills
Start Practicing Python Now
bengaluru
7.0 - 8.0 Lacs P.A.
mumbai
3.0 - 7.0 Lacs P.A.
hyderabad
2.0 - 5.0 Lacs P.A.
hyderabad
2.0 - 5.0 Lacs P.A.
hyderabad
4.5 - 8.5 Lacs P.A.
hyderabad
2.0 - 5.0 Lacs P.A.
hyderabad
2.0 - 5.0 Lacs P.A.
hyderabad
2.0 - 5.0 Lacs P.A.
hyderabad
2.0 - 5.0 Lacs P.A.
thiruvananthapuram
18.0 - 20.0 Lacs P.A.
