Data Engineer

HIRING FOR A USA BASED MNC -

Location - Chennai

Experience : 3-10years

Data Engineer T3

• Bachelor’s or master’s degree with 3+ years of experience in Computer Science, Data Science, Engineering, or a related field.
• 3+ years of experience in working with Python, SQL, PySpark and bash scripts. Proficient in software development lifecycle and software engineering practices.
• 3+ years of experience in developing and maintaining robust data pipelines for both structured and unstructured data to be used by Data Scientists to build ML Models.
• 3+ years of experience working with Cloud Data Warehousing (Redshift, Snowflake, Databricks SQL or equivalent) platforms and experience in working with distributed frameworks like Spark.
• 2+ years of hands-on experience in using Databricks platform for data engineering. Detailed knowledge of Delta Lake, Databricks Workflow, Job Clusters, Databricks CLI, Databricks Workspace etc.
• Solid understanding of machine learning life cycle, data mining, and ETL techniques.
• Familiarity with commonly used machine learning libraries (like scikit-learn, xgboost) in terms of exposure and handling of code base which makes use of these libraries for model training & scoring.
• Proficiency in understanding of REST APIs, experience in using different types of APIs to either extract data or perform a functionality exposed by APIs.
• Familiarity in Pythonic API development frameworks like Flask / FastAPI. Experience in using containerization frameworks like Docker / Kubernetes.
• Hands-on experience in building and maintaining tools and libraries which have been used by multiple teams across organization. e.g. Creating Data Engineering common utility libraries, DQ Libraries etc.
• Proficient in understanding and incorporating software engineering principles in design & development process.
• Hands on experience with using CI/CD tools (e.g., Jenkins or equivalent), version control (Github, Bitbucket), Orchestration (Airflow, Prefect or equivalent)
• Excellent communication skills and ability to work and collaborate with cross functional teams across technology and business.