Job
Description
Job Description & Responsibilities: Work with business and technical leadership to understand requirements Design to the requirements and document the designs Ability to write product-grade performant code for data extraction, transformations and loading using Spark, Py-Spark Do data modeling as needed for the requirements Write performant queries using Teradata SQL, and Spark SQL against Teradata and Implementing dev-ops pipelines to deploy code artifacts on to the designated platform/servers like AWS(pref) or Implement Hadoop job orchestration using Shell scripting, CA7 Enterprise Scheduler and Airflow Troubleshooting the issues, providing effective solutions and jobs monitoring in the production environment Participate in sprint planning sessions, refinement/story-grooming sessions, daily scrums, demos and retrospectives Experience Required: Overall 5 - 8 Years Experience Desired: Experience in Jira and Confluence Health care domain knowledge is a plus Excellent work experience on Databricks OR Teradata as data warehouse Experience in Agile and working knowledge on DevOps tools Education and Training Required: Primary Skills: Spark, Py-Spark, Shell scripting, Teradata SQLs (using Teradata SQLand Spark SQL) and Stored Procedures Git, Jenkins, Artifactory CA7 Enterprise Scheduler, Airflow AWS data services(S3, EC2, SQS) AWS Services- SNS Lambda, ECS, Glue, IAM Cloud Watch Monitoring tool Databricks (Delta lake, Notebooks, Pipelines, cluster management, Azure/AWS integration) Good to have: Unix / Linux Shell scripting (KSH) and basic administration of Unix servers Additional Skills: Exercises considerable creativity, foresight, and judgment in conceiving, planning, and delivering initiatives.
