Data Engineering Consultant - Python

Data Engineering Consultant

Key Responsibilities

Data Pipeline Development and Management:

• Design, develop, and optimize ETL pipelines to extract, transform, and load healthcare data from various sources.
• Build scalable data workflows to process large volumes of structured and unstructured data using Python and pandas.

Database and Data Warehouse Management:

• Develop and maintain SQL databases and data warehouses to support advanced analytics and reporting requirements.
• Collaborate on data modeling and schema design to optimize query performance and storage efficiency.

Big Data and Cloud Infrastructure:

• Architect and implement Big Data solutions using technologies such as Hadoop, Spark, or similar frameworks.
• Leverage cloud platforms (e.g., AWS, Azure, Google Cloud) to design and manage data storage, processing, and integration solutions.

Reporting and Data Accessibility:

• Build data pipelines and integrations to support reporting dashboards and visualization tools (e.g., Tableau, Power BI).
• Ensure data accessibility and security for stakeholders while maintaining compliance with healthcare regulations.

Collaboration and Cross-Functional Support:

• Collaborate with Data Scientists to deliver clean, enriched datasets for machine learning and predictive modeling.
• Work with Product Owners and business stakeholders to understand data requirements and translate them into technical solutions.

Healthcare Data Expertise:

• Manage and process healthcare-related data, including claims, EHRs, and other patient-centric datasets.
• Ensure compliance with healthcare data privacy regulations such as HIPAA and HITECH Act.

Required Skills and Experience:

• 6–10 years of professional experience in Data Engineering or related roles.
• Proficiency in Python and pandas for data manipulation and processing.
• Strong expertise in SQL for querying, database design, and optimization.
• Solid experience building ETL pipelines and integrating data from multiple sources.
• Hands-on experience with Big Data technologies (e.g., Hadoop, Spark, Hive, etc.).
• Familiarity with reporting tools (e.g., Tableau, Power BI) and data visualization workflows.
• Experience with cloud platforms (e.g., AWS, Azure, Google Cloud), including data storage and processing services.
• Knowledge of data governance, security, and compliance, particularly in healthcare environments.

Preferred Qualifications:

• Exposure to healthcare data formats (e.g., HL7, FHIR) and standards.
• Experience with containerization and orchestration tools (e.g., Docker, Kubernetes).
• Basic understanding of machine learning workflows and how data engineering supports them.
• Familiarity with version control systems (e.g., Git) and Agile methodologies.

Soft Skills:

• Strong problem-solving and analytical skills.
• Excellent communication and collaboration abilities across technical and non-technical teams.
• Attention to detail and commitment to delivering high-quality data solutions.
• Ability to thrive in a fast-paced, dynamic environment.

Education:

• Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Systems, or a related field.

  Role & responsibilities