Senior Python Data Engineer

Key Responsibilities

• Design, build and maintain Python plugins that encapsulate business rules and transformation logic such as custom aggregation/disaggregation, rule-based data enrichment, and time-grain manipulations.
• Implement data-cleaning, validation, mapping and merging routines using Pandas, NumPy and other Python libraries to prepare inputs for downstream analytics.
• Profile and optimize Python code pathsleveraging vectorized operations, efficient merges and in-memory transformationsto handle large datasets with low latency.
• Establish and enforce coding standards, write comprehensive unit and integration tests with pytest (or similar), and ensure high coverage for all new components.
• Triage and resolve issues in legacy scripts, refactor complex routines for readability and extensibility, and manage versioning across releases.
• Collaborate with data engineers and analysts to capture requirements, document plugin behaviors, configuration parameters, and usage examples in code repositories or internal wikis.

Must-Have Skills

• 5-7 years of hands-on experience writing well-structured Python code, with deep familiarity in both object-oriented and functional programming paradigms.
• Expert mastery of Pandas and NumPy for transformations (group-by aggregations, merges, column operations), plus strong comfort with Pythons datetime and copy modules.
• Proven experience designing modular Python packages, exposing configuration through parameters or picklists, and managing versioned releases.
• Ability to translate complex, rule-driven requirements (such as disaggregation rules, external-table merges, and priority ranking) into clean Python functions and classes.
• Proficiency with pytest (or equivalent), mocking, and integrating with CI pipelines (e.g., GitHub Actions, Jenkins) for automated testing.
• Skilled use of Pythons logging module to instrument scripts, manage log levels, and capture diagnostic information.
• Strong Git workflow experience, including branching strategies, pull requests, code reviews, and merge management.