-
The Lead Data Engineer is responsible for designing, building, and optimizing data pipelines and platforms that enable scalable, reliable, and high-performance data processing
- This role involves developing end-to-end data solutions using technologies such as Flink, Scala, Kafka , PySpark, AWS, and SQL , while ensuring efficient ETL/ELT workflows to support analytics, reporting, and data-driven decision-making
- The engineer will analyze complex data requirements, model data structures, and implement data integration solutions across diverse systems
- Strong expertise in Streaming, performance tuning, and big data frameworks is essential, along with the ability to document, automate, and operationalize data pipelines for production use
- The position also requires close collaboration with cross-functional teams including data analysts, data scientists, and application developers to ensure data quality, consistency, and availability
- The Senior Data Engineer is expected to apply best practices in data engineering, cloud optimization, and governance , contributing to the continuous improvement and modernization of the data ecosystem
ESSENTIAL RESPONSIBILITIES AND DUTIES:
- Translates business requirements to conceptual solution architecture and high-level project estimates.
Develops, modifies, and implements Software as a Service (SaaS) hosted applications according to business requirements using:
- Scala/Flink - Any Cloud based Streaming Experience
- Kafka
- Pyspark
- SQL
- ElasticSearch
- Python
- ETL
- BigData
- AWS/Azure
- Containers and Container Orchestration
- Docker
- Kubernetes
- Logical data modeling, database definition and manipulation, and data synchronization
- Object oriented design, coding, performance tuning, and unit testing
- Research, extraction and analysis of complex data
- Authors and reviews technical requirements to ensure compliance with business requirements
- Performs proper unit testing and software code writing (including automated unit testing)
- Familiar with appropriate standards and techniques used during the Software Development Life Cycle process (SDLC, security) and applies them appropriately
- Participates in the testing process through test review and analysis, test witnessing and certification of software
- Participates in peer code reviews and develops skill level of others through mentorship
- May work on multiple tasks/projects simultaneously with various team members and/or other groups both internally and externally
- Engages effectively in self-directed time management and prioritization of workload
- Perform such other duties as may be assigned by management
- Essential to be available to work during Omnitracs corporate business hours
QUALIFICATIONS :
- EDUCATION: bachelors degree in Computer Science, Computer Engineering, or Equivalent
- EXPERIENCE: 9+ years of applicable Software Development experience
KNOWLEDGE/SKILLS/ABILITIES:
- Experience in Streaming
- Experience in Pyspark/Scala/Flink
- Experience in Python
- Experience in SQL
- Experience in AWS
- Experience in Database management system
- Experience with Dockers/Containers
- Excellent interpersonal skills
- Excellent written and oral English skills
- Excellent working knowledge of logical data modeling and Database management systems
- Excellent working knowledge of python development environments and process
- Versed in industry standard Software Development Life-Cycle (SDLC) process methodologies and Agile methodologies such as SCRUM
- Knowledge of Agile practices
- Knowledge and understanding of software design patterns
- Knowledge of Open Source Technologies
- Transportation industry knowledge desired but not required
- Experience mentoring less experienced developers
- Experience Leading Data Team
- Approaches software development using industry standard concepts, practices, and procedures
- Ability to work on multiple tasks simultaneously with various team members and/or other groups both internally and externally
- Ability to effectively engage in self-directed time management and prioritization of workload
- Ability to work independently to develop new skills as needed to support new products and applications
- Ability to coordinate and oversee parallel efforts for multiple developers
