Data Engineer

As a Data Exchange Platform Developer, your role involves collaborating with city officials, agencies, system integrators, solution providers, and other data sources to integrate relevant datasets into the data exchange platform. You will be responsible for deploying and managing the data exchange, supporting application providers in building optimized management and citizen solutions. Your key responsibilities include: - Evaluating APIs and datasets - Creating data models - Developing software ETL modules - Performing unit testing - Deploying modules in cloud environments - Developing ETL modules in Python for data ingestion using REST APIs and streaming protocols like AMQP and MQTT - Containerizing adapters - Creating data models and catalogue entries as per data exchange specifications - Following best practices for software development - Adhering to Agile methodology To excel in this role, you should possess the following qualifications and skills: - Excellent software design and programming skills in Python, Java/JavaScript - Proficiency in data model technologies such as JSON-schema, JSON-LD, Protobuf - Strong functional testing skills - Solid understanding of city solution APIs and data sources - Experience in developing ETL modules for data ingestion using REST APIs and streaming protocols - Proficient in working with geospatial data and GIS tools like QGIS and ESRI - Skilled in analyzing, transforming, and visualizing geospatial data - Proficiency in Linux, GIT version control, and cloud computing platforms - Understanding of IoT, GIS, Big Data, Cloud applications - Experience in containerization using Docker - Familiarity with Kubernetes for orchestration - Knowledge of monitoring and logging tools like Prometheus, Grafana, Logstash would be advantageous The company focuses on utilizing technology to improve operational efficiency and service delivery, with a commitment to creating a positive societal impact.,