ETL Developer - Power BI/Big Data

As an experienced ETL Developer, your role will involve developing and optimizing scalable data pipelines for efficient extraction, transformation, and loading (ETL) of data from diverse sources. You will leverage technologies like Spark, Dask, and other modern ETL frameworks to ensure smooth data processing. **Key Responsibilities:** - Developing and implementing data quality controls, including monitoring and remediation processes - Providing technical guidance and mentorship to junior ETL developers - Collaborating with the infrastructure team to ensure the availability, reliability, and scalability of ETL solutions - Participating in code reviews and contributing to the development of coding standards and best practices - Architecting and implementing advanced data models for both NoSQL (e.g., HBase) and relational databases - Establishing robust systems to ensure data quality, implementing monitoring tools and validation frameworks - Designing, building, and maintaining high-performance APIs to provide seamless access to curated data sets - Overseeing the assembly of large, complex data sets that meet functional, non-functional, and business requirements - Conducting root cause analysis on internal and external data and processes to resolve business inquiries **Qualifications Required:** - 2+ years of experience in developing, testing, and maintaining applications using Python, Bash, and SQL - Strong expertise in writing clean, efficient, and maintainable code - In-depth knowledge of SQL and NoSQL databases, including database design and optimization techniques - Extensive experience with parallel processing and distributed computing frameworks like Hadoop, Spark, or Dask - Proven track record in custom ETL design, implementation, and operational maintenance - Familiarity with ETL orchestration frameworks such as Luigi, Apache Airflow, or Apache NiFi - Proficiency in backend frameworks like Django or Flask - Strong experience with cloud platforms (AWS, GCP, Microsoft Azure) - Exposure to data analytics, data science, or machine learning workflows is a plus **Additional Details:** Please note that this job description is provided by hirist.tech.,