Data Engineer

dharani.wikilabs@gmail.com

Data Engineer

for designing, implementing, and maintaining our data infrastructure to support our B2B intelligence platform.

Responsibilities

• Design, build, and maintain scalable data pipelines for collecting, processing, and storing large

volumes of business data

• Develop

  ETL

  processes to integrate data from various sources, including web scraping, APIs, and

third-party data providers

• Implement data quality checks and monitoring systems to ensure data accuracy and integrity
• Optimize data storage and retrieval processes for high performance and scalability
• Collaborate with data scientists to implement machine learning models in production

environments

• Work with the backend team to design and implement APIs for data access
• Collaborate with data scientists to develop and deploy machine learning models.
• Implement data security and privacy measures to protect sensitive information.
• Stay up to date with the latest big data technologies and best practices

Requirements

• Bachelors or Masters degree in Computer Science, Engineering, or a related field
• 4+ years of experience in data engineering roles
• Strong programming skills in Python, Scala and/or Java
• Expertise in SQL and experience with NoSQL databases (e.g., MongoDB, Cassandra)
• Proficiency with big data technologies such as Apache Spark, Hadoop, and Kafka
• Experience with cloud platforms (AWS, GCP, or Azure) and their data services
• Familiarity with data warehousing concepts and ETL processes
• Experience with data warehousing solutions (e.g., AWS Redshift, Snowflake).
• Knowledge of data modelling, data architecture, and data pipeline design
• Experience with version control systems (e.g., Git) and CI/CD practices
• Excellent problem-solving skills and attention to detail

Preferred Qualifications

• Experience in the

  B2B data

  or sales intelligence industry
• Familiarity with web scraping techniques and tools
• Knowledge of data privacy regulations (e.g.,

  GDPR, CCPA

  )
• Experience with real-time data processing and streaming architectures

If you're interested, please reply with your details updated resume in

dharani.wikilabs@gmail.com