Job
Description
As a Big Data Engineer at Findem, you will play a crucial role in building, deploying, and managing data pipelines, data lake infrastructure, and Big data processing solutions using various Big data and ETL technologies. Your expertise in AI-assisted coding, particularly with tools like Cline, Cursor, or similar, will be essential for enhancing the efficiency of modern engineering workflows. In addition to your core engineering skills, you will be responsible for assembling and processing large, complex data sets from diverse sources like MongoDB, S3, Server-to-Server, Kafka, etc., using SQL and big data technologies. Your ability to build analytical tools for actionable insights and data models for performance and scalability will be instrumental in driving key business metrics and enhancing operational efficiency. **Key Responsibilities:** - Build data pipelines, Big data processing solutions, and data lake infrastructure using various Big data and ETL technologies - Assemble and process large, complex data sets from diverse sources like MongoDB, S3, Server-to-Server, Kafka, etc., using SQL and big data technologies - Build analytical tools to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics - Build data models and data schema for performance, scalability, and functional requirements - Research, experiment, and prototype new tools/technologies to drive innovation and success **Qualifications Required:** - Strong proficiency in Python/Scala - Highly proficient in AI-assisted coding with tools like Cline, Cursor, or similar - Experience in Big data technologies like Spark, Hadoop, Athena/Presto, Redshift, Kafka, etc. - Proficiency in various file formats like parquet, JSON, Avro, orc, etc. - Experience in workflow management tools like airflow - Familiarity with batch processing, streaming, and message queues - Exposure to visualization tools like Redash, Tableau, Kibana - Experience working with structured and unstructured data sets - Strong problem-solving skills As a Big Data Engineer at Findem, you will have the opportunity to work with cutting-edge technologies, drive innovation, and contribute to the success of the company. The role is full-time and comes with full benefits. Findem is globally headquartered in the San Francisco Bay Area with its India headquarters in Bengaluru. Please note that Findem may use artificial intelligence (AI) tools in the hiring process to support recruitment activities. These tools assist the recruitment team but do not replace human judgment. Final hiring decisions are made by humans. If you have any questions regarding data processing, feel free to reach out to us.,
