Data Software Engineer DMAIC Academy & Consultancy

5.0 - 12.0 years

0 Lacs

coimbatore, tamil nadu

On-site

You should have 5-12 years of experience in Big Data & Data related technologies. Your expertise should include a deep understanding of distributed computing principles and strong knowledge of Apache Spark. Proficiency in Python programming is required, along with experience using technologies such as Hadoop v2, Map Reduce, HDFS, Sqoop, Apache Storm, and Spark-Streaming for building stream-processing systems. You should have a good understanding of Big Data querying tools like Hive and Impala, as well as experience in integrating data from various sources such as RDBMS, ERP, and Files. Knowledge of SQL queries, joins, stored procedures, and relational schemas is essential. Experience with NoSQL databases like HBase, Cassandra, and MongoDB, along with ETL techniques and frameworks, is also expected. The role requires performance tuning of Spark Jobs, experience with AZURE Databricks, and the ability to efficiently lead a team. Designing and implementing Big Data solutions, as well as following AGILE methodology, are key aspects of this position.,

Posted 15 hours ago

Apply

Data Software Engineer OptimHire

5.0 - 12.0 years

0 Lacs

chennai, tamil nadu

On-site

You should have 5-12 years of experience in Big Data & Data related technologies, with expertise in distributed computing principles. Your skills should include an expert level understanding of Apache Spark and hands-on programming with Python. Proficiency in Hadoop v2, Map Reduce, HDFS, and Sqoop is required. Experience in building stream-processing systems using technologies like Apache Storm or Spark-Streaming, as well as working with messaging systems such as Kafka or RabbitMQ, will be beneficial. A good understanding of Big Data querying tools like Hive and Impala, along with integration of data from multiple sources including RDBMS, ERP, and Files, is necessary. You should possess knowledge of SQL queries, joins, stored procedures, and relational schemas. Experience with NoSQL databases like HBase, Cassandra, and MongoDB, along with ETL techniques and frameworks, is expected. Performance tuning of Spark Jobs and familiarity with native Cloud data services like AWS or AZURE Databricks is essential. The role requires the ability to efficiently lead a team, design and implement Big data solutions, and work as a practitioner of AGILE methodology. This position falls under the category of Data Engineer and is suitable for individuals with expertise in ML/AI Engineers, Data Scientists, and Software Engineers.,

Posted 1 day ago

Apply

Data Engineer Python Scry Analytics

3.0 - 7.0 years

0 Lacs

pune, maharashtra

On-site

You will be part of our Data Engineering Team in Gurgaon, Noida, or Pune, contributing to the development and management of large enterprise data and analytics platforms. Your role will involve collaborating with the data engineering and data science teams to implement scalable data lakes, data ingestion platforms, machine learning analytics platforms, and more. With at least 3 years of industry experience, you will be responsible for creating end-to-end data solutions and optimal data processing pipelines for handling large volumes of diverse data types. Proficiency in Python, including knowledge of design patterns and strong design skills, is essential. You should have expertise in working with PySpark Dataframes, Pandas Dataframes, and developing efficient data manipulation tasks. Experience in building Restful web services, API platforms, SQL, NoSQL databases, and stream-processing systems like Spark-Streaming and Kafka will be crucial. You will collaborate with the data science and infrastructure teams to deploy machine learning solutions in production environments. It would be advantageous if you have experience with testing libraries like pytest, knowledge of Dockers, Kubernetes, model versioning with mlflow, and microservices libraries. Familiarity with machine learning algorithms and libraries is a plus. Our ideal candidate is proactive, independent, and enjoys problem-solving. Continuous learning is encouraged as we are a rapidly growing company. Being a team player and an effective communicator is essential for success in this role.,

Posted 6 days ago

Apply

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.