4 Sparkml Jobs

As a Senior AWS Data Engineer Cloud Data Platform at Teamware Solutions, a division of Quantum Leap Consulting Pvt. Ltd, located in Bangalore, you will be responsible for end-to-end implementation of Cloud data engineering solutions like Enterprise Data lake and Data hub in AWS. Working onsite in an office environment for 5 days a week, you will collaborate with the Offshore Manager and Onsite Business Analyst to understand the requirements and deliver scalable, distributed, cloud-based enterprise data solutions. You should have a strong background in AWS cloud technology, with 4-8 years of hands-on experience. Proficiency in architecting and delivering highly scalable solutions is a must, along with expertise in Cloud data engineering solutions, Lambda or Kappa Architectures, Data Management concepts, and Data Modelling. You should be proficient in AWS services such as EMR, Glue, S3, Redshift, and DynamoDB, as well as have experience in Big Data frameworks like Hadoop and Spark. Additionally, you must have hands-on experience with AWS compute and storage services, AWS Streaming Services, troubleshooting and performance tuning in Spark framework, and knowledge of Application DevOps tools like Git and CI/CD Frameworks. Familiarity with AWS CloudWatch, Cloud Trail, Account Config, Config Rules, security, key management, data migration processes, and analytical skills is required. Good communication and presentation skills are essential for this role. Desired skills include experience in building stream-processing systems, Big Data ML toolkits, Python, Offshore/Onsite Engagements, flow tools like Airflow, Nifi or Luigi, and AWS services like STEP & Lambda. A professional background in BE/B.Tech/MCA/M.Sc/M.E/M.Tech/MBA is preferred, and an AWS certified Data Engineer certification is recommended. If you are interested in this position and meet the qualifications mentioned above, please send your resume to netra.s@twsol.com.,

As a member of the Google Cloud Consulting Professional Services team, you will have the opportunity to contribute to the success of businesses by guiding them through their cloud journey and leveraging Google's global network, data centers, and software infrastructure. Your role will involve assisting customers in transforming their businesses by utilizing technology to connect with customers, employees, and partners. Your responsibilities will include interacting with stakeholders to understand customer requirements and providing recommendations for solution architectures. You will collaborate with technical leads and partners to lead migration and modernization projects to Google Cloud Platform (GCP). Additionally, you will design, build, and operationalize data storage and processing infrastructure using Cloud native products, ensuring data quality and governance procedures are in place to maintain accuracy and reliability. In this role, you will work on data migrations, modernization projects, and design data processing systems optimized for scaling. You will troubleshoot platform/product tests, understand data governance and security controls, and travel to customer sites to deploy solutions and conduct workshops to educate and empower customers. Furthermore, you will be responsible for translating project requirements into goals and objectives, creating work breakdown structures to manage internal and external stakeholders effectively. You will collaborate with Product Management and Product Engineering teams to drive excellence in products and contribute to the digital transformation of organizations across various industries. By joining this team, you will play a crucial role in shaping the future of businesses of all sizes and assisting them in leveraging Google Cloud to accelerate their digital transformation journey.,

As an Artificial Intelligence Specialist at Gen AI, you will be responsible for driving customer conversations, understanding customer requirements, creating Gen AI solution architectures, and developing customer proposals and RFP responses. You will guide solution engineers in creating Gen AI POCs and solutions for various industry verticals. Your role will involve staying updated on the latest technology developments, industry best practices, and incorporating them into Gen AI applications. Additionally, you will design and deploy Proof of Concepts (POCs) and Points of View (POVs) across different industry verticals to showcase the potential of Generative AI applications. To qualify for this role, you should have at least 8 years of experience in software development, with a minimum of 3 years of experience in Generative AI solution development. A bachelor's degree or higher in Computer Science, Software Engineering, or related fields is required. You should be adept at critical thinking, logical reasoning, and have a strong ability to learn new industry domains quickly. Being a team player who can deliver under pressure is essential. Furthermore, you should have experience with cloud technologies such as Azure, AWS, or GCP, as well as a good understanding of NVIDIA or similar technologies. A solid appreciation of AI/ML concepts and sound design principles is necessary for this role. In terms of required skills, you should be extremely dynamic and enthusiastic about technology. Development experience with languages like C++, Java, JavaScript, HTML, C#, Python, or node.js is preferred. You should be able to adapt quickly to new challenges and evolving technology stacks. Excellent written and verbal communication skills in English are essential, along with strong analytical and critical thinking abilities. A customer-focused attitude, initiative-taking, self-driven nature, and the ability to learn quickly are also important qualities for this role. Knowledge of Python, ML Algorithms, Statistics, source code maintenance, versioning tools, Object-Oriented Programming Concepts, debugging, and analytical skills is required. Preferred skills for this position include at least 5 years of experience in ML development and MLOps. Strong programming skills in Python, knowledge of ML, Data, and API libraries, and expertise in creating end-to-end data pipelines are advantageous. Experience with ML models, ModelOps/MLOps, AutoML, AI Ethics, Trust, Explainable AI, and popular ML frameworks like SparkML, TensorFlow, scikit-learn, XGBoost, H2O, etc., is beneficial. Familiarity with working in cloud environments (AWS, Azure, GCP) or containerized environments (Mesos, Kubernetes), interest in understanding functional and industry business challenges, and knowledge of IT industry and GenAI use cases in insurance processes are preferred. Expertise in Big Data and Data Modeling is also desirable for this role.,

This role is for one of Weekday's clients Min Experience: 8 years Location: Pune JobType: full-time Title: Principal Data Engineer Job Description: Job Summary: Lead the design, development, and optimization of enterprise-scale data pipelines and architectures to drive data-driven decision-making. Architect efficient data tables and schemas to enhance query performance, ensuring seamless integration with cross-functional teams Key Responsibilities: Data Pipeline Design and implement high-performance data pipelines using ETL processes, batch/streaming frameworks (e.g., Apache Spark, Airflow), and cloud platforms (AWS). Optimize data ingestion, transformation, and storage workflows to meet scalability, reliability, and latency requirements. Data Architecture & Query Efficiency Architect database schemas and dimensional models (e.g., star/snowflake) to maximize query performance and reduce latency. Implement indexing strategies, partitioning, and materialized views to optimize data retrieval. Collaboration & Governance Partner with data scientists, product managers, and engineers to align data infrastructure with business needs. Establish data governance frameworks, ensuring compliance with security, privacy, and quality standards. Mentor junior engineers and foster best practices in data engineering Innovation & Leadership Stay ahead of industry trends (e.g., AI/ML integration, real-time analytics) and advocate for emerging technologies. Lead technical strategy for data infrastructure, balancing innovation with operational stability. Technical Expertise: 8+ years of experience in data engineering, with a focus on ETL, data modelling, and cloud-based architectures. Proficiency in SQL, Python, and tools like Spark and Snowflake. Strong knowledge of database design, dimensional modeling, and query optimization. Leadership & Collaboration Proven ability to lead teams, mentor engineers, and drive technical initiatives. Experience translating business requirements into scalable technical solutions. Preferred qualification Expertise in machine learning model deployment or real-time analytics. Familiarity with DevOps practices (CI/CD, Docker, Kubernetes). Our Core Values Data Fanatics: Our edge is always found in the data Partner Obsessed: We are obsessed with partner success Team of Doers: We have a bias for action Game Changers: We encourage innovation