Data Migration & Integration: Lead data migration projects to move on-premise data (Oracle, SQL Server) to AWS cloud-based solutions (Redshift, S3, RDS). Coordinate end-to-end processes for seamless data transfers and integrations across platforms. Data Ingestion & Pipelines: Design, implement, and maintain robust, scalable data ingestion pipelines to integrate structured and unstructured data from various sources into cloud-based platforms like AWS. Use tools like Informatica, Apache Airflow, or custom Python solutions. Cloud Infrastructure (AWS): Develop cloud-native data architectures using AWS services (e.g., Redshift, S3, Glue, Lambda, RDS, and Athena). Optimize data storage, processing, and retrieval in AWS to meet performance and cost efficiency goals. ETL/ELT Development: Build and optimize ETL/ELT processes using tools like Informatica, AWS Glue, AWS Transfer Family and custom Python-based solutions. Ensure data flows are automated, efficient, and scalable. Automation & Workflow Orchestration: Implement and manage workflow orchestration with Apache Airflow to automate and schedule ETL tasks, ensuring reliable data delivery to target systems. Collaborate with Stakeholders: Work closely with business users, analysts, and other engineering teams to understand data requirements, propose data solutions, and ensure alignment with business goals. Translate business needs into technical solutions. Data Quality & Governance: Ensure data quality, integrity, and compliance with governance policies. Implement monitoring and logging to ensure pipeline health and detect anomalies or issues proactively. Mentoring & Leadership: Mentor and guide junior data engineers, helping them grow in their technical skills and best practices. Promote a culture of continuous learning and high performance. Performance Tuning & Optimization: Continuously monitor and optimize data pipeline performance, troubleshoot issues, and apply best practices for improving query performance and data processing times

As a Data Engineer at IBM, you'll play a vital role in the development, design of application, provide regular support/guidance to project teams on complex coding, issue resolution and execution. Your primary responsibilities include: Lead the design and construction of new solutions using the latest technologies, always looking to add business value and meet user requirements. Strive for continuous improvements by testing the build solution and working under an agile framework. Discover and implement the latest technologies trends to maximize and build creative solutions Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Design and Develop Data Solutions, Design and implement efficient data processing pipelines using AWS services like AWS Glue, AWS Lambda, Amazon S3, and Amazon Redshift. Develop and manage ETL (Extract, Transform, Load) workflows to clean, transform, and load data into structured and unstructured storage systems. Build scalable data models and storage solutions in Amazon Redshift, DynamoDB, and other AWS services. Data Integration: Integrate data from multiple sources including relational databases, third-party APIs, and internal systems to create a unified data ecosystem. Work with data engineers to optimize data workflows and ensure data consistency, reliability, and performance. Automation and Optimization: Automate data pipeline processes to ensure efficiency Preferred technical and professional experience Define, drive, and implement an architecture strategy and standards for end-to-end monitoring. Partner with the rest of the technology teams including application development, enterprise architecture, testing services, network engineering, Good to have detection and prevention tools for Company products and Platform and customer-facing

As a Data Engineer at IBM, you'll play a vital role in the development, design of application, provide regular support/guidance to project teams on complex coding, issue resolution and execution. Your primary responsibilities include: Lead the design and construction of new solutions using the latest technologies, always looking to add business value and meet user requirements. Strive for continuous improvements by testing the build solution and working under an agile framework. Discover and implement the latest technologies trends to maximize and build creative solutions Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Design and Develop Data Solutions, Design and implement efficient data processing pipelines using AWS services like AWS Glue, AWS Lambda, Amazon S3, and Amazon Redshift. Develop and manage ETL (Extract, Transform, Load) workflows to clean, transform, and load data into structured and unstructured storage systems. Build scalable data models and storage solutions in Amazon Redshift, DynamoDB, and other AWS services. Data Integration: Integrate data from multiple sources including relational databases, third-party APIs, and internal systems to create a unified data ecosystem. Work with data engineers to optimize data workflows and ensure data consistency, reliability, and performance. Automation and Optimization: Automate data pipeline processes to ensure efficiency Preferred technical and professional experience Define, drive, and implement an architecture strategy and standards for end-to-end monitoring. Partner with the rest of the technology teams including application development, enterprise architecture, testing services, network engineering, Good to have detection and prevention tools for Company products and Platform and customer-facing

Technical Lead Budget is open C2H Clint Name:Durapid Tech Location:remote Experience: 10+ years (including at least 2 years in a leadership role) Mode: Remote Contract Duration: 6 months Role Description: We seek a seasoned Technical Lead to guide and mentor a dynamic engineering team. This role is critical in designing, developing, and optimizing high-performance, scalable, and reliable backend systems. The ideal candidate should have deep expertise in Python (Flask), AWS (Lambda, Redshift, Glue, S3), Microservices, and Database Optimization (SQL, RDBMS) . We operate in a high-performance environment, comparable to leading product companies, where uptime, defect reduction, and data clarity are paramount. The candidate must ensure engineering excellence, maintain high-quality standards, and drive innovation in software architecture and development. Key Responsibilities: Lead backend architecture and develop scalable, efficient web applications and microservices. Ensure production-grade AWS deployment with high availability, cost optimization, and security best practices. Design and optimize databases (RDBMS, SQL) for performance, scalability, and reliability. Lead API and microservices development ensuring integration, scalability, and maintainability. Implement high-performance solutions emphasizing low latency, uptime, and data accuracy. Mentor developers and promote a culture of collaboration, disciplined coding, and technical excellence. Conduct technical reviews and enforce coding best practices. Drive automation and CI/CD pipelines to enhance deployment efficiency. Effectively communicate technical concepts to technical and non-technical stakeholders. Provide accurate work estimations and align development efforts with broader business objectives. Key Skills: Programming: Strong expertise in Python (Flask) and Celery. AWS: Lambda, Redshift, Glue, S3, and production-level deployment strategies. Microservices & API Development: Service discovery, API gateway design, observability, and distributed systems best practices. Database Optimization: SQL, PostgreSQL, Amazon Aurora RDS, and performance tuning. CI/CD & Infrastructure: GitHub Actions, GitLab CI/CD, Docker, Kubernetes, Terraform, and CloudFormation. Monitoring & Logging: AWS CloudWatch, ELK Stack, and Prometheus. Security & Compliance: Backend security best practices and performance optimization. Collaboration & Communication: Ability to convey complex technical concepts to international stakeholders and work seamlessly in Agile/Scrum environments. Preferred Experience: Experience in high-performance, product-focused companies emphasizing uptime, defect reduction, and system reliability. Hands-on leadership in scaling cloud infrastructure and optimizing backend services. Proven ability to lead and mentor a development team while driving strategic technical initiatives. Location - Lakshadweep,chandigarh,dadra,daman,delhi,diu,goa,haveli,pondicherry,sikkim

Position Title: Specialist- Data Science Business Title: Engineer II- Data Science We are seeking a highly motivated and enthusiastic Senior Data Scientist with over 8 years of experience to join our dynamic team. The ideal candidate will have a strong background in AI/ML analytics and a passion for leveraging data to drive business insights and innovation. Key Responsibilities: Develop and implement machine learning models and algorithms. Work closely with project stakeholders to understand requirements and translate them into deliverables. Utilize statistical and machine learning techniques to analyze and interpret complex data sets. Stay updated with the latest advancements in AI/ML technologies and methodologies. Collaborate with cross-functional teams to support various AI/ML initiatives. Qualifications: Bachelors degree in Computer Science, Data Science, Statistics, Mathematics, or a related field. Strong understanding of machine learning, deep learning and Generative AI concepts. Preferred Skills: Experience in machine learning techniques such as Regression, Classification, Predictive modeling, Clustering, Computer vision(yolo ), Deep Learning stack, NLP using python Strong knowledge and experience in Generative AI/ LLM based development. Strong experience working with key LLM models APIs (e.g. AWS Bedrock OR Azure Open AI/ OpenAI) and LLM Frameworks (e.g. LangChain OR LlamaIndex OR RAG ) Experience with cloud infrastructure for AI/Generative AI/ML on AWS, Azure. Expertise in building enterprise grade, secure data ingestion pipelines (ETL Gluejob, Quicksight) for unstructured data including indexing, search, and advance retrieval patterns. Knowledge of effective text chunking techniques for optimal processing and indexing of large documents or datasets. Proficiency in generating and working with text embeddings with understanding of embedding spaces and their applications in semantic search and information. retrieval. Experience with RAG concepts and fundamentals ( VectorDBs, AWS OpenSearch, semantic search, etc.), Expertise in implementing RAG systems that combine knowledge bases with Generative AI models. Knowledge of training and fine-tuning Foundation Models (Athropic, Claud, Mistral, etc.), including multimodal inputs and outputs. Proficiency in Python, TypeScript, NodeJS, ReactJS (and equivalent) and frameworks. (e.g., pandas, NumPy, scikit-learn), Glue crawler, ETL Experience with data visualization tools (e.g., Matplotlib, Seaborn, Quicksight). Knowledge of deep learning frameworks (e.g., TensorFlow, Keras, PyTorch). Experience with version control systems (e.g., Git, CodeCommit). Good to have Skills Knowledge and Experience in building knowledge graphs in production. Understanding of multi-agent systems and their applications in complex problem-solving scenarios. ** Title/Designation of role are as per Global Team Job profiling. Local Titles are adjusted for understanding of regional candidates applying for job in India. Designations are dependent on years of relevant work experience and performance during interviews.

- Highly experiences in developing ETL pipelines using AWS Glue and EMR with PySpark/Scala. - Utilize AWS services (S3, Glue, Lambda, EMR, Step Functions) for data solutions. - Design scalable data models for analytics and reporting. - Implement data validation, quality, and governance practices. - Optimize Spark jobs for cost and performance efficiency. - Automate ETL workflows with AWS Step Functions and Lambda. - Collaborate with data scientists and analysts on data needs. - Maintain documentation for data architecture and pipelines. - Experience with Open source bigdata file formats such as Iceberg or delta or Hundi - Desirable to have experience in terraforming AWS data analytical resources. Must-Have Skills: - AWS (S3, Glue, EMR Lambda, EMR), PySpark or Scala, SQL, ETL development. Good-to-Have Skills: - Snowflake, Cloudera Hadoop (HDFS, Hive, Impala), Iceberg

Role: AWS Data Engineer Exp.: 5+ years Location: Gurugram, Noida & Pune (Hybrid 3 days work from Office) Job Description : Candidate should Provide technical expertise in needs identification, data modeling, data movement, and translating business needs into technical solutions with adherence to established data guidelines and approaches from a business unit or project perspective. Good knowledge of conceptual, logical, and physical data models, the implementation of RDBMS, operational data store (ODS), data marts, and data lakes on target platforms (SQL/NoSQL). Oversee and govern the expansion of existing data architecture and the optimization of data query performance via best practices. The candidate must be able to work independently and collaboratively Requirement: 5+ Years of experience as a Data Engineer Strong technical expertise in SQL is a must Strong knowledge of joins and common table expressions (CTEs) Strong experience with Python Experience in Data brick, Pyspark Strong expertise in ETL process and with various data model concepts Knowledge of star schema and snowflake schema Good to know about AWS services such as S3, Athena, Glue, EMR/Spark with a major emphasis on S3 and Glue Experience with Big Data Tools and technologies Key Skills: Good Understanding of data structures and data analysis using SQL or Python Knowledge of Insurance Domain is an addition. Knowledge of implementing ETL/ELT for data solutions end-to-end Understanding requirements, and data solutions (ingest, storage, integration, processing) Knowledge of analyzing data using SQL Conducting End to End verification and validation for the entire application Responsibilities : Understand and translate business needs into data models supporting long-term solutions. Perform reverse engineering of physical data models from databases and SQL scripts. Analyze data-related system integration challenges and propose appropriate solutions. Assist with and support setting the data architecture direction (including data movement approach, architecture/technology strategy, and any other data-related considerations to ensure business value)

PySpark with experience in architecting high throughput data lakes. Understanding of CDC and respective tools like Debezium, DMS. Workflows – Airflow or Step Functions, or Glue Workflow. Glue ecosystem – Glue Jobs, Data Catalog, Bookmarks, Crawlers.

Mandatory Skills : Data engineer , AWS Athena, AWS Glue,Redshift,Datalake,Lakehouse,Python,SQL Server Must Have Experience: 6+ years of hands-on data engineering experience Expertise with AWS services: S3, Redshift, EMR, Glue, Kinesis, DynamoDB Building batch and real-time data pipelines Python, SQL coding for data processing and analysis Data modeling experience using Cloud based data platforms like Redshift, Snowflake, Databricks Design and Develop ETL frameworks Nice-to-Have Experience: ETL development using tools like Informatica, Talend, Fivetran Creating reusable data sources and dashboards for self-service analytics Experience using Databricks for Spark workloads or Snowflake Working knowledge of Big Data Processing CI/CD setup Infrastructure-as-code implementation Any one of the AWS Professional Certification

Job Description AWS Data engineer Hadoop Migration We are seeking an experienced AWS Principal Data Architect to lead the migration of Hadoop DWH workloads from on-premise to AWS EMR. As an AWS Data Architect, you will be a recognized expert in cloud data engineering, developing solutions designed for effective data processing and warehousing requirements of large enterprises. You will be responsible for designing, implementing, and optimizing the data architecture in AWS, ensuring highly scalable, flexible, secured and resilient cloud architectures solving business problems and helps accelerate the adoption of our clients data initiatives on the cloud. Key Responsibilities: Lead the migration of Hadoop workloads from on-premise to AWS-EMR stack. Design and implement data architectures on AWS, including data pipelines, storage, and security. Collaborate with cross-functional teams to ensure seamless migration and integration. Optimize data architectures for scalability, performance, and cost-effectiveness. Develop and maintain technical documentation and standards. Provide technical leadership and mentorship to junior team members. Work closely with stakeholders to understand business requirements, and ensure data architectures meet business needs. Work alongside customers to build enterprise data platforms using AWS data services like Elastic Map Reduce (EMR), Redshift, Kinesis, Data Exchange, Data Sync, RDS , Data Store, Amazon MSK, DMS, Glue, Appflow, AWA Zero-ETL, Glue Data Catalog, Athena, Lake Formation, S3, RMS, Data Zone, Amazon MWAA, APIs Kong Deep understanding of Hadoop components, conceptual processes and system functioning and relative components in AWS EMR and other AWS services. Good experience on Spark-EMR Experience in Snowflake/Redshift Good idea of AWS system engineering aspects of setting up CI-CD pipelines on AWS using Cloudwatch, Cloudtrail, KMS, IAM IDC, Secret Manager, etc Extract best-practice knowledge, reference architectures, and patterns from these engagements for sharing with the worldwide AWS solution architect community Basic Qualifications: 10+ years of IT experience with 5+ years of experience in Data Engineering and 5+ years of hands-on experience in AWS Data/EMR Services (e.g. S3, Glue, Glue Catalog, Lake Formation) Strong understanding of Hadoop architecture, including HDFS, YARN, MapReduce, Hive, HBase. Experience with data migration tools like Glue, Data Sync. Excellent knowledge of data modeling, data warehousing, ETL processes, and other Data management systems. Strong understanding of security and compliance requirements in cloud. Experience in Agile development methodologies and version control systems. Excellent communication an leadership skills. Ability to work effectively across internal and external organizations and virtual teams. Deep experience on AWS native data services including Glue, Glue Catalog, EMR, Spark-EMR, Data Sync, RDS, Data Exchange, Lake Formation, Athena, AWS Certified Data Analytics – Specialty. AWS Certified Solutions Architect – Professional. Experience on Containerization and serverless computing. Familiarity with DevOps practices and automation tools. Experience in Snowflake/Redshift implementation is additionally preferred. Preferred Qualifications: Technical degrees in computer science, software engineering, or mathematics Cloud and Data Engineering background with Migration experience. Other Skills: A critical thinker with strong research, analytics and problem-solving skills Self-motivated with a positive attitude and an ability to work independently and or in a team Able to work under tight timeline and deliver on complex problems. Must be able to work flexible hours (including weekends and nights) as needed. A strong team player

Bachelor's or Master's degree in Computer Science, Information Technology, or a related field. Strong focus on AWS and PySpark. Knowledge of AWS services, including but not limited to S3, Redshift, Athena, EMR, and Glue. Proficiency in PySpark and related Big Data technologies for ETL processing. Strong SQL skills for data manipulation and querying. Familiarity with data warehousing concepts and dimensional modeling. Experience with data governance, data quality, and data security practices. Excellent problem-solving skills and attention to detail. Strong communication and collaboration skills to work effectively with cross-functional teams.

Role & responsibilities Technical Skills : AWS Glue, Athena, SQL, Python • Will need to be able to understand existing data models. • Be able to build data pipelines using Glue and provision cloud resources. • Be able to perform data analysis and data quality checks and fix the data quality issues working with the business stakeholders or POs • Will need to follow leading practices to build pipelines in accordance with Vanguard patterns. • Will be working with stakeholders when there are critical issues that needs to be addressed Preferred candidate profile Must Have skills : Pyspark; AWS Glue; SQL. Good to Have : Redshift; Arora; Dynamo. Perks and benefits Mode: 5 Days WFO Shift: 1:30 PM 10:30 PM, During Daylight Saving: 12:30 PM – 9:30 PM

Experience 10+ Years. Must have architect experience. Data Architect : 1.AWS : PySpark with experience in architecting high throughput data lakes Glue ecosystem – Glue Jobs, Data Catalog, Bookmarks, Schema Registry, and Crawlers. Workflows – Airflow or Step Functions, or Glue Workflow Data Modeling – including medallion architecture, star schema (for architect) Understanding of Parquet and Iceberg file formats – including optimizing, partitioning and snapshotting (for architect) Good understanding of Data lineage, Data Governance, and DQ and AWS services around the same like Glue DQ, Deequ, GE, Lake Formation Redshift – Nice to have (since it is part of their existing infrastructure) Understanding of CDC and respective tools like Debezium, DMS AWS Certified Data Engineer Preferred OR 2. Databricks : PySpark with experience in architecting/building high throughput data lakes Workflows Autoloaders Understanding of Parquet and Iceberg/Hudi file formats – including optimizing, partitioning and snapshotting (for architect). Should understand DLTs Good understanding of Data lineage, Data Governance, and DQ including Unity Catalog Understanding of CDC and respective tools like Debezium, DMS Databricks certified engineer preferred

About the Role: We are seeking a skilled QuickSight Developer to design and implement advanced business intelligence (BI) solutions using Amazon QuickSight. The ideal candidate will have a strong background in data visualization, data modeling, and analytics, with the ability to translate complex data into actionable insights. Key Responsibilities Develop, design, and maintain interactive dashboards and reports using Amazon QuickSight. Collaborate with stakeholders to gather requirements and translate them into actionable BI solutions. Optimize and enhance existing QuickSight dashboards for performance and usability. Connect and integrate QuickSight with various data sources such as Redshift, S3, RDS, and other AWS services. Build robust data models to support analytical requirements. Ensure data quality, consistency, and accuracy in all reporting and visualization efforts. Provide training and support to end-users for effective use of QuickSight dashboards and reports. Stay updated on QuickSight features and industry best practices to implement innovative BI solutions. Qualifications Bachelors degree in Computer Science, Information Technology, or a related field. Proven experience in designing and developing dashboards using Amazon QuickSight . Strong understanding of data visualization principles and best practices. Hands-on experience with AWS services, especially Redshift, S3, RDS, and Athena. Proficiency in SQL for data extraction and transformation. Experience with ETL processes and data modeling techniques. Familiarity with scripting languages like Python or tools like AWS Glue is a plus. Strong analytical and problem-solving skills. Excellent communication and collaboration abilities. Preferred Skills Experience in BI tools like Tableau, Power BI, or Looker. Knowledge of data governance and security best practices in AWS. Ability to handle large datasets and optimize queries for performance. Certifications in AWS or data analytics are an added advantage. What We Offer Opportunity to work on cutting-edge BI solutions. Competitive salary and benefits package. A collaborative and innovative work environment. Professional development and training opportunities.

Job Title: Product Engineer - Big Data Location: Mumbai Experience: 3 - 8 Yrs Job Summary: As a Product Engineer - Big Data , you will be responsible for designing, building, and optimizing large-scale data processing pipelines using cutting-edge Big Data technologies. Collaborating with cross-functional teams--including data scientists, analysts, and product managers--you will ensure data is easily accessible, secure, and reliable. Your role will focus on delivering high-quality, scalable solutions for data storage, ingestion, and analysis, while driving continuous improvements throughout the data lifecycle. Key Responsibilities: ETL Pipeline Development & Optimization: Design and implement complex end-to-end ETL pipelines to handle large-scale data ingestion and processing. Utilize AWS services like EMR, Glue, S3, MSK (Managed Streaming for Kafka), DMS (Database Migration Service), Athena, and EC2 to streamline data workflows, ensuring high availability and reliability. Big Data Processing: Develop and optimize real-time and batch data processing systems using Apache Flink, PySpark, and Apache Kafka . Focus on fault tolerance, scalability, and performance. Work with Apache Hudi for managing datasets and enabling incremental data processing. Data Modeling & Warehousing: Design and implement data warehouse solutions that support both analytical and operational use cases. Model complex datasets into optimized structures for high performance, easy access, and query efficiency for internal stakeholders. Cloud Infrastructure Development: Build scalable cloud-based data infrastructure leveraging AWS tools. Ensure data pipelines are resilient and adaptable to changes in data volume and variety, while optimizing costs and maximizing efficiency using Managed Apache Airflow for orchestration and EC2 for compute resources. Data Analysis & Insights: Collaborate with business teams and data scientists to understand data needs and deliver high-quality datasets. Conduct in-depth analysis to derive insights from the data, identifying key trends, patterns, and anomalies to drive business decisions. Present findings in a clear, actionable format. Real-time & Batch Data Integration: Enable seamless integration of real-time streaming and batch data from systems like AWS MSK . Ensure consistency in data ingestion and processing across various formats and sources, providing a unified view of the data ecosystem. CI/CD & Automation: Use Jenkins to establish and maintain continuous integration and delivery pipelines. Implement automated testing and deployment workflows, ensuring smooth integration of new features and updates into production environments. Data Security & Compliance: Collaborate with security teams to ensure data pipelines comply with organizational and regulatory standards such as GDPR, HIPAA , or other relevant frameworks. Implement data governance practices to ensure integrity, security, and traceability throughout the data lifecycle. Collaboration & Cross-Functional Work: Partner with engineers, data scientists, product managers, and business stakeholders to understand data requirements and deliver scalable solutions. Participate in agile teams, sprint planning, and architectural discussions. Troubleshooting & Performance Tuning: Identify and resolve performance bottlenecks in data pipelines. Ensure optimal performance through proactive monitoring, tuning, and applying best practices for data ingestion and storage. Skills & Qualifications: Must-Have Skills: AWS Expertise: Hands-on experience with core AWS services related to Big Data, including EMR, Managed Apache Airflow, Glue, S3, DMS, MSK, Athena, and EC2 . Strong understanding of cloud-native data architecture. Big Data Technologies: Proficiency in PySpark and SQL for data transformations and analysis. Experience with large-scale data processing frameworks like Apache Flink and Apache Kafka . Data Frameworks: Strong knowledge of Apache Hudi for data lake operations, including CDC (Change Data Capture) and incremental data processing. Database Modeling & Data Warehousing: Expertise in designing scalable data models for both OLAP and OLTP systems. In-depth understanding of data warehousing best practices. ETL Pipeline Development: Proven experience in building robust, scalable ETL pipelines for processing real-time and batch data across platforms. Data Analysis & Insights: Strong problem-solving skills with a data-driven approach to decision-making. Ability to conduct complex data analysis to extract actionable business insights. CI/CD & Automation: Basic to intermediate knowledge of CI/CD pipelines using Jenkins or similar tools to automate deployment and monitoring of data pipelines. Required Skills Big Data,Etl, AWS

About Client Hiring for One of Our Multinational Corporations! Job Description Job Title : Data Engineer Relevant Experience : 5 to 8 years Must Have Skills : Python (advanced proficiency) PySpark AWS (Amazon Web Services) , including services like S3, Lambda, EC2, and CloudWatch. SQL (advanced skills in writing complex queries) AWS Glue (experience in ETL processing and managing data pipelines). Good to Have Skills : Familiarity with Big Data frameworks (e.g., Hadoop, Spark) Experience with Data Warehousing Knowledge of Containerization and orchestration (Docker, Kubernetes) Experience with CI/CD pipelines Understanding of Machine Learning concepts and tools. Roles and Responsibilities : Design, develop, and implement data pipelines for processing and transforming large datasets. Work closely with data scientists, analysts, and other stakeholders to ensure data availability, integrity, and performance. Manage and optimize data workflows on AWS using services like Glue, S3, and Lambda. Collaborate in the design of cloud-based architecture and ensure scalability and efficiency of data processing systems. Troubleshoot and resolve data-related issues, ensuring high-quality data delivery. Maintain data security and governance best practices. Location : Bangalore, Hyderabad, Chennai. Notice Period : Immediate to 15 days preferred. Nushiba Taniya M HR Analyst Black and White Business solutions PVT Ltd Bangalore, Karnataka, INDIA. 08067432408 |Nushiba@blackwhite.in|www.blackwhite.in

Hi We are looking for AWS Data Engineer Main Skills: AWS, AWS Glue, Spark, Apache, Airflow, Spark. Exp: 5-10Yrs Location: Hyderabad, Bengaluru, Chennai Work Mode: Hybrid Immediate joiners Notice Period: 0-15 Days Excellent Communication Share your CV to hrd5@ontimesolutions.in or Call 9036023373

Experience: Minimum of 10+ years in database development and management roles. SQL Mastery: Advanced expertise in crafting and optimizing complex SQL queries and scripts. AWS Redshift: Proven experience in managing, tuning, and optimizing large-scale Redshift clusters. PostgreSQL: Deep understanding of PostgreSQL, including query planning, indexing strategies, and advanced tuning techniques. Data Pipelines: Extensive experience in ETL development and integrating data from multiple sources into cloud environments. Cloud Proficiency: Strong experience with AWS services like ECS, S3, KMS, Lambda, Glue, and IAM. Data Modeling: Comprehensive knowledge of data modeling techniques for both OLAP and OLTP systems. Scripting: Proficiency in Python, C#, or other scripting languages for automation and data manipulation. Preferred Qualifications Leadership: Prior experience in leading database or data engineering teams. Data Visualization: Familiarity with reporting and visualization tools like Tableau, Power BI, or Looker. DevOps: Knowledge of CI/CD pipelines, infrastructure as code (e.g., Terraform), and version control (Git). Certifications: Any relevant certifications (e.g., AWS Certified Solutions Architect, AWS Certified Database - Specialty, PostgreSQL Certified Professional) will be a plus. Azure Databricks: Familiarity with Azure Databricks for data engineering and analytics workflows will be a significant advantage. Soft Skills Strong problem-solving and analytical capabilities. Exceptional communication skills for collaboration with technical and non-technical stakeholders. A results-driven mindset with the ability to work independently or lead within a team. Qualification: Bachelor's or masters degree in Computer Science, Information Systems, Engineering or equivalent. 10+ years of experience

As a Data Engineer at IBM, you'll play a vital role in the development, design of application, provide regular support/guidance to project teams on complex coding, issue resolution and execution. Your primary responsibilities include: Lead the design and construction of new solutions using the latest technologies, always looking to add business value and meet user requirements. Strive for continuous improvements by testing the build solution and working under an agile framework. Discover and implement the latest technologies trends to maximize and build creative solutions Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Design and Develop Data Solutions, Design and implement efficient data processing pipelines using AWS services like AWS Glue, AWS Lambda, Amazon S3, and Amazon Redshift. Develop and manage ETL (Extract, Transform, Load) workflows to clean, transform, and load data into structured and unstructured storage systems. Build scalable data models and storage solutions in Amazon Redshift, DynamoDB, and other AWS services. Data Integration: Integrate data from multiple sources including relational databases, third-party APIs, and internal systems to create a unified data ecosystem. Work with data engineers to optimize data workflows and ensure data consistency, reliability, and performance. Automation and Optimization: Automate data pipeline processes to ensure efficiency Preferred technical and professional experience Define, drive, and implement an architecture strategy and standards for end-to-end monitoring. Partner with the rest of the technology teams including application development, enterprise architecture, testing services, network engineering, Good to have detection and prevention tools for Company products and Platform and customer-facing

Project Role :Data Platform Engineer Project Role Description :Assists with the data platform blueprint and design, encompassing the relevant data platform components. Collaborates with the Integration Architects and Data Architects to ensure cohesive integration between systems and data models. Must have skills :AWS Glue

*******This is work from office role******* *******Apply only if you are an early joiner******* *******Candidates with career Gaps and year downs are requested to update their CVs accordingly******* Role & responsibilities Have strong data engineering knowledge and cloud development exposure. Proficiency in python Proficiency in both RDBMS (Mysql preferred) and NoSql datastores Spark, Cassandra, AWS data pipeline stack (Athena, S3, Glue data catalog etc.), Airflow etc are technologies you have used. Very comfortable with data lakes, warehouses, and ETL/ELT paradigms Worked in an agile development environment.• Optional Basic knowledge of statistical analysis, mathematical modelling and machine learning Experience Have used or are very hands-on with Microservices, Docker, Kubernetes, Gradle/Ant,Kafka, GIT/bitbucket in an agile workplace. Develop high quality code with strong unit/integration tests. Comfort with test-driven development is a plus. Comfortable with exploring proven open-source tech stack like Grafana, Kibana, Jira, Prometheus, caches like Redis/Memcached, task queues like celery, to name a few. Proficiency in SQL, Python, Java, Spring boot, Hibernate, REST API development will be a good plus. Preferred candidate profile Last organization preference: Product based , Fintech, NBFC Minimum 3 years of relevant experience in Data Engineering. Candidates from Premium Tier 1 and Tier 2 college will be preferred.

#Hiring #AWSClouddeveloper #DataEngineering Hello Connections....! Hope you are doing well !!! We are hiring for Level 5 MNC Client Role : AWS Cloud developer (C2h (Contract)) Exp : 8+ Years Notice Period : Immediate-20 Days location : Pune/Trivandrum(WFO) MOH : C2h (Contract) Interested candidates can share updated cv (or) any references to mansoor@burgeonits.com Job Description:- Key Responsibilities: Design, develop, and implement data pipelines and #ETL processes using AWS services. Collaborate directly with our #FinOps teams and market units across the organization to understand data requirements and deliver solutions that meet business needs. Optimize and manage data storage solutions using AWS services such as S3, RDS, and NoSQL databases. Ensure data quality, integrity, and security across all data engineering projects. Monitor and troubleshoot data workflows to ensure high availability and performance. Design and build advanced and interactive dashboards using tools such as AWS Quick Sight and Power BI. Create and oversee a cloud billing dashboard to track, manage and optimize cloud costs and Reserved Instance purchases Build a dashboard that provides secure selfservice capabilities to all teams on cloud spend Knowledge of DevOps practices and CI/CD pipelines. Stay updated with the latest AWS technologies and best practices to continuously improve our data infrastructure. Solve technical problems and create viable tooling Design and implement shared services in cloud infrastructure. Use best appropriate infrastructure automation tools to provision cloud infrastructure components Attend important Agile events and finish assigned work packages / tasks Ensure smooth handover of project deliverables to internal and external customers Actively contributing to internal projects such as tooling and documentation Skills: Proven experience as a Cloud Developer or #DataEngineer , with a focus on AWS. Strong proficiency in AWS services #EKS , #EC2 , #S3 , #Lambda , #Glue . Solid understanding of data modeling, #ETL processes, and data warehousing concepts. Strong proficiency in #Python Familiarity with infrastructure as code tools like #CloudFormation or #Terraform . Excellent problem-solving skills and attention to detail. Strong communication and collaboration skills. Experience in building and utilizing #REST APIs. Experience building and running #Kubernetes Background Tasks with Batch-Jobs. Hands on experience #GitHub Action Preferred Qualifications: AWS Certified DevOps Engineer - Professional and or AWS Certified Data Engineer - Associate Knowledge of DevOps practices and #CICD pipelines. Knowledge in using #Flask framework to create dynamic websites, APIs or microservices is a plus.

As a Data Engineer at IBM, you'll play a vital role in the development, design of application, provide regular support/guidance to project teams on complex coding, issue resolution and execution. Your primary responsibilities include: Lead the design and construction of new solutions using the latest technologies, always looking to add business value and meet user requirements. Strive for continuous improvements by testing the build solution and working under an agile framework. Discover and implement the latest technologies trends to maximize and build creative solutions Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Design and Develop Data Solutions, Design and implement efficient data processing pipelines using AWS services like AWS Glue, AWS Lambda, Amazon S3, and Amazon Redshift. Develop and manage ETL (Extract, Transform, Load) workflows to clean, transform, and load data into structured and unstructured storage systems. Build scalable data models and storage solutions in Amazon Redshift, DynamoDB, and other AWS services. Data Integration: Integrate data from multiple sources including relational databases, third-party APIs, and internal systems to create a unified data ecosystem. Work with data engineers to optimize data workflows and ensure data consistency, reliability, and performance. Automation and Optimization: Automate data pipeline processes to ensure efficiency Preferred technical and professional experience Define, drive, and implement an architecture strategy and standards for end-to-end monitoring. Partner with the rest of the technology teams including application development, enterprise architecture, testing services, network engineering, Good to have detection and prevention tools for Company products and Platform and customer-facing

As a Data Engineer at IBM, you'll play a vital role in the development, design of application, provide regular support/guidance to project teams on complex coding, issue resolution and execution. Your primary responsibilities include: Lead the design and construction of new solutions using the latest technologies, always looking to add business value and meet user requirements. Strive for continuous improvements by testing the build solution and working under an agile framework. Discover and implement the latest technologies trends to maximize and build creative solutions Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Design and Develop Data Solutions, Design and implement efficient data processing pipelines using AWS services like AWS Glue, AWS Lambda, Amazon S3, and Amazon Redshift. Develop and manage ETL (Extract, Transform, Load) workflows to clean, transform, and load data into structured and unstructured storage systems. Build scalable data models and storage solutions in Amazon Redshift, DynamoDB, and other AWS services. Data Integration: Integrate data from multiple sources including relational databases, third-party APIs, and internal systems to create a unified data ecosystem. Work with data engineers to optimize data workflows and ensure data consistency, reliability, and performance. Automation and Optimization: Automate data pipeline processes to ensure efficiency Preferred technical and professional experience Define, drive, and implement an architecture strategy and standards for end-to-end monitoring. Partner with the rest of the technology teams including application development, enterprise architecture, testing services, network engineering, Good to have detection and prevention tools for Company products and Platform and customer-facing

Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Databricks Unified Data Analytics Platform Good to have skills : Python (Programming Language) Minimum 7.5 year(s) of experience is required Educational Qualification : Any graduation Summary :As an Application Developer, you will be responsible for designing, building, and configuring applications to meet business process and application requirements using Databricks Unified Data Analytics Platform. Your typical day will involve working with Python and utilizing your expertise in software development to deliver impactful solutions. The ideal candidate will work in a team environment that demands technical excellence, whose members are expected to hold each other accountable for the overall success of the end product. Focus for this team is on the delivery of innovative solutions to complex problems, but also with a mind to drive simplicity in refining and supporting of the solution by others About The Role : & Responsibilities: Be accountable for delivery of business functionality. Work on the AWS cloud to migrate/re-engineer data and applications from on premise to cloud. Responsible for engineering solutions conformant to enterprise standards, architecture, and technologies Provide technical expertise through a hands-on approach, developing solutions that automate testing between systems. Perform peer code reviews, merge requests and production releases. Implement design/functionality using Agile principles. Proven track record of quality software development and an ability to innovate outside of traditional architecture/software patterns when needed. A desire to collaborate in a high-performing team environment, and an ability to influence and be influenced by others. Have a quality mindset, not just code quality but also to ensure ongoing data quality by monitoring data to identify problems before they have business impact. Be entrepreneurial, business minded, ask smart questions, take risks, and champion new ideas. Take ownership and accountability. Experience Required:3 to 5 years of experience in application program development Experience Desired: Knowledge and/or experience with healthcare information domains. Documented experience in a business intelligence or analytic development role on a variety of large-scale projects. Documented experience working with databases larger than 5TB and excellent data analysis skills. Experience with TDD/BDD Experience working with SPARK and real time analytic frameworks Education and Training Required:Bachelor's degree in Engineering, Computer Science Primary Skills: PYTHON, Databricks, TERADATA, SQL, UNIX, ETL, Data Structures, Looker, Tableau, GIT, Jenkins, RESTful & GraphQL APIs. AWS services such as Glue, EMR, Lambda, Step Functions, CloudTrail, CloudWatch, SNS, SQS, S3, VPC, EC2, RDS, IAM Additional Skills: Ability to rapidly prototype and storyboard/wireframe development as part of application design. Write referenceable and modular code. Willingness to continuously learn & share learnings with others. Ability to communicate design processes, ideas, and solutions clearly and effectively to teams and clients. Ability to manipulate and transform large datasets efficiently. Excellent troubleshooting skills to root cause complex issues Qualifications Any graduation