Freelance Agentic AI & Databricks Engineer

Company Description

ThreatXIntel is a cybersecurity startup focused on safeguarding businesses and organizations from evolving cyber threats. Our team specializes in cloud security, web and mobile security testing, and DevSecOps, offering tailor-made and cost-effective solutions to meet the unique needs of our clients. With a proactive approach, we continuously monitor and assess digital environments to prevent vulnerabilities and ensure robust protection. Our mission is to provide businesses with the confidence to focus on growth by safeguarding their digital assets. ThreatXIntel is committed to making high-quality cybersecurity services accessible to organizations of all sizes.

Role Description

Freelance Agentic AI & Databricks Engineer

Key Responsibilities:

• Agentic AI Implementation:

• Develop and integrate

  Agentic AI

  frameworks to build intelligent systems that can autonomously make decisions based on real-time data.
• Collaborate with AI/ML teams to design self-learning models and intelligent automation solutions that adapt and improve over time.
• Leverage

  Agentic AI

  techniques (e.g., reinforcement learning, decision agents) to create dynamic decision-making systems that drive operational efficiency and innovation.

• Databricks and Data Engineering:

• Design and build data pipelines on

  Databricks

  to process large datasets, ensuring high performance, scalability, and data integrity.
• Utilize

  Apache Spark

  and

  MLlib

  in

  Databricks

  to develop advanced machine learning models and predictive analytics for business intelligence.
• Work on integrating

  Databricks

  with Azure cloud services (e.g.,

  Azure Data Lake

  ,

  Azure SQL Database

  ,

  Azure Blob Storage

  ) for seamless data management and real-time analytics.

• Cloud Integration and Optimization:

• Implement

  Azure

  cloud services to manage the full data lifecycle from ingestion, storage, processing, and analytics.
• Optimize

  Databricks

  workflows and processes, improving execution speed and resource utilization within the

  Azure

  ecosystem.
• Collaborate with cloud architects and DevOps teams to ensure the secure, scalable deployment of Agentic AI systems in

  Azure

  .

• Automation and Scaling Solutions:

• Develop and automate the deployment of machine learning models, ensuring they are integrated within continuous integration and continuous deployment (CI/CD) pipelines.
• Create monitoring and alerting systems to ensure the consistent performance and quality of deployed AI-driven models and workflows.

• Collaboration and Stakeholder Engagement:

• Collaborate with business analysts, product managers, and technical teams to identify AI-driven solutions and requirements for business use cases.
• Provide regular updates on project status, challenges, and achievements to stakeholders, ensuring alignment with business goals and objectives.

Required Skills & Qualifications:

• Education:

  Bachelor’s or Master’s degree in

  Computer Science

  ,

  Data Engineering

  ,

  AI

  ,

  Machine Learning

  , or a related field.

• Experience:

  5+ years of experience in

  Agentic AI

  development,

  Databricks

  , and

  Azure cloud services

  .

• Core Skills:

• Proficiency in

  Agentic AI frameworks

  , including experience with reinforcement learning and autonomous decision-making systems.
• Strong experience in building and optimizing

  Databricks

  environments using

  Apache Spark

  and

  MLlib

  for large-scale data processing and machine learning.
• Expertise in integrating

  Azure cloud services

  (e.g.,

  Azure Data Lake

  ,

  Azure SQL Database

  ,

  Azure Blob Storage

  ) with machine learning pipelines and data processing workflows.
• Deep understanding of

  cloud architecture

  ,

  data lakes

  , and real-time data processing.
• Proficient in programming languages such as

  Python

  ,

  Scala

  , and

  SQL

  .

• Tools & Technologies:

• Databricks

  : Expert in using

  Databricks

  for machine learning, data pipelines, and big data analytics.

• Azure

  : Proficient with

  Azure Data Lake

  ,

  Azure Synapse Analytics

  , and

  Azure ML Studio

  .

• Agentic AI frameworks

  : Strong experience in developing decision-making systems with

  Agentic AI

  techniques.

• ML Frameworks

  : Familiarity with

  TensorFlow

  ,

  PyTorch

  , and other popular machine learning libraries.

• CI/CD

  : Experience in

  CI/CD

  pipelines for automation and deployment of AI models and services.

• Soft Skills:

• Strong communication skills, capable of conveying technical concepts to both technical and non-technical stakeholders.
• Proactive and solution-oriented with a focus on delivering high-quality AI-driven systems and solutions.
• Strong collaboration skills to work cross-functionally with teams in a fast-paced, agile environment.