Generative AI

Introduction

The Generative AI service on the Crewdle Mist platform, currently available in preview, is designed to harness the power of machine learning within a decentralized computing environment. This innovative service facilitates the execution of both pre-built models, including Large Language Models (LLMs), and custom models directly within a mist cluster, offering several key benefits for developers and businesses alike.

Key Features

  • Private Retrieval-Augmented Generation (RAG) Pipeline: Ensures that uploaded documents never leave the private environment of the business, maintaining strict data privacy and security while leveraging machine learning capabilities.
  • Deployment of Machine Learning Models in a Cluster: Developers can easily deploy pre-built and custom machine learning models to specific mist clusters, supporting a wide range of applications from data analysis to AI-driven interactions.
  • Real-Time Model Execution: Execute machine learning models in real-time within the cluster, ensuring efficient processing and quick response times, crucial for dynamic and interactive applications.
  • Custom Model Support: Provides the flexibility to deploy custom machine learning models tailored to specific business needs and unique use cases.

Use Cases

  • Predictive Analytics: Ideal for applications requiring real-time data analysis and predictions, such as market trend analysis, risk assessment, and customer behavior prediction.
  • Personalized User Experiences: Enhance user engagement by offering personalized experiences based on machine learning insights, like recommendation systems or tailored content.
  • Natural Language Processing: Run LLMs to enable advanced natural language processing capabilities, including chatbots, language translation, and sentiment analysis.

Setup

To setup your Crewdle Generative AI, there are a few key steps to follow:

  1. Create a new Generative AI Context.

  2. Setup your Mistlet as shown in the Mistlet Guide

  3. Access the configure tab in your new Mistlet and associate the VPM created in step one as well as the Generative AI Context created in step 2.

  4. Download the Mistlet and the configuration file via the deploy tab, install the mistlet and upload the configuration file.

  5. You can now choose to upload (and then delete) documents to the context by clicking the upload document in you newly created generative AI context.

Now to make full use of this service, you have to instantiate the Crewdle SDK (see Getting Started section). Make sure the cluster ID matches your VPM ID.

Then, open your AI context with the correct label in your app.

const generativeAIContext = await cluster.openGenerativeAIContext('CONTEXT_ID');

With that setup it is possible to prompt the AI.

Use .prompt() to receive the full answer to the prompt in one batch.

const prompt = 'How do you instantiate a Crewdle Generative AI Context?';

const output = await generativeAIContext.prompt(prompt);

// You can now use the output as you see fit i.e:
console.log('AI Service Output:', output);

Use .stream() to get a progressive result of the answer to help simulate a constant conversation.

let output

for await(const token of generativeAIContext.stream(prompt)) {
  // You can use the stream to simulate constant writing
  output += token
}

// You can now use the output as you see fit i.e:
console.log('AI Service Output:', output);

The AI can now be closed

generativeAIContext.close();

Accessing the Service

As the Machine Learning service is currently in preview, access is limited. Interested users and developers need to request access to ensure quality and gather valuable feedback for further improvements. To request access, simply click the "Request Access" button provided on the service page.

With these features, Crewdle Mist's Generative AI service is poised to transform the way developers and businesses leverage machine learning, offering a powerful and flexible platform for deploying and executing models in a decentralized environment.

Additional Resources

This service and more are accessible via the Crewdle Developer Console

To see a full example of a Generative AI Context being used, checkout our Generative AI sample.