Architecture#
The diagram below illustrates the architecture of the Alauda AI platform.
#TOC
Component DescriptionComponents in Alauda Container Platform LayerComponents in AI Platform LayerComponents in GenAI Platform Layer#Component Description
#Components in Alauda Container Platform Layer
| Component | Description | Type | License |
|---|---|---|---|
| GPU (Alauda Build of Nvidia GPU Device Plugin) | Provides GPU resources for AI workloads | Open source | Apache Version 2.0 |
| HAMi (Alauda Build of Hami, Alauda Build of Hami-WebUI) | GPU resource slicing, sharing and scheduling | Open source | Apache Version 2.0 |
| Alauda Build of DCGM-Exporter | GPU monitoring | Open source | Apache Version 2.0 |
| Alauda Build of NPU Operator | Provides NPU resources for AI workloads | Open source | Apache Version 2.0 |
| Alauda Build of Node Feature Discovery | Detects hardware features of cluster nodes | Open source | Apache Version 2.0 |
| DRA (Alauda build of NVIDIA DRA Driver for GPUs) | Dynamic Resource Allocation for GPU sharing | Open source | Apache Version 2.0 |
| Volcano (Alauda support for Volcano) | Batch job scheduling for AI workloads | Open source | Apache Version 2.0 |
| Kueue (Alauda Build of Kueue) | Job scheduling for AI workloads | Open source | Apache Version 2.0 |
| Milvus (Alauda Build of Milvus) | Vector database for embedding storage and retrieval | Open source | Apache Version 2.0 |
| PGVector (Alauda support for PostgreSQL) | PostgreSQL extension for vector similarity search | Open source | The PostgreSQL License |
#Components in AI Platform Layer
| Component | Description | Type | License |
|---|---|---|---|
| Model Catalog (Alauda AI/Alauda AI Essentials) | Centralized repository for managing AI models and their metadata | Proprietary | Commercial |
| Model Registry (Alauda support for Kubeflow Model Registry) | Keep track of AI model versions and metadata for each namespace | Open source | Apache Version 2.0 |
| Datasets (Alauda AI/Alauda AI Essentials) | Centralized repository for managing datasets and their metadata | Proprietary | Commercial |
| Labeling (Alauda support for Label Studio) | Data labeling tool for creating labeled datasets | Open source | Apache Version 2.0 |
| Feature Store (Alauda support for FeatureForm) | Centralized repository for managing and serving machine learning features | Open source | Mozilla Public License (MPL) |
| Workbench (Alauda AI Workbench) | Web-based interface for managing AI projects, including model training and inference | Proprietary | Commercial |
| Training Jobs (Alauda support for Kubeflow Trainer v2) | Kubernetes-native training job management | Open source | Apache Version 2.0 |
| Kubeflow Pipelines (Alauda support for Kubeflow Base & Alauda support for Kubeflow Pipeline) | Workflow orchestration for AI pipelines | Open source | Apache Version 2.0 |
| Guardrails (Coming soon) | AI safety and governance framework | Open source | Apache Version 2.0 |
| Drift & Bias Detection (Alauda support for Evidently) | Monitoring for model performance degradation and bias | Open source | Apache Version 2.0 |
| Experiment Tracking (Alauda support for MLFlow) | Tracking and comparing machine learning experiments | Open source | Apache Version 2.0 |
#Components in GenAI Platform Layer
| Component | Description | Type | License |
|---|---|---|---|
| Kserve (Alauda AI Model Serving/Alauda Generative AI) | Kubernetes-native model serving framework | Open source | Apache Version 2.0 |
| vLLM (Alauda AI Model Serving/Alauda Generative AI) | High-performance model inference engine for large language models | Open source | Apache Version 2.0 |
| llm-d (Alauda Generative AI) | Distributed inference engine for large language models | Open source | Apache Version 2.0 |
| Model as a Service (Alauda build of Envoy AI Gateway) | API gateway for serving AI models as a service | Open source | Apache Version 2.0 |
| Fine-tuning | Tools integrated with the workbench for fine-tuning large language models, e.g. transformers, accelerate, llama-factory etc. | Open source | - |
| Training (Alauda support for Kubeflow Trainer v2) | Kubernetes-native training job management | Open source | Apache Version 2.0 |
| Model Quantization | Tools integrated with the workbench for model quantization, e.g. llm-compressor etc. | Open source | - |
| Evaluation | Tools integrated with the workbench for evaluating model performance, e.g. lm-evaluation-harness etc. | Open source | - |
| Llama Stack (Alauda build of Llama Stack) | Framework for building applications with large language models | Open source | MIT |
| Langchain | Tools integrated with the workbench for building LLM applications using Langchain | Open source | MIT |
| Dify (Alauda support for Dify) | Platform for building AI assistants and chatbots | Open source | Apache Version 2.0 (modified) |
| MCP Servers | Can integrate with various MCP servers | - | - |
| Agent Tracing (Alauda support for MLflow) | Tracing and monitoring for AI agents | Open source | Apache Version 2.0 |
| Agent Evaluation | Tools integrated with the workbench for evaluating AI agents, e.g. RAGAS etc. | Open source | - |