Description
Create a new component called OneLLM for LLM inference, composed of:
- GUI tab integrated into OpenNebula Sunstone
- REST API for backend logic
- Hugging Face Hub driver to sync/download models into OpenNebula datastores
Use case
- Administrators
- Define reusable Instance Types in a dedicated UI.
- Configure and run model pre-download from Hugging Face.
- End users
- Browse pre-downloaded models only
- Deploy by choosing model + instance type
Interface Changes
CLI and GUI
Additional Context
Please feel free to add any other context or screenshots about the feature request here. Or any other alternative you have considered to address this new feature.
Progress Status
Description
Create a new component called OneLLM for LLM inference, composed of:
Use case
Interface Changes
CLI and GUI
Additional Context
Please feel free to add any other context or screenshots about the feature request here. Or any other alternative you have considered to address this new feature.
Progress Status