Is there a way to run it on vLLM?

by Jongsim - opened Mar 6

Mar 6

Which version of vLLM can run this BnB model?
I want to run the API server using a command.
Since this model is for fine-tuning, is it not possible to run it with vLLM?

Jongsim

Mar 6

•

edited Mar 6

No description provided.

Jongsim changed discussion status to closed Mar 6

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment