Qwen

/

Qwen3-14B-GGUF

like 57

Follow

Qwen 60k

vllm serve "Qwen/Qwen3-14B-GGUF"
INFO 11-13 06:19:46 [init.py:216] Automatically detected platform cuda.
(APIServer pid=2882) INFO 11-13 06:19:50 [api_server.py:1839] vLLM API server version 0.11.0
(APIServer pid=2882) INFO 11-13 06:19:50 [utils.py:233] non-default args: {'model_tag': 'Qwen/Qwen3-14B-GGUF', 'model': 'Qwen/Qwen3-14B-GGUF'}
(APIServer pid=2882) Traceback (most recent call last):
(APIServer pid=2882) File "/home/shivanisharma/myvenv_vllm/bin/vllm", line 10, in
(APIServer pid=2882) sys.exit(main())
(APIServer pid=2882) ^^^^^^
(APIServer pid=2882) File "/home/shivanisharma/myvenv_vllm/lib/python3.11/site-packages/vllm/entrypoints/cli/main.py", line 54, in main
(APIServer pid=2882) args.dispatch_function(args)
(APIServer pid=2882) File "/home/shivanisharma/myvenv_vllm/lib/python3.11/site-packages/vllm/entrypoints/cli/serve.py", line 57, in cmd
(APIServer pid=2882) uvloop.run(run_server(args))
(APIServer pid=2882) File "/home/shivanisharma/myvenv_vllm/lib/python3.11/site-packages/uvloop/init.py", line 92, in run
(APIServer pid=2882) return runner.run(wrapper())
(APIServer pid=2882) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=2882) File "/usr/lib/python3.11/asyncio/runners.py", line 118, in run
(APIServer pid=2882) return self._loop.run_until_complete(task)
(APIServer pid=2882) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=2882) File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
(APIServer pid=2882) File "/home/shivanisharma/myvenv_vllm/lib/python3.11/site-packages/uvloop/init.py", line 48, in wrapper
(APIServer pid=2882) return await main
(APIServer pid=2882) ^^^^^^^^^^
(APIServer pid=2882) File "/home/shivanisharma/myvenv_vllm/lib/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 1884, in run_server
(APIServer pid=2882) await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
(APIServer pid=2882) File "/home/shivanisharma/myvenv_vllm/lib/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 1902, in run_server_worker
(APIServer pid=2882) async with build_async_engine_client(
(APIServer pid=2882) File "/usr/lib/python3.11/contextlib.py", line 204, in aenter
(APIServer pid=2882) return await anext(self.gen)
(APIServer pid=2882) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=2882) File "/home/shivanisharma/myvenv_vllm/lib/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 180, in build_async_engine_client
(APIServer pid=2882) async with build_async_engine_client_from_engine_args(
(APIServer pid=2882) File "/usr/lib/python3.11/contextlib.py", line 204, in aenter
(APIServer pid=2882) return await anext(self.gen)
(APIServer pid=2882) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=2882) File "/home/shivanisharma/myvenv_vllm/lib/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 206, in build_async_engine_client_from_engine_args
(APIServer pid=2882) vllm_config = engine_args.create_engine_config(usage_context=usage_context)
(APIServer pid=2882) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=2882) File "/home/shivanisharma/myvenv_vllm/lib/python3.11/site-packages/vllm/engine/arg_utils.py", line 1142, in create_engine_config
(APIServer pid=2882) model_config = self.create_model_config()
(APIServer pid=2882) ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=2882) File "/home/shivanisharma/myvenv_vllm/lib/python3.11/site-packages/vllm/engine/arg_utils.py", line 994, in create_model_config
(APIServer pid=2882) return ModelConfig(
(APIServer pid=2882) ^^^^^^^^^^^^
(APIServer pid=2882) File "/home/shivanisharma/myvenv_vllm/lib/python3.11/site-packages/pydantic/_internal/_dataclasses.py", line 121, in init
(APIServer pid=2882) s.pydantic_validator.validate_python(ArgsKwargs(args, kwargs), self_instance=s)
(APIServer pid=2882) pydantic_core._pydantic_core.ValidationError: 1 validation error for ModelConfig
(APIServer pid=2882) Value error, Invalid repository ID or local directory specified: 'Qwen/Qwen3-14B-GGUF'.
(APIServer pid=2882) Please verify the following requirements:
(APIServer pid=2882) 1. Provide a valid Hugging Face repository ID.
(APIServer pid=2882) 2. Specify a local directory that contains a recognized configuration file.
(APIServer pid=2882) - For Hugging Face models: ensure the presence of a 'config.json'.
(APIServer pid=2882) - For Mistral models: ensure the presence of a 'params.json'.
(APIServer pid=2882) 3. For GGUF: pass the local path of the GGUF checkpoint.
(APIServer pid=2882) Loading GGUF from a remote repo directly is not yet supported.
(APIServer pid=2882) [type=value_error, input_value=ArgsKwargs((), {'model': ...rocessor_plugin': None}), input_type=ArgsKwargs]
(APIServer pid=2882) For further information visit https://errors.pydantic.dev/2.12/v/value_error