errors out with roocode
Why you didn't test it properly.
Works fine with "fake" Qwen3-30B-A3B-Instruct-Coder-480B-Distill-v2-Q8_0.gguf fine.
Using Jan.ai v0.7.1
Errors out with Roocode 3.28.15:
With or without this template:
{% macro render_item_list(item_list, tag_name='required') %}
{%- if item_list is defined and item_list is iterable and item_list | length > 0 %}
{%- if tag_name %}{{- '\n<' ~ tag_name ~ '>' -}}{% endif %}
{{- '[' }}
{%- for item in item_list -%}
{%- if loop.index > 1 %}{{- ", "}}{% endif -%}
{%- if item is string -%}
{{ "" ~ item ~ "" }}
{%- else -%}
{{ item }}
{%- endif -%}
{%- endfor -%}
{{- ']' }}
{%- if tag_name %}{{- '</' ~ tag_name ~ '>' -}}{% endif %}
{%- endif %}
{% endmacro %}
{%- if messages[0]["role"] == "system" %}
{%- set system_message = messages[0]["content"] %}
{%- set loop_messages = messages[1:] %}
{%- else %}
{%- set loop_messages = messages %}
{%- endif %}
{%- if not tools is defined %}
{%- set tools = [] %}
{%- endif %}
{%- if system_message is defined %}
{{- "<|im_start|>system\n" + system_message }}
{%- else %}
{%- if tools is iterable and tools | length > 0 %}
{{- "<|im_start|>system\nYou are Qwen, a helpful AI assistant that can interact with a computer to solve tasks." }}
{%- endif %}
{%- endif %}
{%- if tools is iterable and tools | length > 0 %}
{{- "\n\nYou have access to the following functions:\n\n" }}
{{- "" }}
{%- for tool in tools %}
{%- if tool.function is defined %}
{%- set tool = tool.function %}
{%- endif %}
{{- "\n\n" ~ tool.name ~ "" }}
{%- if tool.description is defined %}{{- '\n' ~ (tool.description | trim) ~ '' }}{%- endif %}
{{- '\n' }}
{% if tool.parameters.properties is defined %}
{%- for param_name, param_fields in tool.parameters.properties|items %}
{{- '\n' }}
{{- '\n' ~ param_name ~ '' }}
{%- if param_fields.type is defined %}
{{- '\n' ~ (param_fields.type | string) ~ '' }}
{%- endif %}
{%- if param_fields.description is defined %}
{{- '\n' ~ (param_fields.description | trim) ~ '' }}
{%- endif %}
{{- render_item_list(param_fields.enum, 'enum') }}
{%- set handled_keys = ['type', 'description', 'enum', 'required'] %}
{%- for json_key in param_fields.keys() if json_key not in handled_keys %}
{%- set normed_json_key = json_key | replace("-", "") | replace(" ", "") | replace("$", "") %}
{%- if param_fields[json_key] is none %}
{{- '\n<' ~ normed_json_key ~ '/>' }}
{%- elif param_fields[json_key] is mapping %}
{{- '\n<' ~ normed_json_key ~ '>' ~ (param_fields[json_key] | tojson) ~ '</' ~ normed_json_key ~ '>' }}
{%- else %}
{{- '\n<' ~ normed_json_key ~ '>' ~ (param_fields[json_key] | string) ~ '</' ~ normed_json_key ~ '>' }}
{%- endif %}
{%- endfor %}
{{- render_item_list(param_fields.required, 'required') }}
{{- '\n' }}
{%- endfor %}
{% endif %}
{{- render_item_list(tool.parameters.required, 'required') }}
{{- '\n' }}
{%- if tool.return is defined %}
{%- if tool.return is mapping %}
{{- '\n' ~ (tool.return | tojson) ~ '' }}
{%- else %}
{{- '\n' ~ (tool.return | string) ~ '' }}
{%- endif %}
{%- endif %}
{{- '\n' }}
{%- endfor %}
{{- "\n" }}
{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n\n\n\n\n\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...> block must be nested within XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n' }}
{%- endif %}
{%- if system_message is defined %}
{{- '<|im_end|>\n' }}
{%- else %}
{%- if tools is iterable and tools | length > 0 %}
{{- '<|im_end|>\n' }}
{%- endif %}
{%- endif %}
{%- for message in loop_messages %}
{%- if message.role == "assistant" and message.tool_calls is defined and message.tool_calls is iterable and message.tool_calls | length > 0 %}
{{- '<|im_start|>' + message.role }}
{%- if message.content is defined and message.content is string and message.content | trim | length > 0 %}
{{- '\n' + message.content | trim + '\n' }}
{%- endif %}
{%- for tool_call in message.tool_calls %}
{%- if tool_call.function is defined %}
{%- set tool_call = tool_call.function %}
{%- endif %}
{{- '\n\n<function=' + tool_call.name + '>\n' }}
{%- if tool_call.arguments is defined %}
{%- for args_name, args_value in tool_call.arguments|items %}
{{- '<parameter=' + args_name + '>\n' }}
{%- set args_value = args_value if args_value is string else args_value | string %}
{{- args_value }}
{{- '\n\n' }}
{%- endfor %}
{%- endif %}
{{- '\n' }}
{%- endfor %}
{{- '<|im_end|>\n' }}
{%- elif message.role == "user" or message.role == "system" or message.role == "assistant" %}
{{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
{%- elif message.role == "tool" %}
{%- if loop.previtem and loop.previtem.role != "tool" %}
{{- '<|im_start|>user\n' }}
{%- endif %}
{{- '\n' }}
{{- message.content }}
{{- '\n\n' }}
{%- if not loop.last and loop.nextitem.role != "tool" %}
{{- '<|im_end|>\n' }}
{%- elif loop.last %}
{{- '<|im_end|>\n' }}
{%- endif %}
{%- else %}
{{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>\n' }}
{%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
{{- '<|im_start|>assistant\n' }}
{%- endif %}
Just to be clear I DO NOT test these models . Unfortunately I can only requantize models as is. I have not got the resources to do anything more; zero income from this! If you want tested models then Unsloth gguf files will be better tested as they also provide a service around this.
That said. I have never used RooCode and so have no idea how it works under the hood. But you may want to look into this; from the model card "NOTE: This model supports only non-thinking mode and does not generate blocks in its output. Meanwhile, specifying enable_thinking=False is no longer required.
" . If you use llama.cpp and provide output I may be able to help.
Thank you for your hard work friend. The KAD-DEV model you made is the only one that works ! (there is no official quantization as you know) it beats Qwen3-Coder-30B-A3B, GLM4.5-Air that needs 120GB VRAM.
I tested them all and nobody else model works. Your model works with Jan.ai and Roocode only for some reason. Jan.ai comes with empty template and it simply works, however the lmstudio does not for some reason, any ideas how to make it work too?
I did not wanted to sound rude, I fixed the Qwen3-Coder-30B-A3B problem with this template:
{% macro render_item_list(item_list, tag_name='required') %}
{% if item_list and item_list | length > 0 %}
{% if tag_name %}<{{ tag_name }}>{% endif %}
[
{% for item in item_list %}
{% if not loop.first %}, {% endif %}
{% if item is string %}
{{ item }}
{% else %}
{{ item }}
{% endif %}
{% endfor %}
]
{% if tag_name %}</{{ tag_name }}>{% endif %}
{% endif %}
{% endmacro %}
{# ---- Handle system message and loop messages ---- #}
{% if messages and messages[0].role == "system" %}
{% set system_message = messages[0].content %}
{% set loop_messages = messages[1:] %}
{% else %}
{% set loop_messages = messages %}
{% endif %}
{% set tools = tools or [] %}
{% if system_message %}
<|im_start|>system
{{ system_message }}
<|im_end|>
{% elif tools | length > 0 %}
<|im_start|>system
You are Qwen, a helpful AI assistant that can interact with a computer to solve tasks.
<|im_end|>
{% endif %}
{% if tools | length > 0 %}
You have access to the following functions:
{# --- force required parameters for Roo built-in tools --- #}
{% if t.name == "write_to_file" %}
<required>[`path`, `content`]</required>
{% elif t.name == "apply_diff" %}
<required>[`path`, `diff`]</required>
{% else %}
{{ render_item_list(t.parameters.required, 'required') }}
{% endif %}
</parameters>
{% if t.return %}
<return>{{ t.return | tojson if t.return is mapping else t.return }}</return>
{% endif %}
</function>
{% endfor %}
If you choose to call a function ONLY reply in the following format with NO suffix:
value_1 This is the value for the second parameter that can span multiple lines Reminder: - Function calls MUST follow the specified format: an inner block must be nested within XML tags - Required parameters MUST be specified - You may provide optional reasoning BEFORE the function call, but NOT after - If there is no function call available, answer normally with your knowledge {% endif %}{# ---- Render loop messages ---- #}
{% for message in loop_messages %}
<|im_start|>{{ message.role }}
{{ message.content }}
<|im_end|>
{% endfor %}
{% if add_generation_prompt %}
<|im_start|>assistant
{% endif %}
Ah yes it's a template issue. Well done for fixing it. I don't use templates in my code so don't have so much experience with fixing them. My code is interacting with llm using my own templating system using .net . So my approach to fixing templates is to work with the raw input output and compare that to what the model expects, which is not always obvious. But it gives good flexibility. So to answer your question how to fix with Llmstudio. I think it uses llama.cpp under the hood. So running llama.cpo in non conversational mode (--nocnv ??) and give it the raw token input output that the template would produce.