A 20B parameter model specialized in function calling capabilities, supporting nested calls, function chaining, parallel functions, and parameter detection across multiple quantization formats (GGUF).
A specialized language model for function calling tasks, based on IBM's granite-20b-code-instruct and trained on seven fundamental function calling tasks.
This model introduces advanced function calling abilities through multi-task learning of granular tasks including:
The model is available in multiple GGUF quantization formats (Q2_K through Q8_0) ranging from 7.38GB to 20.01GB, allowing you to balance quality and resource usage.
When using this model, follow these steps to implement function calling capabilities:
1. **Define Your Functions**
- Structure functions with name, description, and parameters schema
- Use JSON Schema format for parameter definitions
- Clearly specify required parameters
2. **Format Your Prompt**
- Serialize functions as JSON strings
- Apply the model's chat template with your query and function list
- Use the following structure:
```
{
"functions_str": [serialized_function_definitions],
"query": "user_query_here"
}
```
3. **Parse Model Output**
- Function calls are preceded by the `<function_call>` token
- Output format: `{"name": "function_name", "arguments": {"arg_name": "arg_value"}}`
- Handle multiple function calls in a single response for parallel execution
4. **Choose Appropriate Quantization**
- Q2_K/Q3_K: Smallest size, lower quality (7-10GB)
- Q4_K/Q5_K: Balanced quality/size (10-14GB)
- Q6_K/Q8_0: Higher quality, larger size (15-20GB)
5. **Model Loading**
- For local inference, use llama.cpp, Ollama, or compatible tools
- Ensure sufficient RAM/VRAM for your chosen quantization
- GPU acceleration recommended for models above Q4_K
```python
query = "What's the current weather in New York?"
functions = [
{
"name": "get_current_weather",
"description": "Get the current weather",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA"
}
},
"required": ["location"]
}
}
]
```
Leave a review
No reviews yet. Be the first to review this skill!
# Download SKILL.md from killerskills.ai/api/skills/ibm-granite-20b-function-calling/raw