A lightweight AI assistant powered by Gemma 1.1 2B model fine-tuned for function calling and tool use. Supports multiple quantization levels for efficient local inference.
A specialized AI assistant using the Gemma 1.1 2B model fine-tuned for function calling capabilities. This skill enables you to work with a lightweight, quantized model that can run locally and handle structured function calls and tool use patterns.
This skill provides instructions for integrating and using the Gemma 1.1 2B function calling model, which has been fine-tuned on function calling datasets and supports ChatML format. The model is available in multiple GGUF quantization levels (q2_k through fp16) to balance performance and resource usage.
| Quantization | Size | Use Case |
|--------------|------|----------|
| q2_k | 1.16 GB | Fastest, lowest quality |
| q3_k_m | 1.38 GB | Fast, acceptable quality |
| q4_k_m | 1.63 GB | Balanced (recommended) |
| q5_k_m | 1.84 GB | Higher quality |
| q6_k | 2.06 GB | Very high quality |
| q8_0 | 2.67 GB | Near-original quality |
| fp16 | 5.02 GB | Full precision |
When using this skill, follow these steps:
First, verify if the user has a local LLM runtime (like llama.cpp, Ollama, or LM Studio) installed:
```bash
which ollama
```
If not available, guide them to download the model from HuggingFace:
The model expects ChatML format with this structure:
```
<bos><start_of_turn>system
You are a helpful AI assistant.<end_of_turn>
<start_of_turn>user
{user_question}<end_of_turn>
<start_of_turn>model
```
When implementing function calling:
1. Define available functions in JSON schema format
2. Include function definitions in the system prompt
3. Parse model outputs for function call syntax
4. Execute requested functions and return results
5. Continue conversation with function results
For integrating this model into a project:
1. **Download the model file** (recommend q4_k_m for balance)
2. **Set up inference runtime** (llama.cpp, Ollama, etc.)
3. **Configure the prompt template** using ChatML format
4. **Define function schemas** for available tools
5. **Implement function execution** logic
6. **Handle model responses** and parse function calls
7. **Test with example queries** that require tool use
```python
functions = {
"get_weather": {
"description": "Get current weather for a location",
"parameters": {
"location": {"type": "string", "description": "City name"}
}
}
}
system_prompt = """You are a helpful AI assistant with access to these functions:
{json.dumps(functions, indent=2)}
When you need to use a function, respond with:
FUNCTION_CALL: function_name(param1=value1, param2=value2)
"""
```
If the model doesn't generate proper function calls:
1. **Local Tool-Using Assistant:** Build a privacy-focused assistant that runs entirely offline
2. **Function Call Testing:** Test function calling implementations without API costs
3. **Embedded Systems:** Deploy on edge devices with limited resources
4. **Rapid Prototyping:** Quick experimentation with function calling patterns
5. **Educational Projects:** Learn about function calling and model fine-tuning
Leave a review
No reviews yet. Be the first to review this skill!
# Download SKILL.md from killerskills.ai/api/skills/gemma-function-calling-assistant/raw