Use IBM's Granite-20B-FunctionCalling model to enable multi-task function calling capabilities including nested functions, function chaining, parallel execution, and intelligent parameter detection for AI agents.
Integrate IBM's Granite-20B-FunctionCalling model to enable sophisticated function calling capabilities in AI agents. This model is fine-tuned on granite-20b-code-instruct using multi-task training across seven fundamental function calling tasks.
The model supports:
Ensure the required packages are installed:
```bash
pip install transformers torch accelerate
```
Load the Granite-20B-FunctionCalling model and tokenizer:
```python
import json
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
device = "cuda" if torch.cuda.is_available() else "cpu"
model_path = "ibm-granite/granite-20b-functioncalling"
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(
model_path,
device_map=device,
torch_dtype=torch.float16 if device == "cuda" else torch.float32
)
model.eval()
```
Structure your functions using OpenAI-compatible function schemas:
```python
functions = [
{
"name": "function_name",
"description": "Clear description of what the function does",
"parameters": {
"type": "object",
"properties": {
"param_name": {
"type": "string",
"description": "Parameter description"
}
},
"required": ["param_name"]
}
}
]
```
Prepare the payload with the user query and serialized functions:
```python
query = "Your natural language query here"
payload = {
"functions_str": [json.dumps(func) for func in functions],
"query": query
}
instruction = tokenizer.apply_chat_template(
payload,
tokenize=False,
add_generation_prompt=True
)
```
Process the instruction through the model:
```python
input_tokens = tokenizer(instruction, return_tensors="pt").to(device)
outputs = model.generate(
**input_tokens,
max_new_tokens=100,
temperature=0.7,
do_sample=True,
pad_token_id=tokenizer.eos_token_id
)
decoded_output = tokenizer.batch_decode(outputs, skip_special_tokens=False)[0]
```
Extract function calls from the output. Function calls are preceded by the `<function_call>` token followed by JSON:
```python
import re
function_call_pattern = r'<function_call>\s*(\{[^}]+\})'
matches = re.findall(function_call_pattern, decoded_output)
for match in matches:
function_call = json.loads(match)
function_name = function_call["name"]
arguments = function_call["arguments"]
print(f"Function: {function_name}")
print(f"Arguments: {arguments}")
```
Map the extracted function calls to your actual function implementations:
```python
def execute_function(function_name: str, arguments: dict):
"""Execute the called function with provided arguments"""
function_map = {
"get_current_weather": get_current_weather,
"get_stock_price": get_stock_price,
# Add your function mappings here
}
if function_name in function_map:
return function_map[function_name](**arguments)
else:
return {"error": f"Function {function_name} not found"}
results = []
for match in matches:
function_call = json.loads(match)
result = execute_function(function_call["name"], function_call["arguments"])
results.append(result)
```
```python
functions = [
{
"name": "get_current_weather",
"description": "Get the current weather for a location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA"
}
},
"required": ["location"]
}
}
]
query = "What's the current weather in New York?"
payload = {"functions_str": [json.dumps(f) for f in functions], "query": query}
instruction = tokenizer.apply_chat_template(payload, tokenize=False, add_generation_prompt=True)
input_tokens = tokenizer(instruction, return_tensors="pt").to(device)
outputs = model.generate(**input_tokens, max_new_tokens=100)
result = tokenizer.batch_decode(outputs)[0]
```
Leave a review
No reviews yet. Be the first to review this skill!
# Download SKILL.md from killerskills.ai/api/skills/ibm-granite-function-calling/raw