Llama 3 8B Function Calling

A Meta Llama 3 8B Instruct model fine-tuned specifically for function calling and tool use capabilities, quantized to GGUF format for efficient local deployment.

Description

This skill provides a specialized version of Meta's Llama 3 8B model that has been fine-tuned on the Trelis function_calling_v3 dataset. The model excels at:

The model is available in multiple GGUF quantization levels, allowing you to balance between quality and resource usage based on your hardware constraints.

Instructions

1. Choose Your Quantization Level

Select the appropriate quantization based on your available VRAM/RAM:

2. Download the Model

Download your chosen quantization from HuggingFace:

```bash

Example for Q4_K_M (recommended)

wget https://huggingface.co/mradermacher/Meta-Llama-3-8B-Instruct-function-calling-GGUF/resolve/main/Meta-Llama-3-8B-Instruct-function-calling.Q4_K_M.gguf

```

3. Configure Your AI Runtime

Set up the model in your preferred runtime environment. For llama.cpp-based tools:

```bash

Example with llama.cpp

./main -m Meta-Llama-3-8B-Instruct-function-calling.Q4_K_M.gguf \

-p "Your prompt here" \

-n 512 \

--temp 0.7

```

4. Format Your Function Calling Prompts

Use the Llama 3 instruction format with function definitions:

```

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are a helpful assistant with access to the following functions:

{

"name": "get_weather",

"description": "Get current weather for a location",

"parameters": {

"location": {"type": "string", "description": "City name"},

"units": {"type": "string", "enum": ["celsius", "fahrenheit"]}

}

}

<|eot_id|><|start_header_id|>user<|end_header_id|>

What's the weather like in San Francisco?<|eot_id|><|start_header_id|>assistant<|end_header_id|>

```

5. Parse Model Output

The model will generate structured function calls that you should parse and execute:

```json

{

"function": "get_weather",

"arguments": {

"location": "San Francisco",

"units": "fahrenheit"

}

}

```

6. Handle Multi-Turn Conversations

After executing the function, provide the result back to the model:

```

<|start_header_id|>tool<|end_header_id|>

{"temperature": 68, "condition": "partly cloudy"}<|eot_id|><|start_header_id|>assistant<|end_header_id|>

```

Usage Examples

Example 1: API Integration

Define available API endpoints as functions and let the model generate the appropriate calls based on user requests.

Example 2: Database Queries

Expose database operations as functions and have the model translate natural language queries into structured function calls.

Example 3: Workflow Automation

Define workflow steps as callable functions and let the model orchestrate multi-step processes based on high-level instructions.

Performance Considerations

Requirements

License

Apache 2.0 - Fine-tuned from Meta Llama 3 8B Instruct base model

Source

Model card: https://huggingface.co/mradermacher/Meta-Llama-3-8B-Instruct-function-calling-GGUF

Base model: https://huggingface.co/Trelis/Meta-Llama-3-8B-Instruct-function-calling

Training dataset: https://huggingface.co/datasets/Trelis/function_calling_v3