Llama-3-Groq-8B-Tool-Use Model

A quantized GGUF version of the Llama-3 8B model fine-tuned by Groq for tool use and function calling. This model enables AI agents to interact with tools and APIs through structured function calls.

Overview

This skill provides access to multiple quantization levels of the Llama-3-Groq-8B-Tool-Use model, allowing you to balance between model quality, speed, and hardware requirements. The model is based on Meta's Llama-3 8B architecture and has been specifically trained for function calling and tool use scenarios.

Key Features

Instructions

1. Select Appropriate Quantization Level

Choose a quantization based on your hardware and quality requirements:

**Recommended Options:**

**Lower Quality (for resource-constrained systems):**

**Minimal Options (desperate situations only):**

2. Download the Model

Download your chosen quantization from the HuggingFace repository:

```bash

Example for Q4_K_M (recommended)

wget https://huggingface.co/mradermacher/Llama-3-Groq-8B-Tool-Use-i1-GGUF/resolve/main/Llama-3-Groq-8B-Tool-Use.i1-Q4_K_M.gguf

```

3. Load with llama.cpp

```bash

./llama-cli -m Llama-3-Groq-8B-Tool-Use.i1-Q4_K_M.gguf \

-p "You are a helpful assistant with access to tools." \

--temp 0.7 \

-n 512

```

4. Use with Ollama

Create a Modelfile:

```dockerfile

FROM ./Llama-3-Groq-8B-Tool-Use.i1-Q4_K_M.gguf

PARAMETER temperature 0.7

PARAMETER top_p 0.9

SYSTEM "You are a helpful assistant capable of using tools and functions to help users accomplish tasks."

```

Import and run:

```bash

ollama create llama3-tool-use -f Modelfile

ollama run llama3-tool-use

```

5. Function Calling Format

Structure your prompts to enable tool use:

```

Available tools:

User: What's the weather in San Francisco?