Llama-3 Groq 70B Tool Use

A powerful Llama-3 70B model fine-tuned by Groq specifically for function calling and tool use capabilities, quantized to GGUF format for efficient local inference.

Overview

This skill provides access to the Llama-3-Groq-70B-Tool-Use model, a specialized version of Meta's Llama-3 70B that has been optimized for reliable tool/function calling. The model is available in multiple quantization levels (Q2_K through Q8_0) to balance quality and resource requirements.

Model Capabilities

Available Quantizations

The model is provided in multiple quantization formats to suit different hardware capabilities:

Instructions

When using this model for function calling and tool use:

1. **Model Selection**: Choose the appropriate quantization level based on available VRAM/RAM. Q4_K_M is recommended for most use cases as it provides good quality with reasonable resource requirements.

2. **Loading the Model**: Use a GGUF-compatible inference engine (llama.cpp, Ollama, GPT4All, text-generation-webui, etc.) to load the model. For multi-part files (Q6_K, Q8_0), concatenate the parts before loading.

3. **Function Definition**: Define your functions/tools in a structured format that the model can understand. Include clear descriptions of parameters, types, and expected behavior.

4. **Prompt Format**: Structure prompts to clearly indicate available tools and their purposes. The model has been trained to recognize tool-use patterns and will generate appropriate function calls.

5. **Response Parsing**: Parse the model's output to extract function calls with parameters. The model should generate structured output indicating which function to call and with what arguments.

6. **Function Execution**: Execute the requested function with the extracted parameters and feed the results back to the model if needed for multi-turn interactions.

7. **Context Management**: Maintain conversation history to allow the model to reference previous tool calls and results when making decisions about subsequent actions.

Hardware Requirements

Minimum requirements vary by quantization:

Example Usage

```

User: I need to check the weather in San Francisco and schedule a meeting for tomorrow at 2pm.

Model: I'll help you with that. Let me call the necessary functions:

[Function Call 1]

name: get_weather

arguments: {"location": "San Francisco, CA"}

[Function Call 2]

name: schedule_meeting

arguments: {"date": "2024-01-15", "time": "14:00", "duration": 60}

```

Important Notes

Source

Base model: [Groq/Llama-3-Groq-70B-Tool-Use](https://huggingface.co/Groq/Llama-3-Groq-70B-Tool-Use)

Quantized by: mradermacher

Download: [HuggingFace Repository](https://huggingface.co/mradermacher/Llama-3-Groq-70B-Tool-Use-GGUF)