Llama 2 7B Function Calling

A fine-tuned version of Meta's Llama 2 7B Chat model specifically optimized for function calling capabilities. This model enables structured function invocation while maintaining conversational abilities, making it ideal for building AI assistants that need to interact with APIs, databases, or other programmatic interfaces.

Description

This model is an imatrix-quantized variant of the Llama-2-7b-chat-hf-function-calling-v3 model, available in multiple GGUF quantization formats. The quantizations allow you to run the model locally with different memory footprints and performance characteristics.

Model Selection Guide

Choose the appropriate quantization based on your hardware and quality requirements:

Instructions

When using this model for function calling tasks, follow these steps:

1. **Select the appropriate quantization** based on your available memory and quality requirements. Download the GGUF file from the model repository.

2. **Load the model** using a compatible inference engine that supports GGUF format (llama.cpp, Ollama, LM Studio, etc.).

3. **Structure your prompts** to include:

- Clear description of available functions with their parameters

- The user's request or query

- Expected output format (typically JSON)

4. **Function definition format**:

```

Available functions:

- function_name(param1: type, param2: type): description

User request: [user's query]

Respond with the function call in JSON format.

```

5. **Parse the model's response** to extract the function name and parameters, then execute the appropriate function in your application.

6. **Handle the function result** by feeding it back to the model if continuation of the conversation is needed.

Usage Example

```

System: You have access to the following functions:

User: What's the weather like in San Francisco?