Instructions to use atharvapawar/securix_Llama-2-7B-Chat-GGML with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use atharvapawar/securix_Llama-2-7B-Chat-GGML with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="atharvapawar/securix_Llama-2-7B-Chat-GGML")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("atharvapawar/securix_Llama-2-7B-Chat-GGML")
model = AutoModelForCausalLM.from_pretrained("atharvapawar/securix_Llama-2-7B-Chat-GGML")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use atharvapawar/securix_Llama-2-7B-Chat-GGML with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "atharvapawar/securix_Llama-2-7B-Chat-GGML"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "atharvapawar/securix_Llama-2-7B-Chat-GGML",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/atharvapawar/securix_Llama-2-7B-Chat-GGML

SGLang

How to use atharvapawar/securix_Llama-2-7B-Chat-GGML with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "atharvapawar/securix_Llama-2-7B-Chat-GGML" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "atharvapawar/securix_Llama-2-7B-Chat-GGML",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "atharvapawar/securix_Llama-2-7B-Chat-GGML" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "atharvapawar/securix_Llama-2-7B-Chat-GGML",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use atharvapawar/securix_Llama-2-7B-Chat-GGML with Docker Model Runner:
```
docker model run hf.co/atharvapawar/securix_Llama-2-7B-Chat-GGML
```

Model Overview

Library: PEFT
Language: English (en)
Pipeline Tag: Text2Text Generation
Tags: Code Generation (cod)

Model Details

This model has been fine-tuned on the Llama-2 model using a dataset of Python code vulnerability rules.

Training Procedure

The model was trained with a quantization configuration using the bitsandbytes quantization method. Some key configurations include:

Quantization Method: bitsandbytes
Load in 8-bit: False
Load in 4-bit: True
LLM Int8 Threshold: 6.0
LLM Int8 Skip Modules: None
LLM Int8 Enable FP32 CPU Offload: False
LLM Int8 Has FP16 Weight: False
BNB 4-bit Quant Type: nf4
BNB 4-bit Use Double Quant: False
BNB 4-bit Compute Dtype: float16

Framework Versions

PEFT: 0.6.0.dev0

Model Details

This model card provides information about a fine-tuned model using the PEFT library. The model is designed for text-to-text generation tasks, particularly in the field of code generation and vulnerability rule detection.

Intended Use

The model is intended for generating text outputs based on text inputs. It has been fine-tuned specifically for code generation tasks and vulnerability rule detection. Users can input text descriptions, code snippets, or other relevant information to generate corresponding code outputs.

Limitations and Considerations

It's important to note that while the model has been fine-tuned for code generation, its outputs may still require human review and validation. It may not cover all possible code variations or edge cases. Users are advised to thoroughly review generated code outputs before deployment.

Training Data

The model was trained on a dataset of Python code vulnerability rules. The dataset includes examples of code patterns that could potentially indicate vulnerabilities or security risks.

Training Procedure

The model was trained using the PEFT library. The quantization method used was bitsandbytes, with specific configurations mentioned earlier. The model underwent multiple training epochs to optimize its performance on code generation tasks.

Model Evaluation

The model's performance has not been explicitly evaluated in this model card. Users are encouraged to evaluate the model's generated outputs for their specific use case and domain.

Framework Versions

PEFT: 0.6.0.dev0

Downloads last month: 11

Safetensors

Model size

0.1B params

Tensor type

I64

F32

atharvapawar
/

securix_Llama-2-7B-Chat-GGML

Model Overview

Model Details

Training Procedure

Framework Versions

Model Details

Intended Use

Limitations and Considerations

Training Data

Training Procedure

Model Evaluation

Framework Versions

Space using atharvapawar/securix_Llama-2-7B-Chat-GGML 1