Instructions to use gorkemgoknar/gpt2-turkish-writer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use gorkemgoknar/gpt2-turkish-writer with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="gorkemgoknar/gpt2-turkish-writer")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("gorkemgoknar/gpt2-turkish-writer")
model = AutoModelForCausalLM.from_pretrained("gorkemgoknar/gpt2-turkish-writer")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use gorkemgoknar/gpt2-turkish-writer with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "gorkemgoknar/gpt2-turkish-writer"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "gorkemgoknar/gpt2-turkish-writer",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/gorkemgoknar/gpt2-turkish-writer

SGLang

How to use gorkemgoknar/gpt2-turkish-writer with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "gorkemgoknar/gpt2-turkish-writer" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "gorkemgoknar/gpt2-turkish-writer",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "gorkemgoknar/gpt2-turkish-writer" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "gorkemgoknar/gpt2-turkish-writer",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use gorkemgoknar/gpt2-turkish-writer with Docker Model Runner:
```
docker model run hf.co/gorkemgoknar/gpt2-turkish-writer
```

Turkish AI Writer based on GPT2-Small

Türkçe Yapay Zeka Yazarı

Model description

This model is enhanced version of gpt2-small-turkish finetuned version. In addition to 28-10-2020 Wikipedia Turkish article dump this model is trained with more than 400 classic novels and plays in Turkish (Including Dostoyevski, Shaekspeare, Dumas)

Base work has been done on Pierre Guillou tutorial as on this page. (https://github.com/piegu/fastai-projects/blob/master/finetuning-English-GPT2-any-language-Portuguese-HuggingFace-fastaiv2.ipynb)

Note that Since Turkish language is not close to English as in Porteguese instead of training last 2 layers, last 3 layers are trained.

Code is converted to work with Fastai 2.X . Using Google Colab for training.

Current accuracy 36.3 % , Perplexity : 44.75

Demo (using CPU inference) is available on: http://www.metayazar.com

Models are available:

[gpt2-small-tuned-tr] (https://huggingface.co/gorkemgoknar/gpt2-small-turkish)
[gpt2-small-turkish-writer] (https://huggingface.co/gorkemgoknar/gpt2-turkish-writer)

Intended uses & limitations

How to use

Install

from transformers import AutoTokenizer, AutoModelWithLMHead
import torch

tokenizer = AutoTokenizer.from_pretrained("gorkemgoknar/gpt2-turkish-writer")
model = AutoModelWithLMHead.from_pretrained("gorkemgoknar/gpt2-turkish-writer")

# Get sequence length max of 1024
tokenizer.model_max_length=1024 

model.eval()  # disable dropout (or leave in train mode to finetune)

Generate 1 word

# input sequence
text = "Bu yazıyı bilgisayar yazdı."
inputs = tokenizer(text, return_tensors="pt") 

# model output
outputs = model(**inputs, labels=inputs["input_ids"])
loss, logits = outputs[:2]
predicted_index = torch.argmax(logits[0, -1, :]).item()
predicted_text = tokenizer.decode([predicted_index])

# results
print('input text:', text)
print('predicted text:', predicted_text)

# input text: 
# predicted text:

Generate Full Sequence

# input sequence
text = "Bu yazıyı bilgisayar yazdı."
inputs = tokenizer(text, return_tensors="pt")

# model output using Top-k sampling text generation method
sample_outputs = model.generate(inputs.input_ids,
                                pad_token_id=50256,
                                do_sample=True, 
                                max_length=50, # put the token number you want
                                top_k=40,
                                num_return_sequences=1)

# generated sequence
for i, sample_output in enumerate(sample_outputs):
    print(">> Generated text {}\n\n{}".format(i+1, tokenizer.decode(sample_output.tolist())))

# >> Generated text
#

Limitations and bias

The training data used for this model come from Turkish Wikipedia and books. We know it contains a lot of unfiltered content from the internet, which is far from neutral. Also not much pre-processing was done on books hence chapter names and page numbers can be seen on some cases. This is a work in progress.

Training data

Wikipedia Turkish article dump as of 28-10-2020 Turkish book dataset of >400 classic novels

Training procedure

Eval results

epoch	train_loss	valid_loss	accuracy	perplexity	time
0	4.497828	4.549605	0.277328	94.595070	2:09:58
1	4.503929	4.519456	0.275071	91.785645	2:04:30
2	3.612716	3.921146	0.344802	50.458256	2:03:22
3	3.777645	4.072006	0.326130	58.674530	1:56:14
4	2.934462	3.801303	0.363719	44.759476	1:58:55

Note: 1cycle rule training is used and epochs are at different times

Downloads last month: 26

gorkemgoknar
/

gpt2-turkish-writer