| | --- |
| | language: en |
| | tags: |
| | - exbert |
| |
|
| | license: gpl |
| | --- |
| | |
| |
|
| | # TCP 2023 for NTU students |
| |
|
| | Fine tuning pre-trained language models for text generation. |
| |
|
| | Pretrained model on Chinese language using a GPT2 for Large Language Head Model objective(GPT2LMHeadModel). |
| |
|
| | ## Model description |
| |
|
| | TCP 2023 is a transformers model that has undergone fine-tuning using the GPT-2 architecture. |
| | It was initially pretrained on an extensive corpus of Chinese data in a self-supervised manner. |
| | This implies that the pretraining process involved using raw text data without any human annotations, allowing the model to make use of a wide range of publicly available data. |
| | The model leveraged an automatic process to derive inputs and corresponding labels from these texts. |
| | To be more specific, the pretraining aimed at predicting the subsequent word in sentences. |
| | it was trained to guess the next word in sentences. |
| |
|
| | ### How to use |
| |
|
| | You can use this model directly with a pipeline for text generation. Since the generation relies on some randomness, we |
| | set a seed for reproducibility: |
| |
|
| | ```python |
| | >>> from transformers import GPT2LMHeadModel, AutoTokenizer, pipeline |
| | |
| | >>> model_name = "DavidLanz/tcp2023" |
| | |
| | >>> model = GPT2LMHeadModel.from_pretrained(model_name) |
| | >>> tokenizer = AutoTokenizer.from_pretrained(model_name) |
| | |
| | >>> text_generator = pipeline('text-generation', model=model, tokenizer=tokenizer) |
| | >>> generated_text = text_generator(input_text, max_length=max_len, num_return_sequences=1) |
| | >>> print(generated_text[0]['generated_text']) |
| | |
