Text Generation

This example uses text generation. Text generation can be very powerful, but using it correctly takes a bit of practice.

For most use cases based on text generation, especially if you use Dolphin, Yi-34B, or Mixtral 8x7B, it is crucial to understand the few-shot learning concept. Basically, in order for the generative model to understand what you want, you should give it a couple of examples in your input request.

If you use a model that understands natural language instructions like ChatDolphin, Dolphin Yi-34B, Mixtral 8x7B, or Fine-tuned LLaMA 3 70B, your requests can be made with simple human instructions.

If you're still not getting good results even when using few-shot learning, it might be a sign that you need to fine-tune your own generative model. You can easily do it on NLP Cloud.

Feel free to play with the text parameters like top p, temperature, repetition penalty... and see the documentation about how to use these parameters if needed.

If you need advice, please contact us!

Fine-tuned LLaMA 3 70B: A fine-tuned version of LLaMA 3 70B that understands human instructions without having to use few-shot learning, but that also works very well with few-shot learning. It supports a 4096 tokens context (input + output). Not all the parameters are supported for the moment. It natively understands the following languages: Albanian, Arabic, Armenian, Awadhi, Azerbaijani, Bashkir, Basque, Belarusian, Bengali, Bhojpuri, Bosnian, Brazilian Portuguese, Bulgarian, Cantonese (Yue), Catalan, Chhattisgarhi, Chinese, Croatian, Czech, Danish, Dogri, Dutch, English, Estonian, Faroese, Finnish, French, Galician, Georgian, German, Greek, Gujarati, Haryanvi, Hindi, Hungarian, Indonesian, Irish, Italian, Japanese, Javanese, Kannada, Kashmiri, Kazakh, Konkani, Korean, Kyrgyz, Latvian, Lithuanian, Macedonian, Maithili, Malay, Maltese, Mandarin, Mandarin Chinese, Marathi, Marwari, Min Nan, Moldovan, Mongolian, Montenegrin, Nepali, Norwegian, Oriya, Pashto, Persian (Farsi), Polish, Portuguese, Punjabi, Rajasthani, Romanian, Russian, Sanskrit, Santali, Serbian, Sindhi, Sinhala, Slovak, Slovene, Slovenian, Spanish, Swahili, Swedish, Tajik, Tamil, Tatar, Telugu, Thai, Turkish, Turkmen, Ukrainian, Urdu, Uzbek, Vietnamese, Welsh, and Wu.

Dolphin: An NLP Cloud foundational in-house model. It works best when used with few-shot learning. It supports a 2048 tokens context (input + output). It works best in English.

ChatDolphin: An NLP Cloud in-house model based on Dolphin, fine-tuned to understand human instructions without having to use few-shot learning. It supports 2048 tokens (input + output). No text generation parameters are supported by this model for the moment, but the "###" end sequence is automatically implemented under the hood in case you want to use few-shot learning. It works best in English.

Yi 34B: A model by 01 AI. It works best when used with few-shot learning. It supports a 2048 tokens context (input + output). It works best in English.

Dolphin Yi 34B: An instruct version of Yi 34B, by Eric Hartford, fine-tuned to understand human instructions without having to use few-shot learning. It supports 2048 tokens (input + output). No text generation parameters are supported by this model for the moment, but the "###" end sequence is automatically implemented under the hood in case you want to use few-shot learning. It works best in English.

Mixtral 8x7B: A model by Mistral AI. It works best when used with few-shot learning. It supports a 2048 tokens context (input + output). It works best in English.

Dolphin Mixtral 8x7B: An instruct version of Mixtral 8x7B, by Eric Hartford, fine-tuned to understand human instructions without having to use few-shot learning. It supports 2048 tokens (input + output). No text generation parameters are supported by this model for the moment, but the "###" end sequence is automatically implemented under the hood in case you want to use few-shot learning. It works best in English.

Control whether you want to use the model on a GPU. Machine learning models run much faster on GPUs.

Whether min_length and max_length should not include the length of the input text. If false, min_length and max_length include the length of the input text. If true, min_length and max_length don't include the length of the input text. Defaults to false.

Whether you want to remove the input text form the result. Defaults to false.

The maximum number of tokens that the generated text should contain. If length_no_input is false, the size of the generated text is the difference between max_length and the length of your input text. If length_no_input is true, the size of the generated text simply is max_length. Defaults to 50.

A specific token that should stop the text generation. For example it could be `.`, `\n`, or `###` or anything else below 10 characters. If you use few-shot learning, we recommend that you use `###` as your end sequence.

Whether you want to remove the end sequence form the result. Defaults to false.

The number of independently computed returned sequences. Defaults to 1.

(Optional float). Prevents the same word from being repeated too many times. 1.0 means no penalty. Above 1.0, repetitions are less likely to happen. Below 1.0, repetitions are more likely to happen. Should be between -1.0 and 3.0. Defaults to 1.0. Not supported by ChatDolphin/Dolphin Yi-34B/Dolphin Mixtral 8x7B.

Number of beams for beam search. 1 means no beam search. Defaults to 1. This is an advanced parameter, so you should change it only if you know what you're doing.

Top K sampling (optional float). The number of highest probability vocabulary tokens to keep for top k filtering. The lower this value, the less the model is going to generate off-topic text. Should be between 1 and 1000. Defaults to 50. Not supported by ChatDolphin/Dolphin Yi-34B/Dolphin Mixtral 8x7B.

Top P sampling (optional float). If below 1, only the most probable tokens with probabilities that add up to top_p or higher are kept for generation. The higher this value, the less deterministic the result will be. It's recommended to play with `top_p` if you want to produce original content for applications that require accurate results, while you should use `temperature` if you want to generate more funny results. You should not use both at the same time. Should be between 0 and 1. Defaults to 1.0. Not supported by ChatDolphin/Dolphin Yi-34B/Dolphin Mixtral 8x7B.

Temperature sampling (optional float). It modules the next token probabilities. The higher this value, the less deterministic the result will be. If set to zero, no sampling will be applied at all and greedy search will apply instead. For example if `temperature=0` the output will always be the same, while if `temperature=1` each new request will produce very different results. It is recommended to play with `top_p` if you want to produce original content for applications that require accurate results, while you should use `temperature` if you want to generate more funny results. You should not use both at the same time. Should be between 0 and 1000. Defaults to 0.8. Not supported by ChatDolphin/Dolphin Yi-34B/Dolphin Mixtral 8x7B.

AI models don't always work well with non-English languages.

We do our best to add non-English models when it's possible. See for example Fine-tuned LLaMA 3 70B, Dolphin, ChatDolphin, XLM Roberta Large XNLI, Paraphrase Multilingual Mpnet Base V2, or spaCy. Unfortunately not all the models are good at handling non-English languages.

In order to solve this challenge, we developed a multilingual module that automatically translates your input into English, performs the actual NLP operation, and then translates the result back to your original language. It makes your requests a bit slower but often returns very good results.

Even for models that natively understand non-English languages, they actually sometimes work even better with the multilingual addon.

Simply select your language in the list, and from now on you can write the input text in your own language!

This multilingual add-on is a free feature.