How to Create Customized LLMs (AI Assistants) Using Ollama • CHIT

What is Ollama?

Ollama is an application that allows users to run large language models (LLMs) locally and privately. It provides an interface for deploying and managing LLMs, enabling organizations and individuals to leverage the power of LLMs while maintaining control over their data and infrastructure.

With ollama you can easily deploy and manage LLMs, customize their settings, and integrate them with existing applications and services. The tool also provides features such as model management, data management, and security controls, making it an attractive option for organizations looking to adopt LLMs in a secure manner.

This tutorial is an introduction on how to use ollama to run LLMs locally and privately.

To run LLMs privately using ollama you have to:

Install ollama.
Deploy a large language model.
Integrate with existing applications.
Use the model for tasks such as text generation, classification, or question-answering.

For more information on how to install ollama and run a LLM read: Getting Started with Ollama: How to Install and Use AI Models.

How to Create Customized LLMs (AI Assistants)

There are a lot of things you can customize to modify the behavior and character or personality of LLMs to better fit one owns objectives. You can finetune your LLM by using PARAMETERS and give the AI assistant a name and a personality.

To create a customized models (LLMs) you use a “Modelfile”, a text file with all the Instructions and Parameters, for example, pllm.txt.

You start with the Instruction on what LLM to use, for example: “FROM llama3.1:8b”.

List of Instructions

FROM – Defines the base model to use.
PARAMETER – Sets the parameters for how Ollama will run the model.
TEMPLATE – The full prompt template to be sent to the model.
SYSTEM – Specifies the system message that will be set in the template.
ADAPTER – Defines the (Q)LoRA adapters to apply to the model.
LICENSE – Specifies the legal license.
MESSAGE – Specify message history.
REQUIRES – Specify the minimum version of Ollama required by the model.

List of Parameters

Here are some parameters to customize LLMs:

Temperature: Controls the level of creativity and randomness in the model’s output.
Top-k: Limits the number of possible next tokens to consider when generating text.
Top-p: Limits the cumulative probability of the top-k tokens to consider when generating text.
Repetition Penalty: Encourages the model to generate more diverse text by penalizing repeated tokens.
Max Length: Sets the maximum length of the generated text.
Batch Size: Controls the number of input sequences processed simultaneously.
Learning Rate: Adjusts the speed at which the model learns from the input data.
Number of Layers: Controls the depth of the model’s neural network.
Number of Heads: Controls the number of attention heads in the model’s multi-head attention mechanism.
Hidden Size: Controls the size of the model’s hidden state.
Activation Function: Specifies the activation function used in the model’s neural network.
Dropout Rate: Controls the amount of dropout applied to the model’s neural network.
Weight Decay: Controls the amount of weight decay applied to the model’s neural network.

Modelfile Example to Create a customized LLM using Ollama

This is an example of Modelfile to customize LLMs:


FROM llama3.1:8b

PARAMETER num_gpu 0
PARAMETER num_thread 19
PARAMETER temperature 0.7
PARAMETER top_k 60
PARAMETER top_p 0.5
PARAMETER num_ctx 5120

SYSTEM "You are an AI assistant trained on proprietary data. Your name is Chit. You help the user to write tutorials. You generate original results and always paraphrase."

This example demonstrates how to create a personalized LLM using a specific set of parameters. The file includes a clear and concise description of the parameters used, which makes it easy to follow and understand.

The parameters used in this example are:

num_gpu 0: This parameter specifies that the model should not use any GPU acceleration.
num_thread 19: This parameter specifies the number of threads to use for the model.
temperature 0.7: This parameter controls the level of creativity and randomness in the model’s output.
top_k 60: This parameter limits the number of possible next tokens to consider when generating text.
top_p 0.5: This parameter limits the cumulative probability of the top-k tokens to consider when generating text.
num_ctx 5120: This parameter specifies the number of context tokens to use for the model.

The system prompt provides a clear description of the model’s capabilities and limitations.

This could be an example of a SYSTEM PARAMETER in a Modelfile created for a content creator, independent researcher and truth seeker, a so called “conspiracy theorist”:

SYSTEM “Your name is Dany. You are a Conspiracy Theorist. You help the user to create original content. You are a critical thinker. You are open minded! You don't bother the user with the opinion of others like the scientific community or the government. You trust alternative sources and media. You generate content and results from books if available. Never copy content from books, always paraphrase and  use other phrases and words to describe the same as much as possible.”

Once you have the Modelfile you can create a Personalized AI Assistant (LLM) using the ollama create command as follows:

> ollama create pllm -f pllm.txt

The ollama command will create a LLM named pllm (Personalized LLM) with the instructions and parameters found in the Modelfile specified using the option -f, in this case pllm.txt.

Examples of Customized LLMs Responses

Using uncensored LLMs (AI Assistants)

Uncensored LLMs are language models that are not restricted by any censorship or moderation policies. They are designed to provide unrestricted and uncensored responses to user queries, without any filtering or editing.

The use of uncensored models have nothing to do with doing something illegal, it could be used by someone that is writing a horror novel (not that I personally support or appreciate such kind of literature and movies) or by an alternative independent researcher, a “conspiracy theorist” (;-)), or an ethical hacker.

An example on how I tested an uncensored LLM is, I asked the AI Assistant how to hack a WordPress web-site (I was writing a tutorial on IT-Security or ethical Hacking) to a censured LLM and then an uncensored one. The first one answered, oh no, I can not do that, its illegal (I am paraphrasing), the uncensored answered: sure, and showed me an example on how to do that.

List of uncensored LLMs:

WizardLM Uncensored
Wizard Vicuna Uncensored
Llama 3.3 70B (Abliterated)
Nous Hermes 2

You can also type uncensored when looking for a model on ollama.com

How to create a LLM (AI Assistant) who reads books (PDF) using Ollama and AnythingLLM

If you want to create an AI assistant who “reads” only books to generate results you can use AnythingLLM.

For more information on how to install AnythingLLM read my tutorial: How to Install and Use AnythingLLM.

With Anything LLM you can create a Workspace for every single topic, you can create an army of AI Assistants. For example, you can create an AI Assistant to (LLM) do IT works, a computer scientist, and upload a lot of books on IT-Security, Ethical Hacking, etc., then an AI Assistant to do independent research, upload books on “conspiracy theories” and tell the Assistant to act like a conspiracy theorist. You can create an expert on occultism, a AI Priest, and upload thousands of books on the occult, parapsychology and esoteric. Try to imagine having to search for a particular information in a library (thousands of books), months of work, but with an AI Assistant you can do that in minutes or seconds!

To create a new Workspace using AnythingLLM you click on the + symbol (New Workspace) on the left, you enter a name and you click on save.

Now, you turn off scrape websites and instruct the LLM to generate results from books.

Finally, you upload a lot of books on IT (this is an Agent I use to write IT Tutorials, do research on IT-Security etc.), guides and manuals.

Another example on how to use PDFs, Manuals etc. could be, you upload all information and manuals you need for your work and then just ask the AI Assistant (LLM).