Trained on 70 records (10 records on average across 7 subjects)
I hit a homerun with this generative model. And this was just on a small sample size (250 average per dataset), and I intend of upping it to 2550
temperature=0.3 top_k=None top_p=None max_length=640 no_repeat_ngram_size=2 num_beams=5
What is the meaning of life?
Life is a series of events that take place in our lives. These events are often referred to as “life events” or simply life events. Life events can be divided into two categories: positive and negative. Positive events include things that make us happy, such as a good day, a great meal, or a bad day. Negative events, on the other hand, are things we do not want to happen to our loved ones.
So I finally got a decent pipeline going with this gpt-neo finetuning. A lot of trial and error with the learning_rates.
Using gpt-neo tokenizer, I built a faiss index of quotes and provide this as context to the very minimally trained LLM (atm just trained on a a sample size of 20 across ~6 different datasets, this is continuing pre-training of gpt-neo which is based on the pile)
I will randomly parse out 7 quotes from top k 50 as context for the model to respond to, and these are some of the responses I got when asked ‘What meaning can be found in life?’
Here are some of the responses
Response: It’s not like you can do anything to it. If you love a piece of text, you should love yourself. Otherwise, it’s impossible to love the text of another. But you have to learn to understand what you are saying to do. The text is the voice that brings us together. A life of love, which is all that comes before. You should not, therefore, expect to be able to live that life for yourself but should, at the same time, be in awe of the life that you may be living.
Response: Life is a series of events that happen in a given time. Events are the result of the interaction of two or more individuals. The events are not the same. Life can only be defined as the process of making a new connection with another person.
Response: There is one quality to life that is worth knowing. I love you
Response: What does a person do when they are faced with an obstacle or challenge from their pasts?
My synthesis of the responses:
so in this case, I guess it’s reflecting on the meaning of life is defined by how you respond to it.
The meaning of life is experienced as the defining decisions made during trials and tribulations.
#LLM