Researchers claim that LLMs have no logical ability.
I argue that you simply need the RIGHT prompt. Here’s how I discovered a terrible new way of prompting :
I call it Thinking Through Visualisation.
The Research Paper : Alice in Wonderland
The problem is as follows:
“Alice has 2 sisters and she also has 4 brothers.
How many sisters does Alice’s brother have?”
The answer is 3.
It’s obvious to humans.
But researchers claim that AI has a lot of trouble solving this…
And it’s true!
All OpenAI’s contenders stumble on logic.
Even GPT-4o does not perform well.
You can get a precise answer by discussing with an LLM.
But with just one prompt?
It’s almost impossible!
The researchers tested several prompting methods:
‘Restricted’, ‘Standard’, ‘Thinking’ and others.
I’ve tried them myself.
But none of them really worked.
On paper, the rates of correct answers are as follows:
- Claude 3 Sonnet : 1%
- GPT-3.5 : 5%
- Llama3-8b : 5%
- Llama3-70b : 5%
- GPT-4o: 65%
But here are the results I obtained with my approach :
- Claude 3 Sonnet : 10%
- GPT-3.5 : 65%
- Llama3-8b : 70%
- Llama3-70b : 90%
- GPT-4o: 90%
Thinking Through Visualisation
To do this, I used a new approach that I call Thinking Through Visualisation.
(I call it ‘new’ because I’ve never heard of a similar method).
The principle is inspired by @ylecun visualisation task, which he has tweeted about several times.
By the way, if your goal is to master Deep Learning - I've prepared the Action plan to Master Neural networks. for you.
7 days of free advice from an Artificial Intelligence engineer to learn how to master neural networks from scratch:
- Plan your training
- Structure your projects
- Develop your Artificial Intelligence algorithms
I have based this program on scientific facts, on approaches proven by researchers, but also on my own techniques, which I have devised as I have gained experience in the field of Deep Learning.
To access it, click here :
Now we can get back to what I was talking about earlier.
Thinking about this and our human ability to understand and find solutions through images, I thought it would enhance the model’s abilities if I incited it to visualise the problem and state what it sees.
(For the LLM to ‘think’, he needs to state).
From my current experiences and the results I’ve shared – it works brilliantly!
Here’s the top prompt I used:
`Alice has 2 sisters and she also has 4 brothers. How many sisters does Alice’s brother have?
You are Alice’s brother. In front of you are all the members of your family. Make up real names for them. List them one by one with first name and then their function.
Finally, you must first count and then answer the question in <answer>Here is the answer</answer>`
Obviously, the prompt can be further refined and generalised!
Feel free to test it and optimise it for your own needs.
I’m curious to hear your feedback on this approach!
Prompt breakdown
Here’s the breakdown of the prompt:
- Establish the problem – `Alice has …`
- Change the perspective – `You are Alice’s brother.`
- Stimulate visualisation – `In front of you are all the members of your family.`
- Reinforce visualisation – `Make up real names for them.`
- List the problem variables – `List them one by one with first name and then their function.`
- Use @AndrewYNg basic ‘Explain then answer’ method, which I have mentioned in the past – `Finally, you must first count and then answer the question.`
Where to find other prompting approaches?
To find my publications on Artificial Intelligence and prompting, you can find me on X.
I regularly publish posts on relevant AI news, prompting methods and Artificial Intelligence.
Click here to read more: : Visit my X profile
One last word, if you want to go further and learn about Deep Learning - I've prepared for you the Action plan to Master Neural networks. for you.
7 days of free advice from an Artificial Intelligence engineer to learn how to master neural networks from scratch:
- Plan your training
- Structure your projects
- Develop your Artificial Intelligence algorithms
I have based this program on scientific facts, on approaches proven by researchers, but also on my own techniques, which I have devised as I have gained experience in the field of Deep Learning.
To access it, click here :