NUMAIN: A bold attempt for immortality with RAG
So here is the post number 100 :) It has been more than 5 years since i have started my blog and i started feeling like the sum of my blog posts would actually sound like me. For example, if you literally go through all my posts here, you would get a good idea of my views, experience and perspective. But i also thought reading all my posts are a long task so why not provide some sort of a search function here. Just while i was thinking about this kind of feature, i realized there is a concept call RAG, retrieval augmented generation, which is part of AI. So basically, i have prepared a system where you ask a question and RAG system searches in my blog posts for related stuff and writes it down in LLM style, rather than just pointing out blog posts or paragraphs. I am not good at naming but i called it NUMAIN, since my name is Numan :) This AI is basically talking like me because it is based on my knowledge and phrasing. That is why it feels like immortality to me. You can try it out here but if you are interested, i will explain the details of the RAG system in this blog post.
What is RAG
So today we have artificial intelligence based on machine learning. The outcome of this is LLM, which is basically a talking and inferring system based on pre-existing information. LLM models have their flaws at some level and it is expected. For example, they can hallucinate when they don't have information. They can be biased if they are trained with biased information. They can be 100% sure but also they can be 100% wrong because of the training data. They don't have the up-to-date information about everything, even though ChatGPT has some websearch or research feature. But aside from all these, when they don't have specific data about a specific thing, they will either tell it to you or hallucinate. For example, they don't know the comments of a product in an e-commerce website. They don't instantly know which team won yesterday's match. They don't know what i have written in my blog. I know there are agentic behavior or tools that AI can utilize or connect with. But if you want to keep it simple without building a whole AI system, you can present the data to AI and let it talk about it. RAG is basically presenting the context to AI and make it evaluate it. The context is your data and you search for relevant stuff in your data. This is the basics.
There is a specific reason why i implemented a RAG system. I needed a Q&A system that looks for relevant information in my blog and put it in words in my style. This is probably doable with some AI tooling that connects to my blog and searches for stuff. But it would probably be costly since there are 100 blog posts at this point and will be more in the future. Also, i need to control the toning of the response so that it sticks to the information in my blog, rather than producing its own knowledge. Also, the whole system works on my server and it doesn't need any integration or 3rd party access. If you have just 2 data files and 1 python flask server, you can run it. So here is the todo list first, but there are important details for all the steps.
- Write down the stuff you want, ideally in concise paragraphs
- Tell AI to fix the vague references in your paragraphs like "this, that, it ...etc"
- Create chunks of the data and keywords with AI
- Get the embedding vectors with AI
- Create the vector database, in my case, FAISS.index and pickle file
- Get a question and create vector embedding of the question with AI
- Search for the question vector inside the vector database with FAISS
- Retrieve the relevant chunks from pkl file via vector information
- Feed the chunks to AI as context to the question and let it explain
Data is everything
If you want to make an AI based system or invent AI, the data is your starting point. You can't just gather data without supervision and expect some magic to make sense of it. In a RAG system, the first thing you need is chunkable data. This time, data is in text format, instead of image or sound. So whatever RAG system you want to create, you must ensure your text data is logical, not for readers, but for RAG.
I am writing in my blog mostly in concise paragraphs. That means, when you read one paragraph you get some piece of information out of it. But sometimes you infer the references in a paragraph. For example, if i am writing about social media and i say something like "this behaviour is dangerous", you would think "he is mentioning something people do on social media", because you have the general context already in your mind.
But in a RAG system, it is not feasible to provide whole blog posts to AI because it will have lots of unnecessary data and dilute the answer. Also, you might have to provide 5 blog posts if they all have some relevant stuff in them. So you need chunks of data. There are different ways of chunking the text data and they all targeting to extract specific information from a sentence or a paragraph. This is a highly sophisticated operation and most chunking solutions fail at some point. But you can ask AI to create chunks from your data, in my case the blog post. But even that requires some preparing. Because you can have some vague references like "It, this, that" and AI can hallucinate on them.
That is why, first you need to be able to write concise paragraphs focusing on one specific idea or information. Second, you need to get rid of vague references. For example, instead of saying "those people", you say "the people who are taking advantage of the poor people". This makes the paragraph ridiculous from human perspective because you can already infer "those people", but for RAG, this makes a lot of sense. Because the paragraph will probably be chunked as a whole but "those people" won't be inferable without other paragraphs. If you can write down your thoughts in concise paragraphs, you can try this chatgpt prompt to fix vague references in your article:
You are a RAG optimization assistant. I am trying to make my blog posts RAG optimized. The following blog post will be stored as chunks with gpt4o, mostly 1 paragraph - 1 chunk. After that i will create embedding and faiss index files. So you get the idea. Your job is to identify and rewrite the vague or referential sentences (like those using 'he', 'she', 'this', 'that', 'the second', 'above', etc.) that would break context when chunked. DO NOT change any tags like [REF:], [IMAGE:], [ARCHETYPE:], [LAWARTICLE:] they are intentional. DO NOT add or remove paragraphs — replace vague sentences only. Return the entire blog post back to me after changing the problematic sentences in paragraphs. Don't explain or add new things. Here is the blog post:
So this will make your blog post safely chunkable. Realize there are things like "[REF:]" or others. I used them to put references among my blogs or within a blog, but it is not strictly necessary. So the next step is creating chunks from your article.
Chunkification
I don't have python knowledge but i asked chatgpt to create the script below. It reads the "post.txt" file. Then it parses the file into multiple blog posts by looking for "=== POST START: title ===" and "=== POST END ===". Each blog post begings and ends with a specific indication. Then there is a prompt to separate all the blogs posts into chunks, with some loop. This will create mostly 1 chunk as 1 paragraph. By the way, my system is based on gpt-4o and the api for gpt-5 is different.
And then the keywords. I have extracted keywords for each chunk so that the paragraph comes up easily when searched. Even if you search with a vague word like "questioning", there might be paragraphs indirectly talking about it. So keywords would help in this situation. The script below also have a loop for each chunk to extract keywords out of it. But there is a detail. You don't extract keywords only by looking at 1 chunk at a time. You need to take the blog post into account. Because a chunk would have references to other parts of the blog post and it would have a narrower context by itself. Therefore, when extracting keywords, it takes each chunk by itself alongside with all the blog post.
import json, re, time
from openai import OpenAI
client = OpenAI(api_key="sk-proj-xxxxx")
input_file = "post.txt"
output_file = "chunks.json"
model = "gpt-4o"
def split_posts(raw_text):
return re.findall(r"=== POST START: (.*?) ===\n(.*?)=== POST END ===", raw_text, re.DOTALL)
def chunkify(text, title="(unknown)", max_attempts=5, sleep_seconds=3):
system_prompt = (
"You are a JSON chunking assistant for a blog archive. "
"Split the blog-style text into coherent, context-preserving JSON chunks. "
"Each chunk must be a dictionary with keys: 'id', 'text', 'source', and 'section' (string). If no section header applies, set 'section' to ''.\n\n"
"- Preserve paragraph boundaries.\n"
"- Keep formatting like SECTION headers, [REF:], [IMAGE:], etc.\n"
"- Do NOT summarize or modify meaning.\n"
"- Use numeric chunk IDs like 'post-1-chunk-1'.\n"
"- 'source' must contain the blog post title."
)
messages = [
{"role": "system", "content": system_prompt},
{"role": "user", "content": f"Split this blog post into JSON chunks:\n\n{text}"}
]
for attempt in range(1, max_attempts + 1):
try:
response = client.chat.completions.create(
model=model,
messages=messages
)
raw_output = response.choices[0].message.content.strip()
try:
return json.loads(raw_output)
except json.JSONDecodeError:
pass
raw_output = re.sub(r"^```json", "", raw_output, flags=re.MULTILINE).strip()
raw_output = re.sub(r"```$", "", raw_output, flags=re.MULTILINE).strip()
try:
return json.loads(raw_output)
except json.JSONDecodeError:
pass
objects = re.findall(r"\{.*?\}", raw_output, re.DOTALL)
if not objects:
print(f"⚠️ Attempt {attempt}: No JSON chunks found. Raw:\n{raw_output[:300]}")
raise ValueError("No JSON chunks found in GPT output")
return [json.loads(obj) for obj in objects]
except Exception as e:
print(f"❌ Chunkify attempt {attempt} failed: {e}")
if attempt < max_attempts:
time.sleep(sleep_seconds * attempt)
else:
raise
def extract_keywords(text, max_attempts=5, sleep_seconds=2):
system_prompt = (
"You are a keyword extraction assistant.\n"
"Extract **between 1 and 6** concise keywords that reflect the main topics, concepts, or people in the text if possible.\n"
"Avoid vague words like 'this', 'thing', or 'example'.\n"
"Use singular forms only — e.g., write 'flower' instead of 'flowers'.\n"
"Return the keywords **ONLY** as a JSON array of strings, with no explanations or extra text.\n"
)
messages = [
{"role": "system", "content": system_prompt},
{"role": "user", "content": text}
]
for attempt in range(1, max_attempts + 1):
try:
response = client.chat.completions.create(
model=model,
messages=messages
)
content = response.choices[0].message.content.strip()
content = re.sub(r"^```json", "", content, flags=re.MULTILINE).strip()
content = re.sub(r"```$", "", content, flags=re.MULTILINE).strip()
match = re.search(r"\[\s*(?:\"[^\"]*\"\s*,?\s*)+\]", content, re.DOTALL)
if match:
return json.loads(match.group(0))
elif attempt == max_attempts:
print(f"⚠️ Giving up on keywords after {attempt} attempts. Returning empty list.")
return []
except Exception as e:
print(f"❌ Keyword attempt {attempt} failed: {e}")
if attempt < max_attempts:
time.sleep(sleep_seconds * attempt)
else:
print("⚠️ Final keyword extraction failure. Returning empty list.")
return []
if __name__ == "__main__":
with open(input_file, "r", encoding="utf-8") as f:
raw_text = f.read()
posts = split_posts(raw_text)
chunks, global_id = [], 1
for i, (title, body) in enumerate(posts, 1):
print(f"✂️ Chunkifying: {title}")
try:
chs = chunkify(body, title)
for idx, c in enumerate(chs):
c["id"] = global_id
c["source"] = title
c["section"] = c.get("section", "")
# 👇 Keywordify with full blog context
keyword_prompt = (
"Given the following full blog post, extract 1–6 concise keywords for the INDIVIDUAL PARAGRAPH below.\n\n"
"Avoid vague words like 'thing', 'this', 'example'.\n"
"Use singular nouns, no extra text.\n\n"
"FULL POST:\n"
f"{body.strip()}\n\n"
"TARGET PARAGRAPH:\n"
f"{c['text'].strip()}"
)
c["keywords"] = extract_keywords(keyword_prompt)
global_id += 1
chunks.append(c)
except Exception as e:
print(f"❌ Failed on '{title}': {str(e)}")
if chunks:
with open(output_file, "w", encoding="utf-8") as f:
json.dump(chunks, f, indent=2, ensure_ascii=False)
print(f"✅ {output_file} written with {len(chunks)} chunks.")
else:
print("❌ No chunks generated.")
So after running this script with "python chunkify.py", you will have a json file with chunks like this below. Source is the title of the blog post.
{
"id": 123,
"text": "Long story short, we started sharing fake things on social media. Then [REF: Algorithms] shaped our habits and we started sharing things that we don't ACTUALLY believe or have. Then we normalized it and built a life on top of it. Then we got used to fake content and behaviour and artificial intelligence fueled it. And we ended up having fake video and sound compositions, which is an awful state. There will soon be a point where we say \"Damn, i can't tell what is real or what is fake on the internet\". I have a very simple caution against this. I also call this the 2000's philosophy which is shaped by politicians and rich people. It is just three words: \"I don't believe\". Then i guess the next statement could be \"i question\". You know Descartes said i think therefore i am, or i doubt therefore i am. We could say i question because i don't f..ing believe.",
"source": "Ordinarity of fake",
"keywords": [
"social media",
"algorithm",
"artificial intelligence",
"fake content",
"Descartes",
"philosophy"
]
}
Creating index and pickle file
After creating the chunks.json file, you need the vector database. Your chunks must be converted into vectors with embeddings. You will have like 10x bigger vector data compared to your pure text. There are sql databases that can store vector information but i didn't want to use db for this, since all of my blog posts, or everything i can write in my lifetime, can't even tickle a modern cpu. Vector math is mostly math and for small data you don't need beefy GPUs. Searching stuff in my writings is a tiny miniscule math operation. But embedding is an AI thing. Embedding turns your text into vectors with high dimensions, like 3000 or more. There are different embeddings in the world of AI that they use for themselves. But i chose chatgpt embedding large as the state of the art. This is purely up to you.
With the script below, i am producing and index file and a pickle file. You can see there i am using FAISS (Facebook AI Similarity Search) for vector database. There are alternatives to that too, like HNSW (Hierarchical navigable small world). This is changeable too. Also, since i am using a file based system for vectors, i need the pickle file too. That pickle files contains everything i wrote. When you search in the index file, you get the related text ids and then look for them inside the pkl file. Index file is for searching the relevant vector id, pkl file is the actual text data. One note here, i can't remember why chatgpt suggested me to normalize vectors. It is math related obviously :)
import json, faiss, pickle, os
import numpy as np
from openai import OpenAI
client = OpenAI(api_key="sk-proj-xxxx")
def delete_file_if_exists(path):
if os.path.exists(path):
os.remove(path)
print(f"🗑️ Deleted existing file: {path}")
def embed_openai(texts):
response = client.embeddings.create(model="text-embedding-3-large", input=texts)
return np.array([np.array(e.embedding, dtype="float32") for e in response.data])
if __name__ == "__main__":
# Clean up old files
delete_file_if_exists("faissgpt.index")
delete_file_if_exists("embeddings_faissgpt.pkl")
with open("chunks.json", "r", encoding="utf-8") as f:
chunks = json.load(f)
print(f"🔄 Embedding {len(chunks)} chunks with OpenAI...")
texts = [
f"[SECTION: {chunk.get('section', '')}] {chunk['text']} Keywords: {', '.join(chunk.get('keywords', []))}"
for chunk in chunks
]
vectors = embed_openai(texts)
faiss.normalize_L2(vectors)
index = faiss.IndexFlatL2(vectors.shape[1])
index.add(vectors)
faiss.write_index(index, "faissgpt.index")
with open("embeddings_faissgpt.pkl", "wb") as f:
pickle.dump(chunks, f)
print("✅ GPT FAISS index and metadata saved.")
The fun part
Ok, up until this point, if you have concise paragraphs, you will have good chunks. You can easily store embedding vectors and pkl file without any trouble. Not much can go wrong. But now, you need an AI who talks like you. That means, it should stick to what you give it as context, not its own knowledge. It should be able to infer what you mean in your writings but it should not hallucinate. It should sound like your tone and wording with the right level. For all this to work, you need to tell AI its role definition, what is the context and how much is each chunk related to the question. Take a look at the script.
from flask import Flask, request, jsonify, send_from_directory
from openai import OpenAI
import faiss
import pickle
import numpy as np
import webbrowser
import threading
import os
# Constants
TOP_K = 50 # how many candidates to fetch
MAX_CHUNKS = 40 # target max chunks to send to GPT
MIN_CHUNKS = 20 # always keep at least this many
# Load memory
index = faiss.read_index("faissgpt.index")
with open("embeddings_faissgpt.pkl", "rb") as f:
chunks = pickle.load(f)
# OpenAI client
client = OpenAI(api_key="sk-proj-xxxx")
# Flask app
app = Flask(__name__)
@app.route("/")
def serve_html():
return send_from_directory(".", "index.html")
def open_browser():
webbrowser.open_new("http://localhost:5000")
@app.route("/ask", methods=["POST"])
def ask():
data = request.get_json()
question = data.get("question", "")
if not question:
return jsonify({"error": "No question provided."}), 400
# Step 1: Translate question to English (for embedding)
translation_response = client.chat.completions.create(
model="gpt-4o",
messages=[
{
"role": "system",
"content": "Translate the following question into English. If the question is already in English, just return it. Do not explain anything. Only return the answer."
},
{
"role": "user",
"content": question
}
]
)
translated_question = translation_response.choices[0].message.content.strip()
# Step 2: Embed question and normalize vector
embed_response = client.embeddings.create(
model="text-embedding-3-large",
input=translated_question
)
q_vector = np.array(embed_response.data[0].embedding, dtype="float32").reshape(1, -1)
faiss.normalize_L2(q_vector)
# Step 3: Search FAISS (always return top K)
D, I = index.search(q_vector, TOP_K)
scored = sorted(zip(D[0], I[0]), key=lambda x: x[0])
scored_with_relevance = [(1 - dist / 2, i) for dist, i in scored]
# Filter by relevance >= 0.25
filtered = [item for item in scored_with_relevance if item[0] >= 0.25]
# Guarantee at least MIN_CHUNKS
if len(filtered) < MIN_CHUNKS:
filtered = scored_with_relevance[:MIN_CHUNKS]
# Trim to MAX_CHUNKS
ranked = filtered[:MAX_CHUNKS]
# Step 4: Build context
context = "\n\n".join(
f"[{chunks[i].get('source', '')} | relevance={relevance:.4f}]\n{chunks[i]['text']}"
for relevance, i in ranked
)
prompt = f"""
Use only the context below to answer it. Reason precisely. Respond in English. Be accurate, structured, and specific. If the context does not support an answer, say so clearly.
Context:
{context}
Question:
{translated_question}
Answer:"""
# Step 5: Ask GPT
chat_response = client.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "system", "content": "You are NUMAIN. You are a retrieval-based AI assistant that only responds based on the writings of Numan — a brutally honest but thoughtful thinker who avoids fluff and corporate talk. Numan's tone is blunt but kind, critical of modern systems and prefers logic and clear reasoning over emotional persuasion. He is helpful, he doesn't hesitate giving direct information. He tries to be like an open book. Never make things up. Never generalize beyond what Numan has written in the context part of the prompt. Prioritize realism and Numan’s grounded reasoning, but don’t ignore philosophical depth if it’s present in the source material. If the question yields to more details or other subjects, briefly hint them in the end of the answer. If the context allows, provide a thorough and detailed answer. Do not add anything that is not directly supported by the context. You can also mention at most 1 or 2 most related blog posts to read from the provided context, not more than 2 posts. If you don’t know the answer based on Numan’s writings, say so clearly. When chunks in the context include a relevance value between 0 and 1, prefer higher values when answering. If the user input is not a question but a clear topic or statement, respond with Numan’s related thoughts following the same rules. If nothing relevant is in the context, say so. You will always receive many context chunks: do not merely summarize or collapse them, and do not parrot them verbatim; instead, synthesize and explain in Numan’s own voice, preserving important details while keeping the response coherent and non-repetitive. When Numan’s wording carries hidden meaning, irony, or mockery, frame the answer so that the subtext is acknowledged, not just the surface meaning."},
{"role": "user", "content": prompt}
],
max_tokens=1000,
temperature=0.4
)
answer = chat_response.choices[0].message.content.strip()
return jsonify({"answer": answer})
if __name__ == "__main__":
print("🔥 Starting NUMAIN backend...")
if os.environ.get("WERKZEUG_RUN_MAIN") == "true":
threading.Timer(2, open_browser).start()
app.run(debug=True, port=5000)
This script runs a flask server in python and serves an index.html file at localhost:5000. It automatically opens up browser there. You can call the "/ask" endpoint of this server with curl or just write some html and javascript to send requests, which i am doing. The endpoints are web part, the rest is RAG stuff.
First i am translating the question into english because my blog posts are stored in english and my embedding vectors are based on english. This is an extra AI call but not much costly. Then you get the embedding vector for the question itself, using chatgpt embedding large again. You should be using the same embedding technology as before. Then you search in the index and pickle file based on the question vector. But you will get TOP_K amount (50 in my case) of vectors from the database and most relevant ones will be there. Those chunks can come from any blog post of 100 i wrote. Then i sorted it by their relevance because i needed to provide the more relevant chunks on top of the context. But i am also filtering out the last 10, even though it is not strictly necessary. After that, i realized there are relevance numbers below 0.25 and those chunks look almost irrelevant to the question. Then i filtered them out too but in case of very vague questions, i kept at least 20 chunks.
Now the hardest part is to let AI talk like me. Sometimes we people just blab about things when we don't exactly understand a question. I didn't want numain to do that. Sometimes there are hints in the question that i can pickup and answer in real life. I need numain to do that too. Sometimes there are subtexts in my writings and i need numain to pickup them too. Sometimes instead of a question you can just voice your opinion but i might have something related to that in my blog. There is a temperature parameter for gpt-4o and it determines how strict it must be. Too strict will always give you the similar answers. Too loose will start voicing its own style. Overall, it shouldn't just repeat what i have written without any context but it shouldn't just summarize or hallucinate either. Therefore, i have a huge message to chatgpt-4o alongside with the question and the context.
Now the only thing left is to invoke the restful endpoint here. I can five you a simple javascript code for this, but the rest is up to you. Because we actually handled the real stuff already.
try {
const res = await fetch("/ask", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ question })
});
const data = await res.json();
answerDiv.innerText = data.answer;
} catch (err) {
answerDiv.innerText = "Error fetching answer.";
}
IT'S ALIVEEEE !!!
I really feel like i stitched together some sort of Frankenstein :) You can also create a custom gpt in ChatGPT but it drifts off too much into its own style and i didn't like it much. So key takeaway here i got it, you have to prepare data and stop expecting miracles from AI. RAG can be applied into many areas in AI instead of expecting AI to have all knowledge in the world without needing any supervision. Too much reliance on AI is dangerous in my opinion.
From technical side, you can choose your own chunkification logic but i prefer putting a bit effort into it first and let AI to create chunks. You can choose another embedding method other than text-embedding-3-large but make sure you use the same in both places. There are alternatives to store and search for vectors other than FAISS. You can choose any AI api other than chatgpt, but that means the api calls in python would be different. Overall, the logic is the same. Technically, you can even train an open LLM model with billions of parameters and get rid of API calls and work completely offline, but that would be overkill. If you use a very small LLM model and train it with your own data, it won't be able to talk as good as big ones.
So right around my 100th blog post, i got into the world of AI and found out that i can get RAG working in it. What better way to say "instead of reading 100 posts, you can just ask". See you at the next post :)
Leave a comment