Skip to main content

Build a Q&A Assistant over Your Notes

Level: Intermediate · Time: 45–55 min · Category: Command line

Tags: Python · OpenAI SDK · Retrieval · RAG

Build a CLI that answers questions from a folder of notes. It splits your files into chunks, finds the most relevant chunks with a simple local search, and sends only those to the Research Computing OpenAI-compatible API. This is the retrieval-augmented generation (RAG) pattern, kept small enough to finish in one sitting.

Unlike the Lab Onboarding Assistant, which reads a single file, this project retrieves from many documents.

Prerequisites​

Confirm Python 3 and pip are available before you start:

python3 --version      # expect 3.9 or newer
python3 -m pip --version

If Python is not found, install it from python.org. On the supercomputer, load a Python module instead (for example module load python), then run the checks again.

What you will do​

  • Create a Python CLI workspace with the OpenAI SDK and scikit-learn.
  • Load and chunk a folder of text and markdown notes.
  • Retrieve the most relevant chunks for a question.
  • Send only those chunks to the API and answer with citations.

Build it step by step​

1. Create the CLI workspace​

mkdir notes-assistant
cd notes-assistant
python3 -m venv .venv
source .venv/bin/activate
pip install openai python-dotenv scikit-learn rich
Keep this out of Git

If you track this project with Git, do not commit your virtual environment or any secrets. Create a .gitignore file in the project folder with at least these lines:

.gitignore
.venv/
.env
__pycache__/

.venv/ is large and specific to your machine, and .env holds your API key — neither belongs in a shared repository. New to Git? See Track a New Project with Git.

2. Add API connection values​

Your program needs three values to reach the API: your API key, the gateway URL, and a model ID. You keep them in a small text file named .env so they stay out of your code — and out of Git.

What is a .env file, and where does it go?

A .env ("dot-env") file is a plain text file that stores settings as NAME=value lines, one per line. Programs read it at startup so you never have to paste secrets like API keys directly into your code.

  • The filename is literally .env — a leading dot with nothing before it (not config.env or .env.txt).
  • It goes in your project folder: the same directory you just created, where the .venv lives and where you run your commands.
  • The python-dotenv package loads it with load_dotenv(), which looks for .env in the folder you run the program from.
  • Because it holds a secret, never commit it to Git (see the note below).

Create a new file named .env in your project folder — in VS Code use File → New File, or run code .env in the terminal — then paste these three lines and replace the placeholders:

.env
OPENAI_API_KEY="paste-your-key-here"
OPENAI_BASE_URL="https://openai.rc.asu.edu/v1"
RC_LLM_MODEL="paste-model-id-here"
Where do the values come from?

Get your API key and a valid model ID from the LLM API guide. The OPENAI_BASE_URL above is already correct for the Research Computing gateway.

  • Use the /v1 base URL, not the full /chat/completions URL, for the Python SDK.
  • Use a model ID that appears in the portal model list for your account.
  • Keep .env out of Git: add a line containing .env to your .gitignore (see Track a New Project with Git).

3. Collect a notes folder​

Create a notes/ folder with a few markdown or text files. Real content works best, but a couple of short sample files are enough to test the workflow.

mkdir notes
printf "# Storage\nScratch is temporary and is purged. Use project storage for durable data.\n" > notes/storage.md
printf "# Jobs\nSubmit batch work with sbatch. Check the queue with squeue.\n" > notes/jobs.md

4. Ask for chunking and an index​

Retrieval quality depends on how you split and index the text.

You are helping me build a beginner Python RAG CLI that uses an OpenAI-compatible API.

Project goal (this step):
- Load every .md and .txt file in a notes/ folder.
- Split each file into overlapping chunks of roughly 800 characters, keeping the source filename with each chunk.
- Build a TF-IDF index over the chunks with scikit-learn.

Constraints:
- Keep the code in one file named ask.py.
- Explain how the chunks and index are stored in memory.

Please create ask.py with the loading, chunking, and indexing in place.

5. Retrieve and answer​

Now use the index to retrieve context and send only the top chunks to the model.

Please add question answering to ask.py:

Behavior:
- Command: python ask.py "Where should I keep durable data?"
- Use the TF-IDF index to retrieve the top 3 most relevant chunks (cosine similarity).
- Send only those chunks plus the question to the chat completions API through the OpenAI SDK.
- Instruct the model to answer only from the provided chunks, and to say so if the answer is not present.
- Read OPENAI_API_KEY, OPENAI_BASE_URL, and RC_LLM_MODEL from a .env file.

Constraints:
- Print a "Sources" list with the filenames of the chunks used.
- Keep the code beginner-readable.
- Show the command to run a test question.

6. Add one feature of your own​

Please add one focused feature to ask.py:

Feature:
- Add a --k option to control how many chunks are retrieved (default 3).

Constraints:
- Keep the default behavior unchanged when --k is not passed.
- Print the similarity score next to each source.
- Explain how changing k affects the answers.

More prompts to try​

  • Add a --show-context flag that prints the retrieved chunks before the answer.
  • Support PDF notes by extracting text with pypdf during loading.
  • Cache the index to disk so startup is faster on large note sets.

Troubleshooting​

  • If answers miss obvious content, increase --k or reduce the chunk size so relevant text is not split awkwardly.
  • If retrieval returns unrelated chunks, the notes may be too short or too similar; add more content.
  • If scikit-learn fails to install, upgrade pip with pip install --upgrade pip and try again.
  • Always keep the "answer only from the provided chunks" instruction so the model does not invent facts.

Next steps​