The first lesson here is the importance of reading documentation. I decided to pick the Lang Chain framework to power the RAG part of the system. I was able to build such a system without having to rely on half-baked tutorials that usually fill the internet. Seriously, such tutorials are not bad since they help me get started easily, I normally take it upon myself to understand what the authors of the framework or library I am using are thinking and why they decided to implement it in the way they did. The outcome of reading the Langchain-provided documentation is an in-depth understanding of what Langchain can do and a better decision when implementing features.

QACLI is an interesting project for me, this allows me to think of possible ways of enhancing the project to the best possibility I can. I decided to use concurrency (I miss Java concurrency 🥹). Reading the Python documentation helped me to (re)understand concurrent.futures better. After reading this, I understand what ChatGPT is talking about (🤷🏿‍♂️ well, who doesn't rely on such assistance these days?) and can validate its suggestion. Why do I have to dig into the documentation? this piece of code below would likely take time and I want them to happen quickly and concurrently.

A simple refactor in the function above is to change the variable name fromall_userstousers. The reason is because thelist_all_usersfunction is not really returning all users if the email is provided. A second refactor is to change the function name tolist_usersor to change the name entirely, you know why already 😉

Random stuff I ask ChatGPT 🫤. (Did you see that mistake in the prompt at the end? Yet, GPT can figure things out, isn't that concerning? 🫠)

For the database that powers both regular data and is used as a vector store, I used PostgreSQL (and pgvector extension) via the supabase. I read the superbase documentation too, though I have to take a clue from its JS version of docs to better understand the community Python library.

A big takeaway is that, I should continually improve my written documentation to help and assist other developers when they are looking at my codebase or trying to use some functions I wrote.

The last three lessons are my favourite because they show how important different pieces of a system contribute to the overall success of such a system.

The above discusses how I am loading the documents. For my use case, I found out that loading the PDF manually works best for me. Now this is not to say the default loader is faulty, I am just experimenting and I found a better result.

The two retriever techniques that stood out for my use case are Long Content Re-order and Contextual Compression. I found out that these two techniques work well for long texts. My documents have a lot of pages, this could be too much for the model to figure out an answer. After experimenting with two retriever techniques, and tweaking my base prompt, I found a better response.

And lastly, at some point, my machine was making some noise 😒. I found out it is because the model is probably trying to figure out the golden answer to my question over a high number of relevant documents. Reducing the number of retrieved relevant documents (from 20 🫣 to 5 or less) calmed the machine. The takeaway here is that the more the number of relevant documents the more the memory usage.

It took me less than a week to implement this application, quite impressive to me because most of the technologies such as Lang Chain and supabase, I haven't used them in previous projects. I did this while still combining these with other time-demanding commitments. 💪🏿

And finally gentlemen and ladies, I was able to reduce my cost, build a custom RAG-powered system (ChatGPT if you will) and learn a couple of topics. In this blog, I did not go deep into what LLM I used and other too technical details of the system, the reason being that I am still doing more research on these across multiple use cases, a deeper dive-in will deserve its blog, stay tuned.

Thanks for reading.

Below is a screenshot of the expenses I incurred after chatting with several PDFs. Spending so much on conversing with my PDFs 😮, I could have just read them myself 😔.

For this experiment, I employed a simple pre-processing technique in NLP, removing stopwords from the corpus of each PDF to reduce the length of the prompts I would be passing to GPT. Of course, this is not the only option for this kind of task; I chose it as a mini-project to reflect on my newly acquired NLP power 💪 The image below displays the sum of all the tokens used, the original length of the PDFs, and the final length after the removal of stopwords.

I also need to mention that I could only pass 5-8 pages at most for each PDF, as the context window limit applies to some PDFs for documents with lengthy pages. In essence, I could only chat with a few of the documents I have.

With these results and my $7 debt, I conclude that using existing ChatWithPDF sites can be helpful, as I don't want to keep accumulating so much debt. Simultaneously, taking a closer look at this problem and my newfound superpower in NLP (due to a growing interest in machine learning, hence the reason for the mini-project), I challenge myself to figure out all the available and suitable techniques in NLP (RAG 😜 ?) I could employ it to reduce the cost of chatting with my PDFs.

In this series of blogs, I will be sharing my progress, and what I am learning, and hopefully, I won't incur much debt.

Upon reading Robert C. Martin's "Clean Code" book, I gained a newfound appreciation for Refactoring Guru's ability to convey the concept of refactoring in straightforward terms. Furthermore, Refactoring Guru's explanations of design patterns in plain language intrigued me, prompting a desire to enhance my understanding of these patterns. As a result, I opted to prioritize refining my grasp of refactoring by completing the Refactoring section before delving into the realm of design patterns.

In the coming days, my focus will be on studying Code Refactoring under the guidance of Refactoring Guru. I intend to document my learning journey on this blog, sharing both my insights and the methods I employ to acquire this knowledge. To gain a more comprehensive understanding of the topics discussed here, I encourage you to refer to https://Refactoring.Guru. This blog series serves as my initial step into the world of blogging, where I will be taking notes and sharing my learning experiences.

What is a Clean Code?

Clean code refers to well-structured, readable, and maintainable computer code that is easy to understand and modify. It is a coding philosophy and a set of principles and practices aimed at producing high-quality software. Clean code is important because it makes software development more efficient, reduces the likelihood of bugs and errors, and makes it easier for developers to collaborate on a project over time. The primary goal of refactoring is to reduce technical debt. It turns a jumble into tidy code and a basic design.

Features of Clean Code

1. Clean Code is obvious to other programmers: Clean code does not have poor variable naming, bloated classes, methods and magic numbers.

2. Clean Code doesn’t contain duplication: Duplicating code means you will have to make changes to it in every place you have the duplication.

3. Clean Code contains a minimal number of classes: Less code is fewer bugs. Code is a liability, keep it short.

4. Clean Code passes all tests, 100%.

5. Clean Code is easier and cheaper to maintain

Technical Debt

Technical debt is like postponing chores at home. Imagine you don't clean your house regularly, and over time, it becomes messier and harder to clean. Similarly, in software, when you take shortcuts or rush to finish coding without doing it properly, it creates "messy" code that's harder to maintain and can cause problems later on. You'll eventually need to spend more time cleaning up and fixing issues (paying off the debt) than if you had done it right from the start. For example, when you temporarily speed up without writing tests for new features, it will slow progress every day until you eventually pay it off by writing the tests.

Causes of Technical Debt

According to Refactoring Guru, these are the causes of technical debts:

1. Business Pressure: This is when business circumstances force you to roll out features before they are finished.

2. Lack of Understanding of Consequences of Technical Debt

3. Failing to combat the strict coherence of components.

4. Lack of Tests

5. Lack of Documentation

6. Lack of Interaction Between Team Members

7. Delayed Refactoring

8. Lack of Compliance Monitoring

9. Incompetence: This is when the developer just doesn’t know how to write decent code.

When to Refactor

1. When you add a new feature, consider taking that opportunity to clean up any existing unclean code related to that feature.

2. When you are fixing a bug.

3. During a code review, which can be an ideal moment to initiate a code refactor. This review provides an opportunity to enhance code quality before it's finalized.

Checklist for Refactoring Code

1. After refactoring your code, the improved code should be cleaner and more maintainable than the code you initially encountered.

2. Avoid introducing new functionality while refactoring. It's advisable to separate the introduction of new features and the refactoring of existing code into distinct commits or steps.

3. If there are existing tests for the code, ensure that all tests pass to confirm that the code is clean and maintains its expected behaviour.

In this article, I established what the concept of Refactoring Code is, what technical debts are and how they come to be. See you in the next article!