Ilya Papers

Ilya Papers Website

Ilya Papers Github Repo

I saw that these papers went viral on Twitter, and the first thought that came to my mind was that there should be an easier way to grasp this information.

My brother and I wanted to create a tool to condense and summarize academic papers for easier reference. We started with RAG (Retrieval Augmented Generation), but we quickly found that it didn't give us the accurate results we needed to feel confident about sharing it with others.

We decided not to share this on Twitter. However, it's just another one of those projects that still feel needed to be out there.

Lesson Learned:

• Most people think that RAG can be used to create more accurate responses, but more often, we see that RAG is good for generalizing and creating more context.
• In order to grasp more accurate responses using RAG, we would need to utilize and try a number of techniques depending on the data we use (rerankers, embedding quantization, colBERT, etc.)
• Apps that are merely "GPT-wrappers" could stand out by being not only more accurate but also more efficient and tailored to specific domains.