tribler/dsi-search-on-toy-dataset

DSI Search on Toy Dataset

This is a simplified demonstration of the search engine presented in De-DSI: Decentralised Differentiable Search Index.

For this example, we fine-tuned the T5-small model on a dataset comprised of 526 distinct documents, including:

• URLs to YouTube videos featuring movie trailers
• Magnet links for accessing CC-licensed music
• Bitcoin wallet addresses belonging to various artists

The train data consisted solely of the respective titles of the documents (i.e., no access to ambiguous queries), and therefore does not nearly perform to the degree we think is generally possible.

For demonstration purposes, however, this model can be tested with queries like "spider man", "oceans 13", "sister staarlightt", or "xileno bitcoin address" (to give some examples).