New pipeline for zero-shot text classification

馃Transformers
67

Hey @charly, here鈥檚 a previous thread about that. The main tricks are going to be:

  • Use one of these distilled models which are smaller and faster but with similar results
  • Run with the ONNX Runtime. One way you can do this is with this project created by @valhalla before he joined Hugging Face
  • If you have long sequences you鈥檙e classifying, you can try truncating to just part of the sequence. That鈥檒l give you a speedup but you鈥檒l have to evaluate how it impacts your performance.
  • If you have a large # of candidate labels, try to come up with a heuristic or use a super lightweight classifier to identify the most likely candidates, and then just feed in those more likely candidates rather than all of them.

Btw if it鈥檚 public would you mind linking to your streamlit app? It鈥檚 always fun to see the ways that people are using it :blush:

1 Like