Building an End-to-End Chat Bot with ONNX Runtime and Rust
Table of Contents Introduction Prerequisites Project Setup Architecture Overview Exporting Models to ONNX Loading an ONNX Model Text Generation Pipeline Building the CLI Chat Interface Going Further Conversation Memory Temperature & Top-p Sampling Streaming Tokens Performance Optimizations Testing Deployment Considerations Conclusion TLDR ...