Benchmarking LLM Inference Back Ends: VLLM, LMDeploy, MLC-LLM, TensorRT-LLM, TGI
15 points by chaoyu 3 months ago | 1 comments
  • iAkashPaul 3 months ago |
    Nice, I'll bench server.cpp as well