The A.I. Daily
NVIDIA Launches AIPerf to Benchmark LLM Inference at Scale
NVIDIA launched AIPerf, a ground-up rewrite replacing GenAI-Perf, using a multiprocessed ZMQ architecture to bypass the Python GIL for accurate LLM inference benchmarking. It supports over 15 endpoint types and datasets like ShareGPT.