Intel 发布面向 Xeon CPU 的 AI 推理软件 Intel Inference Microservice
Intel Unveils AI Inference Software Optimized for Xeon CPUs < Semiconductor < 기사본문 - The Elec Inc.
Intel 于 10 月 16 日发布 Intel Inference Microservice，这款容器化软件用于在 Xeon 服务器 CPU 上最大化 AI 推理性能，可快速为 CPU 优化 LLM。软件启动时会检测 Intel 处理器并套用预先验证调优的推理设置，默认推理引擎为开源 vLLM，并支持 OpenAI SDK、LangChain、LlamaIndex、LiteLLM 和 Haystack 等框架。Intel 计划将支持扩展到其全部 GPU 产品线，包括面向 AI 推理的 D 系列 GPU。
正文
当前显示原文，中文全文译文尚未就绪。
가
The body content of the article will be changed to this letter size.
스크롤 이동 상태바
Intel unveiled Intel Inference Microservices, software designed to optimize AI inference performance on Xeon processors. (Image: Intel)
Intel on Oct. 16 unveiled Intel Inference Microservice, software designed to maximize artificial intelligence (AI) inference performance on its Xeon server central processing units (CPUs).
The containerized software enables rapid optimization of large language models (LLMs) for CPUs. It includes libraries containing the code, engines and configurations needed to run open-source LLMs.
When launched, the software detects the Intel processor and applies prevalidated and tuned AI inference settings, reducing reliance on specialists in AI inference optimization. Major AI frameworks, including the OpenAI SDK, LangChain, LlamaIndex, LiteLLM and Haystack, support the OpenAI API specification. The software uses the open-source vLLM as its default inference engine.
Optimizing existing LLMs for hardware previously took a week. Settings for computational precision, workload distribution across CPU cores, memory allocation across non-uniform memory access (NUMA) nodes and hardware instruction set selection vary depending on the combination of AI model and processor.
Intel plans to expand Inference Microservice support to its entire graphics processing unit (GPU) lineup, including the D-series GPUs designed for AI inference.
JEONG IL JOO 1week@thelec.kr
My comment