GPU 管理分析：为何闲置 GPU 如同停飞的飞机

Models mentioned in this article 1

Dharma-AI/Dharma-OCR-LITE

Image-Text-to-Text • 4B • Updated Apr 17 • 486 • 22 Spaces mentioned in this article 1 Running on Zero Agents 6 DharmaOCR Lite Demo 📄 6 Extract text and tables from document images

More from this author

20 August 17, 2026 Newer Models, Same Advantage 58 July 16, 2026 Community The next frontier is managing an entire fleet of agents and there is nothing stopping it. \n","updatedAt":"2026-07-30T17:54:18.152Z","author":{"_id":"69efeb839255a6a81398f6af","avatarUrl":"/avatars/31cf43fab2e1aaa5c4a6aa2310c8e193.svg","fullname":"Suranto","name":"Mikeleton","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9737027287483215},"editors":["Mikeleton"],"editorAvatarUrls":["/avatars/31cf43fab2e1aaa5c4a6aa2310c8e193.svg"],"reactions":[],"isReport":false},"replies":[{"id":"6a6d0a73f2f02aadd53c048e","author":{"_id":"67b22efa9b36260fe286e99f","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/67b22efa9b36260fe286e99f/vm2fyO1NSqmjcIuyeF6PQ.jpeg","fullname":"Gabriel Pimenta de Freitas Cardoso","name":"GabrielPimenta99","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":23,"isUserFollowing":false,"primaryOrg":{"avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/67b22efa9b36260fe286e99f/5XWR_0JJv0gy6PRN9WDMF.png","fullname":"Dharma-AI","name":"Dharma-AI","type":"org","isHf":false,"details":"Specialized Small Language Models; Optimized Fine-Tuning; Efficient Inference","plan":"team"}},"createdAt":"2026-07-31T20:49:55.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Agreed. And the part that isn't priced in yet: a fleet of agents makes per-token cost scale linearly with the business. At some point large enterprises stop paying that curve and bring the models in-house — which doesn't remove the cost, it just trades a vendor price list for a utilization number. And agent fleets are brutal on utilization: real-time inference under latency SLAs, batch jobs, retraining, evals — all bursty, all fighting over the same finite pool. Managing the fleet is going to be, in large part, a GPU scheduling problem.","html":" Agreed. And the part that isn't priced in yet: a fleet of agents makes per-token cost scale linearly with the business. At some point large enterprises stop paying that curve and bring the models in-house — which doesn't remove the cost, it just trades a vendor price list for a utilization number. And agent fleets are brutal on utilization: real-time inference under latency SLAs, batch jobs, retraining, evals — all bursty, all fighting over the same finite pool. Managing the fleet is going to be, in large part, a GPU scheduling problem. \n","updatedAt":"2026-07-31T20:49:55.496Z","author":{"_id":"67b22efa9b36260fe286e99f","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/67b22efa9b36260fe286e99f/vm2fyO1NSqmjcIuyeF6PQ.jpeg","fullname":"Gabriel Pimenta de Freitas Cardoso","name":"GabrielPimenta99","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":23,"isUserFollowing":false,"primaryOrg":{"avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/67b22efa9b36260fe286e99f/5XWR_0JJv0gy6PRN9WDMF.png","fullname":"Dharma-AI","name":"Dharma-AI","type":"org","isHf":false,"details":"Specialized Small Language Models; Optimized Fine-Tuning; Efficient Inference","plan":"team"}}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9328820705413818},"editors":["GabrielPimenta99"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/67b22efa9b36260fe286e99f/vm2fyO1NSqmjcIuyeF6PQ.jpeg"],"reactions":[{"reaction":"👍","users":["ErickvL"],"count":1}],"isReport":false,"parentCommentId":"6a6b8fcace3e4a2fd6c3dc17"}}]},{"id":"6a6c42007f3b7929687d3a8f","author":{"_id":"66f15a00f7d3e809429a0d0e","avatarUrl":"/avatars/76248e5f0f20e0b0e8e95336e5abb52c.svg","fullname":"blacker","name":"blacker521","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false},"createdAt":"2026-07-31T06:34:40.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Scheduling is the core bottleneck of this problem. A CPU-like isolation mechanism, or a sandbox-based architecture on GPUs, could significantly alleviate this issue.","html":" Scheduling is the core bottleneck of this problem. A CPU-like isolation mechanism, or a sandbox-based architecture on GPUs, could significantly alleviate this issue. \n","updatedAt":"2026-07-31T06:34:40.079Z","author":{"_id":"66f15a00f7d3e809429a0d0e","avatarUrl":"/avatars/76248e5f0f20e0b0e8e95336e5abb52c.svg","fullname":"blacker","name":"blacker521","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9256368279457092},"editors":["blacker521"],"editorAvatarUrls":["/avatars/76248e5f0f20e0b0e8e95336e5abb52c.svg"],"reactions":[],"isReport":false},"replies":[{"id":"6a709886abfb77b0f07169d3","author":{"_id":"6825e93ede3594c18b8d97ac","avatarUrl":"/avatars/506583318c7aa57f91ddfaa554e5a377.svg","fullname":"Breno de Almeida Beleza","name":"BrenoBeleza","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":10,"isUserFollowing":false,"primaryOrg":{"avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/67b22efa9b36260fe286e99f/5XWR_0JJv0gy6PRN9WDMF.png","fullname":"Dharma-AI","name":"Dharma-AI","type":"org","isHf":false,"details":"Specialized Small Language Models; Optimized Fine-Tuning; Efficient Inference","plan":"team"}},"createdAt":"2026-08-03T13:32:54.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Good point, isolation is a real part of the picture. CPU-style boundaries (and the GPU sandboxing now landing, like the Kata Containers work) are what make it safe to co-locate unrelated workloads on the same card without fault or memory-contention risk, which widens the set of placements a scheduler is even allowed to make.\n\nTaking that a step further: once co-location is safe, we have to decide which workload runs on which GPU, at what time, at what priority, given that each one wants something different from the hardware (memory footprint, latency tolerance, duration). Isolation gives you a bigger, safer decision space; something still has to pick the right point in it. And rigid partitioning can even cut utilization, since an idle slice can't lend capacity to a busy neighbor.\n\nThat's exactly the direction we're taking next: an intelligent solution for job scheduling. We've got a follow-up article on it in the works, which will be published next week. Would be curious to get your read when it's out.","html":" Good point, isolation is a real part of the picture. CPU-style boundaries (and the GPU sandboxing now landing, like the Kata Containers work) are what make it safe to co-locate unrelated workloads on the same card without fault or memory-contention risk, which widens the set of placements a scheduler is even allowed to make. \n Taking that a step further: once co-location is safe, we have to decide which workload runs on which GPU, at what time, at what priority, given that each one wants something different from the hardware (memory footprint, latency tolerance, duration). Isolation gives you a bigger, safer decision space; something still has to pick the right point in it. And rigid partitioning can even cut utilization, since an idle slice can't lend capacity to a busy neighbor. \n That's exactly the direction we're taking next: an intelligent solution for job scheduling. We've got a follow-up article on it in the works, which will be published next week. Would be curious to get your read when it's out. \n","updatedAt":"2026-08-03T13:32:54.414Z","author":{"_id":"6825e93ede3594c18b8d97ac","avatarUrl":"/avatars/506583318c7aa57f91ddfaa554e5a377.svg","fullname":"Breno de Almeida Beleza","name":"BrenoBeleza","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":10,"isUserFollowing":false,"primaryOrg":{"avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/67b22efa9b36260fe286e99f/5XWR_0JJv0gy6PRN9WDMF.png","fullname":"Dharma-AI","name":"Dharma-AI","type":"org","isHf":false,"details":"Specialized Small Language Models; Optimized Fine-Tuning; Efficient Inference","plan":"team"}}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9595561623573303},"editors":["BrenoBeleza"],"editorAvatarUrls":["/avatars/506583318c7aa57f91ddfaa554e5a377.svg"],"reactions":[{"reaction":"👍","users":["ErickvL","GabrielPimenta99"],"count":2}],"isReport":false,"parentCommentId":"6a6c42007f3b7929687d3a8f"}}]},{"id":"6a7703cfc961dfc90e3e3ecb","createdAt":"2026-08-08T10:24:15.000Z","type":"comment","data":{"edited":true,"hidden":true,"hiddenBy":"","latest":{"raw":"This comment has been hidden","html":"This comment has been hidden","updatedAt":"2026-08-19T14:05:19.080Z"},"numEdits":0,"editors":[],"editorAvatarUrls":[],"reactions":[]}},{"id":"6a7b322f18c8c3ca299106af","author":{"_id":"6a5f540e8a30e0660988ea8f","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/U9AwsZyvT017uoE5buqC0.jpeg","fullname":"Mike Coville","name":"scifiscrivener","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":2,"isUserFollowing":false},"createdAt":"2026-08-11T14:31:11.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"I expect we will see GPU rental systems become a thing. There are a lot of high-end GPUs sitting in consumer computers doing absolutely nothing for 90% of the day. Imagine if individuals could allow a training model to access their GPU remotely and in return pay them something in return. Yes, remote GPUs is not optimal, but with the consumer taking on the maintenance, power bill, and ownership of the GPU, any use the training model gets is a net positive.","html":" I expect we will see GPU rental systems become a thing. There are a lot of high-end GPUs sitting in consumer computers doing absolutely nothing for 90% of the day. Imagine if individuals could allow a training model to access their GPU remotely and in return pay them something in return. Yes, remote GPUs is not optimal, but with the consumer taking on the maintenance, power bill, and ownership of the GPU, any use the training model gets is a net positive. \n","updatedAt":"2026-08-11T14:31:11.484Z","author":{"_id":"6a5f540e8a30e0660988ea8f","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/U9AwsZyvT017uoE5buqC0.jpeg","fullname":"Mike Coville","name":"scifiscrivener","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":2,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.964832603931427},"editors":["scifiscrivener"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/U9AwsZyvT017uoE5buqC0.jpeg"],"reactions":[],"isReport":false}}],"status":"open","isReport":false,"pinned":false,"locked":false,"collection":"community_blogs"},"contextAuthors":["ErickvL","GabrielPimenta99","gustavolucchetti"],"primaryEmailConfirmed":false,"discussionRole":0,"acceptLanguages":["en","zh"],"withThread":true,"cardDisplay":false,"repoDiscussionsLocked":false,"hideComments":true}"> Mikeleton Jul 30 The next frontier is managing an entire fleet of agents and there is nothing stopping it.

1 reply

GabrielPimenta99 Article author Jul 31 Agreed. And the part that isn't priced in yet: a fleet of agents makes per-token cost scale linearly with the business. At some point large enterprises stop paying that curve and bring the models in-house — which doesn't remove the cost, it just trades a vendor price list for a utilization number. And agent fleets are brutal on utilization: real-time inference under latency SLAs, batch jobs, retraining, evals — all bursty, all fighting over the same finite pool. Managing the fleet is going to be, in large part, a GPU scheduling problem.

Scheduling is the core bottleneck of this problem. A CPU-like isolation mechanism, or a sandbox-based architecture on GPUs, could significantly alleviate this issue.

1 reply

BrenoBeleza Aug 3 Good point, isolation is a real part of the picture. CPU-style boundaries (and the GPU sandboxing now landing, like the Kata Containers work) are what make it safe to co-locate unrelated workloads on the same card without fault or memory-contention risk, which widens the set of placements a scheduler is even allowed to make.

Taking that a step further: once co-location is safe, we have to decide which workload runs on which GPU, at what time, at what priority, given that each one wants something different from the hardware (memory footprint, latency tolerance, duration). Isolation gives you a bigger, safer decision space; something still has to pick the right point in it. And rigid partitioning can even cut utilization, since an idle slice can't lend capacity to a busy neighbor.

That's exactly the direction we're taking next: an intelligent solution for job scheduling. We've got a follow-up article on it in the works, which will be published next week. Would be curious to get your read when it's out.

I expect we will see GPU rental systems become a thing. There are a lot of high-end GPUs sitting in consumer computers doing absolutely nothing for 90% of the day. Imagine if individuals could allow a training model to access their GPU remotely and in return pay them something in return. Yes, remote GPUs is not optimal, but with the consumer taking on the maintenance, power bill, and ownership of the GPU, any use the training model gets is a net positive.

Reply Edit Preview Upload images, audio, and videos by dragging in the text input, pasting, or clicking here . Tap or paste here to upload images Comment · Sign up or log in to comment

Upvote 97 +85 Models mentioned in this article 1 Dharma-AI/Dharma-OCR-LITE Image-Text-to-Text • 4B • Updated Apr 17 • 486 • 22 Spaces mentioned in this article 1 Running on Zero Agents 6 DharmaOCR Lite Demo 📄 6 Extract text and tables from document images