When Alibaba Cloud AI Gateway is layered with a third-party large model API aggregation platform, which layer should handle routing and retries?

When enterprises use both Alibaba Cloud AI Gateway and a third-party large model API aggregation platform, if both layers perform routing, retries, and caching, it will cause retry amplification, model drift, and bills that cannot be reconciled. The article recommends dividing responsibilities according to a "single policy owner" principle: the in-cloud gateway handles task-level model selection, token budgets, and auditing, while the aggregation platform only switches resource channels for the same model and returns actual execution information. The retry limit is recommended to be set to 1, and request_id and attempt_id should be used to align logs and bills.