There are only 10h left for this Openledger event on Cookie ~

@cookiedotfun @OpenledgerHQ

I was lazy for a long time, always posting about Sapien, and finally the snapshot was fixed at over 80... In the last 10 hours, I will post one more about Openledger.

Today I'm going to talk about how I just finished reading the technical documentation from @OpenledgerHQ, and I can't help but share my thoughts.
I admit that this framework has made breakthroughs in AI deployment efficiency, but it still falls short of being "revolutionary."

1. The reduction of memory usage from 40-50 GB to 8-12 GB is a highlight, especially for small and medium-sized enterprises.
However, what is not mentioned in the document is that this optimization largely relies on hacks in the CUDA kernel, and the long-term maintenance costs could be quite high.
You must have seen similar projects before, with stunning performance metrics in the early stages, but starting to encounter various strange OOM errors three months later.

2. Model switching time <100 ms?
In a real production environment, considering network latency and cold start issues, reaching 200 ms is already a blessing.
The benchmarks in the document are all measured in ideal conditions, and if I'm not mistaken, even basic stress test data is not provided. Whether there is a reference line for less than 100ms still needs to be validated through practice.

Three, is the GPU expansion plan just a pie in the sky?
The basic topological structure design has not been announced yet. It is important to know that in a distributed environment, the synchronization of the LoRA adapter will be a nightmare.
A similar project died on this last year, @KaitoAI should still remember that.

4. The support for edge devices is indeed a real demand.
I was impressed when I saw the optimization of Jetson Nano, as the solutions currently on the market are either too heavy or suffer too much loss in precision.
However, the quantitative techniques mentioned in the document are, to be honest, just ordinary QAT with a different name, something that the @cookiedotfun team has already played with two years ago.

Five, those who have seen the blockchain part understand it.
AI decision-making on the blockchain sounds great, but the documentation doesn't mention how to solve the gas fees at all. A simple inference request requires writing dozens of on-chain records; who can bear this cost in the long run?
It is more practical to use centralized logs combined with periodic Merkle root on-chain.

Of course, it's not a complete denial.
The design of the dynamic loading adapter is indeed clever, especially the idea of zero-shot automatic fine-tuning. Although the name sounds a bit juvenile, the technical approach is correct.
If the memory fragmentation issue can be resolved, this feature could become a killer app!!!

Overall, achieving a complete transformation of the AI service model as stated in the white paper is not something that can be accomplished overnight; it requires ongoing optimism~

#Openledger