Earlier this year, I mentioned Kimi, a Chinese LLM that I liked using, at least for some things. At the time, it didn't seem to be much on anyone's radar (outside of nerd world). Hence, over the weekend, when there was a lot of news about the high performance (and cheaper price) of its newest model release compared to the American models, I gave myself a pat on the back for being an early adopter.
This also led me to watching quite a few videos on Youtube of people trying out Claude.ai and Kimi on the exact same tasks, and the results were quite interesting. Overall, as might be expected, Kimi takes longer but at about a third of the price, and produces results that are very similar (and certainly not, say, a third of the quality.)
But the most interesting result was that one video which showed that when given a relatively open ended job, such as inventing a game, with next to nothing very little in terms of instructing on the style, both LLMs produced near identical results. Well, there were some differences of course, but the core aspects would make anyone think they are extremely similar. [I made a mistake in my first version of this post - on rewatching the video, the instructions were not as open ended as I thought.]
As far as I can tell, the reason for this is not 100% clear, although I think some people are pointing to the "distillation" issue that is upsetting the American companies. They say that Kimi, and other Chinese models, conduct a large part of their training by running fake accounts on the American LLMs, and the results they get are hence influenced strongly by the outcomes of the American
Update: I have deleted most of the rest of this post, as it was based on my careless watching of the video over breakfast this morning. I thought the prompt had been "make a compelling game, your choice" (more or less), but it wasn't like that. Hence the similarities in the games are not as surprising as I thought.
I thought what it showed was that LLMs didn't have much in the way of human imagination, but seem very constrained in their "thinking". And the way they are fiddled with to give the "right" answers on contentious cultural issues (like how Musk has spent so much time fiddling with Grok to try to make it lean Right and not be "woke", but without being a Hitler admirer) is another good example of this.
Anyway, I must watch videos more carefully before I post about them!
%20_%20X.png)