87flowers ~ $

demotivation, llms, and chess engines

chess llm

This is a rant.

I will not be naming names.


I love figuring out and solving technical problems. It is my lifeblood. I spend a lot of time outside of work doing this, because I find joy in the puzzle and the search for the solution.

I am fortunate enough to not use LLMs at work, but I have seen it seemingly take over the software industry.

I am an advocate for the open sharing of knowledge. I have built on the knowledge from others, and my work is not possible without the efforts of those who came before. I wish to give back and build upon the edifice of human achievement.

I have been demotivated to properly document new innovations recently.

In the chess engine community, we have seen the influx of LLM-generated engines.

Some of these are one-shot engines and thus not very good.

Some use modern testing processes and thus are able to self-improve.

The issues I have with them are:

  1. I have seen search patches added with comments that are outright wrong. An example is how LLMs simply don't understand how hindsight extensions or reductions work. I suspect in cases like this is because it is a recent invention and there is very little documentation on it other than communication on Discord.
  2. I have seen whitewashing of the sources of the patch. I have seen an LLM generated engine with a search feature which was clearly lifted from Stockfish but credits the Chess Programming Wiki for the source, which has a very outdated implementation on the page. I have seen the automated scouring of other engines for patches, and subsequent hiding of the receipts when this was pointed out. I have seen some novel SIMD I wrote lifted, under the justification that it is "common" - when the concepts used to write that SIMD didn't exist as far as I can tell prior to my invention of them.
  3. I have seen LLM engines generated, entered into ranking lists, then abandoned, polluting these with low-effort engines while bringing no novelity to the table.

Fundamentally, reading an LLM generated engine is uninteresting. I enjoy taking crazy leaps and gambles, without knowing if I will make the landing. Sometimes these pay off. Sometimes they don't.

LLMs do not truly innovate. It is self-evident - looking at the engines produced - that if you ask an LLM to produce an engine it will be very cookie cutter, and will probably just lift everything from every open source engine it can get its hands on.

Perhaps the issue is the driver of the LLM. Would it be possible for innovation to be driven if you direct an LLM and keep it closely leashed? Probably. But writing the code has never been the bottleneck in my experience. Why bother babysitting a machine when it is easier to do it yourself?


I worry.

I enjoy novelity. I enjoy invention. I worry about the stagnation that would occur if we allow ourselves to submit to the machine. The machine does not invent, it fills in the gaps. If the driver takes their hands off the wheel it produces bland mush. It makes the average decision. The common decision. The uninteresting decision. The safe decision.

I worry we will lose human understanding. I worry we will lose the spark of invention. I worry we will lose the joy of creation of knowledge.

I have seen this consume other hobbies.

I don't draw as much as I used to. The machines do it better than I now. My art is subpar to theirs. I have no motivation to continue.

I am still able to run ahead of the machines in software. I am not sure for how long.


I am in the middle of writing an article for an acceleration technique for chess neural networks. I have looked at a couple of LLM-generated engines. They do not have this technique yet. It feels demotivating to know that as soon as I document this clearly they will likely replicate it without understanding. I want to share new knowledge and yet I feel the machine is a leech.

What about a human who copies? A human who copies would at least be able to invent. I have seen this already a top engine that adopted "lilytech" and implemented it in their way has given me good insight and fresh new ideas! I love these remixes. I learn more by sharing with other humans than I do by not sharing.

The LLM engines I have seen that have copied my code do not innovate. They do not provide me with new insight. They care so little about copying, they use the exact same idosyncratic variable names I do. Their output is uninteresting.

Would my opinion change if AI is able to become a true partner in innovation? It might do. I feel current generation LLMs are incapable of the on-line learning required. But I feel I will also only truly know how I feel about that if/when they roll that out.

© 2024. All rights reserved. 87flowers