AI rule summaries are full of errors

[tbhp]

Errors made by humans are quite different from AI slop. Humans tend to miss corner cases, get timing wrong, forget options. AI just makes up things based on what "could have been" the reality. And it is very verbose and confident about it.

Yes, AI mistakes tend to be more baffling in that they reveal the AI doesn't truly understand anything they are talking about. But my advice remains: don't bother with what is not official anyway.

Would you trust what someone says in a forum post about a game? You would read it, but you would take it with a grain of salt. The same goes for those rule summaries.

[cigma]

Why would you bother with those summaries anyway? ... When you are new to a game, you should read the official booklet.

For some languages and games, the rules summaries are the only available rules in that language, and they are often based on the English rules summaries.

[Jellby]

One thing I agree with: they should not be trusted, no more than a random Wikipedia page. The facts that they're the only source in a language, and they're based on the English version, are just additional reasons why they should not be trusted.

[Thomas]

Don’t blame BGA. Don’t blame AI. Blame me.

It was me, that used AI as a basis for summarizing rules. First, I did it for my personal use only, for a game where rules are available only in a language, that I did not understand. I was impressed by the result and I tried to use AI as a basis for creating summaries for games with rules in English or my native language.

I know, that AI makes mistakes and I never plant to use the plain result of the AI. I spend quite some time to verify the result against the original rules and to correct and edit the summary. It worked well for some games, but it failed for some others.

Looking back, I can no longer tell whether I overlooked something while reviewing the summary against the original rulebook, whether I accidentally clicked “Preview” instead of “Save” (and thus didn’t save my changes), or whether I accidentally applied an incorrect, unfinished version that I had open in a text editor at the same time.

learned from this. I stopped using this method of creating summaries after just a few days. If I ever have to create summaries again in the future, I will proceed more carefully and rely more on my human intelligence.

I‘m not a robot. I‘m a human, that makes mistakes, too. And I‘ sorry for that. Please do not blame BGA, they have nothing to do with it. And don’t blame AI, It's not AI‘s fault that I made mistakes. I’ve already corrected a few of my summaries in the past (and checked/verified some more, where I did not find any diff to the original rules), and if you can tell me specific games where you noticed errors in the summary, I’d be happy to take another detailed look at them. I will then correct them manually (or temporarily delete them until I’ve revised them, so they don’t cause any more confusion).

By the way, someone has already edited the Wiki entry with the summary for Pondscape. Thanks for fixing it!

[DrunkCat1337]

Yeah this GenAI infection is running deep. It's time premium subscriptions start getting canceled.

[SiRan90]

Yeah this GenAI infection is running deep. It's time premium subscriptions start getting canceled.

A tad dramatic. Rules summary has little to no impact on the game or who you are playing against. If even that insults you you can just leave, you don't have to announce it on every AI post

[drewco]

Don’t blame BGA. Don’t blame AI. Blame me.

It was me, that used AI as a basis for summarizing rules. First, I did it for my personal use only, for a game where rules are available only in a language, that I did not understand. I was impressed by the result and I tried to use AI as a basis for creating summaries for games with rules in English or my native language.

I know, that AI makes mistakes and I never plant to use the plain result of the AI. I spend quite some time to verify the result against the original rules and to correct and edit the summary. It worked well for some games, but it failed for some others.

Looking back, I can no longer tell whether I overlooked something while reviewing the summary against the original rulebook, whether I accidentally clicked “Preview” instead of “Save” (and thus didn’t save my changes), or whether I accidentally applied an incorrect, unfinished version that I had open in a text editor at the same time.

learned from this. I stopped using this method of creating summaries after just a few days. If I ever have to create summaries again in the future, I will proceed more carefully and rely more on my human intelligence.

I‘m not a robot. I‘m a human, that makes mistakes, too. And I‘ sorry for that. Please do not blame BGA, they have nothing to do with it. And don’t blame AI, It's not AI‘s fault that I made mistakes. I’ve already corrected a few of my summaries in the past (and checked/verified some more, where I did not find any diff to the original rules), and if you can tell me specific games where you noticed errors in the summary, I’d be happy to take another detailed look at them. I will then correct them manually (or temporarily delete them until I’ve revised them, so they don’t cause any more confusion).

By the way, someone has already edited the Wiki entry with the summary for Pondscape. Thanks for fixing it!

Thank you for replying! It's good to know that you have learnt from it, and this isn't some site-wide thing. My gaming group use the summaries to get into a new game, so even a short primer on the rules is enough to through the first few turns. It doesn't need to be a complete transcription.

[lefalaf]

Yeah this GenAI infection is running deep. It's time premium subscriptions start getting canceled.

A tad dramatic. Rules summary has little to no impact on the game or who you are playing against. If even that insults you you can just leave, you don't have to announce it on every AI post

Even after having read the rules, watch videos, etc., it can have an impact if you are double checking something quickly to jog your memory and it confidently states something wrong. A human created summary, pre-LLM, at least tried in good faith to be accurate with what was stated. Omissions were fine as you'd just go to the book then, but misinformation is bad. Better to not state anything than to mislead that 2+2=5.

Once something is polluted, it then becomes difficult to clean up or avoid without massive effort. Same as search results, the mass/mess of AI sites, AI images in searches, AI images on image sharing sites, all drowning out the real thing and devaluing the overall site/tool/etc.

I give Thomas credit for admitting the mistake, that is more than AI-fans usually do, but he still had to go an defend AI (LLMs) countless times in his post. Why? It's delusional. It's like politics or religion.

[Resolute445]

Yeah this GenAI infection is running deep. It's time premium subscriptions start getting canceled.

A tad dramatic. Rules summary has little to no impact on the game or who you are playing against. If even that insults you you can just leave, you don't have to announce it on every AI post

By the same token, it's important that Asmodee understand what IT is risking by going neck deep into this morass. AI slop games, AI slop rules summaries. Where does it end, and why should anyone be expected to pay for the privilege of finding out?

I appreciate that Thomas at least recognizes he goofed. I don't appreciate that he still tries to defend tools that clearly have made numerous mistakes and just are not suitable for this kind of work. He has one block of five rules summaries created in a half hour, and another of TEN created in about the same time. There is no chance whatsoever that he reviewed those summaries for accuracy. That is a problem that someone has to fix. And lets face it, it's not going to be the person who used the clanker in the first place. It rarely is.

[Resolute445]

Slight aside, I just noticed "Users browsing this forum: Amazon [Bot]" at the bottom. I expect to and do see things like "Bing [Bot}" there from time to time to index the forum for search engine use, but Amazon doesn't do search, so it must be feeding their gen-AI our posts.

EDIT: Bing just joined the party after I sent this, lol.
EDIT: and now Google.

At this point, assume every major website has sold you out to train AI. It's become a truly unfortunate and repugnant core aspect of the internet and the only way to escape it is to go completely offline. It is certainly too much to ask that some of that money Asmodee sold us out for go toward improving the servers.