AI accuracy

[claidheamhmor]

A few days ago I needed to check which way the propellers rotated on my Hornet model kit, so I asked Chatgpt. It told me they rotated outward, and that the top of the blade tips rotated toward the cockpit (and it offered up an ASCII-drawn diagram that looked like a lozenge). That seemed contradictory, so I asked more questions. I was asked which kit I was building, and Chatgpt offered up some suggestions for improving the kit's accuracy - like lengthening the nose wheel. The Hornet did not have a nose wheel. 

 

Then it talked about the clean lines, and how the tail wheel doors closed. The Hornet did not have tail wheel doors. It suggested lengthening the tail wheel so that the model would sit more nose-up. That is not how it works.

 

At this point I couldn't trust a single thing it was saying, so I went to other sources to check which way the propellers rotate. They rotate inward, not outward as Chatgpt said. 

 

How much do you trust AI? I picked up those errors only because I'd been researching the aircraft.

 

Incidentally, I've tried Deepseek, which got the answer drastically wrong, saying the propellers are not handed, despite knowing the Hornet had different model engines on left and right. 

 

Copilot says the props turn inward, quoting a pilot reference. Copilot is right. It got the different engines right too. 

 

Gemini was correct too, and provided a video as reference.

 

Comments

[filialucis]

AI talks an awful lot of rubbish. It seems to read maybe ten websites and take whatever it finds there as fact. I slightly prefer Deepseek to Chatgpt because it isn't quite as sycophantic in tone, but in my experience there's nothing to choose between them in terms of accuracy of information.

I haven't tried Gemini or Copilot yet, but maybe I should...

[claidheamhmor]

Gemini and Copilot can be decent. Deepseek sometimes has outdated info.

[filialucis]

Oh boy, does it ever. I once had an argument with it about a public figure who had died not too long before; Deepseek claimed the death was a "conspiracy theory" debunked by the fact that the person had been active on social media after the alleged death reports.

It was a bit bewildering until I realised that it was going off its latest training info, which was more than a year old at that point and it thought the day's date was something like 14 months in the past. When I clicked the Search button to allow it to search the Internet, it very quickly confirmed that the person in question had indeed died. But until then, it stuck to its guns and would not be gainsaid that the whole thing was a conspiracy theory! 😂

[eve_prime]

I ask the Google AI things, which it supports with links. Jonathan uses Claude frequently; he pays for a more advanced version than is available publicly.

[claidheamhmor]

I quite like Claude too.

[kaishin108]

Oh gosh, this is pretty confusing to a novice or to someone who knows nothing about the subject they are looking up. I do notice a lot of mistakes in the Google AI that comes up automatically. Maybe that is Chatgpt, I'm not sure.

p.s. I just read that what I see is Gemini, it does seem like it has a lot of mistakes and in the first sentence says one thing and the next sentence contradicts that.

[claidheamhmor]

They just present a summary of what they see, and it can be contradictory.