How to Know If You Can Trust an AI's Answer to Your Question
Anonymous Coward writes:
How to know if you can trust an AI's answer to your question
I recently typed a simple question into Google search: How much screen time is too much for teenagers? Instead of presenting links, as Google had been doing for many years, it gave me an AI-generated answer. The artificial intelligence agent cited a number, then complicated that reply, noting that quality and balance of time could matter more than the number of hours, and that "too much" time could depend on a teenager's sleep, exercise, school demands and mood.
I tried another search: Should I take a daily aspirin? This time the AI answer presented me with medical information, warned about risks and offered more tailored guidance if I provided my age and medical history.
These were good replies. What interested me was that they were different kinds of replies.
Debate about AI answers has focused on accuracy: Did the system get the answer right? That matters, but accuracy is only one test. Each kind of answer requires a user to judge something different.
I find it useful to sort AI answers into an "answer typography" of four broad types: factual, interpretive, constructive and strategic. A factual claim can often be checked against a source. An interpretation can be accurate and still reflect choices about which evidence matters. A construction can be well reasoned and still be wrong for the person receiving it. A beautifully written strategic document may not be true. Yet AI presents all four types of answers in much the same fluent, authoritative form; the differences are easy to miss.
The four categories are not airtight boxes. A response from an AI agent can reflect several types. That said, I describe each type of answer below, and offer guidance for deciding whether a reply is ready to use or needs more investigation.
[...] Before asking whether an AI answer is right, ask a more basic question: What kind of answer is this? The type will tell you what to do next.
Fallacy of Appeal to Authority, Ad Verecundiam. If you are not smart enough to know whether the AI is answering correctly, you are not smart enough to deploy AI. A simple solution.
Read more of this story at SoylentNews.