Test of 12 AI decision models finds bias and sequence effects

Researchers made tests on 12 decision models for bias with 3 known tests. A decision model gets a question and a list of alternatives. It gives a probability score for each one. Software uses the answers with no person to check them. The test had 1,638,000 decisions. It found bias that favors stereotypes and answers that change with the sequence of the alternatives.

How it works

The team used 3 bias tests: BBQ, CrowS-Pairs and SilverSpoon. The team changed them to fit models that give typed answers. A typed answer comes from a set list. Each item with an effect from sequence had 2 tests, with the alternatives in a different sequence.

The numbers

• In BBQ, 97 of 143 scores are clear results, and 94 of these show stereotype bias.
• A change of the sequence changed 0.0% to 72.6% of the selections.
• Letter prompting with no tuning changed 73% of the answers.
• A model of a vendor did not answer 99.5% of items with no clear answer.

What is not known

The paper does not show the effect of this bias on products. The authors tell that confidence follows instability, not bias. They tell that tests must always change the sequence of the alternatives.

Insight: Buyers can now set the test for sequence

The authors tell that buyers must have scoring with no effect from sequence in the contract. Sarvam tells that its Prism model puts alternatives in one set sequence before it gives a score, and Sarvam did all the tests. This reads as a step to put the buyer in control of the proof. Buyers that make this test first can set the standard.