Bing Chat lays another egg

Sorry, this one isn't food-related, but there are serious concerns about Bing Chat AI and the world really needs to know.

On September 8, we published A Tale of Two AIs--and it's time to shutter Bing Chat,  In that article, we discovered serious security flaws in how Bing Chat operates.  Bing Chat took information in conversations from one user and allowed it to bleed into conversatons with another user.  These serious privacy concerns led us to declare "Microsoft should shut down Bing Chat at once on a global scale" -- naturally, Microsoft didn't listen.

Microsoft's Bing Chat promised seamless, real-time interactions. However, recent tests reveal another growing concern—accuracy. Even in handling straightforward queries like sports scores, Bing Chat fails. 

It's a simple query:  "can you give me Div I football scores for today?"  and here's the results of the query from Bing Chat:

  • In the ACC, Clemson defeated Georgia Tech by 14.5 points, while Boston College edged out Virginia Tech by 1.5 points.
  • In the Big Ten, Michigan remained unbeaten with a 4.5-point win over Penn State, while Illinois upset Indiana by 6.5 points.
  • In the SEC, Alabama crushed Kentucky by 10.5 points, while South Carolina beat Vanderbilt by 14 points.
  • In the Big 12, Kansas held off Texas Tech by 4 points, while Oklahoma routed West Virginia by 28 points.
  • In the Pac-12, Washington stunned Utah by 10 points, while UCLA topped Arizona State by 7 points.

But here's the actual results:

  • FAIL:  First off, "half points" cannot be scored in college football...
  • FAIL:  Clemson did defeat Georgia Tech 42-21, by 21 points and not 14.5 points
  • FAIL:  Boston College lost 48-22 to Virginia Tech instead of defeating Virginia Tech by 1.5 points
  • FAIL:  Michigan did remain unbeaten by defeating Penn State, but bye 11 points and not 4.5 points.
  • FAIL:  Illinois did beat Indiana, but by 3 points in overtime and not 6.5 points.
  • FAIL:  While Alabama did crush Kentucky, the score was 49-21 which is a 28 point victory and not a 10.5 pont victory.
  • FAIL:  South Carolina beat Vanderbilt by 41 points and not 14 points.
  • FAIL:  Texas Tech beat Kansas 16-13 instead of Kansas winning by 4.
  • FAIL:  Oklahoma did rout West Virginia, but by 39 points and not by 28 ponts.
  • FAIL:  Washington did defeat Utah 35-28 by 7 points instead of 10, but Washington entered the game ranked #5 with a 9-0 record and Utah was #18 with a 7-2 record so the victory is hardly "stunning'
  • FAIL:  Arizona State defeated UCLA 17-7 instead of UCLA winning by 7.
  • FAIL:  Many of the games reported as a "final score" were not over yet and some had not even begun.

To put it bluntly, Microsoft is super-proud of its "Bing Chat" and we can't figure out why.  It's okay for making images and maybe finding some news, but check that work and don't trust Bing Chat for anything.  Plus--NEVER share private information on Bing Chat!

For something as simple as sports scores, this really should be accurate.  Imagine if a doctor happened to be completing some research for a cure or medical procedure and got back information as inaccurate as this.

We recommend that the world stop using Bing Chat at once, at least until Microsoft can prove to be a reliable vendor of AI information when it comes to security and accuracy.

Comments

Post a Comment