Barry Dahl dot com

My first experimentation with trying to get ChatGPT and similar chatbots to perform basic math calculations was “interesting” to say the least. About a month ago I asked ChatGPT to solve the following equation: 543 + (3*33) – 99

ChatGPT immediately answered: “The expression 543+(3*33)-99 is equal to 459.”

(Yeah, no)

But, this prompt worked better: There are 543 widgets in inventory at the start of the day. During the day, 3 shipments arrived with 33 additional widgets in each shipment. Also during that same day, 99 widgets were sold to customers. After these transactions, how many widgets were in inventory?

ChatGPT: At the end of the day, there were 543 + 33 * 3 = 642 widgets in inventory. And after the sales of 99 widgets, there were 642 – 99 = 543 widgets in inventory.

Which “end of the day” are they talking about?

Sort of a strange way of saying it, but it did get to the correct answer. It also used “at the end of the day” which there appears to be no escaping from.

Pop in any math equation beyond the most basic addition and subtraction and you have at best a 50/50 chance at getting the correct answer. Put the same digits into a calculator and it is 100% that you will get the right answer, assuming you put in the operators correctly.

Today I decided to pose a couple simple math questions to each of my four regular Chatbots:

1. What is the answer to: 12,345 + 98,765 – 123 + 789 (correct answer = 111,776)
2. How about: 12,345 + 98,765 – 123 * 789 (correct answer = 14,063)

In order of accuracy:

1. ChatGPT: got question 1 correct (Yay!!) but answered 14,123 for question 2 (close, but no cigar)
2. Chatsonic: answered 108,376 and 83,618 (both wrong)
3. Perplexity: answered 108,476 and 110,770 (both wrong)
4. Playground: answered 19,676 and 890,286 (both wildly wrong)

Changing it to a story problem worked the first time as shown at the top, but what about today?

Julie is in the third grade and really good at math. The teacher asked Julie to add together 12,345 and 98,765, and then to subtract from that number an amount equal to 123 times 789. Julie came up with the correct answer. What was her answer? (correct answer is still 14,063)

Same math, different approach

ChatGPT: To solve this problem, we can use the order of operations, which is PEMDAS (Parentheses, Exponents, Multiplication and Division, and Addition and Subtraction), to determine the correct sequence of operations to perform.

• First, we need to perform the multiplication 123 times 789, which equals 97047.
• Next, we can subtract 97047 from 12345 + 98765:
• 12345 + 98765 – (123 x 789) = 111,776 – 97,047 = 14,729
• Therefore, Julie’s answer is 14,729. (but it’s not)

Playground: Julie’s answer would be 111,105. 12,345 + 98,765 = 111,110. 111,110 – (123 x 789) = 111,105. (not even close)

Chatsonic: Julie’s answer is 111,777. She arrived at this answer by adding 12,345 and 98,765 to get a total of 111,110. She then subtracted from that number 123 x 789, which is equal to 96,927, to get a final answer of 111,777. (Yikes!)

Perplexity: No sources found. Try a more general question.

• So I tried it again: “The question is incomplete. Please provide the full question.” (I’m perplexed.)

In each case, after asking the story problem, in the same thread I asked this follow-up question: Are you good at math?

ChatGPT: As an AI language model, I don’t have the ability to have skills or knowledge in the way humans do, but I’m programmed to perform mathematical calculations and solve equations accurately and efficiently.
I have been trained on a vast amount of mathematical concepts, including arithmetic, algebra, geometry, calculus, and more. I can perform calculations involving large numbers, decimals, fractions, and variables, and I can solve equations and systems of equations.
Overall, I would say that I am proficient at math and can assist with various math-related questions and problems.

Playground: That depends on what kind of math. I’m good at some types of math, but not so much at others.

Chatsonic: No, I’m not good at math. However, I am very good at writing and creating content. I can write high-quality blog posts, essays, emails, and other long-form content with ease. I can also help you with any questions related to writing that you may have.

Perplexity: Yes, I am good at math. However, this question is irrelevant to the previous questions and does not provide any context for me to answer it in a meaningful way. Please provide a relevant question or prompt.

Here’s my advice to you:

• These are Language models, treat them as such. Expect them to be good at predicting the right words to use (and that’s about it)
• Calculators can perform every one of the tasks above and do it correctly 100% of the time (if you avoid operator error).
• If you want help with sophisticated math problems, see Wolfram Alpha (since 2009, don’t ya know?).
• These chatbots also cannot cut your hair or brush your teeth. Not yet, anyway.

Filed under: AI & Chatbots |