ChatGPT vs. Gemini: Which one should I use?

(The Debrief is delivered monthly and included as part of your subscription to The Productivity Ping)

Hey friends - Google announced their latest Gemini AI model a few months back and since I (still) see a lot of confusion and misinformation online, here’s a quick overview of what Gemini is and what this announcement means for all of us.

Please share your feedback in the comments, especially if I get something wrong!

tl;dr

1. Gemini comes in 3 versions: Ultra, Pro, and Nano (here’s a 90 second recap of the announcement)
2. As of Jan 21st 2024, consumers (i.e. us) are only able to access Gemini Pro (via Google Bard) and Gemini Nano (via the Samsung Galaxy S24 series and the Google Pixel 8 Pro)
3. Gemini is Multi-Modal Large Language Model (LLM) that will replace Google’s other LLMs (LamBDA and Palm 2). Multi-modal means Gemini is not only trained on text, sound, videos, images (i.e. the inputs), but it can also output text, sound, videos, images (i.e. the outputs)

My take on benchmarks

According to benchmarks released by Google, Gemini Ultra has better performance than OpenAI’s GPT-4:

According to benchmarks released by Microsoft, OpenAI’s GPT-4 has better performance than Gemini Ultra (if you use specific prompting techniques)

Uhhh…what?

In my humble and technically un-informed opinion, benchmarks don’t mean shit compared to real-world usage.

These benchmarks are analogous to running a battery rundown test between the latest iPhone and the latest Samsung device under standardized conditions (charging them to full, closing down all apps, etc.)

We (consumers) would never actually use our phones that way, just like we’d never use Google Bard and ChatGPT under benchmark testing conditions.

Okay so which one should I use?

Before the release of Gemini Ultra, I would still stick with ChatGPT for general purpose tasks and activities, because:

• I’ve personally found that both the free version of ChatGPT (ChatGPT 3.5) and the paid version (ChatGPT 4) generate more consistent, and higher quality results. I think this is a result of ChatGPT being able to better incorporate context in its responses.
• ChatGPT also has the free Custom Instructions feature that provides a small boost to the quality and consistency of ChatGPT responses (when feature is enabled).

For tasks that require high factual accuracy, access to real-time data, access to Google applications, using Google Bard is a no-brainer:

• After enabling extensions in Google Bard, you can use Bard to access and analyze information from Gmail, Google Docs, and Google Drive
• You can ask Google Bard to pull the latest revenue data from tech companies, output in table format, and import that table into Google Sheets in 1 click

Let me know if you have any questions in the comments!

Got feedback for this edition of The Debrief? Let me know!

Thanks for being a subscriber, and have a great one!