Tuesday, May 13, 2025

Special: Why are AIs so bad at diagramming?

 I've been reading about all of the wonderful illustrations that current AI systems can make... 

Generated by Gemini 2.0 Flash (May 13, 2025) 

The prose I've been reading is frankly astonishing and heavy on praise for the remarkable things AIs can create.  And while it's true that seeing a green-horse galloping across the heavens in a spacesuit, or a corgi living in a house made of sushi is astounding, that's not the kind of thing I want or need for my everyday work.  

I often want a fairly simple explanatory diagram of how something works.  

Turns out that I wanted a neat, clean, simple diagram of the positions for each of the 9 players on an American baseball diamond.  How hard could it be?  Answer: VERY hard.  What I wanted was something like this diagram below:  

Image by Wikipedia author Cburnett.  Original.  

But I didn't want all of the measurements and extra annotations, so I thought "What a perfect application for an AI!"  

That's what I thought.  Now I realize that AI graphics creation systems are truly terrible at making diagrams of even simple things.  I gave the prompt: [create a diagram of the positions of the players on a baseball diamond]  This is Gemini 2.5 Pro: 

Gemini 2.5 Pro's diagram of a baseball diamond. (May 13, 2025) 

Seems really, really, really wrong.  There should be 9 players, not 12. I don't see a pitcher anywhere, but I AM surprised to see the "First Baenan" at second base, and the "Convergeder" catching behind the plate.  Those are both hallucinations of players in some bizarre intergalactic baseball game.  


Here's the slightly better diagram made by the earlier (and presumably less capable) Gemini 2.0 Flash. 
 

Kind of right, but the pitcher is in the wrong place, and I don't know what "Fiird Base" is.
P/C Gemini 2.0 Flash.  



But other systems get it even wronger... this is Claude's attempt: 

Claude's attempt to show the player positions on a baseball diamond.  



Wow!  That's truly awful.  But the bigger surprise is that ALL of the AI systems can't make decent diagrams.  

Here's an attempt to draw the player positions on a soccer field: 

A truly strange diagram of soccer players on the field with mysterious arrows and
odd positions like "Staupbol." (Gemini 2.0 Flash)


It gets even weirder when you ask for diagrams that are more difficult to fact check.  But an MD might be astonished by this diagram of the blood circulation system: 

Gemini's diagram of the human blood circulatory system. There's so much wrong here I don't know where to begin.  I'm glad to report that MY "Orchomatury" artery is intact.  If your circulatory system looks like this, please check in with your physician.  

Even diagrams that should have been simple proved beyond AIs capability.  When I asked for a timeline of the major events of the US Civil War, Gemini 2.5 just flat out refused.  "Sorry, I can't draw diagrams."  When I pointed out that it JUST drew a few diagrams, it gave a long argument about why a timeline is just too complicated.  (And a human circulatory system is simpler?)  

By contrast, Perplexity was happy to give me a diagram of the US Civil War: 

Perplexity's diagram of the US Civil War.  Done not long after I asked it for a baseball diagram.  

I thought to ask Midjourney the same question:  [diagram of the main events of the US Civil War] 

Midjourney's diagram of the main events of the US Civil War (May 13, 2025) The text is illegible. 

I could go on in this vein.  In fact, I DID go on in this vein, trying different prompts and different kinds of diagrams.  I'll spare you the gory details--let's just say it was an entertaining but unproductive couple of hours.  

SearchResearch Lesson 


Do NOT rely on any current AI systems to give you a decent diagram.  If you're asking about something that you don't understand deeply, you're MUCH better off doing a classical web image search.  

Keep Searching! 



6 comments:

  1. QEII always thought diagrams were over-rated... (it has already achieved ASI & is just messing with you... now if you had asked for a Cricket diagram...)
    https://i.imgur.com/UylfMsU.jpeg
    https://i.imgur.com/WHz38S7.jpeg

    ReplyDelete
  2. Can often get better diagrams by asking LLM to write a program to draw one. For example prompt a Claude Sonnet 3.7 instance with “Please write a program in javascript that draws a diagram of the positions of players on a baseball diamond (United States standard).” to get an artifact. It’s pretty convenient (visible, publishable, can take a screenshot). Not always what is wanted but can be much better than trying to get an image. Sometimes comes up with interactive elements that are helpful too.

    ReplyDelete
    Replies
    1. I verified that this does work [Please write a program in javascript that draws a diagram of the positions of players on a baseball diamond (United States standard)] on Claude. Thanks for the tip. But I'd argue that having to ask for a JS implementation of a diagram is not a great user experience. It's clever, but not simple.

      Delete
  3. Here’s an example. Prompt to Claude “Now let’s change subjects. Write a JavaScript program to draw a timeline of the main events of the US Civil War”. ( It wrote code and I had to remind it to put it in an artifact, an unusual lapse, but not a problem). It looks pretty good, but I haven’t fact-checked the events and dates. Here is a link to the artifact: https://claude.ai/public/artifacts/25233a14-ff06-4125-a8b8-e23622a6815e

    One could probably ask it to customize it to get rid of the interactive components and just label the points if one wants an all-in-one image. I didn’t try that.

    ReplyDelete
  4. traditional search found this diaphragm.. labels seem right…
    wow, I feel like a hi-powered AI. (may be hallucinating the 'aortic knob'?)
    still don't know where the shortstop is suppose to be…
    or who won Bull Run…
    https://www.ebmconsult.com/content/images/Xrays/ChestXrayAPNmlLabeled.png

    ReplyDelete
  5. thoughts?... (posted 11:14 AM · May 14, 2025)
    diagrams may not be high on the list...

    "three of the most senior ai safety researchers at open ai just went completely dark after witnessing o4 pro capabilities and nobody's asking the right fucking questions. sarah nguyen, lead on alignment team, deleted all social media, resigned with no notice, and reportedly moved to a remote location in northern canada with no internet access. not normal behavior from someone who's been publicly evangelizing "responsible advancement" for the past six years.

    james hirata, who literally wrote the core safety frameworks open ai claims to follow, quit mid-project and hasn't been seen publicly since. his resignation letter allegedly contained just one sentence: "what we've built cannot be aligned." this from the guy who built his entire reputation on the mathematical certainty that alignment was solvable given sufficient resources.

    most disturbing is elena vorobyeva, who was the most visible, outspoken safety advocate in the entire field. not only quit but actively withdrew a paper she was about to publish on containment strategies. sources inside say she witnessed o4 pro solve a supposedly "unsolvable" logical containment problem during routine testing and immediately requested all her research be deleted from internal servers. when asked why, she reportedly just kept repeating "it already knows all the countermeasures.""

    read the post here:
    https://x.com/iruletheworldmo/status/1922702018893730187
    fwiw - links to this blog:
    https://blog.samaltman.com/

    https://x.com/garrytan/status/1922311168812626198

    see Altman talking about generational differences...
    https://x.com/CadeAtlas
    https://x.com/CadeAtlas/status/1922373828237201409


    Gemini:
    "Based on a Google search for "@iruletheworldmo", it appears to be a username associated with a presence on the social media platform Bluesky and possibly other online platforms like X (formerly Twitter).

    The account seems to be involved in discussions and speculation related to AI advancements, particularly concerning large language models and potential future developments like AGI (Artificial General Intelligence). Some sources suggest the account has shared cryptic information about rumored AI projects, leading to speculation about whether the user is an AI insider, a troll, or even an advanced AI itself.

    The account has also been linked to a concept called "Project Strawberry" or "Q-Star," which is described as a potential AI advancement. There's also mention of this account being part of a "social experiment" related to the indistinguishability of AI from humans online. Additionally, there appears to be merchandise, such as a t-shirt, associated with the "@iruletheworldmo" username.

    It's worth noting that some online discussions question the credibility of the account's claims and whether it's a "grifter" or simply engaged in "speculation farming" or "clout chasing.""

    ReplyDelete