What if your AI collaborator could see the same exact visuals with similar information density as you do? Wouldn't that light collaboration on fire?
Have you ever wondered why is it said that 'a picture is worth a thousand words'?
Visual Semiotics has some answers.
I am experimenting with Diagrams-as-Code as a high-synergy way of collaborating with AI to keep both humans and AI on track when innovating and maintaining software.
While it is not clear to me that Donella Meadows ever formally encountered the field of "Semiotics", her holy writ of systems theory directly echoes key concepts of visual semiotics and cites it as necessary to communicating and collaborating over systems (emphasis is mine):
"I have made liberal use of diagrams and time graphs in this book because there is a problem in discussing systems **only with words**. Words and sentences must, by necessity, come only one at a time in linear, logical order. Systems happen all at once. They are connected not just in one direction, but in many directions simultaneously. To discuss them properly, it is necessary somehow to use a language that shares some of the same properties as the phenomena under discussion. Pictures work for this language better than words, because you can see all the parts of a picture at once. I will **build up systems pictures gradually**, starting with very simple ones. I think you’ll find that you can understand this graphical language easily." - Donna Meadows, Thinking In Systems.
The field of Semiotics creates this same "sequential versus at-once" distinction as a foundational difference between visuals and text when we are attempting to assess meaning.
Diagrams-as-Code embody a interesting blend of both worlds as are progressively disclosed visuals that are so common in explaining complex systems. What is more interesting about Diagrams-as-Code is that each type of agent can leverage it's preferred method of ingestion of "meaning mappable" information. Machines (AI) can sequentially read a "Code language" and humans can consume visuals. Coding languages themselves are a form of distilled meaning - all the ambiguities of human language are purposely removed. Every vocabulary word has highly specified meaning and there are no extras. There are no idiomatic meanings - only direct meanings.
Using raster images as diagrams for AI perceptions is not impossible, but a much broader and unnecessarily ambiguous assignment of meaning to elements is necessary - after non-deterministic recognition of elements is complete. At the very least this would massively effect the cost of perceptual inference, the repetitious inferencing implied and the consistency of derived meaning across time and model versions.
This is the foundational reason that I feel Diagrams-as-Code may have a special place and be worth the extra effort.
Interestingly, I found authoring Diagrams-as-Code excruitiating before AI - especially since the layout engine (graphviz), does not reflect human concepts of gestalt in it's layout priorities. It can be very hard to force important relative layout of the system aspects being visualized. AI authoring has both made this much easier and makes diagrams-as-code authoring much more accessible to those who have not had the great pleasure of hand-coded diagrams-as-code.
I have my own pet theory that the long history of "knowledge encoding" has given a massive bias to those who can create and maintain internal visuals, across time, from source materials consumed as sequential words. Recording pictures over words takes more time and has historically been more difficult to reproduce (study the emergence of "moveable type" compared to "wood cuts" in printing technology).
All humans have visual perception as their foundation, fewer humans excel at creating visuals from words over long periods of time and many tracts of text.
These observations are a part of my research on whether visual specifications as diagrams-as-code might be an efficient and/or necessary collaborative mediator between humans and AI when building systems.
More at [Visual Specs Index](https://fastlinks.io/visualspecs/index) and [Visual Specs AI Plugins](https://fastlinks.io/visualspecs/aiplugins) that work with all AI agents and harnesses.
[Photo by ThisIsEngineering on Pexels](https://www.pexels.com/photo/female-engineer-designing-equipment-3861938/)
