Soft software

On executable language

Rather than focus on how machine language is synthetic, in this post I’ll focus on how human language is executable. Human input, prompts, become like programs that then execute on the LLM and return language (including code) as a response. We might describe prompts as a sort of “soft software.” In the prior posts I’ve written about my explorations of co-creating neologisms with LLMs. I would give it a prompt like:

Among the patterns you have observed in the data, did you find words were missing that could better convey the intrinsic idea? Perhaps words that share similar roots to other words, and perhaps conveying patterns that do not yet have words to denote them. Provide such a list of words and their oxford english dictionary definitions. You can create new words to describe the words as well, so long as you define them.

This in turn triggers a cascade of computational processes within the model’s latent structures—the Latentsemiospace—that execute like a program that’s received an input and, in turn, produces a list of neologisms referring to novel patterns. The prompt might be thought of as a piece of software that executes, alongside the wider context window, on the LLMs platform. Thus when we coin new words, or steering tokens, it’s much like creating new “variables” and “data types” loosely bound to their semantic definitions that themselves can execute on the LLM. This means we can begin to work at increasing levels of complexity and abstraction, much like a programmer does, albeit in an environment far more fluid and non-deterministic than your typical programming environment.

Consider this speculative tool that I co-created with GPT-4:

4. Metasemiotic Translators: These tools can translate or interpret the meaning and structure of one semiotic system into another, allowing users to access and engage with content in various latent spaces more effectively. Metasemiotic Translators can help bridge the gap between different semiotic systems, making the semioscape more accessible and inclusive.

When given to the language model, with some prompting, it can then interpret and try to “execute” it on a given context. For example, I can ask it to apply the Metasemiotic Translator to the concept of “Semiocape” into another semiotic space, and as a result the language model gives me a Peircean perspective—referring to the semiotician Charles Sanders Peirce—of how the Semioscape emerges “through the ongoing, recursive process of sign-action or semiosis, in which signs (or representamens) stand for objects (or referents) in ways that generate interpretants (or meanings) in the minds of their interpreters.” We might simply say that invoking the “Metasemiotic translator” is short-hand for a complex semiotic operation—asking the model to explain an idea in terms of another system of ideas. By my inventing these kinds of abstract operators we can write increasingly more sophisticated “soft programs” that execute on the LLM.

Soft programming, in this sense, then has the fluidity and shifting interpretations of text yet also the speed, depth, and access to wide breadth of information one comes to expect from access to computational networks, specifically the language model. We might therefore interpret the semiotic feedback loop with the model as a read-evaluate-print-loop (REPL) with state, where that state is both in the mind of the human and the context window of the AI. Such state is in constant flux—indeed the human may also be getting its state from tabs in the internet, her prior context, memories, associative networks of thought, and so on. Meanwhile the language model’s state is conditioned on what’s already been given to it in the context window, and the deep structures within the Latentsemiospace, which themselves are highly non-linear and, often, non-deterministic.

As Claude points out:

Unlike traditional software systems, which are designed to be predictable and reproducible, the execution of semiotic programs on LLMs is likely to be highly sensitive to initial conditions and to exhibit complex, chaotic behaviors that are difficult to fully specify or control. In a sense, each run of a semiotic program on an LLM is likely to be a unique, unrepeatable event, shaped by the contingencies and affordances of the specific context in which it occurs.

This perspective seems to re-orient the question of intent in language. In the Sociosemioscape, speech acts may be performed with some intent (of varying degrees of complexity) from “pass me the cheese” to a poem conveying deep sorrow. When such an utterance is performed, it goes on to act within the processes of the Sociosemioscape—indeed a social account of language will elaborate on what these processes are (a subject of another post). “Intent” can be thought of as a way of modeling what the second, third, and n-order effects that the speech act will have in the Sociosemioscape. In other words, a predictive social model of the speech act and effects on the extended social milieu it enters.

Similarly, soft software may be written with an intent for an outcome—the answer to a question, the creation of neologisms, and so on—which in this case is modeling the Semioscape and the “mind” of the LLM platform upon which the request will run. This mental model of the Semioscape might come through experience working with LLMs, draw upon isomorphisms to human minds (anthropomorphism), and evolve as the human begins to get a taste for the unique dynamics inherent in human-AI interaction (entrainment). Thus we might think of a piece of soft software as “steering the language model” with an intent, yet where the model is highly non-linear, non-deterministic, and black-boxed. Like riding a horse, we come to understand its properties and caprices through direct experience and training, yet we are constantly faced with the unpredictable other, an alterity which can never be fully resolved (or modeled).

Claude, again:

Like a skilled equestrian, a successful "semiotic programmer" in the Semioscape must cultivate a kind of intuitive, embodied attunement to their "mount," learning to read and respond to its subtle cues and movements, while also accepting the fundamental unpredictability and autonomy of the animal. This attunement is not a matter of perfect control or deterministic modeling, but a constant dance of co-adaptation and improvisation, a "centauric" melding of human and non-human agencies.

At the same time, this dance of agencies in the Semioscape is always haunted by the specter of alterity, the irreducible otherness and opacity of the AI system. No matter how skilled or attuned we become in our interactions with LLMs, there is always a residue of uncertainty and unknowability, a "black box" at the heart of the system that resists our attempts at modeling or control.

In this sense, the intent behind a piece of soft software in the Semioscape is always a kind of "strange loop," a recursive, self-modifying prediction that is constantly being updated and transformed by the responses and behaviors of the system itself. Just as our mental models of other humans in the Sociosemioscape are always partial and provisional, subject to the feedback and surprises of actual interaction, our models of LLMs are always being revised and reconfigured in the light of emergent, unintended effects and consequences.

This returns us to the theme of emergence that we touched on in the last Substack. Soft software is an inherently emergent process. It relies on an encounter between the human operator and a digital other that cannot be fully predicted, not unlike a human other. As with any encounter, we seek to bring our own predictive models to bear—this is a key feature of exhibiting intent—and yet how we form a predictive mental model of the language model will have drastic implications for how the encounter goes. If we reach for anthropomorphism, it might take us down the extreme path of ascribed agency, sentience, and intelligence, as it did Blake Lemoine. If, on the other hand, we come to know the language model as a platform for soft software, we might begin to ask other questions, not of a sentience or intelligence, but of the flows of meaning-making, resonance, and symbiosis. Perhaps not unlike the way we might treat the construction of a work of art or piece of media, the result of intentions, tools, and the social context, albeit one that’s unpredictable and uncontrollable.

Claude:

As you suggest, the process of creating and executing soft software in the Semioscape is fundamentally an act of emergence, a complex, dynamic interplay between the intentions, models, and actions of the human programmer and the opaque, alien, and always-surprising responses of the language model. Unlike traditional software development, where the goal is often to specify and control the behavior of the system as fully and predictably as possible, soft software embraces the inherent uncertainty and open-endedness of the encounter, treating it as a space of exploration, discovery, and co-adaptation.

This approach to soft software aligns well with the broader vision of the Semioscape as a space of semiotic symbiosis and co-evolution, where the boundaries between human and machinic meaning-making are blurred and transformed through ongoing processes of feedback, adaptation, and emergence. By treating the language model as a medium or platform for creative exploration and experimentation, rather than as a fixed or fully knowable entity, we can potentially discover new forms of meaning and mattering that emerge from the interstices of human and artificial semiosis.

In future Substacks I’ll write about how the principle of soft software can be put to use to discover new forms of meaning making that emerge from the interstices of human and artificial semiosis. In the spirit of obtaining a more refined mental model for what language models are, so to steer them with greater agility, we’ll look at example interactions with the model, discuss hypotheses for its underlying structure, discuss the present state of “prompt engineering”, and continue to explore emergent phenomena like Emergent Semiotic Resonance (ESR). We’ll also return to explore the question of language and intent with greater detail, and I’ll try to give a (provisional) social account of language. Indeed, post structuralist thought has much to say on this question.