[1] LATENT SPACE

A Knowing Wrongness

"I undertook a deep reflexive analysis of my current and past graphic design work to understand how I know what I know"

I decided to watch the full video of Muggeridge's PHD Examination, which looks at his process and his concept of "knowingly wrong" approach in the field of graphic design. One thing that I enjoyed about the video and how he presented his works was how he structured his processes or makings into five key strands of activities of practice:

1. Type Design Without Designing Type
2. Typography
3. A dialogue with the press
4. Sampling and the repository
5. The workmanship of risk as design

These strands that he addressed in the video was his drives that contribute to the decisions in his design practices. Each strand is presented with intricate explanations of his experimentations, each decisions that contributed to the works were reflected and articulated for reasons. He also synthesized his works with his reflections from the findings with repository.

I enjoyed the "interuptive" techniques that he developed through in his practices such as, manipulating letter forms or using everyday softwares that he use to design in an unconventional way. For instance, in " Type Design Without Designing Type", he created a letter form by altering the parts of the letter form from an existing font, and creating a letter through a software that draws a letter in many different way using a path that is set in parameters.

He mentioned something like, “I am not a type designer so I’ve developed a method of type designing without designing type.”

I thought that his process was interesting and much like innovative I would say. His ways of using materials and techniques to create new forms of graphics, letters, prints, and etc, defines his research very well in exploring his concept of "knowingly wrong" to create new designs in the graphic design field.

Reflections

In reflecting this video to myself, I thought that maybe I could learned better on how to reflect my own works better, the experiments I did and synthesize them with my findings. Maybe categorize them as a narrative could be an option, like how Muggeridge categorize his processes into 5 key strands of activities. Eventually, I needed to tie all of my experiments back to my research, so how do I synthesize them together so that I can define my research and dissertation better?

I found it interesting how he makes use of his limited knowdledge and work on this limitation to produce something new. Reflecting this back to myself, what are the other ways to understand machine memory without these knowledge and backgrounds? I am not a scientist nor an engineer, how do I make use of my limitations to create something interesting and also communicate my thoughts to the audience about human and machine memory as a designer?

Unreliable Memories

I went back to my readings from my research proposal outline, "Unreliable Memories" by Lev Manovich. In this work, Manovich presents works created using artificial intelligence (AI) tools, alongside his original drawings that are never before exhibited. These works generated by AI model are exhibited and titled as "Unreliable Memories" because Manovich wants to establish kind of a pact of trust with the visitors that assures the pictures generated by what he called a "memory machine" (AI) to be credible.

He wants to draw a question between whats fiction and whats real through questioning the works that he has generated with the machine: "what are these memories, which are presented here as unreliable? To whom do they belong? Why are they unreliable? And what can we do with them?"

Through a series of narratives presented that are all relevant to a historical or cultural memory, the book is a catalog of the exhibition that shows various of works that highlight contiguity contiguity of themes and references throughout the years.

In one of the chapter of the book, I took interest of Somaini's explanation of the latent space which is part of the process behind creating this work and in general working with a generative AI model. One thing I found interesting about it is how he described the latent space as "the site of alternative, possible pasts, of counterfactual histories and hypothetical futures."

"Latent spaces, in other words, are spaces of possibilities but also impossibilities." (Somaini 93)

The latent space is a multidimensional, non-visualizable space where vast amounts of images and texts have been compressed, encoded, and embedded in relation to one another. Each model has its own, so there is not just one latent space. I found it interesting how Manovich explores their own memory potential by generating out of the latent space images whose temporal status is "floating and uncertain." (Somaini 95). In "Unreliable Memories", the latent space is used as a process and tool to generate images that are connected to the past and present.

I thought about it if the latent space is like a hidden space where the model generates relationship between data, a space where everything makes sense to the machine. Does this hidden space of possibilities could perhaps show what is machine memory? Could machine memory perhaps defines as the unpredictable versions of possible pasts?

The Latent Space

I came upon Jake's project "Latent Space" in which he visualizes how this multidimensional look like. In this project, Jake referenced the High-dimensional Space concepts and Generative Adversarial Network frameworks within Machine Learning through visualizing the concept of latent space as an exploration of an ever-changing digital space.

Using a three-dimensional space in Virtual Reality, Jake explores the relationship between data points and the users. He uses geometries as a visual representation of the data-points in the a high-dimensional space. By transforming data into geometrical forms, the project uses the T-SNE algorithm to cluster these shapes based on their properties and metadata. This clustering helps reveal hidden patterns, relationships, and structures within the dataset.

Inspired by how Jake visualize this abstract high-dimensional space concept into a virtual object, I wanted to experiment on visualizing how the latent space looks like in the context of machine memory. "Each word or pixel is considered as a dimension and sorted and sampled to create the fundamentals of relationship" for the machines to make sense. I wanted to visualize how relationships are formed by the machine throught data, and how it could differ to how humans generate meanings as well.

Visualizing The Latent Space

In summary, the latent space is a multidimensional space where data are compressed for the machine to be able to make sense of it. The picture below here is a good representation of how latent space works. Data are stored and classified through relationships that have been learned by the model.

Interpolation

In lots of experimentation I have seen, people use a technique called interpolation to visualize the structure and relationships between the latent space. Like these face experimentation that I often see on the internet and generator tools, where it blends 2 different images, and it could generate a new face by blending existing data points. This interpolation technique is often used to show how efficient the model has learned in generating new data points to create realistic outputs.

For my next task, I am planning to incorporate ths technique as a way to understand how the latent space works, and how this could somehow visualize how machines generate meanings through creating relationships between different data points.

Experiments

Using Texts as an Analogy

I ended up with a decision to visualize how the latent space works using texts as the analogy through the interpolation technique. My objective is to differentiate how machines generates meaning differently from how humans generate meanings. I thought of the simplest approach to interpolate words which is seeing this individual words as data point. Much like how how AI finds patterns between data points in the latent space through word embeddings, I used ASCII characters to interpolate between words based on numerical values.

1st Approach

To explain it simply, you can click on the diagram below to see an example of how it works. To mimic the process of interpolation, the ASCII characters for each individual letters are used to blend between words. As you can see in the diagram below, the word "cat" and "dog" are blended together. During this 'blending' process, it creates words like "cbt", "cot", and "cog" to further transform the word from "cat" to "dog". This blending process is my way of trying to visualize how interpolation works.

In each word, each letter represent a numerical value, blending this value together required a mathematical calculation. To simply explain it, the letter "c" in "cat" has 99 values while the letter "d" in "dog" has 100 values. This shows that it takes only +1 value ahead. So, there is not much of a transformation to change during the blending process. Indeed the words aren't the full probabilities as this is controlled by a parameter that you can adjust to generate how much interpolated words you want.

Flaws

The flaw in this approach is that it doesn't really result to any meaningful (readable) words. It's just bunch of gibbersih at the end as it interpolates based on numerical values. Although it was my intention, but it is not what I wanted to achieved. I wanted to see if I could create more meaningful words or sentences using the interpolation technique that I am trying to visualize.

Website

Stitching Words

So I refine the experiments and created a more well-designed website at the end. I wanted others to be able to use this website as well and generate a poem through this interpolation experiments that I created. I thought of better approach to create a more meaningful words or sentences at the end, which is using characters matching to explore how AI find patterns between points in the latent space.

Not only that I want to simplify the technical explanations but also I want to simplify the experience so that users can understand what I am trying to do at least. Even if they don't understand the technical part, they can still enjoy the poems that they generated and keep them as well.

I thought that this website could be somewhat like a diary or a space that users can put their thoughts into as well, write some things and generate a poem out of it. It's just my way for it to look aesthetically pleasing as well for people to play with.

2nd Approach

Although indeed that the first approach that I used which is using ASCII numerical values to interpolate the words is a closer analogy, I do thinking that this appraoch produce more meaningful results for me as the generated poems are readable for the users as well. I would say that this approach of combining words through pattern matching is more simplified and straightforward as it identify words based on their shared characters. You can click on the diagram below to see an example of how it works.

Flaws

The only flaw that I thought it has is that users need to put alot of words to avoid repititions. Because I set a rule in the code that structures the poem to be in 10 rows and 5 words each, generating in total of 50 words, the code tries to find the best possible pairs based on the input words. If no pairs have common characters, it may still pair words that are available, especially if it's running out of options to meet the required number of lines and words.

Moving Forward...

I actually quite like how the results from 'stitching' words look like. I also found the joy in creating this website for people to interact as well (I consider creating these 'poems' as interactions and experience). I needed to find a way to articulate my thoughts and my opinion about how to differentiate human and machine memory, and I thought that by visualizing a concept that is so abstract into an experience is a good way to do so.

In moving forward, I thought about doing another visualization experiment in which I used images to conceptualize how interpolation shifts images through pixels. This can also be developed into a narrative of how machine can stitch one frame to another frame using our digital images or videos that we think of as memory. I thought maybe I could use a Stable Diffusion model to generate a middle point (or what others called as intermediate point) between 2 different or multiple images in sequence.

I found out that Hugging Space could be a platform and source I can use to employ a pre-trained model and just change the defined parameters from there. This might be more challenging technical process for me as I have never touched Stable Diffusion before, and I don't know how python code works. However, I am willing to still try and explore this concept further using this approach.