QUANTIFYING MOVEMENT

What does a hug look like?

In Matter and Memory, Bergson emphasizes that memory is not just a passive retention of the past but an active process that influences perception and action in the present (Bergson 34-38).

Bergson’s theory suggests that the past is not something that remains fixed but is always present, actively shaping our perceptions, decisions, and experiences. I believe that every small action—whether it's as intimate as a hug, as engaging as reading, or as routine as brushing teeth—is not isolated in time. Instead, it draws upon the entirety of our past experiences, memories, and emotions.

Experiments

Quantifying Movement

In this experiment, I used an accelerometer sensor to collect data from movements and convert them into RGB pixel values to create images. The objective of this experiment is inspired by my previous experiment, which is to quantify something that cannot be quantified, which in this case, is our movements and interactions.

To do this, I used an accelerometer sensor with the arduino and p5js to convert the collected data into pixel data, where it can create images and videos of each movement I have captured. For us these movements can be counted in moments of seconds or minutes, or meaningful interaction like hugging someone, reading a favorite book, or even routines like brushing your teeth which is considered as part of the procedural memory; whereas for a computer, it can be counted as arrays of numerical data.

I recorded 5 simple movement that naturally come in our everyday movements, such as writing, reading, walking and eating, in a short period of time with the sensor on my hand. The captured movement are then converted into pixel values, presented as images and recording videos.

Problem

The only problem I had in recording these movements were where to put the sensor to get dynamic results that are comparable at the end. Initially I thought of putting the sensor attached to my necklace as I am hugging another person. However, this didn't seem to produce dynamic results as the accelometer measures the rotations, and my body doesn't rotate that much when I do something. So, at the end, I decided to put it on my wrist as it is the best location I could think of.

I also realized that if I were to put it on my necklace, thinking I could measure the pressure of the hugging, maybe I could've went with light sensor instead of the accelometer. However, the sensor would only works with hugging, not with the other movements I wanted to capture.

Another problem that occured was that because the accelometer was solder to the wires, its very fragile. I had to tape the wires to the breadboard so it doesn't disattached easily. This made the movements I captured somehow restrained because I kept worrying about whether the arduino getting disconnected during the data collection.

Additional Experiments

Visual Explorations

I also did some additional experiments where I tried fixing the X and Y position to create a whole image without blank spots to see what it would look like if I just compare their colors.

I found the results interesting as you can see how some movements are smoother than the others by looking at the color gradients. Actions such as hugging and flipping the book has a more smoother transitions compared to actions such as brushing and rubbing. Visually, I also think that it is quite aesthetic, showing my ways of manipulating the technique for visual explorations.

Combining all of the images

Additional Experiments

Creating Images through Movements

In this particular experiment, I did more visual explorations in which I used other images for its color values and matching it with the mapped values from the movement data. The idea behind it was to "sample my memory" using images of my childhood photos to reconstruct a new image that maybe could resemble these memories I had.

Although the results were quite abstract and more like for the sake of visual explorations, I found this particular experiment enjoyable as it felt like I was using the sensor as more like a brush tool or like painting when interacting with the sensor.

It wasn't what I had expected. In a bad way, I thought I could create a more visible articulation, however, the results are more abstract and more like a visual representation of the data. In a good way, it was still visually interesting and I enjoyed the process of creating these images.

Feedbacks & Insights

Although the experiments show great outcomes, Andreas told me that at the end, I should be able to find applications to my experiments. How are my experiments relevant to others? How can I synthesize my findings and make them applicable to others? This is something that I have been struggling since day one, and also on my research proposal outline.

And talking about reflections, I haven't done much of reflections so far on each of my experiments. So, it is a good note to be more reflective when doing the experiments. This is something I will keep in mind when doing the next experiments. Although Andreas mentioned that I don't need to have the answers right now, but it is good to have some ideas how to proceed.

During the RPO consultation this week, Andreas mentioned a reference I could look into as it reminded him of my experiments and the unconventional ways of how I explore machine memory. It is a video essay of Fraser Muggeridge' PHD examination, showing the thought processes behind his concept of "knowingly wrong approach"(unconvential approach) that contribute to his practice in graphic design.

By showing compilation of his works and practices he has done for years, he synthesized how his works explore this "knowingly wrong approach" through "interuptive" techniques and processes (manipulations) to create unexpected and innovative outcomes, contributing to the field of graphic design. For future reference, and to refine my approach and methodology further, I should look into this concept and reflect on how I can synthesized my works to be relevant to others.