Skip to content

**Digital Prototyping For Design: Prototyping for interaction and extended bodies

Presentation

In this seminars we had the opportunity to test different open source tools such as Wekinator (real-time machine learning), code tools such as Processing or real-time communication protocols as OSC. Also some commercial tools as Max8. The objective was to provide ourselves with tools to create interactions from body data.

Face tracker

We were also able to build a wearable textile sensor from scratch. With the help of a development board with connectivity, in this case Barduino, we saw how the data taken by the sensor could be sent in real time over the Internet through a Wi-Fi network (in this case internal) for processing and graphical and audio representation. or for training your own artificial intelligence model.

Flow

Exercices

With conductive and semiconductive fabrics we build a sensor for the finger. After checking its resistivity, we built a simple circuit to be able to send the data through the ESP32 and an internal Wi-Fi, to the computer where we played with different representation and machine learning training possibilities with Wekinator i Processing.

Interactions system

The code 
```
/*---------------------------------------------------------------------------------------------

Open Sound Control (OSC) library for the ESP8266/ESP32

Example for sending messages from the ESP8266/ESP32 to a remote computer
The example is sending "hello, osc." to the address "/test".

This example code is in the public domain.

--------------------------------------------------------------------------------------------- */

/*
OSC Library documentation: https://github.com/CNMAT/OSC/tree/master
This example has been adapted by Citlali Hernández, for MDEF Class 2024, Barcelona.
*/

#if defined(ESP8266)
#include <ESP8266WiFi.h>
#else
#include <WiFi.h>
#endif
#include <WiFiUdp.h>
#include <OSCMessage.h>

const int sensorPin = 1;
float sensorValue;
float mapValue;

char ssid[] = "whiteRabbit";              // your network SSID (name)
char pass[] = "goodlife";  // your network password

WiFiUDP Udp;  // A UDP instance to let us send and receive packets over UDP
//192.168.1.137
//const IPAddress outIp(10,40,10,105);        // remote IP of your computer
const IPAddress outIp(192, 168, 8, 182);
const unsigned int outPort = 9999;    // remote port to receive OSC ()
const unsigned int localPort = 8888;  // local port to listen for OSC packets (actually not used for sending)

void setup() {
Serial.begin(115200);

// Connect to WiFi network
Serial.println();
Serial.println();
Serial.print("Connecting to ");
Serial.println(ssid);
WiFi.begin(ssid, pass);

while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
}
Serial.println("");

Serial.println("WiFi connected");
Serial.println("IP address: ");
Serial.println(WiFi.localIP());

Serial.println("Starting UDP");
Udp.begin(localPort);
Serial.print("Local port: ");
#ifdef ESP32
Serial.println(localPort);
#else
Serial.println(Udp.localPort());
#endif
}

void loop() {
presSensor();
sendingOSC();
}

void sendingOSC() {
OSCMessage msg("/testalbant");  // The ID of the OSC Message
// msg.add("hello, osc!"); // Test hello message
msg.add(sensorValue); // Send sensor value
Udp.beginPacket(outIp, outPort);
msg.send(Udp);
Udp.endPacket();
msg.empty();
delay(150);  //500 works ok
}

void presSensor() {
sensorValue = analogRead(sensorPin);
mapValue = (map(sensorValue, 100, 4000, 0, 1000) / 1000.0);
mapValue = constrain(mapValue, 0, 1);
Serial.println(mapValue);
}
//
```

In the module 5 we used the same wearable soft sensor and the circuit of module 4 and we tryed to convert the inputs into a more or less artistic output.

In this case we play with the pressure on the sensor as a symptom of stress and we try to achieve an inversely proportional output. For example, for high values ​​of the sensor, that is, high stress levels, a decrease in the speed and volume of the music was caused and for low values, that is, low pressure or low stress, it caused a sound output at speeds. and higher volumes to activate.

Note

No code or fabrication files are included because the final result wasn’t very achieved.

Reflections

  • Short and productive seminars, in my opinion, very interesting and that provides us with more resources for our future projects.

  • Building the diy sensor ourselves increased our understanding of how it works and enables us for future more complex soft sensor designs.

  • I really liked the possibility of working on these topics exclusively with open source software. Although with these softwares you can build simple things relatively quickly, there are so many options that I need more practice time to get comfortable.

  • Despite Max 8 is commercial, it is also very interesting and powerful as it offers a wide range of capabilities.

  • I really want to continue experimenting with topics of interaction, processing and representation. I see that they open up a lot of very interesting possibilities from an artistic point of view but also from a product design point of view. I feel that these modules have been just a small bite an attractive and interesting topic, interaction.

Tools and references