For my final project, I built off of a project I began in high school. Since my senior year, I have been working to build a sleep tech stuffed animal called the Sleepy Stuffy. The animal tracks how many hours and minutes the user sleeps throughout the night as well as their movements. Additionally, I would like it to be able to play audio just by pressing the stuffed animal’s paws. While I was able to get basic sleep and movement tracking implemented in my last prototype using Microbits, I was not able to implement the audio functionality. Thus, in this prototype I focused on the audio aspect.
While at first I wanted to be able to upload the audio files to the ESP32, upon further research this did not seem like the best approach due to the small amount of memory space that the microcontroller has. The goal of this project became to create a stuffed animal that plays music from a computer by squeezing its paws. I decided to use Processing. I used the Minim library and AudioPlayer to load and play the audio files. The user presses the front left paw to pause and play the songs. The user presses the front right paw to switch to the next song; the user can only switch songs while the music is currently playing. Force sensors are used to detect the squeezing of the paws and the ESP32 is used for collection of sensor data and serial communication.
I originally intended to use wireless communication for this project. I found it more desirable for the user to not have to connect the stuffed animal to their computer when playing the audio. This would give the user more flexibility regarding how they physically interacted with my project.
I tried two different ways to implement wireless communication. The first attempt I made was starting a web server on my ESP32 and then setting up a client in Processing to connect to that server. Sadly, this was not successful. The second attempt I made was to start the server using Processing and have the ESP32 connect to it as the client. While, in office hours with Professor Tseng we were able to get the code to work on her computer, I could not get it to work on my laptop. Processing was not producing the correct IP address for my laptop, thus the ESP32 could not connect to the correct server. I updated the OS of my computer and redownloaded Processing, but this still did not fix the problem.
Thus, I had to pivot and switched to using wired (serial) communication.
To see the code and the Fritzing diagram please check out the following Github repository:
https://github.com/alexbaril/sleepy-stuffy/tree/main
My process for completing this project can be broken down into two stages. The first stage consisted of breadboarding and coding. This tutorial from ESP32 I/O helped guide my materials selection, code, and wiring. I first got the right paw force sensor working. When the force sensor was pressed, the Processing code would switch the song. I have four video game songs that the user can listen to which I downloaded from the internet.
This is what my electronics looked like with only the right force sensor implemented.
I then added a second force sensor. This force sensor was for the left paw. Now the user needed to press this force sensor to play and pause the songs. I also updated the right force sensor code so that the song could only be switched when the music was currently playing. Thus if the user pressed the right paw while the audio is paused, nothing happens; when the user next presses the left paw to play the same song will be playing.
Having all my code and electronics functioning, I next added extension wires to the clincher connectors and then attached the clincher connectors to the force sensors. I used clincher connectors because in doing research on force sensors I found that soldering extension wires to them directly can cause them to break. I also added tape to the wires so that they would not get tangled.
I then began stage two: inserting the electronics into my enclosure. I already had the stuffed animal dog from my previous prototype. It has a zipper on the side which made inserting the half-sized breadboard into the stuffed animal easy. I just had to remove some of the stuffing to make space for the breadboard. However, the paws were a different story.