Difficulty in Detecting Swallowing

Today we have been trying to use different sensors to detect whether a person is swallowing. When we compare the data from swallowing to resting or talking, we can never see any significant difference.

Professor Mangharam sent us some studies a while ago where the researchers tried to classify swallowing. With significant time, they were able to do so. However, they did so solely by isolating swallowing; they did not consider when the user would be talking.

According to one study, "a limitation of the current study was the need to have a quiet environment. If recording included noise originating from talking, body movements, occasional intrinsic sounds (e.g., coughing), and background noise of different sources, a more complicated algorithm would be needed." Since our product is intended for the user to wear throughout the day, detecting swallowing through audio or electrical signals would not be feasible.

We will continue to test different avenues for detecting swallowing and report on their effectiveness.

EMG Circuit

Today I finished creating the hardware for the EMG circuit. I also sent in a PCB design that should arrive mid-next week. In the chance the PCB design does not work, this weekend, I will also prepare a circuit made on perf-board. I also began creating the actuation component of the project, in providing feedback to the user with vibration motors.

We decided to purchase a fanny pack to store all of the equipment, since we have three bulky batteries which would not be feasible to store in a necklace. Right now we need to figure out how well the user would be able to perceive any haptic feedback from vibration motors. Most likely, we  place the vibration motors on the outside of the fanny pack but resting against the wearer's t-shirt.

MBED

Last night we were able to get the classification working, with some bugs, on the mbed. In order to do this we needed to translate all of our python functions in to C++ functions. This was a little more difficult that I had expected because I have little knowledge of C++ and the iterator type hung me up for a little while. In the end I was able to figure this out and test our tree on the mbed It was able to differentiate between talking and chewing, but for some reason the transition from chewing to talking took the mbed longer than expected. The transition from talking to chewing was almost immediate.  I did not take any video of this part because the only response that we get right now is a blinking LED so it was not super exciting.  Here is a link to the code that is currently on our MBED.

Summary of Data Capture and Classification

Over the past few days I have worked on capturing data from our mbed and processing it through a python script and then using a tool in order to classify the data that we captured.

We started by using an accelerometer since we did not have our emg sensor yet.
Here is a matlab plot of the data we collected from the accelerometer. We were able to classify this data based only on the entropy of the signal.

After we got our emg sensor we began capturing data. We had to figure out where to put the electrodes. We decided that the side of the face would be the best location.

Here is an example of 5 seconds of talking followed by 5 seconds of chewing.

Once we found that we could reliably capture the EMG data. We then captured a full minute of resting and talking data and a full minute of chewing data. We then put it through a machine learning program and produced the following tree.

 

Our next step is to recreate the tree on the mbed and classify the data.