Monday, December 28, 2015

From Arduino to MATLAB


EMG signals acquired by the SpikerShield and Arduino Uno can be send to the Macbook Air through serial port (USB). The current problem is how can we read and even display the data from MATLAB. After trying a few possible solutions from the internet. There are two working solutions you can base on for your next-step development:
  1. Serial Monitor (Debugger): It can be downloaded from http://www.mathworks.com/matlabcentral/fileexchange/45839-serial-monitor--debugger-. It is an app that you can install, and then run it by clicking the APPS tab -> MY APPS -> Serial_Monitor. It will show the serial data in a pop-up window. You can save the data to the "SerialData" variable for further processing.
  2. Real Time Serial Data Logger: It's a piece of code that you can run in the MATLAB. It can be copied from https://billwaa.wordpress.com/2013/07/10/matlab-real-time-serial-data-logger/. You can just copy and paste it into the MATLAB, and a pop-up Serial Data Log window will appear to show the plotted serial data.
The followings are the screen shots and videos of these two solutions in action.

Serial Monitor (Debugger)
 

Serial Data Logger
 

 

Finger Gestures vs. EMG Electrode Placement

Congrates to be able to gather raw EMG data from the SpikerShield! While you are picking up MATLAB, we also need to figure out the EMG electrode placement as soon as possible such that we can start acquiring meaningful data. There are three documents in our the Project Resource page, which potentially provide us the info we need.
  1. Finger Motion Decoding Using EMG Signals Corresponding Various Arm Postures (2010), Kyung‐Jin You, Ki‐Won Rhee and Hyun‐Chool Shi
  2.  Relating Forearm Muscle Electrical Activity To Finger Forces (2014), Jennifer Keating. [page 22-23, 62-84]
  3. Forearm Muscle Labels.
The simple questions we want to start with are: "In order to detect the movement (bending and stretching the finger) of individual finger,
  1. what are the most important muscle(s) related to the movement (bending and stretching) of individual fingers?
  2. How many electrodes (or electrode pairs) do we need? And,
  3. where should we place those electrodes (or electrode pairs)?"
Can your team look into them, extract useful info, and summarize the results into two tables (bending and stretching)? The tables should at least consist of the following columns: Finger, Muscles (names & symbols), Category (deep or superficial), Type (extension or flexion), Location (anterior or posterior), Electrode Placement (labels matching the diagrams), and Comments.
The exact electrode placement (position and orientation) should be labeled and indicated in the following tow diagrams (anterior view & posterior view) and matching the electrode placement in the tables.
Forearm muscles of anterior compartment: superficial, middle, and deep

Forearm muscles of posterior compartment: superficial, middle, and deep
You can contact me through email/blog whenever you have any question. Don't forget to post your results onto your blog and we will discuss our next step (data acquisition) based on the results after the break.

Tuesday, December 22, 2015

Progress Report: 11/30/15

We were having a lot of trouble getting MATLAB to work on the school computer, but we managed to download it onto one of our computers. In addition, we have figured out how to gather data on Arduino. We then plotted it using the spreadsheet application on Google Drive. The picture below is just a proof of concept to show that we can gather data. The values are a sign of strength of muscle movement, and each data point is collected every 100ms. The graph is for a very long time, being about 16 and a half seconds.

Some problems we faced this week were getting Matlab to connect to Arduino on the computer to which we downloaded Matlab. For some reason it didn't work. Another problem we encountered was understanding Matlab, as looks like we will have to gain an understanding of coding and algorithms.

The plan for us is to do isolated tests of each finger for a much shorter period of time. We will repeat a specific motion, targeting a specific muscle, and through this, hopefully we will be able to identify patterns through this approach. We also hope to gain a better understanding of how to use Matlab through tutorials. We will also try to learn some coding in Arduino, although it looks like this will not be as necessary as previously thought. We have made some progress this week, we feel, and are excited to continue.

Sunday, December 6, 2015

STEM Patent Research: 11/30/15


An excellent list of patents, and nice summary and analysis. Based on your description in "PLACES WE CAN INNOVATE", you might want to look further into a field called Functional electrical stimulation (FES) or NeuroMuscular Electrical Stimulation (NMES).

Tuesday, December 1, 2015

STEM Patent Research: 11/30/15

PROCESSING EMG:
Assessment of EMG signal acquired from parts of the body where there is a lot of activation of muscles involved in motor skills (ability to do complex muscle and nerve act to produce movement). It can compare the pattern observed to known patterns obtained during a controlled activity.

The invention is a circuit that can take EMG signals and transfer them via USB to a computer.

A way and apparatus that can be used to produce a model EMG signal from a measured one by a series of filters. The EMG signal that results is separate from the EMG and EKG signal that is measured.

EMG signals are detected from several different locations on the hand. These locations of electrodes are where precise movements occur. The EMG signals are registered and processed to be used for biometric assessment.

Acquiring uterine EMG signals. Signal processing device transmits the signal to a relaying device which sends it out to a call center for a doctor to look at. Can potentially help us figure out how to transmit the signal to another arm etc. At least one pair of electrodes used.


ANALYZING EMG:
A machine learning model that has the user do specific gestures so that it learns the signals from that gesture. The machine learning model can then identify specific gestures from specific fingers of the user using the information that it has learned.

A sensing device that allows patients to control an object by using motor unit action potential. It uses an emg sensor which they place on a specific area of the patient. It also configures a signal that represents the motor unit action potential. Then it uses a personal area network transmitting device  that corresponds to a specific signal. Personal area network is used to convert it to an electrical signal which then goes through a processor where the electrical signal is received to generate at least one control signal.


GADGETS TO HELP HUMANS:
Hybrid prosthetic arm controlled by surface EMG (determines electrical activity of the muscle) (sEMG places electrodes on the skin overlaying the muscle and not INTO the skin) and mechanical control from elbow and shoulder of amputee. Device contains mechanical fingers driven by mechanical motors controlled by microcontrollers (a small computer (SoC) on a single integrated circuit containing a processor core, memory, and programmable input/output peripherals.) The instruction sent to the arm is from sEMG signals. It can convert sEMG data into instructions, and movement comes from motion of the shoulder, rotation of the elbow, and sEMG signal.

A lower prosthetic limb that uses EMG signals to determine the user’s specific gait phase. The device recognizes which EMG signals correspond with which type of locomotion to better replicate the user’s motion.

An athletic glove that records EMG signals from electrodes on the inside of the glove. The data is gathered and processed, and then output on an external module of the glove. This data provides feedback on grip to the user.

TENS bandage that uses electrical stimulation to ease and block pain within wounds. Also helps with healing of a wound.

A suit that has electrodes placed on it. Wires connect the electrodes together with a form of stimulation device. The wearer or user can apply electrical stimulation to certain muscle groups or parts of the body, inhibiting a response.


STIMULATION:
Stimulating nerves using electrodes with an electrically insulating back layer. Increases electrical current through surrounding tissues. Increases impedance of electrical path through blood in lumen of blood vessel.

A surface probe with a conductive tip that can apply a local high voltage. The electricity applied causes a stimulation of the targeted muscle fibers, eliciting a forceful movement from those muscles. The force and number of twitches can be altered.

A device that can send electrical stimulation to the extremities specifically. It braces the hands in place on a flat surface and then provides the shock with a TENS unit.

An automated system that is able to deliver electrical stimulation to the user. It can then also detect the muscle response from the electrical stimulation. Being able to detect muscle response and deliver electrical shocks, the system is able to automatically diagnose one characteristic of a muscle from the response and adjust the electrical stimulation accordingly.

A TENS device that is able to apply electrical stimulation to muscles and also detected changes in the skin. There are two modes, stimulation mode and re-calibration mode. Via a plurality of electrodes, the device can apply an adjusted electrical current to the user based on skin impedance.

A device that contains a pulse generator to mimic MUAPs that are naturally generated. The invention can synthesize the basic wave form, not depending on muscle contraction to achieve any form of stimulation.

A TENS unit can connect to a smartphone via bluetooth. It can receive data as far which signals to transmit, and it can also send data to the smartphone in response to biofeedback from the user’s body. The controller can change pulse width, frequency, and/or intensity.


PLACES WE CAN INNOVATE:
The devices of EMG and TENS remain generally separate in the patents that we found. Both technologies seem to be known, it is just a matter of finding a method of combining these technologies. One place we can innovate is in a device that integrates both technologies together.

Another area that we can innovate is in TENS. Some of the patents that we found were able to adjust the signal, like in frequency or intensity. We could find a way to develop a TENS unit that can accept a changing signal and shock the user with that type of signal.


Friday, November 27, 2015

Virtual Reality Application

Hand-to-hand communication project covers two major technologies: decoding the EMG signals and encoding the TENS signals. Each technology has it's wide applications. Here shows a recent application in virtual reality (VR). Please see the YouTube video or read the article.

Tuesday, November 24, 2015

Progress Report: 11/22/15

A problem we were having was with getting the Arduino application on the computer to show the serial monitor. After a bit of trial and error, we got the monitor to load, so now we can have numerical data to use in deciphering different EMG signals. Because we have data in the form of numbers for the strength of the EMG signal, and the intervals in which the data comes, we can visualize the data, putting it into excel and creating graphs. We also found on the Backyard Brains website a pack of 100 electrodes, since we are running out of the electrodes that came with the Muscle SpikerShield Bundle.

We will need to learn how to filter out extra noise gathered from the SpikerShield. To do this we have to learn stuff about Arduino coding. We still are having trouble in gathering data, because the lights that light up on the SpikerShield seem to be inconsistent, and the electrodes also eventually lose functionality.

Our plan for the next week is to look at examples of Arduino code and gain an understanding of how we can approach filtering out noise from the signals that we gather. We also are going to work on getting real data from the SpikerShield. We are also going to look at the other patents on the subject for the patent project.

Tuesday, November 17, 2015

Progress Report: 11/15/15

We analyzed out SpikerShield box. We ran a test on the SpikerShield box and tried to figure how different moments affected the light popping up on the box (The light indicated muscle activity). We looked at the Arduino code and adjusted it a little bit.

Here you can see the lights are lit up due to his movement.

We had trouble accessing the monitor that showed our data (muscle movement) on Arduino so we had to troubleshoot and also MatLab is still not working on the computer. We might have to find a different software or spend more time on just MatLab. However, we decided to wait to work with MatLab since we need to focus on acquiring a signal.

The plan is to figure out how to display the monitor that acquires our data through Arduino. We also want to find more electrodes that we can use as our electrodes are not reusable and don't last very long.

Tuesday, November 10, 2015

Progress Report: 11/8/15


As far as accomplishments, we finally got the Backyard Brains Muscle SpikerShield bundle. We have been researching and examining the equipment, looking at the materials online regarding the SpikerShield. There are different experiments that they have to try out and we have been reading through them. We have also worked on installing Matlab, and have found a download that will help us to install the software and get started with Matlab. We also downloaded the software for the SpikeRecorder software from Backyard Brains, and tried to get it to function with the SpikerShield.

Some problems we faced were in installing Matlab. Due to the way the software uses the internet and the various firewalls and blocked websites that exist in the school, it is very hard to install the software. We also encountered problems with the Backyard Brains app. We plugged the SpikerShield into the computer and tried to find a way to get the app to switch from the computer microphone to the SpikerShield itself. We have been unsuccessful in it so far.

We plan to find a work around for Matlab this week, and also perform the first actual test with the SpikerShield at home on Wednesday. Because we want to minimize noise and perform the test in a controlled environment.

Sunday, November 1, 2015

Progress Report: 11/1/15

We were able to accomplish one of the most important parts of our project this week. We were able to do all of the final product research on finding the right equipment (Backyard brains) for what we are trying to accomplish with our project. We placed the order this week and we should receive the product sometime next week. We also looked into whether or not we could use the Uno board - Leonardo instead of the Uno and it turns out that we could so we are currently working on that.

We tried to figure out Matlab by downloading the actual program onto our computer but we were having trouble setting up the program on the computer and Matlab seems to think it is a proxy error so we have to look into it and/or contact MatLab

Our plan for next week includes trouble shooting MatLab, trying out the equipment assuming it is delivered and also looking more into the program for the Arduino board.

Sunday, October 25, 2015

Progress Report: 10/25/15

We have decided that the most feasible product is something simple that has integrated software that if need be we can upgrade in the future. Looking at the brainwave groups, which likewise used simpler tools as the starting point to tackle a more advanced problem, we decided this would be the best approach. We have chosen the Muscle SpikerShield from Backyard Brains, since it is relatively cheap and can be upgraded to get up to 5 channels. We also set up an account for Matlab, a data software. 

We still need to place the order with approval from Mr Lin, and this aspect only addresses parts 1 and 2 of the project. We haven't found a TENS unit, but at the moment we are looking to just start. 

From here we will research the equipment that we have chosen, reading manuals and things before it comes. We might also look into how the Arduino works, since that is a part of our project. We should have it by next week at the latest, and once we get it we will begin to run tests on how it works. 






Tuesday, September 29, 2015

RE: Team 7 Project Gantt Chart 9/24/15-11/8/15

Feedback:
  1. Go ahead to start your project accordingly!
  2. It takes too long to research equipment.
  3. You can group ordering and shipping together.
  4. The granularity of "Research Muscles and Nerves" needs to be reduced by dividing them into sub-tasks.
  5. There are many overlapping tasks. You need to assign resource in order to see whether it is feasible. 

Sunday, September 20, 2015

RE: Initial Planning & Coordination


Project Description & Merits
Describe the project in your own words and list the possible contribution your project can have to advance the field of STEM or solve societal problems.
 Description
Mimicking muscular electrical signals by recording the electrical impulse from the action of one person and translating that to another person’s arm.
 Contribution
  • Perform surgery if doctor is not there
  • Learning to play an instrument
  • Recording electrical signal from muscle
 
  • What's the purpose/application of recording electrical signals from muscle?
  • Brainstorm more about the potential applications of this technology?

Group/Team Communication
Who will be involved in your project (team) and relevant projects (group)? How to communicate effectively with each other? What are the tools for project collaboration?
 Group members
  • Julian Wexer
  • Lithu Muralee
 How to Communicate Effectively
  • Text and Message one another
  • Clearly outline goals
 Tools for Collaboration
  • Splitting up work
  •  Tools such as Google doc, Git Hub, etc.
Prior Work/Resource Inventory
Go to the Project Resource website to review the current resource and prior work. Update the page.
  • Need more study about the prior work in this fields, and post them in the Resource page.

Technology Analysis
Identify the scientific and technical knowledge/skills involved in your project.

  • Teamwork (e.g., Git Hub)
  • Problem solving
  • Coding (which language? which IDE?)
  • Biology (e.g., structure and stimulation mechanism of the nerve cells)
  • Muscle structure (and neural structure, especially the upper arm and hand)
  • Electric signals (especially neural signals)
  • EMG device and its operation principles
  • TENS device and its operation principles 
  • Needs to be more specific.
Competence
Identify skill sets, technical competence in your group, and list the missing ones which need to be acquired.
  • Teamwork
  • Problem solving
  • Basic knowledge of human anatomy

Safety
Identify any safety issues regarding the build, use, store, and dispose of materials and tools
  • Wrong electrical signal
  • Electrical wires
  • Glitch
  • Needs to pay special attention to the safety restrictions of Transcutaneous Electrical Neural Stimulation (TENS) signals, and the process of apply the signals to human.
Equipment, materials & budget
List the key materials and equipment required for your project. Identify the key items needed to be purchased and their prices
  • Electrodes
  • EMG device
  • Gel
  • TENS device
 Links to Products

Schedule
What are the goals/milestones/plans of your project in the next week? What are the task assignments for each team member?
 Goals for next week

Get a lot of information on the devices we need to carry our research - Lithu and Julian
  •  Needs to be much more specific, including detailed tasks, schedule, resource, etc.


Sunday, September 13, 2015

Initial Planning & Coordination


Project Description & Merits
Describe the project in your own words and list the possible contribution your project can have to advance the field of STEM or solve societal problems.
 Description
Mimicking muscular electrical signals by recording the electrical impulse from the action of one person and translating that to another person’s arm.
 Contribution
  • Perform surgery if doctor is not there
  • Learning to play an instrument
  • Recording electrical signal from muscle

Group/Team Communication
Who will be involved in your project (team) and relevant projects (group)? How to communicate effectively with each other? What are the tools for project collaboration?
 Group members
  • Julian Wexer
  • Lithu Muralee
 How to Communicate Effectively
  • Text and Message one another
  • Clearly outline goals
 Tools for Collaboration
  • Splitting up work

Prior Work/Resource Inventory
Go to the Project Resource website to review the current resource and prior work. Update the page.

Technology Analysis
Identify the scientific and technical knowledge/skills involved in your project.

  • Teamwork
  • Problem solving
  • Coding
  • Biology
  • Muscle structure
  • Electric signals

Competence
Identify skill sets, technical competence in your group, and list the missing ones which need to be acquired.
  • Teamwork
  • Problem solving
  • Basic knowledge of human anatomy

Safety
Identify any safety issues regarding the build, use, store, and dispose of materials and tools
  • Wrong electrical signal
  • Electrical wires
  • Glitch

Equipment, materials & budget
List the key materials and equipment required for your project. Identify the key items needed to be purchased and their prices
  • Electrodes
  • EMG device
  • Gel
 Links to Products

Schedule
What are the goals/milestones/plans of your project in the next week? What are the task assignments for each team member?
 Goals for next week

Get a lot of information on the devices we need to carry our research - Lithu and Julian


Sunday, July 26, 2015

Research Task 2 (07/27/15 - 08/09/15)

Great to see you make some progresses! Please continue and finish up the left-over assignments from the previous task. Don't forget to conduct a thorough survey of the existing low-cost EMG machines. We are going to purchase one based on your study results.

EMG signal acquisition/processing and functional neuromuscular electrical stimulation are two brand new fields which we have never explored before. However, they are extremely active research fields in the past few years. Sometimes, by combining with robotics, they have started bearing fruits to profoundly impact people who have been amputated or paralyzed! The potential applications are even wider and waiting for you to explore!

* Whenever you come across good papers/websites/reports, don't forget to add them to the "Project Resource" page.

* Please take electronic notes while you are studying the materials, watching the videos, or browsing through the web.

* Make PowerPoint presentations based on your notes. We will start presentation at the beginning of the year.
   

Thursday, July 23, 2015

Notes on EMG so far for Research Task 1

Notes on EMG
Lithu Muralee and Julian Wexer

-EMG stands for Electromyography
          -Electromyography is the experimental technique concerning the development, recording, and analysis of myoelectric signals
                    -Myoelectric signals are signals formed by physiological variations in the state of muscle fiber membranes

-Neurological EMG vs Kinesiological EMG
          -Neurological is an artificial muscle response from external electrical stimulation
          -Kinesiological is activation of muscles within specific tasks
                    -Within postural tasks, functional movements, work conditions, and treatment/training regimes

-Kinesiological EMG is an evaluation tool for applied research, physiotherapy/rehabilitation, sports training, and interactions of the human body to industrial products and work conditions

-Benefits of EMG are that it allows you to look into the muscle, allows you to measure muscular performance, helps decision making before and after surgery, documents treatment and training  regimes, helps patients tain their muscles, allows analysis to improve sports activities, and detects muscle response in ergonomic studies

-The motor unit is the smallest functional unit to describe the neural control of the muscular contraction process

-Motor unit defined as cell body and dendrites of a motor neuron, the multiple branches of its axon, and the muscle fibers that the neurons are applied to

-Brain can excite muscle fibers

-Resting potential of not contracted muscle fiber membranes is about -80 mV

-Neuron induces Na+ ions to flow into the muscle fiber membrane in depolarization, which is followed by repolarization, which restores the exchange of Na+ and K+ ions

-The Na+ influx causes the action potential in the muscle to go from -80 mV to +30 mV, depolarization occurs, and levels return to normal, but the excitation leads to a release of calcium ions, which causes the muscles to contract

-EMG measures the action potentials

-Electrodes are placed on the skin over a muscle and are spaced apart and depending on the space between them, the electrodes form a potential difference between them

-Motor unit action potentials or MUAPS are summed up to get a superposed signal

-Two most important mechanisms influencing  the magnitude and density of the observed signal are the recruitment of MUAPs and their firing frequency, which are the main control strategies to adjust the contraction process and modulate the force output of the involved muscle

-A raw EMG signal is an unfiltered and unprocessed signal detecting the superposed MUAPs.

-When a muscles is relaxed there is a baseline in the graph of raw EMG, but when the muscle is contracted, noise can be seen, with the averaged baseline noise not being higher than 3-5 microvolts, and with the target being 1-2

-Raw EMG spikes are of random shape, and the raw recording burst cannot be precisely reproduced in exact shape

-Tissue characteristics can influence the EMG signal, as electrical conductivity varies with tissue type

-The EMG detected can be from other muscles, which can influence the EMG signal, but this cross talk typically doesn’t exceed 10%-15%

-If the change of distance between the electrodes changes, the EMG signal is affected


-Outside electrical noise can impact the EMG signal reading

Sunday, June 21, 2015

Research Task 1 (06/22/15 - 07/03/15)


EMG
  1. Browse through the internet and look for the low-cost (< $1,000) EMG product information. Your search should include commercially available EMG machines, lab-use EMG boards, and DIY EMG kits. Organize the information into spreadsheet table including columns such as no. of channels, signal amplification, signal delay, frequency range, physical dimension, connection, prices, etc.
EMG Signal
  1. ABC of EMG (2005), Peter Konrad. [signal origin and acquisition, 25 pages] A easy introduction of EMG signal and acquisition.
  2. Fundamental Concepts in EMG Signal Acquisition, Delsys. [31 pages] This tutorial give us an in-depth introduction about EMG signal acquisition. It will help you understand the theory behind the scene.
  3. Relating Forearm Muscle Electrical Activity To Finger Forces (2014), Jennifer Keating. [section 1-3, 40 pages] This thesis covers the first half of our project by detecting the electrical activity from the forearm. It provides an overview of how our project will be structured, conducted, and what are the areas we need to have further study.
Please take electronic notes while you are studying the materials, watching the videos, or browsing through the web. Each team will present their learning later in the summer meeting.