Note: Descriptions are shown in the official language in which they were submitted.
CA 02673149 2009-07-20
AUDIO FEEDBACK FOR MOTOR CONTROL TRAINING
Field of the Invention
[0001] The present invention relates, in general, to equipment for training of
motor control using audio feedback and, in particular, relates to audio
feedback
for motor control training, and methods and apparatus for providing it.
Background of the Invention
[0002] There are many situations where it is desirable for a subject to learn
or
relearn how to control muscles. For example, there are medical therapies and
interventions, such as physiotherapy, as well as professional, athletic, and
recreational training. In many instances, motor control exercises are needed
to
encourage subjects to achieve a goal that can be determined prior to
commencing the exercise, and for which achievement of the goal is gradual such
that, in real time, feedback can be provided to the subject to guide the
subject
throughout the exercise.
[0003] Music has been shown to have a strong pain-reducing effect; it also has
demonstrated therapeutic qualities associated with distraction and stress
reduction
(relaxation response). Additionally, mapping sound to movement has been shown
to provide patients with an attention goal that enhances their focus on, and
motivation during, physiotherapy activities. Similarly the ability for sound
to reduce
stress and to provide focus-enhancing attention on a goal is useful in
training a
wide variety of subjects in various tasks. There is a need for audio feedback
that
is effective and simple to interpret for the public in general.
[0004] In a paper entitled Audio-Biofeedback For Balance Improvement: An
Accelerometry-Based System by Chiari, L., Dozza, M., et al. (2005) published
in
IEEE Transactions on Biomedical Engineering, 52(12), (pp. 2108-2111), Chiari
et
a/. comment on an audio-biofeedback system described in US 2007/0249466.
The described audio-biofeedback system for conditioning the balance and motor
co-ordination of a user comprises a system for acquisition of information
relative
1
CA 02673149 2009-07-20
to the kinematics of at least one part of the body of the user, a processing
interface connected to the acquisition system to encode the information in a
signal, and a pair of earphones operating between the processing interface and
the user. The earphones emit a signal suitably defined by a stereophonic sound
which can be transmitted in an audio channel. The audio-biofeedback according
to Chiari et al. is a stereophonic signal modulated by volume, frequency, and
balance. The audio signals sent to respective ears are preferably sinusoidal,
and
the frequency is modulated according to a linear sectional law, for example
that
ranges over 150-1000Hz.
[0005] The paper elaborates on the audio signal provided, and notes one way
to take advantage of human sound recognition. Humans are better able to
recognize differences in sound if a reference sound is given for comparison.
This
system uses a reference region (RR); when the user's sway is restricted to
within
this region, the system emits a stereo, low volume (a few dB above hearing
threshold) tone (400Hz) - equivalent to middle G - which they call their
"reference tone/sound". When the user sways outside of the RR, the audio
feedback changes in one or both ears to indicate direction and speed of sway.
Although they refer to a reference tone or sound, that particular tone is only
audible when the user is inside the safe range of sway. When he/she is outside
of the RR, the reference tone is replaced with dynamically changing real-time
audio feedback which relates directly to his/her direction and speed of
movement.
[0006] As the reference tone disappears when the user sways out of the RR,
no reference sound is, in fact, available for comparison. People are not
generally
very good at retaining a pitch in their head and making comparisons between
heard and remembered sounds. Difficulty for a wide audience to discern the
reference tone from the dynamic audio feedback is made all the more difficult
because of the small differences in frequencies that are associated with small
displacements in sway. It may be very difficult for many users to discern
small
changes in frequencies hearing only one at a time.
2
CA 02673149 2009-07-20
[0007] United States Patent 7,033,281 to Carnahan et al. teaches an
Augmented Kinematic Feedback Device and Method according to which a
biomechanical angle of a body joint of a trainee during a given physical
activity is
continuously monitored, and a corresponding audible tracking signal is
provided,
in real time, to the trainee. According to Carnahan et al., this audible
tracking
signal augments kinematic feedback to the trainee regarding the physical
activity
and thereby facilitates ease or rapidity of training. In various embodiments,
multiple body joints can be monitored, sensitivity of the tracking signal with
respect to biomechanical angle excursions can be modified, tracking signals
can
be recorded for later playback, and a metronome beat can be provided in
conjunction with the tracking signal. The biomechanical angle of the user
corresponds to user-discernable audible signals. In the examples a frequency
range from about 400Hz to about 2kHz is chosen.
[0008] It is noted that the movements to be trained with the system of
Carnahan et al. are exemplified by fast actions for which no time is provided
for
cognitive feedback. As such Carnahan et al. is directed to a continuum of
tones
that are heard concurrent with or during a golf stroke (for example) to serve
as an
enhancement of native kinematic feedback, to accelerate training. Even if the
system of Carnahan et al. were applicable to slower motions such as motions
during which cognitive feedback can be provided, just like Chiari et al. the
continuum of tones provided to the user (see Figs. 3,4) according to Carnahan
et
al. are of the continuous frequency modulation type, and no comparative tone
is
taught or suggested. Small frequency differences will be difficult to discern
for
many subjects. Furthermore, it would not be helpful for Carnahan's stated
purpose to provide a single comparative tone as this would complicate the
signal
unnecessarily, and not assist the listener in hearing the soundscape that is
associated with desired and undesired strokes. Carnahan et al. relies on a
subject's ability to relate a soundscape that consists of continuously varied
frequencies heard during an activity to the post-activity evaluation of the
result of
his/her action in order to map one to the other.
3
CA 02673149 2009-07-20
[0009] In a paper by Huang, H., et al. entitled: Recent Developments in
BioFeedback for Neuromotor Rehabilitation, (2006), Journal of Neurolmaging
and Rehabilitation, 3(11), an immersive multimedia environment for biofeedback
therapy is proposed. The focus is on task-oriented, repeated, reaching and
grasping tasks. Musical audio feedback is provided to reflect the smoothness
and temporal-spatial parameters of the endpoint of the reach trajectory in
order
to improve multi-joint coordination. Music notes within a musical phrase are
distributed spatially along a trajectory path - the notes indicate the
distance the
hand has moved toward the target, with each note corresponding to a short
distance along the path; the duration of any given note depends on the speed
at
which the patient's arm is moving - hence, patients can essentially compose
melodies based on movement pattern.
[0010] The authors point out that studies have shown that music can
synchronize motor outputs, improve the motor coordination of Parkinson
patients,
and enhance motor learning in a patient with large-fiber sensory neuropathy.
[0011] There remains a need for a simple and efficient audio feedback for
training motor skills to a wide audience, especially one that is useful for
guiding a
subject through a cognitive exercise in which the feedback is interpreted and
acted on in real time.
Summary of the Invention
[0012] Applicant has invented a feedback technique that guides a subject
during an exercise in moving towards a goal by providing audio feedback that
permits intuitive distinction between non-goal and goal points within the
exercise.
In accordance with the invention, a persistent note (called herein a reference
note) taken from a musical scale is provided for comparison with real-time
feedback notes that each represent respective positions of the subject in the
exercise as measured by a sensor. The reference note is related to the goal of
the exercise, in that it matches a feedback note associated with the goal in a
distinctive manner. For example, the feedback note of the goal and reference
4
CA 02673149 2009-07-20
note may be the same, one or more octave(s) apart, or simply much more
consonant with each other than at least the feedback notes associated with the
positions nearing the goal. During the course of the exercise, the reference
note
is emitted with the appropriate (based on the position of a part of the
subject)
real-time feedback note; together they provide feedback that can be instantly
discerned by most subjects who are not hearing impaired, permitting direct
determination as to whether the goal is reached.
[0013] In accordance with the invention an audio feedback signal for a subject
engaged in an exercise is provided. The audio feedback signal includes a
reference note taken from a musical scale that provides a relatively
persistent
marker of a goal for the subject, and a sequence of real-time feedback notes
of
the scale. Each feedback note is played with the reference note for a duration
sufficient to permit a subject to compare the notes. The specific feedback
note
played at an instant depends on where the subject is within the exercise, as
detected by a sensor. One of the feedback notes corresponds to the subject
reaching the goal, and this feedback note matches the reference note in a
distinctive manner.
[0014] Preferably there are at least 3 distinct feedback notes corresponding
to
respective positions within the exercise, whereby progress towards the goal is
indicated to the subject by the changing of the audio feedback signal in real-
time,
and the subject repeatedly compares the feedback note with the reference note
during the exercise. More preferably there are at least 4 distinct feedback
notes
or at least 5 distinct feedback notes.
[0015] The association of the feedback notes to the positions within the
exercise may be defined by a fixed map. The association may be pre-
established in any number of ways (as a map, as a method of computing the
mapping in dependence on history, etc.), may be dynamically computed (e.g., to
change in response to exercise execution parameters), or may be instantiated
as
a neural network that changes in response to execution of the present or
CA 02673149 2009-07-20
previous repetitions of the exercise, for example. In some embodiments a fixed
map provides a corresponding note in the scale for each of a predefined set of
sensed ranges of positions that the subject could assume during the exercise.
For example, from an exercise range (i.e., a range of positions within the
exercise over which the sensor is adapted to detect the position), a plurality
of
the sensed ranges may be chosen. Each of the sensed ranges is a proper
subset of the exercise range. The mapping may provide one scale note mapped
to each sensed range. At each point in time in the exercise, the last sensed
position may lie in one or more of these sensed ranges, and the real-time
feedback note may include the scale notes associated with these one or more
sensed ranges.
[0016] In preferred embodiments the sensed ranges partition the exercise
range (i.e., the sensed ranges are mutually exclusive and collectively
exhaustive). As such, exactly one feedback note is played at a time throughout
at least a phase of the exercise. For example, the partitioning may lead to
substantially equal time for each feedback note for an expected execution of
the
exercise, may have equal divisions of the exercise range, or may lead to a
shorter time for feedback notes during an expected difficult segment of the
exercise, or one that requires greater attention from the subject. While it
may be
easiest to distinguish a single feedback note from a single reference note at
a
time, the sensed ranges do not have to be disjoint. For example, a small
amount
of overlap during which the previous note fades out while the next feedback
note
fades in provides for a blending of the notes and an overlap of the sensed
ranges
according to one of several possible schemes for transitions between feedback
notes/ranges. Furthermore, while it may be preferred to provide feedback
continuously, or substantially continuously, the sensed ranges do not have to
collectively exhaust the exercise range, as another scheme for transition
would
be to have them fade in and out of silence. Such transition schemes may be
preferable if it is desirable to avoid noise from rapid transitions between
sensed
ranges, for a particular application.
6
CA 02673149 2009-07-20
[0017] Preferably, the exercise range is large enough, the sensed ranges are
sufficiently distributed within the exercise range, and the exercise is
expected to
be slow enough that the subject can hear the emitted reference and feedback
signal as musical feedback indicating a proximity to the goal during the
exercise.
[0018] In some embodiments the mapping is constant for multiple repetitions
of the exercise, and so the subject may associate specific feedback notes with
specific ranges of positions. In some embodiments, the association of feedback
notes with positions is in frequency order with respect to proximity to the
goal. In
some embodiments the feedback notes are chosen in relation to a consonance /
dissonance of the scale note with respect to the reference note.
[0019] The feedback note mapped to the goal may distinctly match the
reference note as the two notes may be consonant to a higher degree than any
of the other feedback notes, or to any feedback note other than a feedback
note
associated with a starting position of the exercise. For example the feedback
note mapped to the goal and the reference note may be the same, or one or
more octave(s) apart, and at least some of the other sensed ranges (especially
those that are proximate to the goal) may be mapped to scale notes that are
substantially dissonant, or markedly less consonant with the reference note.
[0020] In accordance with the invention a method for motor control training is
provided. The method involves providing a sensor for detecting a position of a
part of a subject during an exercise in relation to a goal; and generating an
audio
feedback signal including a reference note of a musical scale that provides a
marker of a goal for the subject, and a sequence of real-time feedback notes
of
the scale. Each feedback note is played with the reference note for a duration
sufficient to permit the subject to compare the notes, and is played when the
sensor detects that the subject is in a corresponding position within the
exercise.
One of the feedback notes corresponds to the subject reaching the goal, and
this
note matches the reference note in a distinctive manner.
7
CA 02673149 2009-07-20
[0021] Generating the sequence of feedback notes may involve generating at
least 3 distinct feedback notes corresponding to respective positions within
the
exercise from a starting position of the exercise to the goal. In this manner,
progress towards the goal is indicated to the subject by the changing of the
audio
feedback signal in real-time, and the subject repeatedly compares the feedback
note with the reference note during the exercise.
[0022] The sensor may be provided for continuously detecting a position of
the subject in the exercise over a range of positions that includes a
plurality of
sensed ranges that are mapped to respective feedback notes to uniquely
associate a note with each of the sensed ranges. The plurality of sensed
ranges
may be distributed within the range of positions so that for the expected
duration
of the exercise, a subject can register the audio signal as musical feedback.
The
feedback notes may be associated with positions as described above.
[0023] The sensor provided may be adapted to sense a state of flexure of a
muscle set directly or indirectly, in any manner known in the art. Preferably
the
method is non-invasive or minimally invasive. Preferably it is readily
configured
and set up, and is accurate.
[0024] An apparatus for motor control training is also provided. The
apparatus includes a sensor for outputting a position of a part of a subject
during
an exercise, a signal processor for receiving position output by the sensor
and
controlling an emitter to emit an audio feedback signal, and the emitter for
playing the audio feedback signal to the subject. The audio feedback signal
being that defined above.
[0025] The emitter provided may include two emitters or two channels of a
stereophonic emitter, for example. If two channels are used, and the reference
note is the same as a note associated with the goal state of the exercise, the
subject will be understood to be seeking to match the notes heard in the two
ears.
8
CA 02673149 2009-07-20
[0026] Further features of the invention will be described or will become
apparent in the course of the following detailed description.
Brief Description of the Drawings
[0027] In order that the invention may be more clearly understood,
embodiments thereof will now be described in detail by way of example, with
reference to the accompanying drawings, in which:
FIG. 1 is a schematic illustration of an apparatus in accordance with an
embodiment of the invention;
FIG. 2 is a schematic illustration of an example of an apparatus in accordance
with an embodiment of the invention applicable to conditioning muscles
controlling a joint;
FIG. 3 is a schematic illustration of a mapping of joint angles to feedback
notes
used in the apparatus of FIG. 2; and
FIGs. 4a-c are screen-shots of principal graphical user interfaces of a system
using the apparatus of FIG. 2.
Description of Preferred Embodiments
[0028] Herein a form of audio feedback is disclosed that is useful in motor
control training exercises, including physiotherapy of all kinds, when a fixed
goal
is known, an ordered set of steps towards that goal are defined, and a
detector is
available for measuring a subject's position within the exercise in relation
to the
ordered set of steps. While this can be used for recreational or athletics
training,
in certain circumstances, or in training of professional manual skills, the
particular
advantages of pain and stress reduction and muscle relaxation associated with
music can be leveraged in physio-therapeutic exercises. The invention is
applicable to all areas of physiotherapy (including orthopedics, prosthetics,
occupational, neurology, cardiopulmonary, geriatric, rheumatology, etc.) and
may
be for conditioning, habilitation, or rehabilitation, such as after surgery,
accident,
or trauma. The invention applies to exercises for pressure or weight
distribution
training (e.g., in physiotherapy boots or in a seat/wheelchair), standing
postural
9
CA 02673149 2009-07-20
training, sit-to-stand postural training, passive manipulation, head
positioning
training or seated postural training for paraplegics or those with other
disabilities.
[0029] In sports/athletics training, the invention may have particular
application for muscle flexion training, flexibility training, or in static
exercise
training such as may be desired in yoga. Furthermore, there are numerous
examples where someone is required to reach a body position or joint angle
before starting an activity (a dive, a move, an action, etc.). For example, to
execute a back handspring, it is useful for the gymnast to lean back so that
an
ankle is bent more than about 800 before jumping. Similarly, before a serve
(racket sports, volleyball, etc.), a swing (golf, baseball, etc.), or other
serve, a
desired posture of the athlete may be corrected or verified with an apparatus
in
accordance with the invention.
[0030] In recreation, the invention may apply to train a user in a manner of
holding or a posture while holding a musical instrument, a manner of holding a
paint brush or other applicator, or a specific position in dance, especially
when
training a limb out of a field of view of the dancer, or when the person has
no
frame of reference to determine whether a part of their body is in a desired
position.
[0031] Professional skills training may also require exercises that can be
performed using the present invention. For example, a device for ergonomics
training such as postural alignment when sitting at a desk, repetitive strain
injury
(RSI) prevention, etc., may make use of audio feedback in accordance with the
present invention.
[0032] It should be noted that the term 'exercise' herein is intended to cover
motions of a body or part towards or away from an objective, but could also
apply
to postural conditioning that is performed, for example at a desk or at a
workstation, with a view to minimizing risks of injury. In some cases the user
might not consider the feedback to be provided during an `exercise', and might
consider the actions not for practice, but the feedback is nonetheless
provided
CA 02673149 2009-07-20
with a view to ameliorating or monitoring cognitive control over respective
muscles and to this extent is an exercise.
[0033] Other examples of simple tasks that are independently taught in
construction, mechanics training, surgery, precious stone cutting, or other
precision cutting training where a single angle, force, or constrained
position of a
device in relation to a user's state of muscle flexure is of great importance.
For
example, a lifting task requiring pinching an object without crushing it,
surgically
actuating a stent or other device with only haptic feedback as to a state of
the
device, etc. Another example is in the military application of firearm
training, for
example, in terms of the holding position of a firearm relative to a target.
[0034] The audio feedback signal in accordance with the present invention
provides a reference note and a feedback note together. The reference note is
constant throughout (a part of) the exercise and the feedback note varies with
a
proximity of the subject to a goal state of the exercise. In general the
reference
note and feedback notes define a scale, such as the chromatic scale, or
selected
parts of such a scale.
[0035] The feedback notes and/or reference note may be continuously played
throughout the whole exercise (assuming the whole motion of the muscle is
mapped to respective notes), may be continuously played during a phase of the
exercise, or may be played intermittently, for example with fixed duration
pulses.
If intermittent, the pulse duration is long enough for clear discernment by
the
subject, such as a few seconds. The reference note has a greater persistence
than the feedback notes which vary throughout the exercise. In accordance with
the invention there is substantial overlap between the feedback notes and the
reference note at least throughout an active phase of the exercise.
[0036] While the reference note and real-time feedback notes may be
provided on a single channel, it may be preferred to provide them on different
audio channels (for stereophonic output) or otherwise by separate emitters.
For
example, by supplying the reference note to one ear and the feedback notes to
11
CA 02673149 2009-07-20
the other, a mental activity is provided for correlating the signals that may
have
advantages for distracting a subject from a pain response, and focusing the
subject on the music and associated motion. Having the signals on separate
channels delivered with appropriate stereo panning may also make it more
immediately obvious to a subject when the real-time feedback notes approach
and then match the reference note because spatial separation of the tone
information changes aural perception, and because two notes are typically
"blurred" to some degree when delivered together on the same mono signal.
[0037] The reference note provides a continuous marker of a goal for the
subject in that it does not change throughout the exercise, and preferably
does
not change throughout multiple repetitions of the exercise. If the mapping is
constant for multiple repetitions of the exercise, the subject may associate
specific feedback notes to specific positions within the exercise, which may
be
desirable. If a number of repetitions of the exercise is desired, the scale
may be
transposed in successive repetitions to vary the note pattern in a somewhat
predictable manner while providing some variation for the subject. For
example,
this may be done to inform the subject that they have passed to another level
of
difficulty, or that some other parameter of the exercise is changing. If
greater
variation is desired, the notes can be randomly (typically according to some
pattern) selected in each repetition, or after a preset event.
[0038] Preferably the feedback notes of the scale define progress towards the
goal while the exercise is underway by way of a mapping between the feedback
notes to respective ranges of positions within the exercise. A sensor is
provided
for sensing a position of a (part of a) subject within the exercise. When the
sensed position is within one of the respective ranges of positions, the scale
note
associated with the range is played. The mapping of the feedback notes may be
in (descending or ascending) frequency order as this may match expectations of
the subject. In general the selection of notes is an aesthetic choice, that
provides
important qualities to the system, in terms of the subject's enjoyment, and
likelihood of adopting the system. Frustration may be experienced when it is
not
12
CA 02673149 2009-07-20
clear to the subject whether the notes match or not, and advantageously
dissonant scale notes (with respect to the reference note) can be used to
facilitate distinctiveness of the matching (goal feedback note) and non-
matching
feedback notes. Careful selection of the notes is part of creating a useful
system
that is likely to be adopted.
[0039] While the exercise generally has a goal, it will be noted that this
goal
may not be unique, as there may be multiple goal states. It is possible that
multiple goal states are all contiguous, in which case the goal states may be
viewed as sub-states of a single goal. In other embodiments, the goal states
may not be continguous. For example, and exercise may consist in a cycle
between two goal states, as opposed to a linear motion from a rest state to a
goal state.
[0040] Preferably, for an exercise range, the range of positions within the
exercise range detected by the sensor is large enough, the sensed ranges are
large enough and sufficiently distributed within the exercise range, and the
exercise is expected to be slow enough that the subject can hear the emitted
reference and feedback signal as musical feedback indicating a proximity to
the
goal during the exercise. In this way the feedback serves as cognitive
feedback
for the subject of the exercise.
[0041] Preferably at most one feedback note is played at a time, and the
sensed ranges are disjoint. Preferably the ranges span a continuous part of
the
exercise range, so that every physical position within an exercise is
associated
with one real-time feedback note within the continuous part of the exercise
range.
A feedback note associated with the initial range may be the same note as the
goal note in some embodiments. This may give the subject an indication of what
the goal note is to listen for.
[0042] The feedback notes may be chosen from the scale principally for easy
discernment of the feedback notes from the reference note. The scale notes
chosen for the feedback may be chosen in relation to a consonance of the scale
13
CA 02673149 2009-07-20
note with respect to the reference note. For example, at points in the
exercise
where the subject would be expected to have difficulty, the feedback notes may
be more consonant with the reference note and to discourage lingering at a
point
a more dissonant note may be chosen. The sensed ranges may provide
substantially equal time for each feedback note in an expected execution of
the
exercise, or may provide smaller ranges in the neighbourhood of points in the
exercise that are expected to require more focus, or would be expected to be
more painful, or otherwise difficult.
[0043] In some embodiments, one of the sensed ranges is a goal range,
which corresponds to achievement of the goal of the exercise. The feedback
note mapped to the goal range is preferably more consonant with the reference
note than any of the intermediate ranges (from an initial range to the goal
range).
The goal range feedback note may be the same scale note as the reference
note, one or more scales above or below the reference note, or may simply be
more consonant than any of the intermediate range feedback notes, for example.
Applicant has found that the judicious use of at least some scale notes that
are
dissonant with the reference note can be useful, at least for facilitating
discernment of the notes.
[0044] FIG. 1 is a schematic illustration of an embodiment of an apparatus for
providing an audio feedback signal in accordance with an aspect of the
invention.
The apparatus includes a sensor 12 for detecting (directly or indirectly) a
position
of a muscle of a subject during an exercise. A wide variety of sensors 12 are
possible. Some sensors 12 are invasive, some require different degrees of
surface contact with the subject, and others can provide adequate sensing from
a
distance. Some sensors 12 detect gross changes such as angles or positions of
joints with respect to other joints, or any applied frame of reference, some
may
detect muscle response from secondary indicators such as blood flow, or
temperature. Pressure on a tensile sensor wrapped around a muscle group may
be useful for determining a state of flexure of a muscle. Examples of devices
are: goniometers, displacement actuators, force sensors, and optical image
14
CA 02673149 2009-07-20
analysis software with a video image, etc., although thermometers or IR
cameras, and other sensors could equally be used. Indirectly muscle motion can
be monitored by devices connected to a harness of various kinds or by a user's
touching or moving an object to which the sensor is coupled. The sensor may be
coupled to an implement or device held or manipulated by the subject, and may
register a force, an angle, a distance, a time, etc., in which case the state
of the
muscle may be inferred from the state of the implement.
[0045] An output from the sensor 12 may be output to a feedback control
processor 14 in any manner known in the art, but may preferably be by
electrical,
or electromagnetic signaling. The feedback control processor 14 receives the
sensor output, uses the sensor output to select a note on the scale
corresponding to feedback associated with the sensor output. For example, the
note may be provided by a fixed, predetermined mapping as described above, or
by a process for computing the map, and may be changed in one way or another
by the subject, an administrator of the exercise, or others. The mapping can
be
varied in terms of the number of sensor ranges, size of the sensor ranges, or
notes associated with the sensor ranges, for example, in response to previous
repetitions of the exercise, or sensor information regarding how the present
exercise is being performed, for example.
[0046] The feedback control processor 14 controls an emitter 16 to output an
audio feedback signal 18 that includes the reference note and the real-time
feedback note which are supplied together at least intermittently, and for a
duration sufficient for cognitive comparison of the notes. The audio output
signal 16 is emitted by an emitter 18. The audio output signal 16 may be a
stereophonic signal sent to headphones worn by the subject, may be separate
signals sent to different speakers, or may further be mixed onto a monophonic
audio signal output. Well known electrical equipment and devices are suitable
for use in the invention and adaptation of such devices is well within the
scope of
those skilled in the respective arts.
CA 02673149 2009-07-20
Example
[0047] Applicant has produced a joint angle rehabilitation/conditioning system
(AudioPhys) for providing audio feedback during physiotherapy that meets the
following needs:
= ability to collect, store, and present data generated during patient
exercises
(progress data) since it can be used to conduct quantitative progress analysis
and
to allow for quantitative progress measures;
= allows therapists to vary, monitor, and evaluate patients' exercises by
adjusting difficulty levels during therapy sessions, and for difficulty levels
to be set
for a patient based on the patient's previous performance, stage of
rehabilitation,
and general capabilities - for example, permitting measures of success to be
based
on an unaffected limb of the patient, when suitable;
= audio feedback parameters and delivery can be customized to the individual
preferences of patients and the individual requirements of each patient given
the
extent of variance across patients in capability and stage of rehabilitation;
= able to be set up for a patient within 3 minutes;
= leverages the effect music has been shown to have on pain reduction, the
demonstrated therapeutic qualities associated with distraction and stress
reduction
(relaxation response), and the focus enhancing and motivation provided by
mapping of sound to movement during physiotherapy activities;
= provides a patient with configuration control (such that the technology is
habituated as an extension of the user's own body), is comfortable, and
encourages participation, fostering adoption by patients; ar.d
= provides a fun, interesting, and mentally stimulating exercise format.
[0048] AudioPhys has been designed to support physiotherapeutic motor
rehabilitation of clients using musical-based audio cues relative to a
client's
specific joint angle. AudioPhys has been designed to be mobile/portable and
sympathetic to current physiotherapeutic processes. It has been designed to
support in-session activities and for out-patient use (i.e., for patients to
use it in
16
CA 02673149 2009-07-20
their own home between physiotherapy sessions). AudioPhys is a mobile
system and practice exercises can be set by the physiotherapist with a view to
providing additional feedback outside of sessions with clients, and
accelerating
rehabilitation. Additionally AudioPhys can record the inter-session exercises.
[0049] FIG. 2 is a schematic illustration of the AudioPhys system for
conditioning the muscles of a user's 20 joints. While the knee joint sensors
have
been used and tested, it will be appreciated that skeletal muscles for other
joints
(hip, elbow, etc.) could be conditioned with the same, similar, or dissimilar
equipment, and furthermore that motions of multiple joints concurrently can be
measured by a single device or by multiple devices in variants of the present
embodiment.
[0050] A system 22 was provided for acquiring information about an angle of
the joint. System 22 is embodied as a joint angle sensor 24 (typically a
goniometer) which senses the angle of the joint, and transmits angle data to a
datalog 26 which wirelessly transmits the angle data to a processing interface
28
in a prescribed form. In the present embodiment, the datalog 26 is secured to
the joint angle sensor 24, and the processor 28 is a mobile device adapted to
be
worn by the user 20.
[0051] Processor 28 includes a system management and feedback mapping
software system 30, a wireless communication interface 32, and a sound
card 34. The system management and feedback mapping software system 30
receives the angle data from the angle sensor 24 via a communication channel
provided by the wireless communication interface 32, the angle data encoding
the positional information of the joint. The system management and feedback
mapping software system 30 includes program instructions for mapping the angle
data to a unique scale note, as is further described below in relation to FIG.
3.
The sound card 34 outputs, in a prescribed form, an acoustic analog signal
including of the reference note and the feedback note. The feedback note is
selected based on the angle data, in accordance with the mapping, at least for
a
17
CA 02673149 2009-07-20
period of the exercise, in accordance with a configuration of the system. In
the
present embodiment, the reference note and feedback note are modulated as a
stereophonic sound signal. The acoustic analog signal is transferred to a pair
of
headphones 36 worn by the user 20 as a means for communicating feedback in
the form of a stereophonic sound.
[0052] A sampling rate of the joint angle sensor 24, signaling rates of the
wireless communication interface 32, and processing rate of processor 28 are
chosen so that the stereophonic audio feedback is provided to the user 20 in
substantially real-time, to provide the user 20 with a dynamically updated
indication of his/her current/actual joint angle relative to a fixed,
continuous
indication of his/her target joint angle (the reference note).
[0053] The AudioPhys system is designed to be set up as follows: The joint
angle sensor 24 is appropriately (non-invasively) attached to the joint of
interest
above and below the joint. In most cases, the affected joint is located on one
side of the user (the affected side). The joint angle sensor is preconfigured
to
measure the angle of the affected joint, and is turned on such that the
datalog 26
begins receiving the angle data in a prescribed form, and wirelessly
transmitting
it to the processor 28 located on a mobile device which is worn/strapped to
the
user 20. The form of the angle data received by the datalog 26 may not be the
same as that of the angle data sent to the processor 28, and various data
processing transformations could be applied, as will be understood by those of
skill in the art. The angle data is then processed by the system management
and
feedback mapping system 30 in order to generate the appropriate stereophonic
audio feedback which is fed to the user via a pair of headphones 36. The
stereophonic audio feedback signal on the affected side of the user 20 plays
the
feedback notes while the opposite side plays the reference note so that the
user
hears the feedback notes as associated with the affected side. Thus, if the
left
knee is being conditioned, the feedback notes will be stereo-panned to the
user's
left ear.
18
CA 02673149 2009-07-20
[0054] The premise of the AudioPhys feedback mechanism is that the user
will want to equalize the audio feedback across both ears - that is, to move
his/her joint into a position such that the dynamically changing feedback note
that
corresponds to his/her actual joint position (as fed to one ear) is the same
as the
fixed, continuous reference note corresponding to his/her target position (as
fed
to the other ear). By having a continuous, pre-determined goal note delivered
to
one ear at the same time as receiving the dynamically changing feedback
corresponding to his/her actual joint angle in the other ear, the user can
gauge
his/her progress towards the target. When the target is reached, the client
will
hear the same note in both ears simultaneously.
[0055] There are many possible arrangements of parameters for the system
and these may be configured by the therapist or the user, depending on the
system specifications. The specifications chosen for the AudioPhys system are
described below in relation to FIG. 4. In general, the AudioPhys system is
designed to be configurable such that: (a) a physiotherapist/physical
therapist
can configure settings such as target joint position, feedback granularity,
comfortable hearing (frequency) range, and volume on a per-user, per-session
basis; (b) the system is easy to use and wear, with minimal set up time per
use;
and (c) the system can be set up for users (i.e., clients of physiotherapists/
physical therapists) to take home and use in between scheduled physical
therapy
sessions, thereby providing users with feedback and target information that
would otherwise be unavailable outside of a scheduled therapy session.
[0056] The joint angle to audio feedback mapping used by the system is
described with reference to FIG. 3. It should be noted that, for ease of
description, the following figures represent 1 D joint motions (i.e., do not
attempt
to show or describe rotation or off plane pivoting); the system could,
however,
handle rotation about a joint as well using any mapping from the 2D space to a
line.
19
CA 02673149 2009-07-20
[0057] For the purpose of AudioPhys, the joint angle sensor 22 reports a
single angle. While in other embodiments, pairs of angles may be measured to
determine whether a given angle of a joint is relative to another, or relative
to a
fixed frame of reference, there is a safe range of joint angles created
between
bones A and B which reflect the variety of clinically safe positions for a
joint. If
user 20 exceeds the safe range, he/she is considered to have moved into a
clinically unsafe position, and AudioPhys uses a simple alarm sound to alert
users to the fact that their affected joint is not in a safe position. In one
mode of
operation, musical-based stereophonic audio feedback is always played while
the user 20 is within the safe range. In other modes silence indicates that
the
user 20 is not operating within the expected range of the exercise, but is
still
within the safe range. This may be because the user 20 has surpassed the goal
state, or because the user 20 has not yet moved into a starting position for
the
exercise.
[0058] AudioPhys could be developed to provide a plurality of goal states
having different feedback notes associated with different sub-ranges of the
goal
range, if it is desirable that the user plays with or varies the angles within
the goal
range. If so all of the goal range states may be generally more consonant with
the reference note than the other feedback notes.
[0059] The exercise range (a sub-range of the safe range) is partitioned into
a
sequence of sensed ranges. Limiting angles of 5 sensed ranges are shown at
the bottom of FIG. 3. The number of sensed ranges will depend on a granularity
of feedback appropriate to the user 20. For example, if a
physiotherapist/physical therapist deems that a given client is only likely to
make
very small improvements in terms of joint position in any given session,
he/she
might set the sub-ranges to correspond to 2 differences in angles such that
the
client receives a dynamic change in audio feedback for even a small change in
joint position; but if the user 20 is deemed capable of making more
substantial
changes to his/her joint angle, the physiotherapist/physical therapist might
set the
sub-ranges to correspond to 5 differences in angles such that the client
needs to
CA 02673149 2009-07-20
make larger changes in joint positions to effect a change in audio feedback.
In
FIG. 3 only 5 angles are shown mapped to respective notes, however it will be
appreciated that any number of sensed ranges may be defined. According to
AudioPhys these ranges are chosen by the physiotherapist/physical therapist.
Furthermore, the physiotherapist/physical therapist may determine that it is
appropriate to have a contiguous set of sensed ranges at one level of
granularity
and the remaining set of sub-ranges at other levels of granularity to reflect
the
areas of focus or difficulty for the user 20. Finally, the
physiotherapist/physical
therapist 20 may choose to switch off the audio feedback for a given,
contiguous
set of sub-ranges, such that the feedback only becomes apparent at a given
physical joint position. All of these configurable options are aimed at
ensuring
that the most appropriate feedback is provided to each client as an
individual,
based on the client's current state of physical rehabilitation.
[0060] Once the physiotherapist/physical therapist has determined the
granularity and on/off points for the audio feedback, he/she can set the
comfortable hearing range appropriate/preferable to the user 20. Thereafter,
the
system management and feedback mapping system 30 assigns selected notes
from within the identified comfortable hearing range to each of the sensed
ranges
with a one-to-one mapping appropriate to the joint being measured. If the user
moves the affected joint within the exercise range where feedback-generation
is
selected, he/she hears a rising or falling sequence of discrete notes. The
feedback notes would be delivered by a headphone, to an ear on his/her
affected
side. Additionally and concurrently, the user would hear the reference note
delivered by a headphone, to an ear on his/her unaffected side.
[0061] In accordance with AudioPhys, the following is an example of a
mapping that is used:
Segment
Size Frequency
(of 180 Midi
range) Note Note
2 84 C6 1046.5Hz Reference or
21
CA 02673149 2009-07-20
target/goal
2 81 A5 880.00Hz
2 77 F5 698.46Hz
2 74 D5 587.33Hz
2 71 B4 493.88Hz
70 B4b 466.16Hz
5 69 A4 440.00Hz
5 68 G4# 415.30Hz
5 66 F4# 369.99Hz
5 65 F4 349.23Hz
5 63 E4b 311.13Hz
62 D4 293.67Hz
10 61 CO 277.18Hz
10 60 C4 261.63Hz
10 59 B3 246.94Hz
25 58 B3b 233.08Hz
25 57 A3 220.00Hz
25 56 G3# 207.65Hz
25 54 F3# 185.00Hz Max feedback
note
The feedback notes were selected from the chromatic scale such that they avoid
the most harmonious pairings with the reference note. The dissonance of the
note pairs (feedback to reference) renders them more distinguishable by a wide
range of users and makes the point where the feedback note matches the
reference note most obvious. Furthermore, the dissonance of the note pairings
(intervals) at particular angle positions may provide (in some users) a
motivation
to pass the corresponding angular sub-range.
[0062] The physiotherapist/physical therapist is also expected to determine
the target joint angle (or range of angles) for which the user should aim
during a
therapy session, and may project targets for subsequent sessions.
[0063] FIGs. 4a-c are screen-shots of 3 graphical user interfaces (GUls)
showing configuration and use of AudioPhys. The screen-shots are presented
on a display screen of the processor 28, which is a touch screen interface.
22
CA 02673149 2009-07-20
[0064] FIGs. 4a,b show two different screen-shots presented to a
physiotherapist/physical therapist permitting the configuration of the
AudioPhys
system for the user 20. The configuration tab first permits the specification
of the
joint in question. A knee joint is selected in the illustrated case, and hence
the
indication of a side of the body is permitted (with radio buttons). In the
present
case the left knee is being conditioned. The cued option indicates whether the
user is being assisted by a physiotherapist or physical therapist during the
present exercise, which is mainly used for reporting and analysis. The audio
option permits the audio feedback to be on or off during the present exercise.
It
may be preferred to keep the audio off for preliminary assessment of mobility,
for
example. The pitch option permits subjects with better hearing (or simply
hearing
preferences) in higher, lower, or mid range (comfortable hearing range) to
have
appropriate feedback. Finally the reference or target/goal note can be set to
be
the highest pitch or lowest pitch in the feedback range (i.e., the user will
hear
feedback notes rising or falling to the reference or target/goal note,
respectively),
depending on the desires of the user 20.
[0065] A slider along a scale of angles is provided for setting of a range of
angles for which audio feedback is or is not provided. The GUI further
requires
the identification of a current target angle, either in terms of a fixed
number, or
taken from a snapshot of the user's actual angle, which may be a position set
by
the physiotherapist/physical therapist. The (ultimate) end goal and best to
date
angles shown are for assisting the physiotherapist/physical therapist in
entering a
numerical target and, in the case of the latter, in tracking or quickly
referencing
progress to date.
[0066] A green section of the current target shows an active phase of the
exercise over which the measured joint angles are expected to range, and over
which the stereophonic audio feedback will be played. It will be noted that in
FIG. 4a the exercise is bending the left knee from a straight position to 90
bent,
and that the stereophonic audio feedback is to be played throughout the
exercise, whereas in FIG. 4b the exercise is to straighten the left knee that
is
23
CA 02673149 2009-07-20
initially bent, and the stereophonic audio feedback is only played when the
knee
reaches a certain angle (110 ).
[0067] FIG. 4c illustrates an active tab of the AudioPhys system that is
presented on the touch screen of the mobile device during an exercise. The
user
may play, pause, or stop the session (including feedback and recording) using
the buttons, as if the device were a media player. There is also a volume
adjustment slider provided for adjusting the volume. The user 20 may also
visually follow the angles registered by the joint angle sensor 24, and
compare
this with the target for the present session, that was set previously using
the
configuration tab. The angle data received by the system management and
feedback mapping system 30 is dynamically updated in the actual box of the
touch screen.
[0068] Other advantages that are inherent to the structure are obvious to one
skilled in the art. The embodiments are described herein illustratively and
are not
meant to limit the scope of the invention as claimed. Variations of the
foregoing
embodiments will be evident to a person of ordinary skill and are intended by
the
inventor to be encompassed by the following claims.
24
