Note: Descriptions are shown in the official language in which they were submitted.
KTX0001 CADOO
SYSTEM AND METHOD FOR DETERMINING RUNNING POWER
FIELD
[0001]This document relates to systems and methods for processing data from
sensors
monitoring human movement or human activity. In particular, this document
relates to
determining mechanical running power using force sensor data.
BACKGROUND
[0002] US Patent No. 9,216,320 (Ishii) discloses a method and apparatus for
measuring
power output of exercise motion by a user. The system includes a wireless
motion
sensor and a wireless central mass unit. The wireless motion sensor detects,
measures
and transmits motion data associated with a first motion by the user. The
central mass
unit detects and measures motion data associated with a second motion by the
user.
The central mass unit also computes the power output of the first motion and
the second
motion as a function of the first motion data and the second motion data.
[0003]US Patent No. 10,744,371 (Mohrman et al.) discloses a sensing system and
wearable sensor platform to provide real-time feedback to a user/wearer of
power
expenditure during an activity. In one example, the system includes an
inertial
measurement unit (IMU) for acquiring multi-axis motion data at a first
sampling rate, and
an orientation sensor to acquire orientation data at a second sampling rate
that is varied
based on the multi-axis motion data.
SUMMARY
[0004] The following summary is intended to introduce the reader to various
aspects of
the detailed description, but not to define or delimit any invention.
[0005]A system, method and computer program product for determining running
power
is disclosed. In some examples, force sensor data is acquired from a plurality
of force
sensors positioned underfoot of a user performing an activity such as running
or
walking. Using the force sensor data, force values can be determined for a
plurality of
strides. A corresponding slope, stance time and running speed can also be
determined
1
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
for each stride. The force values, slope, stance time and running speed can be
provided
as inputs to a machine learning model trained to determine the mechanical
running
power output by the user. The mechanical running power can then be provided to
the
user as feedback or stored for purposes such as review and analysis.
[0006] In some examples, the inputs to the machine learning model can be
determined
entirely based off of sensor data received from a wearable device worn by the
user. The
wearable device can include a plurality of force sensors and an inertial
measurement
unit. For example, an insole can be provided with the plurality of force
sensors and the
inertial measurement unit. Sensor data from the force sensors and the inertial
measurement unit can be used to determine the force values, slope, stance time
and
running speed. This may allow the user to perform activities at various
locations while
still tracking and monitoring their mechanical running power output.
[0007]According to some aspects, a method for determining running power using
force
sensor data from a plurality of force sensors positioned underfoot includes:
obtaining a
plurality of force sensor readings from the plurality of force sensors;
determining force
values for a plurality of strides using aggregate force data based on the
force sensor
readings; for each stride in the plurality of strides, determining a slope
value, a stance
time value, and a running speed; and determining a mechanical running power
associated with the plurality of force sensor readings by inputting the force
values, the
slope value, the stance time value and the running speed to a machine learning
model
trained to predict the mechanical running power.
[0008]The plurality of force sensors can be disposed on a wearable device that
is worn
on a foot.
[0009] The wearable device can include a deformable material.
[0010] The deformable material can be a foam.
[0011]The wearable device can be an insole.
[0012] The wearable device can be a shoe.
2
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
[0013]The wearable device can be a compression-fit garment.
[0014]The wearable device can be a sock.
[0015]The method can include outputting an output dataset, which can include
the
mechanical running power and/or the running speed.
[0016]The output dataset can be used as an input to a game.
[0017]The output dataset can be used to execute an action in the game.
[0018]A scaling factor can be applied to the output dataset in the game.
[0019]The scaling factor can be an integer.
[0020]The scaling factor can have a value of 1.
[0021]An avatar can be generated in the game with motion defined according to
the
output dataset.
[0022]The output dataset can be used to model the dynamics of virtual objects
and
surroundings with which a user interacts in the game.
[0023]A game score in the game can be calculated based on the output dataset.
[0024]A training goal can be generated based on the output dataset and/or the
game
score.
[0025]A percentage of progress towards achieving the training goal can be
calculated
based on the output dataset and/or the game score.
[0026]The training goal can be a range of mechanical running power values that
the
user must stay within for a predetermined period of time.
[0027]The training goal can be a range of running speed values that the user
must stay
within for a predetermined period of time.
3
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
[0028]A technique quality of a user performing a movement can be calculated
from the
output dataset.
[0029]A task readiness score can be calculated from the output dataset and/or
the
technique quality.
[0030]A first user can be challenged to replicate the output dataset of a
second user in
the game.
[0031 ]The wearable device can include at least one vibrotactile motor.
[0032]The at least one vibrotactile motor can generate a haptic signal based
on the
output dataset.
[0033]An audio signal can be generated based on the output dataset.
[0034]A visual display can be generated based on the output dataset.
[0035]The visual display can be an output dataset vs. time graph.
[0036] Determining the force values can include determining at least one mean
force
value.
[0037]The aggregate force data may be separated into a plurality of foot
regions, and
determining the at least one mean force value can include determining a region-
specific
mean force value for each foot region.
[0038] Determining the force values can include determining at least one peak
force
value.
[0039]The aggregate force data may be separated into a plurality of foot
regions, and
determining the at least one peak force value can include determining a region-
specific
peak force value for each foot region.
[0040]The slope value can be determined based on inertial measurement data
received
from an inertial measurement unit.
4
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
[0041]The method can include identifying the plurality of strides by:
identifying a plurality
of foot-contact times based on the force sensor readings and/or a plurality of
foot-off
times based on the force sensor readings; and identifying each stride as a
stride period
between adjacent foot-contact times or between adjacent foot-off times.
[0042] For each stride, the stance time value can be identified as the length
of a foot
contact period between the foot contact time and the foot off time.
[0043] For each stride, determining the running speed can include: identifying
a stride
time as the length of time of the corresponding stride period; identifying a
stride length
for the corresponding stride period based on inertial measurement data
received from an
inertial measurement unit; and determining the running speed as the stride
length
divided by the stride time.
[0044] Each stride can include a foot contact period between the foot contact
time and
the foot off time and identifying the stride length can include: determining a
stride
velocity based on the inertial measurement data; integrating the stride
velocity over the
length of time between adjacent foot contact periods to determine a stride
displacement;
and determining the stride length based on a combination of directional
components of
the stride displacement.
[0045] The inertial measurement data can include accelerometer data and
determining
the stride velocity can include: determining transformed accelerometer data by
transforming the accelerometer data to a global reference frame; and
determining the
stride velocity by integrating the transformed accelerometer data.
[0046] The inertial measurement data can include angular velocity data and
transforming
the accelerometer data to a global reference frame can include: determining a
subsequent foot flat time for the subsequent stride based on the inertial
measurement
data; determining a foot angle by integrating the angular velocity data from
the foot-
contact time to the subsequent foot flat time; and transforming the
accelerometer data to
the global reference frame based on the foot angle.
5
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
[0047]The method can further include determining a first foot flat time for
the stride
based on the inertial measurement data; determining a foot flat velocity value
at the first
foot flat time based on the integrated transformed accelerometer data; and
adjusting the
stride velocity by subtracting the foot flat velocity value.
[0048]The inertial measurement data can include gyroscope data and determining
each
foot flat time can include: determining a time of minimum gyroscope energy for
the
corresponding stride from the gyroscope data; and identifying the foot flat
time as the
time of minimum gyroscope energy.
[0049] Determining each foot flat time can include identifying a time of
maximal plantar
force for the corresponding stride based on the force sensor readings.
[0050]The machine learning model can be a regression model. The regression
model
can be a LASSO model.
[0051]The machine learning model can be trained using training data generated
by
collecting training sensor readings and calculating a training average
positive power and
a training average negative power of each stride using a force-instrumented
treadmill.
[0052]The mechanical running power can be determined as a time-continuous
mechanical running power.
[0053]The mechanical running power can include an average positive power and
an
average negative power of each stride.
[0054]The method can include receiving inertial measurement data from an
inertial
measurement unit; for each stride, determining a vertical velocity and an
anterior-
posterior velocity based on the inertial measurement data; and calculating an
estimated
mechanical running power using the vertical velocity, the anterior-posterior
velocity, and
the force values; and determining the mechanical running power by inputting
the
estimated mechanical running power to the machine learning model along with
the force
values, the slope value, the stance time value and the running speed.
[0055]The estimated mechanical running power P can be calculated according to:
6
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
P= vv* vGRF +v ap*apGRF
where vv represents the vertical velocity, vap represents the anterior-
posterior velocity,
vGRF represents a vertical ground reaction force determined from the force
values, and
apGRF represents an anterior-posterior ground reaction force determined from
the force
values.
[0056]According to some aspects, a system for determining running power
includes a
plurality of force sensors positionable underfoot, one or more processors
communicatively coupled to the plurality of force sensors, and a non-
transitory storage
memory storing a machine learning model trained to predict mechanical running
power.
The one or more processors are configured to: obtain a plurality of force
sensor
readings from the plurality of force sensors; determine force values for a
plurality of
strides using aggregate force data based on the force sensor readings; for
each stride in
the plurality of strides, determine a slope value, a stance time value, and a
running
speed; and determine the mechanical running power associated with the
plurality of
force sensor readings by inputting the force values, the slope value, the
stance time
value and the running speed to the machine learning model.
[0057]The system can include a wearable device on which the plurality of force
sensors
are disposed, and the wearable device can be worn on a foot.
[0058]The wearable device can include a deformable material.
[0059]The deformable material can be a foam.
[0060]The wearable device can be an insole.
[0061]The wearable device can be a shoe.
[0062] The wearable device can be a compression-fit garment.
[0063]The wearable device can be a sock.
[0064]The one or more processors can be further configured to output an output
dataset, and the output dataset can include the mechanical running power
and/or the
running speed.
7
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
[0065]The one or more processors can be further configured to use the output
dataset
as an input to a game.
[0066]The one or more processors can be further configured to execute an
action in the
game based on the output dataset.
.. [0067]The one or more processors can be further configured to apply a
scaling factor to
the output dataset in the game.
[0068]The scaling factor can be an integer.
[0069]The scaling factor can have a value of 1.
[0070]The one or more processors can be further configured to generate an
avatar in
the game with motion defined according to the output dataset.
[0071]The one or more processors can be further configured to model the
dynamics of
virtual objects and surroundings with which a user interacts in the game based
on the
output dataset.
[0072]The one or more processors can be further configured to compute a game
score
in the game based on the output dataset.
[0073]The one or more processors can be further configured to generate a
training goal
based on the output dataset and/or the game score.
[0074]The one or more processors can be further configured to calculate a
percentage
of progress towards achieving the training goal based on the output data
and/or the
game score.
[0075]The training goal can be a range of mechanical running power values to
stay
within for a predetermined period of time
[0076]The training goal can be a range of running speed values to stay within
for a
predetermined period of time.
8
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
[0077]The one or more processors can be further configured to determine a
technique
quality of a user performing a movement based on the output dataset.
[0078]The one or more processors can be further configured to determine a task
readiness score based on the output dataset and/or the technique quality.
[0079]The one or more processors can be further configured to challenge a
first user to
replicate the output dataset of a second user in the game.
[0080]The system can include at least one vibrotactile motor.
[0081]The at least one vibrotactile motor can be configured to generate a
haptic signal
based on the output dataset.
[0082]The one or more processors can be further configured to generate an
audio
signal based on the output dataset.
[0083]The one or more processors can be further configured to generate a
visual
display based on the output dataset.
[0084]The visual display can be an output dataset vs. time graph.
[0085]The one or more processors can be configured to determine the force
values to
include at least one mean force value.
[0086]The aggregate force data may be separated into a plurality of foot
regions and
the one or more processors can be configured to determine the at least one
mean force
value by determining a region-specific mean force value for each foot region.
[0087]The one or more processors can be configured to determine the force
values to
include at least one peak force value.
[0088]The aggregate force data may be separated into a plurality of foot
regions and
the one or more processors can be configured to determine the at least one
peak force
value by determining a region-specific peak force value for each foot region.
9
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
[0089]The system can further include an inertial measurement unit associated
with the
plurality of force sensors and the one or more processors can be further
configured to
determine the slope value based on inertial measurement data received from the
inertial
measurement unit.
[0090]The one or more processors can be configured to identify the plurality
of strides
by: identifying a plurality of foot-contact times based on the force sensor
readings and/or
a plurality of foot-off times based on the force sensor readings; and
identifying each
stride as a stride period between adjacent foot-contact times or between
adjacent foot-
off times.
[0091]The one or more processors can be configured to, for each stride,
identify the
stance time value as the length of a foot contact period between the foot
contact time
and the foot off time.
[0092]The one or more processors can be configured to, for each stride,
determine the
running speed by: identifying a stride time as the length of time of the
corresponding
stride period; identifying a stride length for the corresponding stride period
based on
inertial measurement data received from an inertial measurement unit; and
determining
the running speed as the stride length divided by the stride time.
[0093] Each stride can include a foot contact period between the foot contact
time and
the foot off time and the one or more processors can be configured to identify
the stride
length by: determining a stride velocity based on the inertial measurement
data;
integrating the stride velocity over the length of time between adjacent foot
contact
periods to determine a stride displacement; and determining the stride length
based on
a combination of directional components of the stride displacement.
[0094]The inertial measurement data can include accelerometer data and the one
or
more processors can be configured to determine the stride velocity by:
determining
transformed accelerometer data by transforming the accelerometer data to a
global
reference frame; and determining the stride velocity by integrating the
transformed
accelerometer data.
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
[0095]The inertial measurement data can include angular velocity data and the
one or
more processors can be configured to transform the accelerometer data to a
global
reference frame by: determining a subsequent foot flat time for the subsequent
stride
based on the inertial measurement data; determining a foot angle by
integrating the
angular velocity data from the foot-contact time to the subsequent foot flat
time; and
transforming the accelerometer data to the global reference frame based on the
foot
angle.
[0096]The one or more processors can be further configured to: determine a
first foot
flat time for the stride based on the inertial measurement data; determine a
foot flat
velocity value at the first foot flat time based on the integrated transformed
accelerometer data; and adjust the stride velocity by subtracting the foot
flat velocity
value.
[0097]The inertial measurement data can include gyroscope data and the one or
more
processors can be configured to determine each foot flat time by: determining
a time of
minimum gyroscope energy for the corresponding stride from the gyroscope data;
and
identifying the foot flat time as the time of minimum gyroscope energy.
[0098]The one or more processors can be configured to determine each foot flat
time by
identifying a time of maximal plantar force for the corresponding stride based
on the
force sensor readings.
[0099]The machine learning model can be a regression model. The regression
model
can be a LASSO model.
[0100]The machine learning model can be trained using training data generated
by
collecting training sensor readings and calculating a training average
positive power and
a training average negative power of each stride using a force-instrumented
treadmill.
[0101]The one or more processors can be configured to determine the mechanical
running power as a time-continuous mechanical running power.
11
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
[0102]The one or more processors can be configured to determine the mechanical
running power to include an average positive power and an average negative
power of
each stride.
[0103]The one or more processors can be configured to: receive inertial
measurement
data from an inertial measurement unit; for each stride, determine a vertical
velocity and
an anterior-posterior velocity based on the inertial measurement data; and
calculate an
estimated mechanical running power using the vertical velocity, the anterior-
posterior
velocity, and the force values; and determine the mechanical running power by
inputting
the estimated mechanical running power to the machine learning model along
with the
force values, the slope value, the stance time value and the running speed.
[0104] The one or more processors can be configured to calculate the estimated
mechanical running power P according to:
P= vv* vGRF +vap*apGRF
where vv represents the vertical velocity, vap represents the anterior-
posterior velocity,
vGRF represents a vertical ground reaction force determined from the force
values, and
apGRF represents an anterior-posterior ground reaction force determined from
the force
values.
[0105] According to some aspects, there is provided a non-transitory computer
readable
medium storing computer-executable instructions, which, when executed by a
computer
processor, cause the computer processor to carry out a method of determining
running
power. The method includes obtaining a plurality of force sensor readings from
a
plurality of force sensors positioned underfoot; determining force values for
a plurality of
strides using aggregate force data based on the force sensor readings; for
each stride in
the plurality of strides, determining a slope value, a stance time value, and
a running
speed; determining a mechanical running power associated with the plurality of
force
sensor readings by inputting the force values, the slope value, the stance
time value and
the running speed to a machine learning model trained to predict the
mechanical
running power; and outputting the mechanical running power.
12
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
[0106]The non-transitory computer readable medium can store computer-
executable
instructions, which, when executed by a computer processor, cause the computer
processor to carry out the method of determining running power, where the
method is
described herein.
BRIEF DESCRIPTION OF THE DRAWINGS
[0107]The drawings included herewith are for illustrating various examples of
articles,
methods, and apparatuses of the present specification and are not intended to
limit the
scope of what is taught in any way. In the drawings:
[0108]FIG. 1 is a block diagram illustrating an example of a system for
determining
running power;
[0109]FIG. 2 is a diagram illustrating an example of a wearable device
incorporating a
sensing unit that can be used in the system of FIG. 1;
[0110]FIG. 3 is a flowchart illustrating an example of a method for
determining
mechanical running power;
[0111] FIG. 4 is a flowchart illustrating an example of a method for
determining running
speed, which may be used with the method shown in FIG. 3;
[0112]FIG. 5A is a plot illustrating the average positive running power
determined by a
physics-based approach using force sensor data from force sensors mounted on
an
insole (Y-axis) and the average positive running power determined by a force-
plate (FP)
instrumented treadmill (X-axis);
[0113]FIG. 5B is a plot illustrating the average negative running power
determined by a
physics-based approach using force sensor data from force sensors mounted on
an
insole (Y-axis) and the average negative running power determined by a force-
plate
(FP) instrumented treadmill (X-axis);
[0114]FIG. 6A is a diagram illustrating the average positive running power
determined
by an implementation of the method of FIG. 3 using force sensor data from
force
13
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
sensors mounted on an insole (Y-axis) and the average positive running power
determined by a force-plate instrumented treadmill (X-axis);
[0115]FIG. 6B is a diagram illustrating the average negative running power
determined
by an implementation of the method of FIG. 3 using force sensor data from
force
sensors mounted on an insole (Y-axis) and the average negative running power
determined by a force-plate (FP) instrumented treadmill (X-axis);
[0116]FIG. 7A is a diagram illustrating estimated running speed values
determined by
an implementation of the method of FIG. 4 using force sensor data; and
[0117]FIG. 7B is a diagram illustrating the percentage error of estimated
running speed
values determined by an implementation of the method of FIG. 4 using force
sensor
data and the percentage error of estimated running speed values determined
using IMU
data alone.
DETAILED DESCRIPTION
[0118]Various apparatuses or processes or compositions will be described below
to
provide an example of an embodiment of the claimed subject matter. No
embodiment
described below limits any claim and any claim may cover processes or
apparatuses or
compositions that differ from those described below. The claims are not
limited to
apparatuses or processes or compositions having all of the features of any one
apparatus or process or composition described below or to features common to
multiple
or all of the apparatuses or processes or compositions described below. It is
possible
that an apparatus or process or composition described below is not an
embodiment of
any exclusive right granted by issuance of this patent application. Any
subject matter
described below and for which an exclusive right is not granted by issuance of
this
patent application may be the subject matter of another protective instrument,
for
example, a continuing patent application, and the applicants, inventors or
owners do not
intend to abandon, disclaim or dedicate to the public any such subject matter
by its
disclosure in this document.
14
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
[0119]For simplicity and clarity of illustration, reference numerals may be
repeated
among the figures to indicate corresponding or analogous elements. In
addition,
numerous specific details are set forth in order to provide a thorough
understanding of
the subject matter described herein. However, it will be understood by those
of ordinary
skill in the art that the subject matter described herein may be practiced
without these
specific details. In other instances, well-known methods, procedures and
components
have not been described in detail so as not to obscure the subject matter
described
herein. The description is not to be considered as limiting the scope of the
subject
matter described herein.
[0120]The terms "coupled" or "coupling" as used herein can have several
different
meanings depending in the context in which these terms are used. For example,
the
terms coupled or coupling can have a mechanical, electrical or communicative
connotation. For example, as used herein, the terms coupled or coupling can
indicate
that two elements or devices are directly connected to one another or
connected to one
another through one or more intermediate elements or devices via an electrical
element,
electrical signal, or a mechanical element depending on the particular
context.
Furthermore, the term "communicative coupling" may be used to indicate that an
element or device can electrically, optically, or wirelessly send data to
another element
or device as well as receive data from another element or device.
[0121]As used herein, the wording "and/or" is intended to represent an
inclusive-or.
That is, "X and/or Y" is intended to mean X or Y or both, for example. As a
further
example, "X, Y, and/or Z" is intended to mean X or Y or Z or any combination
thereof.
Furthermore, the wording "at least one of A and B" is intended to mean only A
(i.e. one
or multiple of A), only B (i.e. one or multiple of B), or a combination of one
or more of A
and one or more of B.
[0122]Terms of degree such as "substantially", "about", and "approximately" as
used
herein mean a reasonable amount of deviation of the modified term such that
the end
result is not significantly changed. These terms of degree may also be
construed as
including a deviation of the modified term if this deviation would not negate
the meaning
of the term it modifies.
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
[0123]Any recitation of numerical ranges by endpoints herein includes all
numbers and
fractions subsumed within that range (e.g. 1 to 5 includes 1, 1.5, 2, 2.75, 3,
3.90, 4, and
5). It is also to be understood that all numbers and fractions thereof are
presumed to be
modified by the term "about" which means a variation of up to a certain amount
of the
number to which reference is being made if the end result is not significantly
changed.
[0124] Described herein are systems, methods and devices for determining the
running
power of a user while performing an activity or other movement. The systems,
methods
and devices described herein can determine a user's mechanical running power
using a
plurality of sensors positioned underfoot. The systems, methods, and devices
can in
some examples use sensors attached to, or contained within, wearable devices
or
fitness equipment to measure and monitor data relating to movement or activity
of a
user. The measured data from the sensors can be used to calculate the
mechanical
running power.
[0125]The sensors can be force sensors and can be provided in the insole of a
shoe or
within the footwear worn by the user. The force data acquired by the force
sensors can
be used to determine the level of force applied by a user's foot when walking,
running,
jumping, or gaming, for example. This force data can be used to derive
additional force
derivatives or force-based metrics, such as the force output, mean force or
the peak
force for the user. The force data, and other data derived therefrom, can be
used to
determine the mechanical running power generated by the user. This may be
useful for
medical, fitness, athletic, security, gaming, entertainment or other purposes.
[0126]The systems, methods, and devices described herein may be implemented as
a
combination of hardware or software. In some cases, the systems, methods, and
devices described herein may be implemented, at least in part, by using one or
more
computer programs, executing on one or more programmable devices including at
least
one processing element, and a data storage element (including volatile and non-
volatile
memory and/or storage elements). These devices may also have at least one
input
device (e.g. a pushbutton keyboard, mouse, a touchscreen, and the like), and
at least
one output device (e.g. a display screen, a printer, a wireless radio, and the
like)
depending on the nature of the device.
16
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
[0127]Some elements that are used to implement at least part of the systems,
methods,
and devices described herein may be implemented via software that is written
in a high-
level procedural language such as object oriented programming. Accordingly,
the
program code may be written in any suitable programming language such as
Python or
C, for example.
Alternatively, or in addition thereto, some of these elements
implemented via software may be written in assembly language, machine language
or
firmware as needed. In either case, the language may be a compiled or
interpreted
language.
[0128]At least some of these software programs may be stored on a storage
media (e.g.
a computer readable medium such as, but not limited to, ROM, magnetic disk,
optical
disc) or a device that is readable by a general or special purpose
programmable device.
The software program code, when read by the programmable device, configures
the
programmable device to operate in a new, specific and predefined manner in
order to
perform at least one of the methods described herein.
[0129] Furthermore, at least some of the programs associated with the systems
and
methods described herein may be capable of being distributed in a computer
program
product including a computer readable medium that bears computer usable
instructions
for one or more processors. The medium may be provided in various forms,
including
non-transitory forms such as, but not limited to, one or more diskettes,
compact disks,
tapes, chips, and magnetic and electronic storage. Alternatively, the medium
may be
transitory in nature such as, but not limited to, wire-line transmissions,
satellite
transmissions, internet transmissions (e.g. downloads), media, digital and
analog
signals, and the like. The computer useable instructions may also be in
various formats,
including compiled and non-compiled code.
[0130]The present disclosure relates in general to a system, method, and
device that
can be used to determine mechanical running power for a user partaking in an
activity.
Mechanical running power generally refers to the power output by a user as
determined
by the forces underfoot while the user is running. This is in contrast to
metabolic running
power, which is calculated by considering the metabolic cost of an activity
using several
factors such as wind resistance for example. Metabolic running power estimates
can be
17
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
inconsistent, as they vary with a user's food consumption, caffeine intake,
sleep, etc.
Mechanical running power, on the other hand, can provide more accurate and
consistent predictions of the power output by a user.
[0131]The motion that a user goes through while running or walking is
typically referred
to as a gait cycle. The gait cycle generally refers to the time period between
the time
when a user's foot contacts the ground and the subsequent time when the same
foot
contacts the ground again. In some cases, the term gait cycle may also refer
to the
events that occur over that time period.
[0132]The term 'stride' is often used to refer to a single gait cycle for one
foot. A stride
can be divided into two phases: a stance phase and a swing phase. The stance
phase
generally refers to the period when the user's foot remains, at least
partially, in contact
with a surface such as the ground. The swing phase generally refers to the
period when
the user's foot is not in contact with the surface (e.g. as the foot swings in
the air
between periods when the foot is in contact with the ground).
[0133]The systems, methods and devices described herein can be used to
determine
mechanical running power using force sensor data from a plurality of force
sensors
positioned underfoot. The force sensor data can be used to determine force
values for a
plurality of strides. Directly measuring the force (or pressure) applied by a
user using
underfoot force sensors (as opposed to deriving the force data from other
sensors such
as accelerometers) can contribute to more accurate power calculations. As used
herein,
the term "force" is used broadly and can refer to raw force (i.e. with units
of N), or
pressure resulting from a raw force (i.e. with units of N/m2).
[0134] For each stride, a corresponding slope, stance time and running speed
can be
calculated. The force values, slope, stance time and running speed can be
input to a
machine learning model. The machine learning model can be trained to output an
estimate of mechanical running power in response to receiving the force
values, slope,
stance time and running speed as inputs. The output from the machine learning
model
can be stored for review or analysis and or provided to the user as dynamic
feedback on
their running power output.
18
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
[0135]The systems, methods and devices described herein can also include one
or
more inertial measurement units (IMUs). Each IMU can be associated with a
corresponding plurality of force sensors. That is, each IMU can be configured
to collect
inertial measurement data relating to movement of the same foot under which
the force
sensors are positioned.
[0136]lMU data from the one or more IMUs can be used to determine one or more
inputs to the machine learning model. For example, the slope and/or the stance
time
and/or the running speed may be determined wholly or partially using IMU data.
These
IMU-derived inputs can be provided to the machine learning model in
conjunction with
the force sensor derived inputs in order to determine the mechanical running
power for
the user.
[0137]The systems, methods and devices described herein can be used to
determine
mechanical running power for a single foot based on sensor data acquired for
that foot.
The machine learning model can be trained to determine the mechanical running
power
for a single foot based on inputs relating to one or more strides for that
foot.
[0138] Determining the mechanical running power for both feet of a user
performing an
activity can include providing inputs to the machine learning model for each
foot. The
inputs provided to the machine learning model can be determined based on
sensor data
acquired for each foot individually.
[0139] Determining the mechanical running power for both feet can also allow
further
gait metrics to be determined. For example, a bilateral power asymmetry can be
calculated by comparing the mechanical running power determined for the left
foot
during a stride to the mechanical running power determined for the right foot
during its
subsequent stride (or vice versa).
[0140]The systems, methods and devices described herein can also be used to
acquire
sensor data for both of a user's feet at the same time. In some cases, this
may require a
separate plurality of force sensors for each foot (e.g. where the force
sensors are
incorporated into a wearable device). Alternatively, a single plurality of
force sensors
19
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
may be used to acquire force sensor data for both feet (e.g. where the force
sensors are
provided by fitness equipment).
[0141]Where an IMU is included in the system and method, a separate IMU can be
provided for each foot. This allows the IMU to collect inertial measurement
data relating
to movement of that foot. The inertial measurement data specific to the foot
can then be
used to determine inputs to the machine learning model for the individual
strides of that
foot. Collecting separate IMU data for each foot may also allow the force
sensor data to
be distinguished for each foot in cases where a single plurality of force
sensors are used
to acquire force sensor data for both feet.
[0142] Referring now to FIG. 1, shown therein is a block diagram illustrating
an example
system 100 that can be used to determine mechanical running power for a user.
System
100 includes a plurality of sensors positioned underfoot of a user performing
an activity
or other type of movement. The sensors may be provided using a wearable device
and/or fitness equipment.
[0143]System 100 includes an input unit 102 (also referred to herein as an
input
device), one or more processing devices 108 (also referred to herein as a
receiving
device or an output device) and an optional remote cloud server 110. As will
be
described in further detail below, the input unit 102 may for example be
combined with,
or integrated into, a carrier unit such as a wearable device or a piece of
fitness
equipment.
[0144]Input unit 102 generally includes a sensing unit 105. The sensing unit
105 can
include a plurality of sensors 106a-106n. The plurality of sensors 106a-106n
can be
configured to collect force sensor data from underneath a user's foot.
[0145]In the example shown, input unit 102 further includes an inertial
measurement
unit (IMU) 112. IMU 112 can include one or more sensors for measuring the
position
and/or motion of the wearable device. For example, IMU 112 may include sensors
such
as one or more of a gyroscope, accelerometer (e.g., a three-axis
accelerometer),
magnetometer, orientation sensor (for measuring orientation and/or changes in
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
orientation), angular velocity sensor, and inclination sensor. Generally, IMU
112 includes
at least an accelerometer. The IMU 112 also typically includes a gyroscope.
[0146]The IMU 112 can also be positioned underneath a user's foot. However,
the IMU
112 need not be positioned underfoot so long as the IMU 112 can collect
inertial
measurement data relating to the position and/or motion of the foot.
[0147]The carrier unit can be configured to position the sensors 106 in
contact with (or
in close proximity to) a user's body to allow the sensors 106 to measure an
aspect of the
activity being performed by the user. The plurality of sensors 106a-106n may
be
configured to measure a particular sensed variable at a location of a user's
body when
the carrier unit is engaged with the user's body (e.g. when the user is
wearing a
wearable device containing the sensors 106 or when the user is using fitness
equipment
containing the sensors 106). In system 100, the plurality of sensors 106a-106n
can be
arranged to measure force underneath the foot (underfoot) of a user.
[0148] In some examples, the carrier unit may include one or more wearable
devices.
The wearable devices can be manufactured of various materials such as fabric,
cloth, polymer, or foam materials suitable for being worn close to, or in
contact with, a
user's skin. All or a portion of the wearable device may be made of breathable
materials to increase comfort while a user is performing an activity.
[0149] In some examples, the wearable device may be formed into a garment or
form
of apparel such as a sock, a shoe, or an insole. Some wearable devices such as
socks may be in direct contact with a user's skin. Some wearable devices, such
as
shoes, may not be in direct contact with a user's skin but still positioned
within
sufficient proximity to a user's body to allow the sensors to acquire the
desired
readings.
[0150]In some cases, the wearable device may be a compression-fit garment. The
compression-fit garment may be manufactured from a material that is
compressive. A
compression-fit garment may minimize the impact from "motion artifacts" by
reducing
the relative movement of the wearable device with respect to a target location
on the
user's body. In some cases, the wearable device may also include anti-slip
21
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
components on the skin-facing surface. For example, a silicone grip may be
provided
on the skin-facing surface of the wearable device to further reduce the
potential for
motion artifacts.
[0151]The wearable device can be worn on a foot. For example, the wearable
device
may be a shoe, a sock, or an insole, or a portion of a shoe, a sock, or an
insole. The
wearable device may include a deformable material, such as foam. This may be
particularly useful where the wearable device is a shoe or insole.
[0152]The plurality of sensors 106a-106n can be positioned to acquire sensor
readings from specified locations on a user's body (via the arrangement of the
sensors on the carrier unit). The sensors 106 can be integrated into the
material of
the carrier unit (e.g. integrated into a wearable device or fitness
equipment).
Alternatively, the sensors 106 can be affixed or attached to the carrier unit,
e.g.
printed, glued, laminated or ironed onto a surface, or between layers, of a
wearable
device or fitness equipment.
[0153]In some examples, the carrier unit may include fitness equipment. The
fitness
equipment may include various types of fitness equipment on which a user can
exert
force with their foot while performing an activity. For example, the carrier
unit may be
fitness equipment such as an exercise mat or a treadmill (e.g. a force-
instrumented
treadmill).
[0154] For clarity, the below description relates to a carrier unit in the
form of an insole.
The insole carrier unit may be provided in various forms, such as an insert
for footwear,
or integrated into a shoe. However, other carrier units may be implemented
using the
systems and methods described herein, such as the wearable devices and fitness
equipment described above. Incorporating the sensing unit 105 (and optionally
the IMU
112) into a carrier unit in the form of a wearable device may be desirable as
it allows
running power to be determined for a user at various locations and without
requiring
specifically configured fitness equipment.
[0155]The below description relates to an insole in which the plurality of
sensors 106
are force sensors. Various types of force sensors may be used, such as force
sensing
22
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
resistors (also referred to as "sensels" or sensing elements), pressure
sensors,
piezoelectric tactile sensors, elasto-resistive sensors, capacitive sensors or
more
generally any type of force sensor that can be integrated into a wearable
device or
fitness equipment capable of collecting force data underfoot.
[0156]The plurality of sensors 106 may be arranged into a sensor array. As
used
herein, the term sensor array refers to a series of sensors arranged in a
defined grid.
The plurality of sensors 106 can be arranged in various types of sensor
arrays. For
example, the plurality of sensors 106 can be provided as a set of discrete
sensors (see
e.g. FIG. 2). A discrete sensor is an individual sensor that acquires a sensor
reading at a
single location. A set of discrete sensors generally refers to multiple
discrete sensors
that are arranged in a spaced apart relationship in a sensing unit.
[0157]Sensors 106a-106n may be arranged in a sparse array of discrete sensors
that
includes void locations where no sensors 106 are located. Alternatively,
sensors 106a-
106n may be arranged in a continuous or dense sensor array in which sensors
106 are
arranged in a continuous, or substantially continuous manner, across the grid.
[0158]Discrete sensors can provide an inexpensive alternative to dense sensor
arrays
for many applications. However, because no sensors are positioned in the
interstitial
locations between the discrete sensors and the void locations external to the
set of
discrete sensors, no actual sensors readings can be acquired for these
locations.
Accordingly, depending on the desired resolution for the force sensor data,
sensor
readings may be estimated (rather than measured) at the interstitial locations
and at the
void locations external to the set of discrete sensors in order to provide
sensor data with
similar resolution to a dense sensor array. Alternatively, where lower
resolution force
sensor data is sufficient, sensor readings may not necessarily be estimated.
[0159]Various interpolation and extrapolation techniques may be used to
estimate
sensor values at interstitial locations and external void locations. In some
cases, sensor
values may be estimated using methods for synthesizing sensor data.
[0160]According to some aspects, a method for synthesizing sensor data can
include:
obtaining a plurality of sensor readings from a corresponding plurality of
sensors, the
23
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
plurality of sensors arranged in a first predetermined pattern, wherein the
first
predetermined pattern maps each of the plurality of sensors to respective
locations on a
wearable device; and based on the plurality of sensor readings and a plurality
of
estimation weights, estimating a plurality of synthesized sensor readings for
a
corresponding plurality of synthesized sensors, the plurality of synthesized
sensors
arranged in a second predetermined pattern, wherein the second predetermined
pattern
maps each of the plurality of synthesized sensors to respective locations on
the
wearable device.
[0161]The plurality of estimation weights can be predetermined in a
preprocessing
phase, and the preprocessing phase can include: obtaining training data, the
training
data including a plurality of sets of physical sensor readings from physical
sensors
arranged according to both the first and second predetermined patterns;
filtering the
training data to obtain filtered training data; using the filtered training
data, computing an
average sensor reading for each physical sensor to produce an input data set
and a
reference data set, the input data set including average sensor readings for
sensors
corresponding to the first predetermined pattern, the reference data set
including
average sensor readings for sensors corresponding to the second predetermined
pattern; and optimizing the estimation weights.
[0162]Optimizing the estimation weights can include: initially estimating the
estimation
weights; computing estimated sensor values based on the input data set and the
estimation weights; and performing gradient descent optimization to update the
estimation weights, where the gradient descent optimization compares error
between
the estimated sensor values and the reference data set.
[0163] Filtering the training data can include resizing each instance of the
training data
to a common size. Filtering the training data can also include: dividing the
training data
into stance data and swing data; and resizing each instance in the set of
stance data to
a common size.
[0164]The plurality of sensor readings can be associated with an activity, and
the
plurality of synthesized sensor readings can be estimated when the activity is
an activity
24
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
requiring more sensors than can be provided by the plurality of sensors in the
first
predetermined pattern. The activity can be running, jogging, walking, or
cycling.
[0165]The method can include predetermining and optimizing estimation weights
associated with a specific activity.
[0166]The first predetermined pattern can include at least 32 locations.
[0167]The first predetermined pattern can include sensors arranged in a 2-3-4-
4-4-3
arrangement in a forefoot portion. The first predetermined pattern can include
sensors
arranged in a 1-1-1 arrangement in a midfoot portion. The first predetermined
pattern
can include sensors arranged in a 2-1-2-1-2-1 arrangement in a heel portion.
[0168]The second predetermined pattern can include at least 32 locations. The
second
predetermined pattern can include at least 68 locations.
[0169]System 100 can be configured to implement a method of determining
mechanical
running power. The method of determining mechanical running power may be
implemented using a controller of the input device 102, a remote processing
device 108,
or cloud server 110.
[0170]As shown in FIG. 1, input unit 102 includes an electronics module 104
coupled
to the plurality of sensors 106 and to optional IMU 112. In some cases, the
electronics module 104 can include a power supply, a controller, a memory, a
signal
acquisition unit operatively coupled to the controller and to the plurality of
sensors
106 (and to IMU 112), and a wireless communication module operatively coupled
to
the controller.
[0171]Generally, the sensing unit refers to the plurality of sensors 106 and
the signal
acquisition unit. The signal acquisition unit may provide initial analog
processing of
signals acquired using the sensors 106, such as amplification. The signal
acquisition
unit may also include an analog-to-digital converter to convert the acquired
signals
from the continuous time domain to a discrete time domain. The analog-to-
digital
converter may then provide the digitized data to the controller for further
analysis or
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
for communication to a remote processing device 108 or remote cloud server 110
for
further analysis.
[0172]Optionally, the electronics module 104 may include a controller or other
processing device configured to perform the signal processing and analysis. In
such
cases, the controller on the electronics module may be configured to process
the
received sensor readings in order to determine synthesized sensor readings. In
some cases, the controller may be coupled to the communication module (and
thereby the sensing unit) using a wired connection such as Universal Serial
Bus
(USB) or other port.
[0173]The electronics module 104 can be communicatively coupled to one or more
remote processing devices 108a-108n, e.g. using a wireless communication
module
(e.g., Bluetooth, Bluetooth Low-Energy, WiFi, ANT+ IEEE 802.11, etc.). The
remote
processing devices 108 can be any type of processing device such as (but not
limited to) a personal computer, a tablet, and a mobile device such as a
smartphone,
a smartwatch or a wristband. The electronics modules 104 can also be
communicatively coupled to remote cloud server 110 over, for example, a wide
area
network such as the Internet.
[0174]Each remote processing device 108 and optional remote cloud server 110
typically includes a processing unit, an output device (such as a display,
speaker,
and/or tactile feedback device), a user interface, an interface unit for
communicating
with other devices, Input/Output (I/O) hardware, a wireless unit (e.g. a radio
that
communicates using CDMA, GSM, GPRS or Bluetooth protocol according to
standards
such as IEEE 802.11a, 802.11b, 802.11g, or 802.11n), a power unit, and a
memory unit.
The memory unit can include RAM, ROM, one or more hard drives, one or more
flash
drives or some other suitable data storage elements such as disk drives, etc.
[0175] The processing unit controls the operation of the remote processing
device 108
or the remote cloud server 110 and can be any suitable processor, controller
or digital
signal processor that can provide sufficient processing power processor
depending on
the desired configuration, purposes and requirements of the system 100.
26
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
[0176]The display can be any suitable display that provides visual
information. For
instance, the display can be a cathode ray tube, or a flat-screen monitor and
the like if
the remote processing device 108 or remote cloud server 110 is a desktop
computer. In
other cases, the display can be a display suitable for a laptop, tablet or
handheld device,
such as an LCD-based display and the like.
[0177]System 100 can generally be used for determining running power based on
sensor readings received from a plurality of sensors positioned underfoot. In
some
cases, system 100 may also track additional data derived from the sensor
readings.
The sensor readings, running power data, and derived data may be monitored,
.. stored, and analyzed for the user. Aspects of the monitoring, storage and
analysis of
biometric features and other metrics may be performed by one or more of the
input
unit 102, and/or a remote processing device 108, and/or the cloud server 110.
For
example, a non-transitory storage memory of one or more of the input unit 102,
and/or a remote processing device 108, and/or the cloud server 110 can store a
machine learning model trained to predict mechanical running power.
[01781A remote cloud server 110 may provide additional processing resources
not
available on the input unit 102 or the remote processing device 108. For
example,
some aspects of processing the sensor readings acquired by the sensors 106 may
be delegated to the cloud server 110 to conserve power resources on the input
unit
102 or remote processing device 108. In some cases, the cloud server 110,
input unit
102 and remote processing device 108 may communicate in real-time to provide
timely feedback to a user regarding the sensor readings, running power data,
and
other related data.
[017911n the example system 100 illustrated in FIG. 1, a single input unit 102
is
shown. However, system 100 may include multiple input units 102 associated
with
the same user. For example, system 100 may include two separate input units
102,
each input unit 102 associated with one of the user's legs. Sensor data from
an
individual input unit 102 may be used to determine the mechanical running
power for
the user's corresponding leg.
27
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
[0180]In some examples, system 100 may include a separate sensing unit 105
(and
optionally a separate IMU 112) for each foot of a user. This may allow the
mechanical
running power to be determined separately for each of the user's feet.
[0181]Alternatively, a single sensing unit 105 may be used to acquire force
sensor data
for both feet of a user. This may be the case where the sensing unit 105 is
incorporated
into fitness equipment such as an exercise mat or treadmill. In such cases,
the force
sensor data acquired by the sensing unit 105 may be associated with individual
feet
through further processing by electronics module 104 and/or processing device
108.
[0182]The IMU 112 is associated with a single foot. Accordingly, separate IMUs
112
may be provided for both feet. IMU data acquired by the IMU 112 associated
with each
foot may be used to associate the force sensor data acquired by a single
sensing unit
105 with the corresponding foot.
[0183]Certain hardware and software features of the system may be enabled,
disabled,
or changed. For example, a user may enable or disable certain sensors 106.
Enabling or
disabling certain features may be desirable if the user has a foot condition
which inhibits
them from activating certain sensors, such as a broken or missing toe.
Sampling rate
may also be modifiable. Sampling rate may be modified to minimize processing
time and
to save memory, or to increase data output to gain deeper insights. The
location of
processing (the input unit 102, the remote processing device 108, or the cloud
server
110) may also be changed. If additional sensors are included in the wearable
device
(e.g. IMU, temperature sensors, and/or GPS), certain sensor types may be
enabled or
disabled. For example, a GPS system can be disabled to conserve battery power
of a
carrier unit, if a user operates the carrier unit while running on a treadmill
at home.
[0184] Referring now to FIG. 2, shown therein is an example of an insole 200
that
includes a sensing unit 202. The insole 200 is an example of an input device
102 that
may be used in the system 100 shown in FIG. 1. The insole 200 may be the
footwear
insert described in PCT Application No. PCT/CA2020/051520 published on May 20,
2021. The insole 200 may be an Orpyx SIC) Sensory Insole sold by Orpyx Medical
Technologies Inc.
28
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
[0185] The insole 200 includes a sensor unit 202 and an optional liner 204.
The liner
204 can provide a protective surface between the sensor unit 202 and a user's
foot.
The liner 204 may have a slightly larger profile as compared to the sensor
unit 202.
That is, the outer perimeter 203 of the sensor unit 202 may be inwardly spaced
from
the outer perimeter 205 of the liner 204 by an offset 208. The offset 208 may
be
substantially consistent throughout the perimeter of the sensor unit 202 such
that the
sensor unit 202 is completely covered by the liner 204.
[0186]Optionally, the sensor unit 202 can include an IMU (not shown). The
sensor
unit 202 can also include a connector 206. The connector 206 may provide a
coupling interface between the plurality of sensors 106 (and the optional
inertial
measurement unit) and an electronics module (not shown) such as electronics
module 104. The coupling interface can allow signals from the sensors 106
and/or
IMU to be transmitted to the electronics module. In some cases, the coupling
interface may also provide control or sampling signals from the electronics
module to
the sensors 106 and/or IMU.
[0187]Optionally, the sensor unit 202 can also include one or more temperature
sensors (not shown) and/or a GPS system (not shown).
[0188]The arrangement of sensors 106 in the sensor unit 202 is an example of a
sparse sensor array that may be used to collect force sensor data. In
alternative
examples, various different types of force sensors, force sensor arrays, and
arrangements of force sensors may be used. For example, sensor units
containing a
dense force sensor array (e.g. a Pedare insole or Tekscan system) may also be
used.
[0189] Referring now to FIG. 3, shown therein is an example method 300 for
determining
running power using force sensor data from a plurality of force sensors
positioned
underfoot. The method 300 may be used with a plurality of sensors configured
to
measure human movement or human activity, such as sensors 106. Method 300 is
an
example of a method for determining running power in which a machine learning
model
is used to determine the running power based on sensor readings acquired from
the
plurality of force sensors.
29
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
[0190]At 310, a plurality of sensor readings can be obtained from a
corresponding
plurality of sensors. The plurality of sensors can be positioned underfoot
(i.e. underneath
the foot) of a user performing a physical activity. The plurality of sensors
can include a
plurality of force sensors configured to acquire force sensor data.
[0191]The force sensors can be positioned at specified locations on a carrier
unit such
as a wearable device or a piece of equipment. The force sensors can be
configured to
measure force data relating to human activity. As shown in FIG 2, the
plurality of
sensors may be force sensors provided at various locations of an insole. The
force
sensors can measure force applied to the insole during physical activities,
such as
walking, running, jumping, or gaming.
[0192]The plurality of sensors may also include one or more IMUs. Accordingly,
the
plurality of sensor readings acquired at 310 can include IMU sensor data
received from
the one or more IMUs.
[0193] Each inertial measurement unit (IMU) can be associated with the
plurality of force
sensors. For example, the IMU may be incorporated into the same wearable
device as
the plurality of force sensors. More generally, the IMU can be configured to
collect IMU
sensor data about a single foot of a user. This IMU sensor data can be
acquired for the
same foot for which the sensor readings were obtained from the plurality of
force
sensors.
[0194]The sensor readings acquired at 310 may be acquired as a time-continuous
set
of sensor readings. This may provide a time-continuous set of sensor data that
can be
used to determine the mechanical running power as a time-continuous running
power
and/or an average power value for each stride. Depending on the nature of the
sensors
and the signal preprocessing performed, the time-continuous sensor data may be
discretized, e.g. using an analog to digital conversion process. Even where
the sensor
data is discretized, the set of sensor data may allow the mechanical running
power to be
determined as a (discretized) time-continuous running power and/or an average
power
value for each stride.
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
[0195]The sensor readings may be acquired over the course of a plurality of
strides
taken by a user. The sensor readings can be used to determine various data
associated
with individual strides.
[0196]The plurality of strides can be identified using data from the sensor
readings.
Each stride may be identified based on times when the user's foot first
contacts the
ground (a foot-contact time) and/or times when the user's foot leaves the
ground (a foot-
off time).
[0197] Each stride can be defined to correspond to a stride period. The stride
period
generally refers to the time period over which a single gait cycle extends.
The endpoints
of the stride period may vary in different implementations of method 300. For
example,
the stride period can be defined as the length of time between adjacent foot-
contact
times. Alternatively, the stride period can be defined as the length of time
between
adjacent foot-off times.
[0198] A plurality of foot-contact times can be determined based on the sensor
readings.
As noted above, the plurality of foot-contact times may be used to determine
the stride
period for each stride as the time between adjacent foot-contact times.
Various
techniques may be used to identify the foot-contact time. For example, the
foot-contact
times can be determined using the force sensor readings acquired from the
plurality of
force sensors.
[0199] Alternatively or in addition, a plurality of foot-off times can be
determined based
on the sensor readings. As noted above, the plurality of foot-off times may be
used to
determine the stride period for each stride as the time between adjacent foot-
off times.
Various techniques may be used to identify the foot-off time. For example, the
foot-off
times can be determined using the force sensor readings acquired from the
plurality of
force sensors.
[0200]According to some aspects, a method for determining a foot-contact time
and a
foot-off time from a plurality of force sensors positioned underfoot can
include: obtaining
a sensor signal dataset based on sensor readings from the plurality of force
sensors
during a first time period, wherein the sensor signal dataset defines a series
of signal
31
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
values extending over the first time period; based on the series of signal
values,
identifying a pair of interstride interval periods within the first time
period, the pair of
interstride interval periods including a foot contact interstride interval and
a foot off
interstride interval, wherein each interstride interval includes a subset of
signal values
from the series of signal values; identifying a foot-contact time by
identifying an inflection
point in the subset of signal values corresponding to the foot contact
interstride interval;
and identifying a foot-off time by identifying an inflection point in the
subset of signal
values corresponding to the foot off interstride interval.
[0201]The method can further include identifying a foot contact period as a
time period
extending between the foot-contact time and foot-off time identified for the
pair of
interstride intervals, and outputting the foot contact period.
[0202] For each interstride interval, identifying the inflection point in the
corresponding
subset of signal values can include: identifying a threshold crossing value in
the subset
of signal values for that interstride interval; dividing the interstride
interval into a plurality
of segments; identifying a transition signal value in the subset of signal
values for that
interstride interval, where the transition signal value is identified at a
transition point
between adjacent segments in the plurality of segments; tracing a unity line
between the
threshold crossing value and the transition signal value, the unity line
identifying a series
of unity line values within the interstride interval; and identifying the
inflection point as a
point of maximum difference between the unity line values and the subset of
signal
values.
[0203]The foot contact interstride interval can be identified by: identifying
a pair of
subsequent positive threshold crossings in the series of signal values, where
each
positive threshold crossing is identified as a point in the first time period
where the series
of signal values is increasing and crosses a specified threshold value; and
defining the
foot contact interstride interval as a first interstride period extending
between the pair of
subsequent positive threshold crossings.
[0204]The threshold crossing value for the foot contact interstride interval
can be
identified at the second positive threshold crossing in the pair of subsequent
positive
threshold crossings.
32
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
[0205]The transition signal value for the foot contact interstride interval
can be identified
at the transition point at the beginning of the last segment in the plurality
of segments.
[0206]The foot off interstride interval can be identified by: identifying a
pair of
subsequent negative threshold crossings in the series of signal values, where
each
negative threshold crossing is identified as a location in the first time
period where the
series of signal values is decreasing and crosses a specified threshold value;
and
defining the foot off interstride interval as a second interstride period
extending between
the pair of subsequent negative threshold crossings.
[0207]The threshold crossing value for the foot off interstride interval can
be identified at
the first negative threshold crossing in the pair of subsequent negative
threshold
crossings.
[0208]The transition signal value for the foot off interstride interval can be
identified at
the transition point located at the end of the first segment in the plurality
of segments.
[0209]The method can include classifying the sensor signal dataset as a
running
dataset or a walking dataset based on the series of signal values; and
determining a
number of segments in the plurality of segments based on the classification of
the
sensor signal dataset. In response to classifying the sensor signal dataset as
a running
dataset, the number of segments can be determined to be 3 segments.
[0210]The specified threshold value can be defined as 50% of the maximum
signal
value. The first time period can be at least 5 seconds.
[0211]The method can include filtering the sensor signal dataset prior to
identifying the
pair of interstride intervals. The filtering can include applying a low-pass
filter to the
sensor signal dataset.
[0212]The method can include classifying the sensor signal dataset as a
running
dataset or a walking dataset prior to filtering the data.
[0213]The method can include identifying a plurality of foot contact periods
using a
rolling window as the first time period.
33
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
[0214]The method can include calculating one or more temporal gait metrics
using the
series of signal values corresponding to the plurality of foot contact
periods.
[0215]At 320, force values can be determined from the sensor readings acquired
at
310. The force values can be determined for a plurality of strides. Separate
force values
may be determined for each stride in the plurality of strides.
[0216]As explained above, the plurality of strides may be identified based on
corresponding stride periods. Force values may be determined for each stride
based on
the sensor readings acquired at 310 relating to the corresponding stride
period.
[0217]The force values may be determined based on aggregate force data for the
corresponding stride. The sensor readings may include corresponding force
sensor
values from each of the force sensors in the plurality of force sensors. The
sensor
readings may include a plurality of corresponding force sensor values from
each of the
force sensors in the plurality of force sensors at various time points
throughout the time
period (e.g. as time-continuous sensor readings or sensor readings at discrete
time
steps). The aggregate force data may be determined based on the sensor values
received from multiple force sensors in the plurality of force sensors over
the stride
period.
[0218]Determining the force values can include determining at least one mean
force
value. Each mean force value can be determined as the arithmetic mean of the
sensor
values received from multiple force sensors.
[0219]A single mean force value can be determined for the entire stride
period.
Determining the single mean force value may include calculating an arithmetic
mean
across multiple sensor values received from each force sensor. This may be the
case
where, for example, the machine learning model is configured to determine an
average
running power for the entire stride period.
[0220]Alternatively or in addition, a separate mean force value may be
determined for
each time (or time step) in the stride period. Determining the mean force
value for each
time may include calculating an arithmetic mean across the sensor values
received from
34
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
each force sensor at that time. This may be the case where, for example, the
machine
learning model is configured to determine a time-continuous running or where
the
machine learning model is configured to determine an average running power for
the
entire stride period.
[0221]Alternatively or in addition, determining the force values can include
determining
at least one peak force value. Each peak force value can be determined as the
peak or
greatest value of the sensor values received from multiple force sensors.
[0222]A single peak force value can be determined for the entire stride
period.
Determining the single peak force value may include determining the peak or
greatest
value across multiple sensor values received from each force sensor. This may
be the
case where, for example, the machine learning model is configured to determine
an
average running power for the entire stride period.
[0223]Alternatively or in addition, a separate peak force value may be
determined for
each time (or time step) in the stride period. Determining the peak force
value for each
time may include determining the peak or greatest value across the sensor
values
received from each force sensor at that time. This may be the case where, for
example,
the machine learning model is configured to determine a time-continuous
running power
or where the machine learning model is configured to determine an average
running
power for the entire stride period.
[0224]The aggregate force data may be determined based on the sensor values
from all
of the force sensors in the plurality of force sensors. Thus, the
corresponding aggregate
force values (e.g. mean force value and/or peak force value) may be determined
for the
entire sensor array.
[0225]Alternatively, the aggregate force data can be separated into a
plurality of foot
regions 220 (see e.g. FIG. 2). Each force sensor 106 in the plurality of force
sensors can
be assigned to a corresponding foot region 220. The aggregate force data for
each foot
region 220 can then be determined based on the sensor readings from the force
sensors 106 corresponding to that foot region 220.
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
[0226]The sensor unit can be separated into a plurality of foot regions in
various
different ways. In the example shown in FIG. 2, the sensing unit 200 has been
separated into five different foot regions 220a-220e in the anterior-posterior
direction.
However, the number and configuration of foot regions can vary.
[0227] If the aggregate force data is separated into a plurality of foot
regions, the force
values may be determined as region-specific force values. Accordingly,
determining the
at least one mean force value can include determining a region-specific mean
force
value for each foot region. Similarly, determining the at least one peak force
value can
include determining a region-specific peak force value for each foot region.
[0228] At 330, a slope value, stance time value and running speed value can be
determined for each stride. The slope value, stance time value and running
speed value
may be determined using the sensor readings acquired at 310. Alternatively or
in
addition, some of the slope value, stance time value and running speed value
may be
determined based on data acquired from external devices, such as a treadmill
on which
a user is running.
[0229]Determining the slope value may allow the mechanical running power to be
determined for a user over a variety of slopes/inclines. The slope value may
be
determined in various ways. For example, where a user is running or walking on
a
treadmill, the slope value may be determined based on the incline setting of
the
treadmill. The incline setting of the treadmill may be determined
automatically, e.g. by
connecting a processing device 108 to the treadmill using a wired or wireless
communication interface and accessing the incline setting through the
connection (e.g.
through an application distributed in association with the treadmill).
Alternatively, the
incline setting may be input by a user, e.g. through an application running on
processing
device 108.
[0230]Alternatively, the slope value may be determined based on inertial
measurement
data received from an inertial measurement unit. The inertial measurement unit
may be
provided by an input unit 102 worn by a user (e.g. as part of a wearable
device in the
form of a sock, shoe or insole).
36
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
[0231]The slope value may be determined from the inertial measurement data in
various ways. For example, the slope value may be determined by estimating a
step
height value for each stride. The step height may be determined by integrating
the stride
velocity (determined from the inertial measurement data) over the
corresponding stride
period. The result of the integration provides a stride displacement that
includes
components in each of the x, y, and z directions. The vertical component Adz
and
horizontal component Ady of the stride displacement may be used to determine
the
slope value 0 according to:
A d
tan0 = z
Ady
[0232]Determining the slope value by estimating a step height value may be
particularly
desirable for users who are moving at lower speeds (e.g. walking).
[0233]Alternatively, the slope value may be determined by estimating a gravity
vector
value associated with the stride. A foot flat time can be identified for the
stride. The
direction of the gravity vector can be determined from the inertial
measurement data.
The direction of the gravity vector at the foot flat time can then be used to
determine the
slope value.
[0234] Determining the slope value based on the gravity vector value can
include an
initial calibration process in which the inertial measurement data is
calibrated with the
user's foot on flat ground. The difference in orientation between the gravity
vector during
the IMU calibration and the gravity vector at the foot flat time during the
stride can then
be used to determine the slope value.
[0235]The method for determining the slope value may be selected based on user-
specific calibration data provided to a processing device 108. In some cases,
the
method used to determine the slope value may be selected based on the foot
strike
pattern for a given user.
[0236]Certain foot strike patterns may make it more difficult to determine the
foot flat
time. For example, forefoot strike runners may not go through a foot flat time
during a
37
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
stride. In such cases, determining the slope value based on a gravity vector
value may
be less accurate. Accordingly, an alternative method of determining the slope
value may
be selected for users identified as forefoot strike runners.
[0237]Alternatively or in addition, the slope value may be determined based on
force
sensor data from the plurality of force sensors. For example, heel-strike
runners
experience less force in the heel region while running uphill as compared to
running on
flat ground. Accordingly, changes in the slope value may be determined by
identifying
trends in the force values. The changes in the slope value may be used to
determine the
slope value, for instance based on an initial calibration when running on flat
ground. For
example, the slope value may be determined using a machine learning model
trained
using known slope values (e.g. from training data acquired while a user is
running on a
treadmill) and force values for a user wearing insoles equipped with force
sensors.
[0238]The stance time value for each stride can be identified as the length of
the
corresponding stance phase. As noted above, the stance phase refers to the
period
during which a user's foot remains (at least partially) in contact with the
ground.
Accordingly, the stance time value can be identified as the length of a foot
contact
period between the foot contact time (for that stride) and the foot off time
(for that stride).
[0239]As noted above, various techniques may be used to identify the foot
contact time
and the foot-off time.
[0240]The running speed value may be determined in various ways. For example,
where a user is running or walking on a treadmill, the running speed value may
be
determined based on the speed setting of the treadmill.
[0241 ]Alternatively, the running speed value may be determined based on
sensor data
received at 310. For each stride, the running speed may be determined based on
a
stride time and a stride length of the stride period corresponding to that
stride. The stride
time and stride length may be determined using the sensor data acquired at
310. An
example process 400 for determining the running speed using sensor data is
described
in further detail herein below with reference to FIG. 4.
38
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
[0242]At 340, the force values (from 320) and the slope value, stance time
value and
running speed value (from 330) can be input to a machine learning model. The
machine
learning model can be trained to predict the mechanical running power in
response to
receiving the force values, slope value, stance time value, and running speed
value as
inputs. Accordingly, the mechanical running power associated with the
plurality of
sensor readings can be determined by the machine learning model.
[0243]The mechanical running power may be defined in various ways. For
example, the
mechanical running power may be defined to include an average positive power
and an
average negative power for each stride. Positive running power generally
refers to the
portion of running power that is propulsive (at the end of the stance phase).
Negative
running power generally refers to the portion of running power that is
absorptive (at the
start of the stance phase). The machine learning model can be trained to
output the
average positive power and the average negative power for each stride based on
the
force values, slope value, stance time value, and running speed value
determined for
the entire stride.
[0244]Alternatively or in addition, the mechanical running power can be
determined as a
time-continuous mechanical running power. The time-continuous mechanical
running
power may be defined using a fully continuous time scale or a discretized time
scale that
includes time steps.
[0245]The machine learning model can be trained to output the time-continuous
running
power based on the force values, slope value, stance time value, and running
speed
value corresponding to each point in time or time step within a given stride.
This may
provide more granular data for feedback and analysis with the trade-off of
requiring
increased computational expense. In such cases, the force values can be
determined as
time-continuous force values that are input to the machine learning model.
[0246]Various different types of machine learning models may be used to
determine the
mechanical running power. A regression model can be used as the machine
learning
model. For example, the regression model may be a LASSO model. This can
provide a
computationally inexpensive method of determining the mechanical running
power,
39
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
which may be particularly beneficial in providing real-time, or near real-
time, feedback to
the user.
[0247]Alternatively, a different type of machine learning could be used to
determine the
mechanical running power, such as, for example a neural network, a gradient
boosted
decision tree, and so on.
[0248]The machine learning model can be trained using training data that
includes the
set of inputs (e.g. the force values, slope value, running speed value and
stance time
value) determined from sensor data. The inputs can be determined based on
training
sensor readings from a plurality of force sensors positioned underfoot and an
IMU (as at
310) and preprocessing of the training sensor readings as described at steps
320 and
330 above (and FIG. 4 below).
[0249]The training data can also include measured data representing the force
output
by a user. The measured data can be defined as desired outputs for the machine
learning model. For example, a training mechanical running power can be
determined
using a force-instrumented treadmill. A force-instrumented treadmill can
include force
plates that allow the average power to be computed using a physics-based
approach
(as described in the Examples section below). The treadmill also allows the
running
speed to be known from the operating settings of the treadmill while the
training data is
collected (which may be used to evaluate a method of determining the running
speed,
such as method 400 described herein below).
[0250]The training data can be collected from one or more users performing an
activity.
Once the machine learning model is trained using the training data, the
machine
learning model can be applied to determine the mechanical running power of the
same
or different users performing an activity.
[0251]The training process may vary depending on the type of machine learning
model
being implemented. For example, with a LASSO regression model, an optimization
algorithm can be applied to optimize the LASSO regression coefficients. The
optimization algorithm may employ a cost function based on the difference
between the
desired outputs (as calculated from the treadmill force data) and the model
outputs
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
calculated from the given inputs. The trained model can then be used, as at
340, to
determine the mechanical running power based on preprocessed data from the
plurality
of force sensors (and the I M U).
[0252] Different combinations of input data may be used to train and implement
the
machine learning model. For example, the machine learning model may be trained
to
receive 13 inputs, namely a running speed value, a slope value, a stance time
value,
five peak force values (e.g. one for each region 220) and five mean force
values (e.g.
one for each region 220).
[0253]Alternatively, a different number of inputs may be used. For example, a
different
number of inputs may be used when the foot is segmented into a different
number of
regions 220. Alternatively or in addition, a different number of inputs can be
used where
additional inputs (e.g. acceleration, estimated mechanical running power) are
included in
addition to the example inputs described herein above.
[0254] For example, an estimated mechanical running power may be calculated
directly
based on the sensor readings from the force sensors and an IMU. The estimated
mechanical running power may be calculated as an average estimated mechanical
power value for the stride. The average estimated mechanical power can include
separate values for the average estimated positive power and the average
estimated
negative power for a given stride. Each estimated mechanical running power
value can
be provided to the machine learning model as an additional input (e.g. along
with the
running speed value, slope value, stance time value, and force values).
[0255]An estimated mechanical running power signal can be calculated according
to:
P=vv* vGRF +v ap * apGRF
where vv represents the vertical velocity as calculated from the sensor data
acquired at
310 (e.g. the insole IMU data and force sensor data), v0p represents the
anterior-
posterior velocity as calculated from the sensor data acquired at 310 (e.g.
the insole
IMU data and force sensor data), vGRF represents the vertical ground reaction
force as
calculated from the insole sensor data using a recurrent neural network with
force,
41
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
speed, and slope inputs, and apGRF represents the anterior-posterior ground
reaction
force as calculated from the insoles using a recurrent neural network with
force, speed,
and slope inputs. The average estimated positive power and the average
estimated
negative power can then be determined from the estimated mechanical running
power
signal.
[0256]The average estimated positive power (Pa+vtg) for a stride can be
determined by
integrating only the positive position of the estimated mechanical running
power signal
for that stride, and then dividing the integration result by the length (t+66)
of the positive
portion of the estimated mechanical running power signal. In equation form,
the average
estimated positive power for a stride can be determined according to:
vt-f P
avg
[0257]Similarly, the average estimated negative power (Pavtgt) for a stride
can be
determined by integrating only the negative position of the estimated
mechanical
running power signal for that stride, and then dividing the integration result
by the length
(C") of the negative portion of the estimated mechanical running power signal.
In
equation form, the average estimated negative power for a stride can be
determined
according to:
D "'P
avg
[0258]The estimated mechanical running power can be calculated based on the
force
values and components of the stride velocity. For instance, a vertical
velocity component
and an anterior-posterior velocity component can be used to calculate the
estimated
mechanical running power. A vertical velocity and an anterior-posterior
velocity can be
determined for each stride based on the inertial measurement data. The
velocity
components can be determined based on the inertial measurement data received
from
the IMU and the force sensor data received from the force sensors, for
instance as
described herein below at step 430 of method 400.
42
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
[0259]The estimated mechanical running power can be calculated using a
vertical
ground reaction force and an anterior-posterior ground reaction force
determined from
the force values. For example, the estimated mechanical running power can be
calculated according to:
P=v,* vGRF +vap*apGRF
where v, represents the vertical velocity, vap represents the anterior-
posterior velocity,
vGRF represents a vertical ground reaction force determined from the force
values, and
apGRF represents an anterior-posterior ground reaction force determined from
the force
values.
[0260] The vertical ground reaction force and anterior-posterior ground
reaction force
can be calculated from the force sensor data in various ways. For example, the
force
sensor data can be provided as an input to a machine learning model configured
to
output the vertical ground reaction force and anterior-posterior ground
reaction force.
Additional inputs may also be provided to the machine learning model in order
to
determine the vertical ground reaction force and anterior-posterior ground
reaction force,
such as user mass, user speed, and user slope for example. Methods for
determining
the vertical ground reaction force and anterior-posterior ground reaction
force may utilize
a machine learning model.
[0261]According to some aspects, a method for determining a vertical and
anterior-
posterior ground reaction force using force sensor data from a plurality of
force sensors
positioned underfoot can include: obtaining force sensor data for a plurality
of specified
foot regions based on sensor readings from the plurality of force sensors
during a first
time period; based on the force sensor data, identifying at least one foot
contact period
within the first time period; obtaining a user mass, a user speed, and a user
slope
associated with the force sensor data for the first time period; and for each
foot contact
period, determining a corresponding vertical ground reaction force signal and
a
corresponding anterior-posterior ground reaction force signal based on the
user mass,
the user speed, the user slope, and the force sensor data for the plurality of
specified
foot regions during the corresponding foot contact period.
43
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
[0262] For each foot contact period, determining the corresponding vertical
ground
reaction force signal and the corresponding anterior-posterior ground reaction
force
signal can include inputting the user mass, the user speed, the user slope,
and the force
sensor data for the plurality of specified foot regions during the
corresponding foot
contact period to a neural network trained to output the corresponding
vertical ground
reaction force signal and the corresponding anterior-posterior ground reaction
force
signal.
[0263]The neural network can be a recurrent neural network. The recurrent
neural
network can have 9 layers.
[0264]The neural network can be trained by: obtaining concurrent force sensor
training
data and ground reaction force measurement training data; obtaining user mass
training
data, user speed training data, and user slope training data corresponding to
the force
sensor training data; defining the ground reaction force measurement training
data as
desired output data; inputting the force sensor training data, the user mass
training data,
the user speed training data, and the user slope training data to the neural
network to
cause the neural network to output predicted data; and training the neural
network to
minimize a cost function determined based on a difference between the desired
output
data and the predicted data.
[0265]The neural network can be a recurrent neural network and the final layer
of the
recurrent neural network can be a regression output layer and training the
neural
network to minimize the cost function can include optimizing the regression
output layer
to minimize a mean square error of the difference between the desired output
data and
the predicted data.
[0266]The regression output layer can be optimized using Adam optimization.
[0267]The method can include separating the force sensor data for each
specified foot
region into a distinct region dataset; and providing each region dataset to
the neural
network as a separate input.
44
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
[0268]The force sensor data can be acquired from a particular user; the neural
network
model can be initially trained using the force sensor training data, ground
reaction force
measurement training data, user mass training data, user speed training data,
and user
slope training data for a plurality of users; and the neural network can be
enhanced for
the particular user prior to determining the corresponding vertical ground
reaction force
signal and the corresponding anterior-posterior ground reaction force signal.
[0269]The neural network can be enhanced by: obtaining concurrent user-
specific force
sensor training data and user-specific ground reaction force measurement
training data
for the particular user; obtaining user-specific mass training data, user-
specific speed
training data, and user-specific slope training data for the particular user
corresponding
to the user-specific force sensor training data; defining the user-specific
ground reaction
force measurement training data as user-specific desired output data;
inputting adjusted
training data to the neural network to cause the neural network to output user-
specific
predicted data, wherein the adjusted training data includes the user-specific
force
sensor training data, the user-specific mass training data, the user-specific
speed
training data, and the user-specific slope training data in addition to the
force sensor
training data, the user mass training data, the user speed training data, and
the user
slope training data; and re-training the neural network to minimize the cost
function
determined based on a user-specific difference between the adjusted desired
output
data and the user-specific predicted data, wherein the adjusted desired output
data
includes the desired output data and the user-specific desired output data.
[0270]The plurality of specified foot regions can include 5 foot regions.
[0271]The force sensor data can be obtained as continuous-time force sensor
data over
the first time period.
[0272]Alternatively or in addition, some of the inputs may be adjusted or
modified. For
example, acceleration may be used as an input in place of peak force values.
[0273]At 350, the mechanical running power determined by the machine learning
model
at 340 can be output. This may provide a user with data representing the power
output
by their foot when performing an activity like running or walking. The output
data can
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
provide the user with insight into their level of performance while performing
movement
or an activity. The output data can additionally be used as an input to a
game.
[0274]The mechanical running power can be output directly through an output
device to
provide a user with feedback on the activity being monitored. For example, the
running
speed may be transmitted to a mobile application on the user's mobile device
(e.g. a
processing device 108). This may allow the user to continuously improve
mechanical
efficiency while performing the activity. For example, the mechanical running
power can
be used for zone training. Zone training is a type of athletic training which
encourages
users to keep their metrics within a range or "zone" of values over a
predetermined
period of time (e.g. the length of a game). Users may be shown their position
in a zone
in real-time and may be rewarded for staying within the zone and/or penalized
for
leaving the zone. For example, a user may be given a mechanical running power
zone
to stay within for a running game. During the game, the user may be encouraged
to
keep their mechanical running power in the designated zone to achieve maximum
points. The mechanical running power may be transmitted to a mobile
application on the
user's mobile device (e.g. a processing device 108). Alternatively or in
addition, the
mechanical running power may be stored, e.g. for later review, comparison,
analysis, or
monitoring.
[0275]The method 300 generally describes the process of determining mechanical
running power for one leg. Optionally, method 300 may be applied to determine
the
mechanical running power for both of a user's legs, based, respectively, on
data (e.g.
sensor readings and IMU data) collected for each leg. Method 300 may be
performed
concurrently on the data collected for each leg in order to provide a user
with real-
time feedback of the mechanical running power generated by each leg.
[0276] Referring now to FIG. 4, shown therein is an example method 400 of
determining
running speed that may be used with a method of determining mechanical running
power such as method 300. The method 400 may be applied to determine the
running
speed for an individual stride.
[0277]The method 400 is an example of a method for determining running speed
using
sensor data from an IMU in conjunction with sensor data from a plurality of
force sensors
46
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
positioned underfoot. Using IMU sensor data in combination with force sensor
data to
determine running speed can help improve the accuracy of the running speed
estimation. Furthermore, using the IMU sensor data in combination with force
sensor
data can allow running speed to be estimated accurately across the variety of
terrain
.. that a user may encounter while performing an activity such as running.
[0278]At 410, a stride period can be identified for the corresponding stride.
The stride
period may be determined between the foot-contact times of adjacent strides
(i.e. the
stride period for a given stride begins at the time the user's foot first
touches the ground
and ends at the time immediately before the user's foot first touches the
ground again
after the swing phase of the same stride) or between the foot-off times of
adjacent
strides (i.e. the stride period for a given stride begins at the time the
user's foot first
leaves the ground following the stance phase of the preceding stride and ends
at the
time immediately before the user's foot first leaves the ground again after
the stance
phase of the given stride).
[0279]Accordingly, in order to identify the stride period a plurality of foot-
contact times or
a plurality of foot-off times can be identified. In some cases, both the
plurality of foot-
contact times and the plurality of foot-off times can be determined. The foot-
contact
times and/or foot-off times can be determined based on the sensor data
received at 310,
described herein above. As noted above, the foot-contact times and/or foot-off
times can
be determined based on the force sensor data received at 310 from the
plurality of force
sensors.
[0280]At 420, a stride time can be identified. The stride time can be
identified as the
length of time or duration of the corresponding stride period identified at
410.
Accordingly, the stride time can be identified as the length of time between
the foot-
contact times of adjacent strides or between the foot-off times of adjacent
strides.
[0281]At 430, a stride length can be identified. The stride length for the
stride period
(from 410) can be determined based on inertial measurement data received from
an
inertial measurement unit (e.g. as at 310). The stride length can represent
the distance
travelled by the foot over the course of the corresponding stride.
47
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
[0282] Each stride can include a foot contact period (i.e. a period during
which the user's
foot remains on the ground). The foot contact period can be identified as the
period of
time between the foot-contact time and the foot-off time identified for that
stride. The foot
contact periods for multiple strides can be used to determine the stride
length for each
stride.
[0283]A stride velocity can be determined for a given stride based on the
inertial
measurement data received from the IMU. The stride velocity can be determined
using
accelerometer data from the IMU. The accelerometer data can be integrated over
the
stride period to obtain the velocity.
[0284] In some cases, the accelerometer data may need to be transformed to a
global
reference frame prior to integration. Accordingly, determining the stride
velocity can
include transforming the accelerometer data to a global reference frame in
order to
generate transformed accelerometer data. The stride velocity can then be
determined by
integrating the transformed accelerometer data over the stride period.
[0285]The accelerometer data may be transformed to a global reference frame
based
on the angle of the user's foot when the accelerometer data is acquired. The
foot angle
can be determined using angular velocity data received from the IMU. To
determine the
foot angle, a subsequent foot flat time can be determined for the subsequent
stride (i.e.
the stride immediately succeeding the current stride). A foot flat time can
also be
determined for the current stride.
[0286]In some cases, each foot flat time may be determined using the inertial
measurement data. For example, the inertial measurement data can include
gyroscope
data. A time of minimum gyroscope energy (the time when all three axes of the
gyroscope have the least amount of measured angular rotation) for a given
stride can be
determined from the gyroscope data. The foot flat time can be identified as
the time of
minimum gyroscope energy for the corresponding stride.
[0287]Alternatively, the foot flat time may be determined based on the force
sensor
readings from the plurality of force sensors. The foot flat time can be
identified as the
48
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
time of maximal plantar force for the corresponding stride as determined from
the sensor
readings.
[0288] The foot angle can then be determined by integrating the angular
velocity data
from the foot-contact time to the foot flat time of the subsequent stride
(i.e. from the foot-
contact time of the current stride through the foot-flat and foot-off times of
the current
stride and through the foot-contact time of the subsequent stride to the next
foot-flat
time). The accelerometer data can then be transformed to the global reference
frame
based on the foot angle.
[0289]The foot flat time can also be used to determine whether the stride
velocity
includes any drift offset. A foot flat velocity value can be determined at the
foot flat time
of the current stride based on the integrated transformed accelerometer data.
The stride
velocity can then be adjusted based on the foot flat velocity value.
[0290]The velocity of a user's foot should be zero at the foot-flat time.
Accordingly, if the
stride velocity determined from integrating the accelerometer data is not zero
at the foot
flat time, the stride velocity at the foot-flat time can be identified as a
drift offset.
Accordingly, the drift offset can be subtracted from the stride velocity of
the entire stride
in order to adjust the stride velocity.
[0291]Once the stride velocity has been determined, it can be used to
determine the
stride displacement. The stride velocity can be integrated over the length of
time
between adjacent foot contact periods (e.g. from one foot-contact time to the
next) to
determine the corresponding stride displacement. The stride length can then be
determined based on a combination of the directional components of the stride
displacement. The directional components of the displacement can be root-
summed-
squared to calculate the stride length.
[0292]At 440, the running speed can be determined for the stride period. The
running
speed can be determined by dividing the stride length (from 430) by the stride
time (from
420). The running speed may then be provided as an input to the machine
learning
model at 340 to determine the mechanical running power.
49
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
[0293]Optionally, the running speed can be output directly through an output
device to
provide a user with feedback on the activity being monitored. For example, the
running
speed may be transmitted to a mobile application on the user's mobile device
(e.g. a
processing device 108), allowing the user to continuously monitor and improve
while
performing the activity. Like mechanical running power, running speed or pace
can be
used for zone training. A user may select or be assigned a pace zone to
maintain during
an activity, such as a running game (where pace is the inverse of running
speed).
During the game, the user will be encouraged to keep their pace in the
designated zone
to achieve maximum points. Alternatively or in addition, the running speed may
be
stored, e.g. for later review, comparison, analysis, or monitoring.
[0294]The mechanical running power, the running speed, or some combination
thereof
may be outputs (i.e. an output dataset) of the system.
[0295]The outputs may be used as inputs to a game. A game could be a running
training game, where a user is incentivized to improve their running technique
and
achieve new training goals. Alternatively, a game could be a non-running
focused game
that requires a user to run at select times in the game (e.g. a treasure hunt
game where
a user must run from a pirate during one level).
[0296]The way a user runs in real life may be recreated in a game, based on
their
outputs. For example, an avatar in a game may run at the same speed that a
user runs
in real life. The user may need to adjust their speed in real life in order to
dodge
obstacles in a game or to run along certain sections of a virtual track.
[0297]Virtual environments, objects, and avatars may be generated, with which
a user
can interact. The virtual environment and virtual objects can be altered based
on the
movements, gestures, and outputs of the user. Output devices (e.g. a
television screen,
a virtual reality headset, etc.) may be used to display the virtual
environment to users. A
user may visit a variety of virtual environments, including imaginary
environments or
environments that replicate real-life environments (e.g. the Boston Marathon
route).
When a user moves around while wearing the carrier unit, they will move around
in and
interact with the virtual environment accordingly.
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
[0298]A scaling factor may be applied to outputs in a game. The scaling factor
may be
an integer (e.g. 1, 2, 5, 10, etc.) or it may not be an integer (e.g. 0.2,
1.5, 2.6, 6.9, etc.).
In one example, the scaling factor may be 1. In this case, the outputs are
applied
equivalently in a game (i.e. a 1:1 scaling). The running speed of an avatar in
the game is
equivalent to the running speed of a user in real life. In another example,
the scaling
factor may be 5. In this case, outputs are scaled 1:5 from real life to the
game. The
running speed of an avatar in the game is five times faster than the running
speed of a
user in real life. Gaming experiences that are directly based on a user's
outputs allow
users to have a more realistic and immersive gaming experience than games that
are
not based on a user's biometric data (e.g. games played with buttons on a
controller).
Output scaling may allow for superhuman performance enhancements in a game.
For
example, an avatar whose running speed is scaled by a scaling factor of 10 may
be able
to outrun a car in a game, but an avatar whose running speed is scaled by a
scaling
factor of 1 may not be able to outrun it. Different scaling factors may also
be applied to
different outputs. For example, a scaling factor of 2 may be applied to the
running
speed, but a scaling factor of 0.5 may be applied to mechanical running power.
(An
avatar may be able to run faster and longer than a real life user if the
mechanical
running power is scaled).
[0299]Outputs may also be applied to different environmental factors in a
game. For
example, the gravity in a game can be changed. The gravity can be changed to
that of
another planet, such as the gravity of Mars. The outputs can be applied to the
new
environmental factors, so a user can understand how they might perform in a
different
environment. The performance of the user under the original conditions and the
simulated conditions can be shown on a visual display.
[0300]The virtual environment can display or generate an avatar representing
the
portion of a user's body to which the carrier unit is affixed. For example, if
the carrier unit
is a pair of insoles, a user's feet may be rendered in the virtual
environment. A user may
be able to change the skins and/or shoes applied to their feet in the virtual
environment.
For example, a user may choose a special type of running shoe to be shown in
the
virtual environment. Special objects and/or abilities may be associated with
the virtual
skins and shoes. For example, virtual lasers or swords may extend from the
virtual
51
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
shoes that can be used to fight villains in a game. As another example,
virtual shoes
may contain a special feature, where they can build up energy if a user
performs a
certain task or reaches certain goals. The built-up energy can be used to
create a burst
of power for an avatar in a running game.
[0301]Alternatively, the virtual environment can display or generate an avatar
for the
user's entire body. The appearance of the avatar's body may depend on the
user's
outputs. For example, if fast running speeds are frequently recorded for a
user, their
avatar may appear aerodynamic (e.g. wearing tight clothing). An avatar's
appearance
may also be location dependent. For example, if a user lives in a warm, dry
climate, the
avatar may be depicted in shorts and a t-shirt, with dried sand on their skin.
Alternatively, if a user lives in the Arctic, their avatar may be depicted in
a parka and
furry boots. There may be location-dependent virtual items that can be
unlocked. For
example, if a user travels to another country in real life, they may unlock a
special
running shoe from that country. The carrier unit may contain a GPS system or
another
location-sensing system to enable the location-dependent items and features to
be
unlocked.
[0302]The outputs may also be used to model the dynamics of virtual objects
and/or
surroundings within a game, with which the user interacts. For example, if a
user
increases their running speed, their virtual surroundings in a game will
appear to move
faster by them. If a user's running avatar collides with a pylon while
running, the pylon
will be thrown in a trajectory defined by the user's running outputs.
[0303]Additionally, the outputs may be used to control a character in a
lifestyle game.
These games may require a user to virtually embody a certain lifestyle and
complete
tasks involved with the lifestyle. For example, a user may embody the
lifestyle of an
Olympic runner in a game. The user will be required to train like an athlete,
and the
outputs can be used to determine if the user has successfully completed the
training.
They may also be required to complete other tasks relating to the lifestyle of
an Olympic
athlete, such as taking rest days, taking part in competitions, achieving
sponsorships,
going on press tours, going grocery shopping, etc.
52
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
[0304]The system may also contain safety features to prevent collisions while
running or
gaming, using sensors and/or software included in the carrier unit. In one
embodiment,
wherein the carrier unit is an insole, software for the Bluetooth system may
detect if a
user's pair of insoles is in close proximity to another user's pair of
insoles. Alternatively,
GPS systems embedded in the insoles and optionally other objects may detect if
a user
is at risk of colliding with another user or object. The system may alert
users if they are
at risk of a collision. For example, if two running users approach the same
street corner
from different directions, they may be alerted to a possible collision. In
another example,
if a running user approaches a crosswalk at the same time as a car (a user in
the car
may be wearing a pair of sensorized insoles or the car's GPS system may
communicate
with the running user's insoles), the running user and/or the driving user may
be alerted
of a potential collision.
[0305]The system may be paired with other carrier devices in gaming scenarios.
For
example, the insoles may be paired with other wearable devices, such as wrist-
worn
IMUs. A gaming platform comprising multiple wearable game controllers at
different
locations on the body can encourage users to engage with a game using their
full body,
which may increase their workout and fitness during a game. The system may
also be
paired with fitness equipment. For example, the insoles can be paired with a
treadmill for
a running game. The incline of the treadmill can change in response to
different virtual
terrains (e.g. running up a virtual mountain), and the user's outputs, as
determined from
the insoles, can determine how they are performing in the game. Visual display
carrier
units, such as VR headsets, smart glasses, and smart goggles, may also be
paired with
the insoles to increase the immersivity of games.
[0306] The system may also contain additional sensor types, whose data can be
used to
augment gaming experiences. In particular, temperature sensors may provide
various
advantages for health and gaming applications. The system may include one or
more
temperature sensors used to measure body or environmental temperature. In a
first
example, one or more temperature sensors (e.g. thermistors) may be included in
a
flexible printed circuit within the bulk of the insole. The one or more
temperature sensors
can detect temperature changes from the body. The temperature changes may be
used
in an algorithm that adjusts other sensor (e.g. force sensor) readings to
account for
53
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
temperature drift. Alternatively, the one or more temperature sensors may be
used to
measure the body temperature of users for health and gaming calculations (e.g.
calorie
burn calculations or task readiness calculations). In another example, the one
or more
temperature sensors may be affixed to the outside of the shoe or at other
locations away
from a user's body to determine the external temperature. The external
temperature
may be used in gaming to send safety messages and notifications to users (e.g.
if the
external temperature is hot, a user may receive a notification suggesting they
hydrate
more frequently). The external temperature may also be used to adjust health
and
gaming calculations and may be used to adjust the virtual environment in a
game (e.g. if
the external temperature is hot, the game may place the user in a virtual
desert).
[0307]Additionally, outputs may contribute to scoring in a game. For example,
a
performance score may be calculated from the outputs. Improvements in the
outputs
may result in an increase in the number of points earned in a game,
incentivizing users
to improve and safely push their limits when running.
[0308]The outputs and performance scores can be accumulated based on a user's
participation in various activities providing a standardized way to
incentivize or reward
users for movement in non-gaming (e.g. athletic) and gaming activities.
[0309]One or more normalization factors may be defined to allow the outputs
and/or
performance scores to be determined fairly for different users. Normalization
factors may
be applied to account for factors such as mass, weight, age, gender, natural
athletic
ability, game skill, other physical characteristics, or any combinations
thereof.
[0310]The calculation of performance scores can also include modification
factors such
as multipliers and bonuses for successful completion of objectives including
streaks,
skillful movement combinations, and/or other unique game experiences such that
performing the same in-game action may not yield the same performance scores
each
time.
[0311]The performance scores and/or outputs can also be used to determine
other
gaming-related metrics for a user. For example, a user can be associated with
one or
more user levels. The user levels generally refer to the experience of a user
within a
54
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
game. User levels may be used to compare users to one another, or to establish
progression in fitness and experience over time.
[0312] The performance score and the outputs may also be used to assign and to
track
progress towards achieving training goals within a predetermined time period.
For
example, based on a user's mechanical running power values over one week, a
training
goal can be generated for the user to achieve the same or greater mechanical
running
power values the subsequent week. The user's mechanical running power can then
be
tracked the subsequent week to determine the user's percentage of progress
towards
achieving the training goal.
[0313]Training goals can relate to accumulated performance scores, the
outputs,
system usage metrics, or some combination thereof, that should be achieved in
a
predetermined time period (session, day, week, month, year, season, etc.) or
instantaneous values (i.e. a rate) that should be achieved at a certain point
in time.
Training goals may be suggested by the processing system based on previous
activities,
be chosen by the user, or be presented as part of a challenge from another
user or
group of users. Suggested training goals can become increasingly targeted for
users as
additional sensor data is collected by the system over time.
[0314]Training goals can be directed toward weight loss. Wherein the carrier
unit is an
insole containing force sensors, body weight or mass can be measured by the
insoles.
Alternatively, an external device may be used to measure body weight or mass
and
transmit the values to the input device 102, remote processing device 108, or
cloud
server 110. If a user has a training goal to lose a certain amount of weight,
the
processing system may recommend certain activities to help them accomplish
their goal.
In particular, the processing system may recommend fitness-related games that
can be
played with the carrier unit. For example, for an overweight user, the system
may
suggest low impact, high calorie burning running games. The system may create
a
fitness-based game schedule for the user to follow, to encourage increased
activity and
intensity as the user's body weight or mass decreases (i.e. as their
percentage of
progress towards achieving the training goal increases). The system may also
include a
virtual coach to help the user in their weight loss journey. A user may
participate in
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
virtual weight loss groups and/or rooms to encourage participation and support
through
interacting with other users with similar training goals. Weight loss may also
be
encouraged through badges, virtual gifts, streaks, and other virtual
achievements.
[0315] Training goals may also be directed toward education. Specific games
and
activities may integrate educational concepts (e.g. a running game that helps
users
learn a new language). The same social interactions and virtual achievements
in the
weight loss example may also apply to a user's journey with an educational
goal.
[0316]Additionally, the outputs may also be used to assess a user's technique
when
performing an activity or movement (i.e. their quality of movement). Wherein
the carrier
unit is an insole containing force sensors, a user's outputs may be recorded
and stored
in the system memory for an activity, such as running. As further data is
collected for the
user, the system may compare previous data against new data to determine
differences
in technique to notify the user of fatigue or of a potential injury.
Alternatively, the system
may compare data contralaterally (i.e. between opposing limbs) to determine
differences
in technique. To assess technique, a machine learning model may be trained on
data
that includes both "correct" and "incorrect" versions of an activity. In
implementation, the
model can then classify an activity as "correctly" or "incorrectly" performed.
Alternatively,
the model can be trained on data that includes rankings (e.g. by a clinician
or sports
scientist) on technique of certain activities (e.g. a 0 to 5 ranking, where 0
indicates that
an activity was poorly executed and where 5 indicates that an activity was
perfectly
executed). In implementation, the system can reject exercise tasks below a
certain
ranking and/or output the ranked value. In another example, technique can be
assessed
based on conditions or restrictions set for each activity. For example, if
running is the
task being assessed, there may be a cut-off bilateral asymmetry value (e.g. no
more
than 5% difference between the two feet). A user's outputs can be used to
determine if
the condition was met. If the user does not meet the condition or restriction,
their
technique may be deemed unacceptable.
[0317]In a further example, the outputs may also be used to determine a user's
"readiness" to participate in a game or activity. For example, a user's
historical output
data for 10 km runs may be used to determine a user's readiness to run a
marathon.
56
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
Alternatively, other data that can be collected from the insoles (e.g. center
of pressure,
ground reaction force, etc.) can be used to determine a person's readiness to
participate
in a run or a running game.
[0318]Task readiness refers to a user's ability to perform a task at a moment
in time. It
is assessed by having a user perform a task, such as a jump, squat, balance,
sprint,
series of steps, or another physical exercise. The exercise may be included as
part of a
game or challenge or may be separate from game play. Injury potential,
technique,
and/or fatigue state of the user may be incorporated in a task readiness score
or may be
pulled out of the task readiness score and displayed as a separate score. The
task
readiness, injury potential, technique, and/or fatigue state scores may be
recorded over
time and may be displayed in a metrics report. The metrics report may be used
to
quantify improvements and overall fitness. The real-time readiness scores of
the user
may be reported to the user on the input device 102, remote processing device
108, or
cloud server 110. For example, on a display of the remote processing device, a
poor
task readiness score may be reported as a red bar, an average task readiness
score as
a yellow bar, and a good task readiness score as a green bar in the top corner
of the
display. The task readiness feedback may alert the user to a deteriorating
quality of their
movements, which can be used to make an informed decision on continuation of
game
play. The task readiness scores may be used to recommend games that are
appropriate
for the user's physical state (e.g. their fitness level) at a certain point in
time. For
example, consistently high task readiness scores over a period may indicate
that a user
should play more advanced games to improve their fitness level. The system may
recommend more advanced games to the user or higher-level players to compete
against. The task readiness scores may also be used to recommend rest periods
for the
user or to coach the user through auditory means, visual means, tactile means,
or any
combination thereof. For example, a virtual coach may be used to instruct the
user on
how to improve movement quality to gain more points, prevent injury, or
achieve another
goal in the game.
[0319]A virtual coach may be used to assist a user with meeting their training
goals. The
virtual coach may be trained through machine learning or other algorithms to
give
suggestions, notifications, and encouragement to the user relating to the
training goal.
57
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
Alternatively, a personal trainer, physiotherapist or other expert in the
field may assess a
user's historical outputs to develop and suggest training goals and paths to
achieving
training goals within the game.
[0320] Feedback may also be provided to users based on their outputs. For
example, a
user may be assigned a goal of staying within a certain mechanical running
power zone
for a run or series of runs. If the user begins to fall outside of that zone,
they may be
alerted. Feedback may be provided in the form of haptic feedback, such as with
vibrational motors embedded in the carrier unit.
[0321] Feedback may also be provided in the form of an audio signal. A user's
outputs
may be sonified and played in real time or post-activity for the user. For
example, if a
user goes on a run, their mechanical running power can be sonified and played
in real
time. The user can then sonically identify changes in their mechanical running
power,
and they can make real time adjustments to their running technique to maintain
or
improve their performance. Signal processing techniques may be used to
increase the
effects of sonification. For example, signals may be amplified, such that the
sonification
spans a broader range of tones than an unamplified signal, which may make it
easier for
users to identify changes in tone. Signals may also be layered. For example,
the signals
from the right and left foot may be added together prior to sonification, or
the
sonifications from the right and left foot may be played simultaneously.
Signals may also
be filtered to minimize noise, which may be distracting to a user once the
signal is
sonified. Visual feedback may also be provided by the system.
[0322] Users may review their feedback and data (e.g visualizations,
sonifications, and
haptics) during or after an activity. Real-time feedback may encourage users
to continue
to engage with the activity at a higher level of intensity or to increase
their intensity.
Post-activity data reviews may encourage users to understand their activity
and
movement statistics to prepare for improvements in the next activity.
[0323] Sonification of outputs may also be used for artistic purposes. For
example, the
output metrics may correspond to certain musical features, such as notes,
instruments,
tempos, and volumes. In one embodiment, running speed may control the tempo of
the
music. As a user runs faster, the tempo of the music will increase. Mechanical
running
58
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
power may control the volume of the music. As a user's mechanical running
power
increases, so will the volume of the music. Users may work together to create
music. For
example, if two users go running, one user's sonification may create a melody
and the
other user's sonification may create a harmony. In this regard, users can
generate music
in real time with their bodies. Similarly, users, such as DJs, may be able to
mix music in
real time. For example, a DJ may run on a treadmill at a concert while wearing
the
insoles, and by changing their running technique, they can cue tracks and
increase or
decrease the speed of tracks.
[0324]The outputs may also be used to create visualizations. Visualizations
may be
data driven (e.g. graphs) or artistic visuals (e.g. data-inspired paintings).
For example, a
user may be able to "paint" with their insoles by changing their running
technique, and
their foot gestures while running may be used to create different "brush
strokes". A large
display screen may be used to show a user's outputs while they are running,
racing, or
gaming.
[0325]Additionally, information may be communicated to and/or between users
through
visual, audio, or haptic cues. For example, the system may send a haptic cue
to a user's
insoles to prompt them to complete a daily challenge based on their output
data. The
results of their daily challenges may be compared with the results of other
users.
Alternatively, if cues are sent between users, a first user in a game can
challenge a
second user in a game to perform an activity by sending a haptic cue to the
second
user's carrier unit. The communicated information may be based upon the two
users'
outputs. For example, the first user may send a haptic cue to the second user
to
challenge them to a run, where the user with the highest mechanical running
power
during the run will be declared the winner.
[0326] Users may also be able to create levels or challenges for other users
based on
their outputs. For example, a first user may create a challenge that involves
running up a
steep hill. If a second user wishes to perform the challenge, they will be
required to
produce outputs (such as mechanical running power) that indicate that they
performed
an equivalently challenging or more challenging task (e.g. running faster up a
hill of the
same incline).
59
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
[0327] The performance scores and/or outputs may be displayed on an output
device,
as part of the remote processing device 108 or cloud server 110. A user may
also be
able to interact with a visual display via an interactive medium (e.g. a
touchscreen) on
the output device. Examples of data visualizations that may be provided on the
visual
display based on the performance scores and/or outputs of a user using the
carrier unit
include: foot pressure maps to show the pressure distribution on the insoles,
foot
pressure maps to show the movement of the center of pressure, points displays
(e.g.
performance score), pop-up notifications of errors in movement, pop-up
notifications
with suggestions to correct the movement, graphs showing changes in data over
time,
colour codes (e.g. different colour pop-ups for different performance scores
or gestures),
footprints whose shapes are estimated based on a user's outputs, footprints
whose
depths are estimated based on a user's outputs, cumulative displays (e.g.
accumulation
of mechanical running power, which, when a certain level is reached, may be
used to
provide a burst of power for an avatar in a game), or combinations thereof.
The data
visualizations may be altered or enabled or disabled by users, with toggles,
buttons, or
other actions.
[0328]The output device may also display information (such as names, outputs,
etc.) of
other users in the same area using the same type of system. Carrier units may
contain
GPS systems or other location-sensing systems to enable viewing information of
other
users in the same area. Location-sensing may provide opportunities for virtual
social
interactions between users. Examples of social interactions include gift
exchanges,
meet-ups in virtual rooms, messaging, game challenges, cooperative games,
competitive games, combination games (i.e. games with a competitive and
cooperative
aspect), tournaments, leaderboards (e.g. for age groups, geographic locations,
specific
games, etc.), and the ability to "follow" and/or "friend" other users (i.e.
adding users to a
list of "friends" on the system platform). Other virtual social interactions
known in the art,
but not listed here, may also be included.
[0329]Virtual meeting rooms are digital areas where users may send messages or
chats
with one another, play games together, and participate in social interactions
with other
users. The system may have virtual meeting rooms available, or users may
create and
design their own virtual meeting rooms. The owner of a virtual meeting room
may allow
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
open access to the virtual meeting room, or they may restrict access to
certain users.
The owner may invite users to join their virtual meeting room.
[0330]Social interactions may also include competitive races against the same
user (i.e.
their previous metrics), other users, a "computer", celebrities and/or
professionals. For
example, a user may enable a "ghost" mode, where they can view their previous
performances when repeating an activity, to compete against themselves. For
example,
in a game where a user is required to perform a running task, they can view a
"ghost" of
their avatar's best performance while repeating the activity, along with a
display window
showing the ghost's outputs, to encourage the user to match or improve the
run. In
another example, in the running game, the user may enable "ghost" mode to view
the
outputs of a 5000 meter professional runner, who recorded their outputs in the
game for
other users to copy. The user can work towards matching the professional
runner's data
to improve their own performance. In another example, a professional runner
may
create a virtual competition where users can compete against the professional
for a
month-long running challenge. The participating users' outputs can be compared
to the
professional's outputs to determine if any of the users beat the professional.
Users who
participate in and/or win the challenge may receive a virtual reward.
[0331]Although the systems, methods and devices have been described primarily
in the
context of activities such as running or walking, the systems, methods and
devices
described herein can be applied in the context of various activities in which
a user takes
a series of strides (e.g. sports, occupational activities, military
performance etc.).
EXAMPLES
[0332]An implementation of the systems and methods described herein was
tested. In
particular, an implementation of method 300 using a plurality of force sensors
and an
IMU mounted in an insole and a LASSO model trained to generate an average
positive
running power and an average negative running power for a plurality of strides
using 13
inputs (a running speed value, a slope value, a stance time value, five region-
specific
peak force values and five region-specific mean force values) was tested to
evaluate the
accuracy of determining the mechanical running power. The mechanical running
power
61
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
generated by the tested implementation was compared against the average
positive and
negative power of strides as calculated by a force-instrumented treadmill.
[0333]Running data was collected from 16 users, for a total of 19,262 strides.
A physics-
based approach was used to calculate mechanical running power from the sensor
readings acquired from the plurality of force sensors. The physics-based
approach was
defined using the following equation:
P=vv* vGRF +vap*apGRF
[0334]where vv represents the vertical velocity as calculated from the insole
IMU, v0p
represents the anterior-posterior velocity as calculated from the insole IMU,
vGRF
represents the vertical ground reaction force as calculated from the insoles
using a
recurrent neural network with force, speed, and slope inputs, and apGRF
represents the
anterior-posterior ground reaction force as calculated from the insoles using
a recurrent
neural network with force, speed, and slope inputs.
[0335] Figure 5A-5B illustrates the results of determining the mechanical
running power
using the physics-based approach as compared to the mechanical running power
determined by the force-instrumented treadmill. FIG. 5A illustrates a plot of
the average
positive running power of strides (in Watts/kg) while FIG. 5B illustrates a
plot of the
negative running power of strides.
[0336]The example implementation with a LASSO regression model was then
applied to
determine the mechanical running power for the same set of data. Figure 6A-6B
illustrates the results of determining the mechanical running power using an
example
implementation of methods 300 and 400 as compared to the mechanical running
power
determined by the force-instrumented treadmill. In the example implementation
of
methods 300 and 400, the slope value was determined from the setting of the
force-
instrumented treadmill. FIG. 6A illustrates a plot of the average positive
running power of
strides (in Watts/kg) while FIG. 6B illustrates a plot of the negative running
power of
strides.
62
Date Regue/Date Received 2022-09-27
KTX0001 CADOO
[0337]Table 1 shows the overall accuracy of determining the average positive
power
and the average negative power using the physics-based approach (the results
shown
in FIGS. 5A and 5B) and using the example implementation of the methods 300
and 400
(the results shown in FIGS. 6A and 6B). For each method, Table 1 shows the
root-mean
square error (RMSE), correlation and mean absolute percentage error (MAPE) of
determining the average positive power and the average negative power. All of
the
values in Table 1 are shown as mean values standard deviations. As table 1
shows,
the example implementation of methods 300 and 400 provides increased accuracy
even
as compared to a physics-based approach in which the vertical and anterior-
posterior
ground reaction forces were determined using a recurrent neural network.
Average Positive Power Average Negative Power
Correlation RMSE MAPE Correlation RMSE MAPE
[W/kg] [%] [W/kg] [%]
Physics-Based 0.93 0.06 1.52 15.31 0.97 0.02 1.93 12.85
Approach 0.54 5.95 0.95
6.34
LASSO Regression 0.95 0.03 1.23 13.02 0.97 0.22 1.87 13.10
Model 0.56 6.96 0.93
6.60
[0338]An implementation of method 400 using a plurality of force sensors and
an IMU
mounted in an insole was also tested to evaluate the accuracy of determining
the
running speed. The results generated by the implementation of method 400 were
also
compared to the running speed generated using IMU data alone. Both methods
were
tested across a series of different running slopes.
[0339] Test data were collected from 18 users running on a force-instrumented
treadmill
(Bertec, Columbus, OH, USA) while wearing an implementation of the sensor unit
200
including an IMU and a plurality of force sensors. Each user ran on the
treadmill with the
incline set to level ground followed by uphill on a 10.5% grade and downhill
on a 10.5%
grade. Each user ran on level ground at 9.4 km/hr, 10.8 km/hr, 12.2 km/hr and
13.7
63
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
km/hr, uphill at 9.4 km/hr, 10.1 km/hr and 10.8 km/hr, and downhill at 9.4
km/hr, 10.1
km/hr and 10.8 km/hr and 12.2 km/hr for one minute each.
[0340] Running speed was then computed using the implementation of method 400
and
a method based on IMU data alone for each user using the sensor data from the
implementation of the sensor unit. The mean absolute percent errors for each
running
speed and slope were then computed to evaluate both running speed algorithms
against
the treadmill belt speed setting. The running speed percent errors for both
methods
were then statistically compared with paired t-tests.
[0341]FIG. 7A illustrates a plot of the estimated running speed as computed by
the
implementation of method 400 across the different test speeds and slopes (for
a total of
over 21,000 strides). The plot shown in FIG. 7A also includes the mean running
speed
and standard deviation of the running speed calculated from the plurality of
test subjects
400 across the different test speeds and slopes. For each setting, the
treadmill speed
setting is indicated by the horizontal black line.
[0342]FIG. 7B illustrates a plot of the mean absolute percent error and
standard
deviations in error of the estimated running speed computed by the
implementation of
method 400 (shown by the bars without hatching) and the method based on IMU
data
alone (shown by the bars with hatching).
[0343]As shown in the plot of FIG. 7B, the example implementation of method
400
produced an average error below 4.3%. For all tested speeds and slopes, the
example
implementation of method 400 insole reduced the average percent error as
compared to
the method based on IMU data alone by up to 29% (p < 0.001, Fig. 7B). For user
subjects, the error was reduced up to 54%.
[0344] Regardless of the speed or slope, the example implementation of method
400
had a lower running speed error than the method based on IMU data alone.
Additionally,
example implementation of method 400 had an average error of 3.0% on level
ground,
which is better than existing IMU-only based algorithms (3.5 - 6.9%) (see, for
example,
M. Zrenner, S. Gradl, U. Jensen, M. Ullrich, and B. M. Eskofier, "Comparison
of Different
64
Date Recue/Date Received 2022-09-27
KTX0001 CADOO
Algorithms for Calculating Velocity and Stride Length in Running Using
Inertial
Measurement Units," Sensors, vol. 18, no. 12, Nov. 2018, doi:
10.3390/s18124194).
[0345]The improved accuracy provided by the example implementation of method
400
can allow users to better track their workouts and overall performance.
Accurate running
speed estimates can also be used to compute additional biomechanical metrics
such as
ground reaction forces and running power.
[0346]While the above description provides examples of one or more processes
or
apparatuses or compositions, it will be appreciated that other processes or
apparatuses
or compositions may be within the scope of the accompanying claims.
[0347]To the extent any amendments, characterizations, or other assertions
previously
made (in this or in any related patent applications or patents, including any
parent,
sibling, or child) with respect to any art, prior or otherwise, could be
construed as a
disclaimer of any subject matter supported by the present disclosure of this
application,
Applicant hereby rescinds and retracts such disclaimer. Applicant also
respectfully
submits that any prior art previously considered in any related patent
applications or
patents, including any parent, sibling, or child, may need to be re-visited.
Date Regue/Date Received 2022-09-27
