Language selection

Search

Patent 3040851 Summary

Third-party information liability

Some of the information on this Web page has been provided by external sources. The Government of Canada is not responsible for the accuracy, reliability or currency of the information supplied by external sources. Users wishing to rely upon this information should consult directly with the source of the information. Content provided by external sources is not subject to official languages, privacy and accessibility requirements.

Claims and Abstract availability

Any discrepancies in the text and image of the Claims and Abstract are due to differing posting times. Text of the Claims and Abstract are posted:

  • At the time the application is open to public inspection;
  • At the time of issue of the patent (grant).
(12) Patent Application: (11) CA 3040851
(54) English Title: MULTI-WAVELENGTH ENDOSCOPIC SYSTEM AND IMAGE PROCESSING METHOD USING SAME
(54) French Title: SYSTEME ENDOSCOPIQUE A LONGUEURS D'ONDE MULTIPLES ET PROCEDE DE TRAITEMENT D'IMAGE UTILISANT LEDIT SYSTEME ENDOSCOPIQUE A LONGUEURS D'ONDE MULTIPLES
Status: Deemed Abandoned and Beyond the Period of Reinstatement - Pending Response to Notice of Disregarded Communication
Bibliographic Data
(51) International Patent Classification (IPC):
  • A61B 01/06 (2006.01)
  • A61B 01/00 (2006.01)
  • A61B 01/04 (2006.01)
  • A61B 01/31 (2006.01)
(72) Inventors :
  • MYUNG, SEUNG-JAE (Republic of Korea)
  • KIM, SANG-YEOB (Republic of Korea)
  • BAE, SANG MUN (Republic of Korea)
  • DO, EUN-JU (Republic of Korea)
  • BAE, DONG-JUN (Republic of Korea)
(73) Owners :
  • THE ASAN FOUNDATION
(71) Applicants :
  • THE ASAN FOUNDATION (Republic of Korea)
(74) Agent: SMART & BIGGAR LP
(74) Associate agent:
(45) Issued:
(86) PCT Filing Date: 2017-10-18
(87) Open to Public Inspection: 2018-04-26
Examination requested: 2019-04-16
Availability of licence: N/A
Dedicated to the Public: N/A
(25) Language of filing: English

Patent Cooperation Treaty (PCT): Yes
(86) PCT Filing Number: PCT/KR2017/011522
(87) International Publication Number: KR2017011522
(85) National Entry: 2019-04-16

(30) Application Priority Data:
Application No. Country/Territory Date
10-2016-0135305 (Republic of Korea) 2016-10-18

Abstracts

English Abstract

The present invention relates to a multi-wavelength endoscope system for photographing an observation site marked with a plurality of fluorescent materials having different colors, the system comprising: a photographing unit for polarizing incident light reflected at the observation site in a first direction and a second direction perpendicular to the first direction, dividing a spectral region of the incident light polarized in the first direction and the second direction into a plurality of spectral channels, and measuring light intensity of each of the plurality of spectral channels to obtain image data; and a computing unit for storing a single fluorescence spectrum extracted from sample image data photographed by singly processing the observation site with each of the fluorescent materials, and separating and outputting the image data photographed by the photographing unit by using the single fluorescence spectrum so that only one of the plurality of fluorescent materials is displayed.


French Abstract

La présente invention concerne un système d'endoscope à longueurs d'onde multiples destiné à photographier un site d'observation marqué avec une pluralité de matériaux fluorescents de couleurs différentes, le système comprenant : une unité de photographie destinée à polariser la lumière incidente réfléchie au niveau du site d'observation dans une première direction et une seconde direction perpendiculaire à la première direction, à diviser une région spectrale de la lumière incidente polarisée dans la première direction et la seconde direction en une pluralité de canaux spectraux, et à mesurer l'intensité lumineuse de chaque canal spectral parmi la pluralité de canaux spectraux en vue d'obtenir des données d'image ; et une unité informatique destinée à stocker un spectre de fluorescence unique extrait de données d'image d'échantillon photographiées par traitement unique du site d'observation avec chacun des matériaux fluorescents, et à séparer et à délivrer en sortie les données d'image photographiées par l'unité de photographie à l'aide du spectre de fluorescence unique de telle sorte que seulement un matériau fluorescent parmi la pluralité de matériaux fluorescents est affiché.

Claims

Note: Claims are shown in the official language in which they were submitted.


WHAT IS CLAIMED IS:
1. A multi-wavelength endoscopic system for imaging an
observation site labeled with a plurality of fluorescent
materials having different colors, comprising:
an imaging unit configured to acquire image data by
polarizing incident light reflected from the observation site
in a first direction and a second direction perpendicular to
the first direction, dividing a spectrum region of the incident
light polarized in the first direction and the second direction
into a plurality of spectrum channels and measuring an
intensity of light for each of the spectrum channels; and
a computing unit configured to store a single
fluorescence spectrum extracted from sample image data obtained
by single-treating the observation site with each of the
fluorescent materials and configured to separate and output the
image data obtained in the imaging unit using the single
fluorescence spectrum so that each of the fluorescent materials
is displayed separately.
2. The
system of claim 1, wherein the imaging unit includes:
a beam splitter configured to polarize the incident light
in the first direction and the second direction perpendicular
to the first direction;
a first area filter positioned in a path of a light beam
split in the first direction and configured to pass a light
beam falling within a predetermined spectral range;
-24-

a second area filter positioned in a path of a light beam
split in the second direction and configured to pass a light
beam falling within a predetermined spectral range;
a first area camera configured to measure an intensity of
the light beam passing through the first area filter; and
a second area camera configured to measure an intensity
of the light beam passing through the second area filter.
3. The system of claim 2, wherein the computing unit is
configured to store an untreated fluorescence spectrum
extracted from untreated image data obtained by imaging the
observation site not labeled with the fluorescent materials.
4. The system of claim 3, wherein the computing unit is
configured to perform a correction to remove an auto-
fluorescence component contained in the image data obtained in
the imaging unit using the untreated fluorescence spectrum.
5. A multi-wavelength endoscopic system for imaging an
observation site labeled with a plurality of fluorescent
materials having different colors, comprising:
a beam splitter configured to polarize incident light
reflected from the observation site in a first direction and a
second direction perpendicular to the first direction;
a first area filter positioned in a path of a light beam
split in the first direction and configured to pass a light
beam falling within a predetermined spectral range;
-25-

a second area filter positioned in a path of a light beam
split in the second direction and configured to pass a light
beam falling within a predetermined spectral range;
a first area camera configured to measure an intensity of
the light beam passing through the first area filter;
a second area camera configured to measure an intensity
of the light beam passing through the second area filter; and
a computing unit configured to separate and output the
image data obtained using the intensity of the light beam
passing through the first area filter and the intensity of the
light beam passing through the second area filter so that each
of the fluorescent materials is displayed separately.
6. The system of claim 5, wherein the computing unit is
configured to store a single fluorescence spectrum extracted
from sample image data obtained by single-treating the
observation site with each of the fluorescent materials and is
configured to separate the image data using the single
fluorescence spectrum so that each of the fluorescent materials
is displayed separately.
7. The system of claim 6, wherein the computing unit is
configured to store an untreated fluorescence spectrum
extracted from untreated image data obtained by imaging the
observation site not labeled with the fluorescent materials.
8. The system of claim 7, wherein the computing unit is
-26-

configured to perform a correction to remove an auto-
fluorescence component contained in the image data using the
untreated fluorescence spectrum.
9. An image processing method for processing an image using
a multi-wavelength endoscopic system, comprising:
irradiating light on an observation site labeled with a
plurality of fluorescent materials having different colors;
acquiring image data by receiving a light reflected from
the observation site;
separating the image data so that only one of the
fluorescent materials is displayed; and
outputting the separated image data according to a
wavelength band.
10. The method of claim 9, further comprising:
extracting a single fluorescence spectrum from sample
image data obtained by single-treating the observation site
with each of the fluorescent materials.
11. The method of claim 10, wherein in separating the image
data, the image data is separated using the single fluorescence
spectrum so that each of the fluorescent materials is displayed
separately.
12. The method of claim 9, further comprising:
extracting an untreated fluorescence spectrum from
-27-

untreated image data obtained by imaging the observation site
not labeled with the fluorescent materials.
13. The method of claim 12, further comprising:
performing a correction to remove an auto-fluorescence
component contained in the image data using the untreated
fluorescence spectrum.
-28-

Description

Note: Descriptions are shown in the official language in which they were submitted.


4
CA 03040851 2019-04-16
MULTI-WAVELENGTH ENDOSCOPIC SYSTEM AND IMAGE PROCESSING METHOD
USING SAME
TECHNICAL FIELD
The present invention relates to a multi-wavelength
endoscopic system and an image processing method using same.
BACKGROUND
Cancer incidents still occur at a high rate. When cancer
is diagnosed through endoscopy, there is a possibility of
misdiagnosis because the tumor must be detected by the naked
eye.
Particularly, if a polyp has a flat shape rather than a
lump shape, the probability for detecting the polyp is further
lowered.
In recent years, along with the development of molecular
imaging technology, there have been ongoing studies to diagnose
gastrointestinal cancer and to image the molecular
characteristics of cancer using this technology. The first
attempt was to introduce the possibility of applying molecular
imaging that targets Cathepsin B to endoscopy. However, the
images at that time were too simple to be applied at a clinical
level.
Related studies have been carried out in various
institutions. Until recently, a technique for imaging a
-1-

CA 03040851 2019-04-16
specific tumor tissue using a peptide as a probe, an ultra-
small imaging technique capable of high-speed three-dimensional
endoscopic imaging and a small microscope technique have been
developed. A more advanced marker material is being developed
through the development of a Raman amplification probe capable
of ultra-sensitive molecular imaging and an aptamer-based
compact fluorescent probe.
The group of Dr. Goetz of Mainz University in Germany has
developed a probe that can identify an Epidermal Growth Factor
Receptor (EGFR) and has attempted to image the probe using a
special endoscope called a confocal endomicroscope.
Although studies for enabling endoscopy using molecular
imaging have been conducted thus far, there has been little
study that has obtained images at a level applicable to actual
endoscopes. Even in the case of a probe that is very important
in molecular imaging, there is available only a technique at a
level that can only confirm and verify a probe for a single
target.
SUMMARY
Embodiments of the present invention provide a multi-
wavelength endoscopic system capable of processing image data
obtained by imaging an observation site labeled through the use
of multiple probes for a composite target and capable of
providing the processed image data to the diagnosis of disease,
and an image processing method using same.
-2-

1 1
CA 03040851 2019-04-16
In accordance with a first aspect of the present
invention, there is provided a multi-wavelength endoscopic
system for imaging an observation site labeled with a plurality
of fluorescent materials having different colors, including: an
imaging unit configured to acquire image data by polarizing
incident light reflected from the observation site in a first
direction and a second direction perpendicular to the first
direction, dividing a spectrum region of the incident light
polarized in the first direction and the second direction into
a plurality of spectrum channels and measuring an intensity of
light for each of the spectrum channels; and a computing unit
configured to store a single fluorescence spectrum extracted
from sample image data obtained by single-treating the
observation site with each of the fluorescent materials and
configured to separate and output the image data obtained in
the imaging unit using the single fluorescence spectrum so that
each of the fluorescent materials is displayed separately.
The imaging unit may include: a beam splitter configured
to polarize the incident light in the first direction and the
second direction perpendicular to the first direction; a first
area filter positioned in a path of a light beam split in the
first direction and configured to pass a light beam falling
within a predetermined spectral range; a second area filter
positioned in a path of a light beam split in the second
direction and configured to pass a light beam falling within a
predetermined spectral range; a first area camera configured to
measure an intensity of the light beam passing through the
-3-

CA 03040851 2019-04-16
first area filter; and a second area camera configured to
measure an intensity of the light beam passing through the
second area filter.
The computing unit may be configured to store an
untreated fluorescence spectrum extracted from untreated image
data obtained by imaging the observation site not labeled with
the fluorescent materials.
The computing unit may be configured to perform a
correction to remove an auto-fluorescence component contained
in the image data obtained in the imaging unit using the
untreated fluorescence spectrum.
In accordance with a second aspect of the present
application, there is provided a multi-wavelength endoscopic
system for imaging an observation site labeled with a plurality
of fluorescent materials having different colors, including: a
beam splitter configured to polarize incident light reflected
from the observation site in a first direction and a second
direction perpendicular to the first direction; a first area
filter positioned in a path of a light beam split in the first
direction and configured to pass a light beam falling within a
predetermined spectral range; a second area filter positioned
in a path of a light beam split in the second direction and
configured to pass a light beam falling within a predetermined
spectral range; a first area camera configured to measure an
intensity of the light beam passing through the first area
filter; a second area camera configured to measure an intensity
of the light beam passing through the second area filter; and a
-4-

CA 03040851 2019-04-16
computing unit configured to separate and output the image data
obtained using the intensity of the light beam passing through
the first area filter and the intensity of the light beam
passing through the second area filter so that each of the
fluorescent materials is displayed separately.
The computing unit may be configured to store a single
fluorescence spectrum extracted from sample image data obtained
by single-treating the observation site with each of the
fluorescent materials and is configured to separate the image
data using the single fluorescence spectrum so that each of the
fluorescent materials is displayed separately.
The computing unit may be configured to store an
untreated fluorescence spectrum extracted from untreated image
data obtained by imaging the observation site not labeled with
the fluorescent materials.
The computing unit may be configured to perform a
correction to remove an auto-fluorescence component contained
in the image data using the untreated fluorescence spectrum.
In accordance with a third aspect of the present
application, there is provided an image processing method for
processing an image using a multi-wavelength endoscopic system,
including: irradiating light on an observation site labeled
with a plurality of fluorescent materials having different
colors; acquiring image data by receiving a light reflected
from the observation site; separating the image data so that
only one of the fluorescent materials is displayed; and
outputting the separated image data according to a wavelength
-5-

4
CA 03040851 2019-04-16
band.
The method may further include extracting a single
fluorescence spectrum from sample image data obtained by
single-treating the observation site with each of the
fluorescent materials.
In separating the image data, the image data may be
separated using the single fluorescence spectrum so that each
of the fluorescent materials is displayed separately.
The method may further include extracting an untreated
fluorescence spectrum from untreated image data obtained by
imaging the observation site not labeled with the fluorescent
materials.
The method may further include performing a correction to
remove an auto-fluorescence component contained in the image
data using the untreated fluorescence spectrum.
The multi-wavelength endoscopic system according to the
embodiment of the present invention can separate and output an
observation region labeled with a plurality of probes according
to a predetermined wavelength band. This makes it possible to
accurately grasp a disease occurrence region.
The multi-wavelength endoscopic system according to the
embodiment of the present invention can output image data by
removing an auto-fluorescence component contained in the image
data obtained by imaging an observation region. This makes it
possible to reduce false positive errors, thereby reducing the
possibility of misdiagnosis.
-6-

4
CA 03040851 2019-04-16
BRIEF DESCRIPTION OF THE DRAWINGS
Fig. 1 is a configuration diagram of a multi-wavelength
endoscopic system according to an embodiment of the present
invention.
Fig. 2 is a view illustrating a structure of an imaging
unit according to an embodiment of the present invention.
Fig. 3 is a view illustrating an imaging unit according
to an embodiment of the present invention.
Fig. 4 is a view illustrating a result of driving a
variable liquid crystal filter according to an embodiment of
the present invention.
Fig. 5 is a flowchart illustrating an image processing
method using the multi-wavelength endoscopic system according
to an embodiment of the present invention.
Figs. 6A to 6D are views showing a simulation apparatus
for evaluating the performance of the multi-wavelength
endoscopic system according to an embodiment of the present
invention and simulation results thereof.
Fig. 7 is a diagram showing images obtained from an
untreated tissue sample and a single fluorescence-treated
tissue sample by the multi-wavelength endoscopic system
according to an embodiment of the present invention.
Fig. 8 is a view showing an endoscopic imaging result
obtained by imaging a tissue sample treated with a plurality of
fluorescent materials using the multi-wavelength endoscopic
system according to an embodiment of the present invention.
-7-

CA 03040851 2019-04-16
Fig. 9 is a view showing an endoscopic imaging result
obtained by imaging a live colon cancer model mouse using the
multi-wavelength endoscopic system according to an embodiment
of the present invention.
Fig. 10 is a view showing an endoscopic imaging result
obtained by imaging a live colon cancer model pig using the
multi-wavelength endoscopic system according to an embodiment
of the present invention.
DETAILED DESCRIPTION
Hereinafter, exemplary embodiments of the present
invention will be described in detail with reference to the
accompanying drawings, which will be readily apparent to those
skilled in the art to which the present invention pertains.
However, the present invention can be implemented in various
different forms, and is not limited to the embodiments
described herein. In order to clearly illustrate the present
invention, parts not related to the description are omitted,
and like parts are denoted by like reference numerals
throughout the specification
Throughout the specification, when some component
"includes" some element, it should be understood that the some
component can include other elements as well, rather than
excluding other elements unless specifically stated otherwise.
A term such as "part", "unit", "module" or the like disclosed
in the specification indicates a unit for processing at least
-8-

4 ,
,
CA 03040851 2019-04-16
one function or operation, and may be implemented in hardware,
software or in combination of hardware and software.
Hereinafter, a multi-wavelength endoscopic system
according to an embodiment of the present invention will be
described in detail with reference to the drawings.
Fig. 1 is a configuration diagram of a multi-wavelength
endoscopic system according to an embodiment of the present
invention. Fig. 2 is a view illustrating a structure of an
imaging unit according to an embodiment of the present
invention. Fig. 3 is a view illustrating an imaging unit
according to an embodiment of the present invention.
Referring to Fig. 1, the multi-wavelength endoscopic
system 100 is a system using a camera that makes it possible to
visually observe an object to be detected and a camera adopting
a filter that enables multi-fluorescence imaging. The multi-
wavelength endoscopic system continuously images a visible
light region to obtain the hyper-spectral radiation luminance
of each channel.
In this regard, the channel is a unit band for measuring
wavelength. A spectral image of each channel can be obtained
by adjusting a filter.
In the subject specification, an endoscope generally
refers to an instrument for observing the inside of a human
body and includes, e.g., a bronchoscope, a gastroscope, a
laparoscope and an anoscope.
The multi-wavelength endoscopic system 100 includes an
imaging unit 200 and a computing unit 300.
-9-
1

4
CA 03040851 2019-04-16
The imaging unit 200 includes an objective lens 210, a
relay lens 220, a beam splitter 230, a first area lens 240, a
first area filter 242, a first area camera 244, a second area
lens 250, a second area filter 252, and a second area camera
254.
A light source 400 shown in Fig. 2 and located outside
the multi-wavelength endoscopic system 100 irradiates light so
as to excite a region to be imaged. The light source 400 may
include two or more light sources having different wavelengths
so as to image an observation target labeled with fluorescent
samples having different wavelengths.
In this embodiment, the observation target is a marker
expressed in cancer.
In this embodiment, the marker may be
labeled using probes labeled with different fluorescent
material having various wavelength bands.
The objective lens 210 is a lens through which incident
light enters.
The objective lens 210 may provide an image
focused regardless of the wavelength in the spectral region of
the multi-wavelength endoscopic system 100.
The relay lens 220 is a lens for advancing incident light
along an optical axis and is configured to output light in
parallel. The relay lens 220 may be a triplet lens having a
predetermined focal length.
Referring to Fig. 2, the relay lens 220 according to the
present embodiment may be connected to the light source 400
that emits light for exciting a region to be imaged. The relay
lens 220 may be positioned inside the endoscope inserted into
-10-

CA 03040851 2019-04-16
the body to image a diagnosis target.
The beam splitter 230 separates the parallel light into
two light beams. The beam splitter 230 is a polarization-based
beam splitter that processes incident light having a wide-band
spectrum. The beam splitter 230 may cover the spectrum falling
within a visible light region.
Light consisting of electric fields in various directions
is polarized into two light beams called a p-polarized light
beam and an s-polarized light beam. In this regard, the p-
polarized light beam means a light beam parallel to a slit
direction of a polarization plate, and the s-polarized light
beam means a light beam perpendicular to the slit direction of
the polarization plate.
The first area lens 240 and the second area lens 250 are
respectively located in the paths of the light beams split from
the beam splitter 230. In order to adjust the light beams
split from the beam splitter 230 at a predetermined
magnification, the first area lens 240 and the second area lens
250 are disposed perpendicularly to each other so that they can
acquire the light beams split from the beam splitter 230.
The first area lens 240 and the second area lens 250 may
adjust the light beams split from the beam splitter 230 at an
appropriate magnification and may transmit the adjusted light
beams to the first area filter 242 and the second area filter
252, respectively.
The first area filter 242 and the second area filter 252
can pass the light beams falling within a specified spectral
-11-

CA 03040851 2019-04-16
range among the light beams passed through the first area lens
240 and the second area lens 250, respectively.
The first area filter 242 and the second area filter 252
may be, for example, a liquid crystal tunable filter (LCTF),
which is a local band-pass filter for passing a light beam
falling within a specified spectral region.
When an LCTF that passes a channel of a specific
wavelength band in a spectral region (for example, 440 nm to
720 nm) is used as the filter of the present system, it may be
possible to control the filter so as to pass a light beam at,
for example, 10 nm intervals.
The LCTF is capable of electronically converting a
wavelength and, therefore, selecting a wavelength at a high
speed.
Referring to Fig. 4, the LCTF (www.perkinelmer.co.kr) may
be controlled to pass light at predetermined intervals and has
an effect of putting several tens to several hundreds of
filters in one filter. Therefore, it is possible to realize
multi-wavelength imaging in vivid and diverse colors.
Referring again to Fig. 1, the first area camera 244 and
the second area camera 254 are disposed at the ends of the
respective optical paths to measure the intensity of the light
beams passing through the first area filter 242 and the second
area filter 252.
The first area camera 244 and the second area camera 254
may be monochrome cameras. In this case, the first area camera
244 and the second area camera 254 may acquire the intensity of
-12-

11
CA 03040851 2019-04-16
the image focused through respective positive triplet lenses
having predetermined focal distances.
The computing unit 300 may sort the spectral images of
the respective channels acquired by the imaging unit 200 and
may output the radiance corresponding to the wavelength. The
multi-spectral image may be composed of a combination of
spectral images imaged from a plurality of channels.
The computing unit 300 may separate and output the multi-
spectral images captured by the imaging unit 200 according to
the wavelength band.
In this embodiment, in order to accurately diagnose a
disease by accurately detecting various disease-related markers
at a site to be imaged, the site to be imaged may be labeled
with fluorescent materials having different wavelengths.
In the case of labeling a single marker with a single
fluorescent material, it is difficult to accurately determine a
lesion. Therefore, in this embodiment, a complex probe is
labeled with fluorescent materials having different wavelengths,
whereby different probes can be supplemented to accurately
image a lesion.
The spectral image obtained by imaging the region labeled
with fluorescent materials having different wavelengths through
the use of the imaging unit 200 may indicate fluorescent
signals having different wavelength regions.
When a plurality of markers labeled with fluorescent
materials having different wavelengths is used to image an
observation site, the fluorescent materials may generate
-13-

CA 03040851 2019-04-16
interference in the image. This may make it difficult to
distinguish the respective fluorescent materials.
In addition, a material other than the markers labeled
with the fluorescent materials in the region to be imaged may
be irradiated with the excitation light emitted from the light
source 400 so as to emit intrinsic light.
For example, there may be generated an auto-fluorescence
phenomenon, in which collagen, elastin, keratin, NADH, flavin,
porphyrin or the like contained in the biological tissue to be
observed, reflects the excitation light.
There is a possibility of misdiagnosis when diagnosing a
disease through the use of an imaging result in which an auto-
fluorescent material generally distributed inside the body
rather than the marker material to be detected is erroneously
regarded as a marker due to the auto-fluorescence phenomenon.
Accordingly, the computing unit 300 of the multi-
wavelength endoscopic system according to an embodiment of the
present invention is configured to separate and output the
imaging result acquired by the imaging unit 200 depending on
the wavelength bands of the respective fluorescent materials,
so that the user can accurately diagnose a disease using a
multi-spectral image as an imaging result.
At this time, the computing unit 300 may extract an auto-
fluorescence spectrum result indicating the intensity of the
light corresponding to a wavelength band from the auto-
fluorescence image obtained by imaging a non-treated tissue
sample that is not treated with a fluorescent material in
-14-

CA 03040851 2019-04-16
advance.
In addition, the computing unit 300 may extract a single
fluorescence spectrum result indicating the intensity of the
light corresponding to a wavelength band from a plurality of
single treated images obtained by imaging a tissue sample that
is single-treated with a fluorescent material.
First, the computing unit 300 extracts an image spectrum
result indicating the intensity of the light corresponding to a
wavelength band from the image data obtained by the imaging
unit 200, and performs a correction of deleting the auto-
fluorescence portion by attenuating the image spectrum result
by just as much as the intensity of the light corresponding to
each wavelength band according to the spectrum of the auto-
fluorescence image.
The computing unit 300 may calculate a normalized
numerical value indicating the intensity of the light
corresponding to a wavelength band from the single fluorescence
spectrum. For example, the ratio of intensities of the light
corresponding to each wavelength band may be calculated by
setting the intensity of the light corresponding to the entire
wavelength band to 100.
Next, the computing unit 300 separates (unmixes) the
image spectrum result of the image data by the intensity of the
light corresponding to each wavelength band according to the
normalized numerical value calculated from the single
fluorescence spectrum, whereby the image obtained by imaging
the observation site labeled with a plurality of fluorescent
-15-

CA 03040851 2019-04-16
materials may be separated into a plurality of images so that
only the respective fluorescent materials appear.
Accordingly, the computing unit 300 according to an
embodiment of the present invention may perform correction to
remove the auto-fluorescence component from the multi-spectral
image in order to reduce the probability of misdiagnosis when
diagnosing a disease according to the imaging result. This
makes it possible to display only a marker labeled with a
fluorescent material.
In addition, by separating the image of the observation
site labeled with a plurality of markers so that only each of
the markers is displayed, it is possible to accurately diagnose
a cancer lesion by supplementing the different markers.
Fig. 5 is a flowchart illustrating an image processing
method using the multi-wavelength endoscopic system according
to an embodiment of the present invention.
Referring to Fig. 5, the region to be imaged is labeled
with a fluorescent material in various wavelength bands (S110).
In this experimental example, the region to be imaged may be
internal body tissue for cancer screening. A marker expressed
in cancer may be labeled using a probe labeled with a
fluorescent material in various wavelength bands.
Table 1 shows various area probes for multi-wavelength
detection.
Labeling Wavelength
Probe name Marker
material band (nm)
HMRG g-Glutamyl Rhodamine 501-524
-16-

4 ,
CA 03040851 2019-04-16
transpeptidase
Cetuximab EGFR receptor Flamma-553 553-570
Herceptin Her-2 receptor Flamma-675 675-700
The probes may be antibody probes. In this embodiment,
the antibody probes may be Cetuximab and Herceptin, which are
targeted antibodies to EGFR and HER2, frequently expressed in
tumor and colon cancer cells. In this embodiment, Cetuximab
and Herceptin are labeled with fluorescent materials Flamma-553
and Flamma-675, respectively.
Furthermore, the probes may be active probes. In this
embodiment, the active probe may be gGlu-HMRG, which exhibits
fluorescence activity when meeting with GGT (y-
glutamyltranspeptidase), frequently expressed in tumor cells
and colon cancer cells.
In this embodiment, HMRG may be
labeled with Rhodamine.
The antibody probe may be intravenously administered to
the tail of a mouse 48 hours prior to acquiring a multi-
wavelength detection endoscopic image. The active probe may be
applied to the colon 10 minutes prior to performing the multi-
wavelength detection endoscopy.
Then, the excitation light is irradiated on the region to
be imaged, and a captured image is acquired by receiving the
reflected light (S120).
At this time, the light entered through the distal end of
an endoscope excites an observation target, and the light
reflected from the observation target is transmitted to the
first area camera 244 and the second area camera 254 through
-17-
1

4
CA 03040851 2019-04-16
the relay lens 220.
The light source 400 may include two or more light
sources having different wavelengths so as to image an
observation target labeled with fluorescent samples having
different wavelengths.
The first area camera 244 and the second area camera 254
may include a first area filter 242 and a second area filter
252, respectively, which may be realized by an LCTF as a local
band-pass filter for passing the light of a specified spectral
region.
Next, the auto-fluorescence portion included in the
captured image is removed and is separated and outputted
according to a predetermined wavelength band (step S130).
The fluorescence spectrum data obtained through the
endoscope is outputted by being divided for each wavelength
band through the division operation of the computing unit 300.
The auto-fluorescence portion may be removed to finally acquire
the desired image of a wavelength region to be obtained from
the observation target.
The multi-wavelength endoscopic system 100 may store an
auto-fluorescence spectrum and a single fluorescence spectrum
result that represent light intensities according to wavelength
bands of a pre-stored untreated tissue sample image and a
tissue sample image obtained by single-processing using a
fluorescent material.
At this time, it is possible to further store the
normalized numerical value indicating the intensity of light
-18-

i .
, CA 03040851 2019-04-16
according to the wavelength band calculated from the auto-
fluorescence spectrum and the single fluorescence spectrum.
Then, the multi-wavelength endoscopic system 100 performs
correction for removing the auto-fluorescence portion by
attenuating the spectrum of the image data obtained by imaging
the observation site labeled with a plurality of fluorescent
materials by just as much as the normalized numerical value of
the auto-fluorescence spectrum.
Then, the multi-wavelength endoscopic system 100
separates (unmix) the image spectrum result of the image data
by the intensity of light corresponding to each wavelength band
according to the normalized numerical value calculated from the
single fluorescence spectrum, whereby the image of the
observation site labeled with a plurality of fluorescent
materials can be separated and displayed as a plurality of
images so that only each fluorescent material appears.
That is, in the case where one marker is labeled with one
fluorescence material, it is difficult to accurately determine
a lesion. Therefore, in this embodiment, by labeling a lesion
using a complex probe labeled with fluorescent materials having
different wavelengths, it is possible to supplement mutually-
different probes, thereby accurately imaging the lesion.
In addition, the computing unit 300 according to an
embodiment of the present invention may perform a correction to
remove the auto-fluorescence component from the multi-spectral
image in order to reduce the probability of misdiagnosis when
diagnosing a disease according to the imaging result. This
-19-
1

CA 03040851 2019-04-16
makes it possible to display only a marker labeled with a
fluorescent material.
Figs. 6A to 6D are views showing a simulation apparatus
for evaluating the performance of the multi-wavelength
endoscopic system according to an embodiment of the present
invention, and simulation results thereof.
Referring to Fig. 6A, polyethylene tubes (PE-10) having
an inner diameter of 0.28 mm and a length of about 15 mm are
prepared to evaluate the performance of the multi-wavelength
endoscopic system according to an embodiment of the present
invention. Fluorescent dyes having different colors are
injected into the respective tubes.
One end of the tube is attached to a circular metal ring
and the other end of the tube is narrowed toward the center at
which endoscope observation is performed. The fluorescent dyes
used have different colors of a visible light region band and
contain wavelength regions close to each other.
Referring to Fig. 6B, the tubes containing fluorescent
dyes are respectively imaged to acquire image data. As shown
in Fig. 60, the spectra representing the intensities of light
configured to the wavelength bands are obtained from the
respective results of image data. This makes it possible to
identify the separated regions for each wavelength of each dye.
In this case, the set wavelength range read in the multi-
wavelength endoscopic system 100 according to the present
embodiment may be 420 nm to 620 nm.
Referring to Fig. 6D, a complete image file is obtained
-20-

CA 03040851 2019-04-16
by a decomposition process according to a single fluorescence
spectrum result through the computing unit 300.
Fig. 7 is a diagram showing images obtained from an
untreated tissue sample and a single fluorescence-treated
tissue sample by the multi-wavelength endoscopic system
according to an embodiment of the present invention. Fig. 8 is
a view showing an endoscopic imaging result obtained by imaging
a tissue sample treated with a plurality of fluorescent
materials using the multi-wavelength endoscopic system
according to an embodiment of the present invention.
At this time, an active probe (HMRG) is injected by local
application, and antibody probes (Cetuximab-Flamma553 and
Herceptin-Flamma675) are intravenously injected. Then, colon
tissue is extracted, and images corresponding to the respective
wavelengths are acquired using the multi-wavelength endoscopic
system 100.
Referring to Fig. 7, fluorescence images by auto-
fluorescence can be observed in the non-treated colonic tissue
of animals in which probes are not treated for control
experiments.
A single probe-treated tissue sample is imaged at the
observation site, and a single fluorescence spectrum result
indicating the intensity of the light corresponding to the
wavelength band is extracted from the captured image data.
Referring to Fig. 8, in this embodiment, a composite
probe labeled with fluorescent materials having different
wavelengths is used. In the image data obtained by imaging a
-21-

1
CA 03040851 2019-04-16
tissue sample labeled with a complex probe, an image is
separated and outputted so that only each fluorescent material
is labeled according to a single fluorescence spectrum result.
Therefore, it is possible to confirm that the imaging is
performed so as to accurately diagnose a lesion by
supplementing different probes.
Fig. 9 is a view showing an endoscopic imaging result
obtained by imaging a live colon cancer model mouse using the
multi-wavelength endoscopic system according to an embodiment
of the present invention.
Active probes (HMRG) are injected into a colon cancer
model mouse by local application, and antibody probes
(Cetuximab-Flamma553 and Herceptin-Flamma675) are injected
intravenously. Then, an image for each fluorescent material is
acquired through colonoscopy using the multi-wavelength
endoscopic system 100.
While it is difficult for the single probe to image and
accurately determine the sections of cancer, the composite
probe can supplement different probes and can image the
sections of cancer.
Fig. 10 is a view showing an endoscopic imaging result
obtained by imaging a live colon cancer model pig using the
multi-wavelength endoscopic system according to an embodiment
of the present invention.
Active probes (HMRG) and antibody probes (Cetuximab-
F1amma553 and Herceptin-F1amma675) are injected into a human-
like pig. Then, an image for each wavelength is acquired
-22-

a ,
,
CA 03040851 2019-04-16
through colonoscopy using the multi-wavelength endoscopic
system 100.
A fluorescence image is not acquired when the probe is
not processed for control experiments.
When each probe is single-treated with a fluorescent
material, the signal is detected only at the spectrum
wavelength of the fluorescence of each probe. Image data is
separated from the triple-treated image data for three probes
using the single fluorescence spectrum result so that only each
fluorescent material is labeled.
Thus, by supplementing the different probes, it is
possible to reduce false positive errors and to accurately
image the sections of cancer.
The embodiments of the present invention described above
are not implemented only by the apparatus and method, but may
be implemented through a program for realizing the function
corresponding to the configuration of the embodiment of the
present invention or a recoding medium on which program is
recorded.
While the disclosure has been shown and described with
respect to the embodiments, it will be understood by those
skilled in the art that various changes and modifications may
be made without departing from the scope of the disclosure as
defined in the following claims.
-23-
il

Representative Drawing
A single figure which represents the drawing illustrating the invention.
Administrative Status

2024-08-01:As part of the Next Generation Patents (NGP) transition, the Canadian Patents Database (CPD) now contains a more detailed Event History, which replicates the Event Log of our new back-office solution.

Please note that "Inactive:" events refers to events no longer in use in our new back-office solution.

For a clearer understanding of the status of the application/patent presented on this page, the site Disclaimer , as well as the definitions for Patent , Event History , Maintenance Fee  and Payment History  should be consulted.

Event History

Description Date
Application Not Reinstated by Deadline 2021-12-24
Inactive: Dead - No reply to s.86(2) Rules requisition 2021-12-24
Letter Sent 2021-10-18
Deemed Abandoned - Failure to Respond to an Examiner's Requisition 2020-12-24
Interview Request Received 2020-11-24
Common Representative Appointed 2020-11-07
Examiner's Report 2020-08-24
Inactive: Report - No QC 2020-08-14
Inactive: COVID 19 - Deadline extended 2020-07-16
Amendment Received - Voluntary Amendment 2020-07-03
Inactive: COVID 19 - Deadline extended 2020-07-02
Inactive: COVID 19 - Deadline extended 2020-06-10
Interview Request Received 2020-06-05
Examiner's Report 2020-03-03
Inactive: Report - No QC 2020-02-28
Amendment Received - Voluntary Amendment 2020-01-30
Common Representative Appointed 2019-10-30
Common Representative Appointed 2019-10-30
Inactive: S.30(2) Rules - Examiner requisition 2019-09-23
Inactive: Report - QC passed 2019-09-20
Withdraw Examiner's Report Request Received 2019-09-06
Inactive: Office letter 2019-09-06
Inactive: S.30(2) Rules - Examiner requisition 2019-08-22
Inactive: Report - No QC 2019-08-22
Advanced Examination Determined Compliant - PPH 2019-07-12
Amendment Received - Voluntary Amendment 2019-07-12
Advanced Examination Requested - PPH 2019-07-12
Inactive: Cover page published 2019-05-03
Inactive: Acknowledgment of national entry - RFE 2019-05-01
Inactive: First IPC assigned 2019-04-29
Letter Sent 2019-04-29
Inactive: IPC assigned 2019-04-29
Inactive: IPC assigned 2019-04-29
Inactive: IPC assigned 2019-04-29
Inactive: IPC assigned 2019-04-29
Application Received - PCT 2019-04-29
National Entry Requirements Determined Compliant 2019-04-16
Request for Examination Requirements Determined Compliant 2019-04-16
All Requirements for Examination Determined Compliant 2019-04-16
Application Published (Open to Public Inspection) 2018-04-26

Abandonment History

Abandonment Date Reason Reinstatement Date
2020-12-24

Maintenance Fee

The last payment was received on 2020-10-09

Note : If the full payment has not been received on or before the date indicated, a further fee may be required which may be one of the following

  • the reinstatement fee;
  • the late payment fee; or
  • additional fee to reverse deemed expiry.

Patent fees are adjusted on the 1st of January every year. The amounts above are the current amounts if received by December 31 of the current year.
Please refer to the CIPO Patent Fees web page to see all current fee amounts.

Fee History

Fee Type Anniversary Year Due Date Paid Date
Basic national fee - standard 2019-04-16
Request for examination - standard 2019-04-16
MF (application, 2nd anniv.) - standard 02 2019-10-18 2019-10-10
MF (application, 3rd anniv.) - standard 03 2020-10-19 2020-10-09
Owners on Record

Note: Records showing the ownership history in alphabetical order.

Current Owners on Record
THE ASAN FOUNDATION
Past Owners on Record
DONG-JUN BAE
EUN-JU DO
SANG MUN BAE
SANG-YEOB KIM
SEUNG-JAE MYUNG
Past Owners that do not appear in the "Owners on Record" listing will appear in other documentation within the application.
Documents

To view selected files, please enter reCAPTCHA code :



To view images, click a link in the Document Description column. To download the documents, select one or more checkboxes in the first column and then click the "Download Selected in PDF format (Zip Archive)" or the "Download Selected as Single PDF" button.

List of published and non-published patent-specific documents on the CPD .

If you have any difficulty accessing content, you can call the Client Service Centre at 1-866-997-1936 or send them an e-mail at CIPO Client Service Centre.


Document
Description 
Date
(yyyy-mm-dd) 
Number of pages   Size of Image (KB) 
Description 2019-04-15 23 848
Drawings 2019-04-15 10 569
Claims 2019-04-15 5 141
Representative drawing 2019-04-15 1 20
Abstract 2019-04-15 1 26
Representative drawing 2019-05-02 1 18
Description 2019-07-11 23 873
Claims 2019-07-11 5 142
Drawings 2019-07-11 10 531
Description 2020-01-29 24 877
Abstract 2020-01-29 1 26
Claims 2020-01-29 4 139
Description 2020-07-02 24 888
Drawings 2020-07-02 10 561
Claims 2020-07-02 4 151
Acknowledgement of Request for Examination 2019-04-28 1 174
Notice of National Entry 2019-04-30 1 202
Reminder of maintenance fee due 2019-06-18 1 112
Courtesy - Abandonment Letter (R86(2)) 2021-02-17 1 551
Commissioner's Notice - Maintenance Fee for a Patent Application Not Paid 2021-11-28 1 563
International search report 2019-04-15 5 302
Amendment - Abstract 2019-04-15 2 92
National entry request 2019-04-15 3 69
PPH request / Amendment 2019-07-11 14 526
PPH supporting documents 2019-07-11 6 244
Examiner Requisition 2019-08-21 4 177
Courtesy - Office Letter 2019-09-05 1 24
Examiner Requisition 2019-09-22 4 219
Amendment 2020-01-29 14 515
Examiner requisition 2020-03-02 7 374
Interview Record with Cover Letter Registered 2020-06-04 2 23
Amendment 2020-07-02 33 1,706
Examiner requisition 2020-08-23 6 382
Interview Record with Cover Letter Registered 2020-11-23 2 16