Note: Descriptions are shown in the official language in which they were submitted.
TITLE: SELF-POWERED REMOTE CAMERA PROBES FOR AUTOMATED
HORTICULTURAL PEST DETECTION
FIELD OF THE INVENTION
[1] The present invention relates to low cost remote camera probes that can be
deployed in great numbers in greenhouse facilities to provide early warning of
insect
pest infestations.
BACKGROUND
[2] In the field of indoor horticulture, insect pests are a continuous
threat to the
health, yield and quality of a crop growing in a controlled environment. Due
to air
circulation systems actively rotating air throughout a facility infestations
can spread
very quickly. The trend towards increasingly large scale greenhouses for
reasons of
cost efficiency at scale increases the net value of crops at risk of
infestation,
especially if just a single crop type is grown across the entire facility.
[3] For certain crops that are grown organically there are few options to
limit the
spread of insect pests once detected and it is necessary to remove affected
plants as
quickly as possible in the hope of avoiding widespread infestation. It is not
uncommon for entire crops to be lost in large growing facilities due to
uncontrollable
insect populations that were detected too late to contain.
[4] Due to the size of the growing facilities, direct visual inspection of
such a large
surface of stems, leaves, flowers and produce can be extremely labour
intensive and
therefore expensive. As crops mature the at-risk investment in terms of time,
energy,
labour and raw materials increases but so does the difficulty of effectively
inspecting
the plants for signs of infestation. More mature crops have a larger surface
area,
more of that area is obscured or hard to access and older plant tissue may be
more
susceptible to attack.
[5] There is therefore a need for a system that can be cost-effectively
deployed to
provide automated early detection of pest insects. A system that could
directly detect
CA 3021755 2018-10-22
and identify pest insects, their eggs or the damage they cause in a timely
fashion
could be a valuable line of defence against catastrophic crop loss, but would
need to
be sensitive, accurate, easy to deploy and inexpensive enough to provide
effective
coverage across a large physical area on a commercially viable basis.
BRIEF SUMMARY OF INVENTION
[6] The present invention comprises a small, low-cost, self-powered camera
probe
capable of directly imaging an integrated sticky insect trap or exposed leaf
and
wirelessly communicating with a back-end comprising standard wireless
networking
and server infrastructure hosting custom image analysis software.
[7] The probes themselves feature an SOC (System on a Chip ¨ a highly
integrated
microprocessor complex) equipped with wireless communication, an interface to
a
digital camera and the ability to control a Light Emitting Diode (LED) light
source to
illuminate the trap in a controlled way. The probe is powered by an integrated
photovoltaic cell, microwave antenna or a primary battery that charges an
energy
storage device such as a supercapacitor or rechargable secondary battery, or
by direct
electrical connections. Probe activity is periodically triggered by a timer
mechanism.
[8] At one end of the device the body of the probe supports the Printed
Circuit
Board (PCB) containing all the elements except the photovoltaic cell and the
trap.
The camera and LED are oriented towards the trap which is mounted at the other
end
of the device and the photovoltaic cell is mounted between the two ends.
Correct
orientation of the camera and trap may be assured by thin wires that attach
the top of
the two ends together under slight tension to maintain rigidity. The main body
of the
probe may be built from stamped sheet metal such as aluminum or plastic or a
wire
frame.
[9] In operation the energy harvested from the photovoltaic cell
accumulates in the
storage device. A very low power timer device continuously operates, "waking
up"
the SOC at regular intervals. On wake up, the SOC illuminates the trap,
captures a
still image and wirelessly transmits it to waiting server infrastructure. Once
at the
2
CA 3021755 2018-10-22
server, the image is processed and analysed for content. If the system
determines that
a new feature of interest (typically a trapped flying insect) has been found,
it records
the possible infestation in a central database and the system decides if
prompt human
intervention may be requested.
[10] Each probe has a unique serial number for identification, so an insect
warning
report can be associated with its originating probe. By analysing various
properties
of measured radio signals reported from multiple wireless base stations at
known
positions within the building, automatic location of each probe in all three
dimensions is possible, faciliting this process.
[11] The system is designed in such a way that a great many probes could be
deployed in a growing facility, allowing high quality coverage of the crop
during its
entire life cycle.
BRIEF DESCRIPTION OF DRAWINGS
[12] Some embodiments of the present invention are illustrated as an example
and
are not limited by the figures of the accompanying drawings, in which like
references
may indicate similar elements and in which:
[13] FIG. 1 ¨ depicts a side elevation of one example of a probe according to
various embodiments of the present invention.
[14] FIG. 2 ¨ depicts a top view of one example of a probe according to
various
embodiments of the present invention.
[15] FIG. 3 ¨ depicts an end elevation of one example of a probe according to
various embodiments of the present invention.
[16] FIG. 4A ¨ depicts a perspective representation of a probe using a wire-
based
frame.
[17] FIG 4B ¨ depicts a perspective representation of a probe using an
alternative
wire frame configuration suited to attaching the trap with tape.
[18] FIG 5 ¨ depicts a high level electrical schematic of a probe.
3
CA 3021755 2018-10-22
[19] FIG 6 ¨ depicts a probe, the wireless access points and downstream
infrastructure.
DETAILED DESCRIPTION OF THE INVENTION
[20] The terminology used herein is for the purpose of describing particular
embodiments only and is not intended to be limiting of the invention. As used
herein,
the term "and/or" includes any and all combinations of one or more of the
associated
listed items. As used herein, the singular forms "a," "an," and "the" are
intended to
include the plural forms as well as the singular forms, unless the context
clearly
indicates otherwise. It will be further understood that the terms "comprises"
and/or
"comprising," when used in this specification, specify the presence of stated
features,
steps, operations, elements, and/or components, but do not preclude the
presence or
addition of one or more other features, steps, operations, elements,
components, and/
or groups thereof
[21] Unless otherwise defined, all terms (including technical and scientific
terms)
used herein have the same meaning as commonly understood by one having
ordinary
skill in the art to which this invention belongs. It will be further
understood that
terms, such as those defined in commonly used dictionaries, should be
interpreted as
having a meaning that is consistent with their meaning in the context of the
relevant
art and the present disclosure and will not be interpreted in an idealized or
overly
formal sense unless expressly so defined herein.
[22] In describing the invention, it will be understood that a number of
techniques
and steps are disclosed. Each of these has individual benefit and each can
also be
used in conjunction with one or more, or in some cases all, of the other
disclosed
techniques. Accordingly, for the sake of clarity, this description will
refrain from
repeating every possible combination of the individual steps in an unnecessary
fashion. Nevertheless, the specification and claims should be read with the
understanding that such combinations are entirely within the scope of the
invention
and the claims.
4
CA 3021755 2018-10-22
[23] New remote pest detection probes and analytics systems are discussed
herein.
In the following description, for purposes of explanation, numerous specific
details
are set forth in order to provide a thorough understanding of the present
invention. It
will be evident, however, to one skilled in the art that the present invention
may be
practiced without these specific details.
[24] The present disclosure is to be considered as an exemplification of the
invention, and is not intended to limit the invention to the specific
embodiments
illustrated by the figures or description below.
[25] The present invention will now be described by referencing the appended
figures representing preferred embodiments.
[26] FIG 1, FIG 2 and FIG 3 respresent side, plan and end elevations views
respectively of the physical configuration of the remote camera probe in one
variation
of the present invention.
[27] Referring to FIGs 1, 2 & 3:
[28] Printed Circuit Board (PCB) 1 connects all electronic elements of the
probe.
All elements except the Photo Voltaic (PV) cells 3 are physically soldered
onto PCB
1. PV cells 3 are attached to the floor of the device and are connected by two
wires to
PCB 1. Multilayer PCB 1 measures approximately 20mm wide by 30mm tall by
0.5mm thick.
[29] Frame 2 supports and interconnects PCB 1, PV cells 3 and trap 4. The
frame
may be constructed from several different materials:
[30] In one embodiment, the frame is fashioned from stamped and folded
aluminum
sheet. The thin sheet may be stamped with ridges and folds to increase
rigidity. The
frame may be attached to the PCB 1 with screws or adhesive tape or glue. The
frame
may be attached to PV cells 3 with adhesive tape or glue. The frame may be
attached
to trap 4 using bendable tabs or the adhesive properties of the trap paper
itself.
[31] In a second embodiment, the frame is constructed from a molded plastic
part.
In this case the PCB 1, PV cells 3 and trap paper 4 may be attached using the
same
5
CA 3021755 2018-10-22
methods as for the aluminum foil frame case, or may be each clipped into place
using
slots and tabs built into the plastic frame molding.
[32] In the case of these first two construction choices for the frame 2,
tension wires
6 may be used to stablise the frame. These wires connect the top two corners
of PCB
1 with the top two corners of the part of the frame that supports the trap 4.
These
wires are thin filaments which are in tension when installed into place and
have the
effect of both stiffening the structure and pulling the camera and trap ends
of the
frame so they are parallel and thus optimally positioned for imaging the
entire surface
of the trap from the camera avoiding focus errors and trapezoidal image
distortion.
[33] In order to improve wireless communication efficiency (data rate, range
or
energy consumption of the radios), tension wires 6 may be connected as antenna
for
the integrated radios.
[34] Referring to FIGs 4 A, 4B: In a third embodiment, the frame is
constructed of
one or more segments of wire 11 that are soldered onto or otherwise attached
directly
to the PCB 1. The PV cells 3 may be attached in this case to the top of the
device and
serve as the alignment and stiffening element in place of the tensioning wires
used in
FIGs 1, 2 and 3. Stainless steel would be a suitable material for wire 11. As
shown
in FIG 4A, trap 4 could be attached to wire 11 using adhesive tape along each
vertical side. Alternatively as shown in FIG 4B, the vertical elements of the
wire
frame supporting the trap could be angled allowing the rear surface of the
trap to be
attached with adhesive tape 12.
[35] In each case, the wire frame 11 could be formed in such a way as to clip
under
compression into holes and slots created in PCB 1, permitting the assembly of
the
probe from component parts without permanently attaching the PCB 1 with
solder,
glue or tape.
[36] In the case of a FIG 4B, a stiff card-like trap 4 the wire frame 11 could
be
designed to engage curved slots cut into the trap 4 to allow the spring of the
wire to
lock the trap in place.
6
CA 3021755 2018-10-22
[37] In the case of wire frame construction as shown in FIGs 4A and 4B the
wire
frame 11 itself may also serve as the wireless communications antenna for
improved
radio performance.
[38] Referring again to FIGs 1, 2 & 3:
[39] The trap paper 4 is derived from commercially available insect trap
paper,
sized for this application. Such trap paper is coated with a sticky material
that traps
insects 5 that fly or walk onto it. The paper may also contain chemical
attractants to
improve the rate at which insects are caught. The paper may also be coloured
in such
a way as to attract targeted insects.
[40] In some indoor horticultural operations, desirable insects performing
useful
functions such as pollination (example: bees/bumblebees) or preying upon pest
species (example: ladybugs) may be deliberately introduced. In these cases, a
coarse
mesh of wire or plastic filaments may be deployed some distance away from the
sticky surface of the trap 4 to prevent desirable species of insect from
becoming
trapped on it. This strategy works since predators and pollinators are often
significantly larger than the pest species the trap 4 is intended for.
[41] The probe is designed to be installed for the duration of the crop's life
cycle
which means multiple months of continuous operation. During this time the trap
paper 4 may be filled with insects or contaminated with obscuring material
such as
dust and soil and so become unable to continue to further generate useful
data. In
this case, the system decides to generate a request for human assistance in
replacing
the paper trap on the probe. This process is facilitated by a convenient
mechanism
for removing and refreshing the trap paper 4, and also the automatic location
system
based on radio telemetry.
[42] Referring to FIG 5 we now describe the principle of operation from an
electrical perspective:
[43] The probe requires electrical energy to function and this energy is
generally
harvested from the PV cells 3 but they cannot be relied upon to produce
sufficient
instantaeous power to operate the SOC 10 and associated peripherals under all
light
7
CA 3021755 2018-10-22
conditions. When installed under the growth canopy or when operating during
times
of darkness (when insects may be most active) there will certainly be
insufficient
direct power. The PV cells instead produce energy for the energy storage
subsystem
7 which charges up and has the power delivery capacity to operate the probe in
short
bursts. Even under low light conditions, the PV cell 3 is able to make
progress
charging the system when it is only used infrequently and periodically. The
energy
storage system in the preferred embodiment is built around supercapacitors.
Supercapacitors have the advantage of tolerating virtually unlimited numbers
of
charge/discharge cycles without suffering significant capacity, leakage or
internal
impedance degradation. Other embodiments may select rechargeable batteries
such
as those based on Lithium-Ion chemistries. Such batteries have price,
volumetric
energy and volumetric power density advantages over supercapacitors however
they
are more volatile and less durable. Supercapacitors also have a significantly
higher
internal impedance than most battery types which makes it easier to design
systems
that are intrinsically safer and specifically may be fundamentally unable to
malfunction in such a way as to serve as ignition sources. The energy storage
7
subsystem is sized to support operational eletrical requirements in the most
compact
and cost effective way. In practice this means that during the longest period
of
darkness the probe must be able to execute a minimum number of bursts to meet
plant surveillance requirements. This requirement informs the energy storage
capacity and also the probe's self-discharge rate while it is waiting between
activity
bursts.
[44] An energy management circuit ensuring positive charging from the PV cells
3
and minimisation of leakage paths ensures the storage system 7 operates as
efficiently
as possible. When the system is idling between activity bursts it is able to
accumulate electrical energy. It is important that during these intervals the
power
consumption of the probe is as low as possible. For this reason during these
times,
power is electrically isolated from the main power draws (the SOC 10, camera
8,
LED 9, radio 15) using a power transistor and only a dedicated timer device 14
is
8
CA 3021755 2018-10-22
allowed to run. The timer 14 is configured to wait for a specified period of
time
measured in minutes or even hours, and then to trigger a wake up of the SOC 10
to
initiate an activity burst.
[45] An activity burst should be executed as efficiently as possible to
minimise the
drain on the energy storage subsystem 7. Each burst comprises the following
steps:
[46] 1. Timer 14 enables power to the SOC 10 and associated peripherals and
triggers the SOC 10 to boot from internal non-volatile storage ¨ typically
flash
memory.
[47] 2. SOC 10 boots into its Real Time Operating System (RTOS) and loads its
application executable.
[48] 3. The application reads a wake counter from non-volatile storage,
increments
it and writes it back to non-volatile storage. If the counter is still below a
fixed
threshold, the probe decides this is not the time to wake up and jumps
directly to step
10 below. However if the counter has reached the threshold then an image
capture
and transmission will occur. This mechanism is designed to allow finer control
over
the effective wake up interval. For example, the wakeup timer may be
configured by
a resistor value to wake up the SOC every 10 minutes which is very likely too
frequent since it would drain the energy storage subsystem 7 too quickly, but
by
using the counter mechanism longer effective intervals can be implemented with
10
minute granularity with only a slight energy budget penalty.
[49] 4. A TCP/IP or UDP session is opened with the wireless network access
point it
was associated with when first deployed.
[50] 5. The camera 8 is initialised for still image capture.
[51] 6. A series of frames is captured with ramping LED illumination durations
until
a correctly exposed image is received. This approach is necessary because the
ambient illumination conditions will vary dramatically from full sunlight in a
greenhouse during the day to complete darkness during the night or certain
phases of
crop maturation which call for darkness.
9
CA 3021755 2018-10-22
[52] 7. The optimally exposed image is optionally compressed and then
transmitted
to the server infrastructure over the wireless network.
[53] 8. Metadata such as the probe's unique ID and other readings that may be
available are appended to the transmitted data package along with the image
itself.
[54] 9. The SOC 10 resets its wake counter in non-volatile memory to zero.
[55] 10. The SOC 10 initiates a shutdown which causes all power to be removed
from the SOC 10 subsystem until the next wake up event.
[56] Each activity burst lasts well under 1 second and therefore consumes a
minimal amount of energy. Due to the time and energy constraints the probes
are
limited to data capture and transmission ¨ all processing, archival and
analysis occurs
within much faster computer systems downstream of the probes.
[57] Referring to FIG 6:
[58] A probe 16 is within wireless communication range of several wireless
access
points 17(i), 17(ii) & 17(iii). In turn the access points 17(i-iii) are
connected using
wired network links to a standard network switch/router 18. Switch 18 also
connects
local server 19 and provides a link 20 to remote data centres across the
Internet or
dedicated circuits.
[59] When a probe 16 is first deployed and activated, it examines the access
points
it can detect and ranks them in order of signal strength, identifying an
affinity to its
preferred access point which is generally the one located closest to it. This
information is stored in the probe's non-volatile memory so it can
automatically
connect to it during an activity burst.
[60] When the probe 16 is sending broadcast signals to all access points 17,
each
can measure the signal strength and so collectively the access points 17 can
estimate
the physical location of each particular probe as identified by a unique ID or
even the
MAC address embedded in the broadcast packets.
[61] Since signal strength scales inversely with distance squared between two
communicating nodes, the access points 17 can model a volumetric region in
which
each probe 16 resides. Various factors such as signal attenuation due to
obscuring
/0
CA 3021755 2018-10-22
objects, reflections and anisotropic antenna characteristics will distort this
estimate so
a heuristic algorithm complete with the ability to discard outlier readings
that are
inconsistent with near concensus among large populations of access points 17
is
suggested.
[62] The more access points 17 that each probe 16 can reach with radio
signals, the
more accurate the location fix will be.
[63] Possible alternative methods for establishing probe 16 location fixes
include
radio flight time measurements and direct optical detection of strobed lights
from the
probe.
[64] As depicted in FIG 6, local analysis of the images transmitted by
populations
of probes is performed in the local server(s) 19. Local servers 19 collect
probe data
from the access points and decide if the images reported by each probe 16 show
significant differences from the previous image reported by that probe, after
normalising for global image changes due to varying illumination levels. If
the probe
16 is deemed to have new data to report, the image and associated metadata are
in
turn transmitted to a remote data centre for detailed analysis. Such analysis
may
include conventional image processing techniques for normalisation and feature
enhancement, as well as pattern recognition algorithms designed or trained to
identify
insects 5 caught in the traps 4 (FIG 2). This information may simply be
archived or
may also trigger alarms that would result in further investigations by the
operators of
the facility to see if intervention is required.
11
CA 3021755 2018-10-22
