Note : Les descriptions sont présentées dans la langue officielle dans laquelle elles ont été soumises.
1
WEED PICKING AND DISPOSAL MODULE AND METHOD
TECHNICAL FIELD
[001] The technical field generally relates to weed picking and disposal
implements for
use on agricultural vehicles, and more specifically to modules and methods of
optimizing
weed or immature crop disposal with automated implements.
BACKGROUND
[002] Weeds compete with crops for resources, including water, nutrients, and
sunlight.
Picking of weeds from agricultural fields is a continuous process which
improves crop yield
by removing competition for resources. Once weeds are removed from an
agricultural
field, the weeds must be properly disposed of. If, for example, a weed is
simply removed
and thrown arbitrarily in the field, there is a chance that the weed will take
root once more.
There is therefore a desire for methods that allow for proper weed disposal
without
disproportionately adding to the time required to carry out weed removal and
disposal.
SUMMARY
[003] In one aspect, there is provided a method for picking and disposing of
weeds using
an implement provided on an agricultural vehicle. The method may comprise
capturing
images of a field travelled by the agricultural vehicle, the field comprising
weeds and crops.
The method may also comprise generating a map from the captured images,
comprising
coordinates of the weeds and crops; and inputting disposal lanes into the
generated map,
the disposal lanes being regions where the weeds are to be disposed. The
method may
comprise determining, based on the coordinates of the weeds, the coordinates
of the crops
and the disposal lanes, a picking and disposing trajectory for the implement
to follow that
minimizes the time needed to pick and dispose of weeds, while avoiding crops
when
moving to the disposal lanes. The method may also comprise removing weeds with
the
implement and disposing of the weeds in the disposal lanes by following the
picking and
disposing trajectory determined. In possible embodiment, immature crops or
other plants
can picked and disposed of according to this method.
[004] In one embodiment, removing weeds comprises picking one or more weeds at
a
time with the implement.
Date Recue/Date Received 2022-02-14
2
[005] In one embodiment, the picking and disposing trajectory is iteratively
built and
modified as the agricultural vehicle travels the field.
[006] In another embodiment, the implement avoids crop avoidance zones as it
follows
the picking and disposing trajectory to the disposal lanes.
[007] In another embodiment, the implement follows the picking and disposing
trajectory
by a combination of rotational and translational movement.
[008] In another embodiment, avoiding contact with the crops comprises
maintaining a
minimum lateral distance between the implement and the crops.
[009] In another embodiment, avoiding contact with the crops comprises
maintaining a
minimum vertical distance between the implement and the crops.
[0010] In another embodiment, determining the picking and disposing trajectory
comprises applying a travelling salesman algorithm, which reduces travel
distance to the
disposal lanes, modified with a collision avoidance algorithm, which avoids
travelling over
the crops while disposing of the weeds.
[0011] In another embodiment, determining the picking and disposing trajectory
comprises a step of concatenating a plurality of path segments, each path
segment
corresponding to a linear distance between one of the weed coordinates and
disposal
coordinates located within the disposal lanes.
[0012] In another embodiment, determining the picking and disposing trajectory
comprises a step of smoothing an initial path comprising the concatenated path
segments.
[0013] In another embodiment, determining the picking and disposing trajectory
comprises determining a speed of the implement based on the shape of the
trajectory and
based on constraints inherent to a controller controlling movement of the
implement.
[0014] In another embodiment, accelerations and decelerations are determined
based on
torque limits of the controller.
[0015] In another embodiment, the implement is a gripper and removing weeds
comprises
controlling jaws of the gripper to pinch and extract weeds from the ground.
Date Recue/Date Received 2022-02-14
3
[0016] In another embodiment, the images are captured using at least one
camera on the
agricultural vehicle.
[0017] In another aspect, there is provided a weed picking and disposal module
provided
on or attached to an agricultural vehicle. The module, or assembly, comprises
an
agricultural implement translatable along three degrees of freedom; a
controller configured
to control the movement of the agricultural implement; and a processing
module. The
processing module comprises input ports for receiving coordinates of weeds and
crops
determined from images and for receiving positions of disposal lanes; and a
non-transitory
memory and a processor, the non-transitory memory storing the coordinates and
instructions for causing the processor to generate a picking and disposing
trajectory for
the implement to follow that minimizes the time needed to pick and dispose of
weeds,
while avoiding crops when moving to the disposal lanes.
[0018] In another embodiment, the implement is also rotatable along at least
one degree
of freedom.
[0019] In another embodiment, the agricultural implement is a gripper
comprising jaws
configured and adapted to pinch and retrieve the weeds.
[0020] In another embodiment, one of a delta robot or a gantry system
operatively
connected to the agricultural implement, for translating the implement along
the three
degrees of freedom.
[0021] In another embodiment, the controller comprises at least one first
actuator for
controlling movement of the gripper and a second actuator for actuation of the
gripper.
[0022] In another embodiment, the non-transitory memory has stored thereon
instructions
to apply a travelling salesman algorithm to an initial path, which reduces
travel distance to
the disposal lanes, and instructions to modify the initial path with a
collision avoidance
algorithm, which avoids travelling over the crops while disposing of the
weeds.
[0023] In another embodiment, the non-transitory memory has stored thereon
instructions
to determine a speed of the implement based on the shape of the modified path
and based
on constraints inherent to a controller controlling movement of the implement.
Date Recue/Date Received 2022-02-14
4
[0024] In another embodiment, the non-transitory memory has stored thereon
instructions
to determine accelerations and decelerations based on torque limits of the
controller.
[0025] In another aspect, there is provided a method for picking and disposing
of
unwanted plants using an implement provided on an agricultural vehicle. The
method
comprises capturing images of a field travelled by the agricultural vehicle,
the field
comprising unwanted plants and crop; generating a map from the captured
images,
comprising coordinates of the unwanted plants and crops; inputting disposal
lanes into the
generated map, the disposal lanes being regions where the unwanted plants are
to be
disposed, and determining, based on the coordinates of the unwanted plants,
the
coordinates of the crops and the disposal lanes, a picking and disposing
trajectory for the
implement to follow that minimizes the time needed to pick and dispose of
unwanted
plants, while avoiding crops when moving to the disposal lanes; and removing
unwanted
plants with the implement and disposing of said unwanted plants in the
disposal lanes by
following the picking and disposing trajectory determined.
BRIEF DESCRIPTION OF DRAWINGS
[0026] Fig. 1 is a perspective view of an example implementation of an
agricultural
vehicle having a weed or immature crop disposal module (not shown) mounted
underneath, according to a possible embodiment;
[0027] Fig. 2 is a side view of a disassembled portion of the body of the
vehicle of Fig.
1, along with the weed picking and disposal module visible underneath;
[0028] Fig. 3 is a top-down view of the body of the vehicle and the weed
picking and
disposal module Fig. 2 placed side-by-side;
[0029] Fig. 4 is a view of an agricultural implement assembly, specifically a
gripper
assembly, that may be used in accordance with the methods described herein;
[0030] Fig. 5 is a top-down view of an example field shown in map form having
crop
lanes and disposal lanes, with weeds and crops shown in the crop lanes;
Date Recue/Date Received 2022-02-14
5
[0031] Fig. 6 is a top-down view of the example field of Fig. 5 shown in map
form having
crop lanes, intermediate lanes and disposal lanes, with weeds and crops shown
in the crop and intermediate lanes;
[0032] Fig. 7 is a top-down view of the example field of Fig. 5 shown in map
form having
crop lanes and disposal lanes, showing a weed disposal location in the weed
disposal lane;
[0033] Fig. 8 is a top-down view of example lanes of a field, with an
implement picking
and disposing trajectory going from a first weed to a first weed drop disposal
location, then to a second weed while respecting crop avoidance;
[0034] Fig. 9 is a top-down view of the example lanes of the field of Fig. 8,
with an
improved (shorter) implement picking and disposing trajectory going from a
first
weed to a first weed drop disposal location, then to a second weed while
respecting crop avoidance;
[0035] Fig. 10 is a top-down view of the example lanes of the field of Fig. 8,
with an
implement picking and disposing trajectory going from a first weed to a first
weed
drop disposal location, then to a second weed all in straight lines without
respecting crop avoidance;
[0036] Fig. 11 is a top-down view of the example lanes of the field of Fig.
10, with an
implement picking and disposing trajectory going from a first weed to a first
weed
drop disposal location, then to a second weed all in straight lines while
respecting
a crop avoidance zone;
[0037] Fig. 12 is a top-down view of the example lanes of the field of Fig.
11, with an
implement picking and disposing trajectory going from a first weed to a first
weed
drop disposal location, then to a second weed all in one smoothed curve while
respecting the crop avoidance zone;
[0038] Fig. 13 is a top-down view of the example lanes of the field of Fig.
12, with an
improved (shorter) implement picking and disposing trajectory going from a
first
weed to an improved first weed drop disposal location, then to a second weed
all
in one smoothed curve while respecting the crop avoidance zone;
Date Recue/Date Received 2022-02-14
6
[0039] Fig. 14 is a top down view of the example lanes of the field of Fig. 8,
displaying
an example of a full implement picking and disposing trajectory going from a
first
weed to a last weed while stopping at weed disposal locations in between each
weed;
[0040] Fig. 15 is a top down view of the example lanes of the field of Fig. 8,
displaying
another example of a full implement picking and disposing trajectory going
from
a first weed to a last weed while stopping at weed disposal locations in
between
each weed; and
[0041] Fig. 16 is an example flow chart of possible steps carried out to pick
and dispose
of weeds along a picking and disposing trajectory that minimizes the time
needed
to pick and dispose of weeds, in accordance with one implementation.
DETAILED DESCRIPTION
[0042] Broadly described, the method of weed disposal described herein can be
used on
or with an agricultural vehicle, such as an autonomous weeding machine,
capable of
navigating farm fields while identifying weeds and pulling them using a
controllable
implement, such as a robot gripper. The method focuses on disposal of weeds or
other
undesired plants growing close to the crop in an appropriate location after
removal. The
weeds are picked by physically gripping them and pulling them from the earth,
and then
discarding them at a separate location to avoid rerooting. Although reference
is made to
weeds throughout this application, it is understood that crops or other
objects of interest
may similarly be picked or disposed of in accordance with the present
disclosure.
[0043] A key performance indicator of any weeding method is the rate at which
weeds
may be removed. Accordingly, a weed disposal method should have as little
impact on the
weed removal rate as possible. For example, once a weed is removed from the
ground,
the simplest disposal method would be to release the weed in the same spot. As
mentioned previously, however, such a disposal method may result in the weed
rerooting
at the same spot. Accordingly, the present method aims to minimize the impact
on the
weed removal rate while disposing of weeds in an appropriate manner.
Date Recue/Date Received 2022-02-14
7
[0044] With reference to Fig. 1 and in accordance with one implementation,
there is shown
an agricultural vehicle 10 for picking weeds or for thinning crops from an
agricultural field.
The vehicle 10 shown therein is configured to travel along lanes in an
agricultural field,
over longitudinal sections of soil, to pick, remove and dispose of weeds. In
possible
implementations, the vehicle is an autonomous vehicle, comprising cameras,
sensors and
processors to control electric motors and wheels connected to the motor(s). In
other
possible implementations, the vehicle can be a tractor or other agricultural
vehicle, driven
by a driver. The crops can be of different types, including vegetable crops,
such as
lettuces, carrots, and onions.
[0045] With reference to Figs. 2 and 3, some components of the autonomous
vehicle 10
are shown disassembled. The components shown include a frame, or chassis, 12
and a
picking and disposal module 100 mounted thereon, including an agricultural
implement
200 for picking weeds. The picking module can be affixed underneath the
vehicle or
attached to one of its sides and tracked. The agricultural implement may also
be
configured and adapted for thinning the field, where in this case, selected
immature crops
(or otherwise unwanted plants) are removed to provide enough room to allow
stronger
crops to grow. As such, the picking and disposal module can also be referred
to a weeding
and/or a thinning and disposal module. In possible implementations, such as
the one
illustrated, the agricultural implement 200 can be a gripper 210. In the
illustrated
implementation, there is provided three grippers 210a, 210b, 210c, each
capable of weed
picking and removal in a predefined zone underneath the vehicle.
Alternatively, there may
be provided a single gripper for serving the entire zone, or multiple grippers
serving
overlapping zones.
[0046] Still referring to Figs. 2 and 3, and also to Fig. 4, in one
implementation, it is
preferable that the grippers are configured to not only grab weeds from the
above-ground
portion of the weed (e.g., the leaves), but to pinch the roots of the weeds
and fully retrieve
them from the ground. Picking at the root of the weed or plant may reduce the
likelihood
of the weed regrowing whether in place or displaced elsewhere. In the
illustrated
implementation, the grippers 210 are installed at an end of a delta robot 150,
with the delta
robot 150 providing translational and rotational movement, as will be
explained in more
detail. Accordingly, the grippers 210 may grasp weeds at first location and
dispose of them
at a second location.
Date Recue/Date Received 2022-02-14
8
[0047] In a possible implementation, and with reference to the example
illustrated in Fig.
4, the gripper may comprise two jaws or plates 212, which can be controlled
such that
they can be spaced apart or brought together to pinch the weeds or plants,
such as the
stems, preferably near or within the soil, close to the roots. Other implement
or gripper
configurations are possible.
[0048] The picking and disposal module 100 may comprise frame 102 onto which
the
grippers 210 and other parts can be mounted, as well as a processing module
400
comprising one or more processor(s) 414, including a CPU, and non-transitory
memory
412, including volatile and non-volatile memory, for storing and carrying out
computer
instructions (see Fig. 16). The processing module 400 can be mounted within of
the
picking and disposal module 100 or on the vehicle 10, such that several
picking modules
can share the processing and/or storage capacity. In possible embodiments, a
portion of
the processing capacity can be remotely accessed. Different configurations are
possible.
The frame 102 can include structural members, such as tubes, and side plates
to protect
the components of the modules from dust, mud and/or water. The frame 102 may
also
include connecting members (not shown) adapted to connect to the frame 12 of
the
agricultural vehicle 10. In possible implementations, the picking and disposal
module 100
can be a distinct, independent module, mountable onto different types of
agricultural
vehicles, which may or not be autonomous vehicles. In other implementations,
the picking
and disposal module 100 can be provided as part of an integral to the vehicle,
such as in
the illustrated implementation. The picking and disposal module may also be
attached and
tracked behind a tractor or other similar vehicle.
[0049] The picking and disposal module 100 additionally may include cameras
220
mounted thereon for identifying and determining the positioning of weeds,
extra or
immature plants and crops in the agricultural field. The cameras 220 may be
coupled to
lights mounted in proximity thereof for illuminating the field of vision of
the cameras 220,
providing improved images. In the illustrated implementation as shown in Figs.
2 and 3,
there is shown a picking and disposal module 100 with cameras 220 mounted onto
the
frame 102 of the picking and disposal module 100. Alternatively, the cameras
220 may be
mounted proximate to the gripper 210, such as on the end of the delta robot
150. In one
implementation, the cameras 220 may instead, or additionally, be mounted onto
the
vehicle 10. The lights coupled to the cameras 220 may also be mounted either
onto the
picking and disposal module 100, or the vehicle 10.
Date Recue/Date Received 2022-02-14
9
[0050] With the implementation shown, when the autonomous agricultural vehicle
10
passes in an agricultural field, the picking and disposal module 100 is
activated. Once the
picking and disposal module 100 is activated, the cameras 220 begin capturing
images of
the field. In one implementation, the cameras 220 are coupled to a transceiver
module for
transmitting images to a remote site where an artificial intelligence (Al)
system identifies
weeds and crops. In one implementation, the images transmitted by the cameras
220 are
interpreted by an operator at the remote site who marks crops and weeds, and
also
optionally immature crops that are to be removed for thinning. The cameras 220
may
transmit the images through any wireless communication system to the remote
site. In yet
other implementations, the Al module and image analysis can be conducted
locally, in a
processing device provided on the vehicle 10 and in communication with one or
more
controller(s) controlling the weed picking implement 200.
[0051] According to the images generated and marked with respect to weeds and
crops,
and possibly other plants such as immature crops, a map 500 may be generated
identifying the coordinates of weeds 520 and crops 510 in the area captured by
the images
(see Fig. 7). Weeds 520 may represent a singular weed or a group of weeds
which may
be handled by the agricultural implement 200. The weeds and crops appearing in
the
images captured by the image sensors or cameras can be first identified by
pixel analysis.
Their coordinates can then be determined. For example, a Euclidean map can be
generated, using Simultaneous Localization and Mapping (SLAM) algorithms,
which
allows building a map and localizing the vehicle and/or picking modules at the
same time.
In a possible implementation, pixels of the map 500 can be classified as weed
520, crop
510 or other, such as other types of plants or immature (or weak) crops, and
this
classification can be used to determine center coordinates of the plants, such
as weeds
and crops. In possible implementations, different types of weeds and plants
could be
detected, and the weed picking and disposal method could be adapted
accordingly. In a
possible implementation, machine learning models, such as deep neural network
models,
can be used to differentiate between weeds or immature crops and desired
crops. For
example, Al classifiers can be used to distinguish between crops and weeds,
based on
the outer contour of the plant.
[0052] In one implementation, the map 500 may instead, or in addition to the
cameras of
the picking and disposal module 100, be provided by another image capture
system such
as a satellite image. As the vehicle travels through the agricultural field,
the map 500 may
Date Recue/Date Received 2022-02-14
10
be continuously updated with images captured by the cameras. Additionally, a
speed of
the vehicle may be aligned with the camera capture rate and the processor. The
speed of
the vehicle may be calibrated so that the vehicle moves through the field at a
rate that
allows for the cameras and the processor to capture and process images, and to
then
effect weed picking and disposal by the gripper 210.
[0053] The processing module 400 is configured to receive inputs, such as the
map 500
or the weed and crop coordinates, process the inputs, and output data for
controlling the
gripper 210. For example, the map 500 or map data may include coordinates of
weeds
520 and crops 510. The coordinates can be provided or accessed through input
ports 410
to the memory 412 and the processor 414, the processor 414 generating a
position and/or
other parameters for feeding motor drives (such as speed, accelerations,
decelerations)
along a trajectory for the gripper 210 to move and rotate to, based on the
received
coordinates, and sending the position information (or other drive parameters)
via output
ports 416 to one or more controller (such as motor drives) for controlling the
gripper 210.
The controller controls the movement of the grippers 210 based on the
coordinates of the
weeds 520 and crops 510, by adjusting actuators associated with X, Y and Z
motion of
the implement 200 or gripper 210. In one implementation, the controller
comprises one or
more actuators for controlling movement of the gripper 210 and another
actuator for
actuation of the jaws (opening and closing) of the gripper 210. In one
implementation, the
set of actuators comprises four actuators, each one corresponding to
translation along the
x, y and z axes, as well as rotation around the z axis. The controllers can
comprise motor
drives. The controller controls the movement of the grippers 210 based on the
coordinates
of the weeds 520 and crops 510, and more specifically based on a picking and
disposing
trajectory determined by the processing module 400. For thinning purposes, the
position
of the implement can be optimized to maximize the number of immature crops to
remove
near the desired, stronger crops.
[0054] In one implementation, the processor 414 determines crop lanes 550 and
weed
disposal lanes 560 based on the coordinates of the weeds and crops. The
disposal lanes
correspond to regions where the weeds are to be disposed. Many crops are
planted in
lanes to simplify picking and planting and to ensure adequate space is
provided for each
crop. Accordingly, in one implementation a crop lane 550 may be defined as
having a
length spanning a first crop to a last crop generally in a longitudinal (or y-
axis) direction in
an agricultural field, and a width in a lateral (or x-axis) direction which
respects a minimum
Date Recue/Date Received 2022-02-14
11
distance between each crop and an acceptable weed disposal location. For
example, a
minimum distance of 0.5 m may be desired between the crop and a potential weed
disposal location. If the crops are planted in a straight line, the crop lane
will span 1 m in
width, and the disposal lane 560 will comprise any point after that while
respecting the
crop lanes 550 of adjacent crops. In one implementation, a collection vehicle
may then go
through the disposal lanes 560 to collect any weeds disposed thereon. The
above values
have been provided as examples. Any other value as deemed appropriate, for
example
based on the type of crop or local soil conditions, may be selected.
[0055] The disposal of the weeds along an agricultural field is carried out
while respecting
constraints. The constraints may include any one or a combination of the
following. In one
implementation, the constraints include a minimum distance to the crop. A
minimum
distance to the crop prevents the agricultural implement from getting closer
than a
specified distance to the crop, whether the distance is lateral and/or
vertical. This
constraint may avoid dirt being dropped on the crop or ensure collision
avoidance with the
crop (see crop avoidance described further below). In one implementation, the
constraints
include setting of the crop lane width. The crop lane width determines a
minimum distance
that the weed must be dropped away from the crops. In one implementation, the
constraints include a maximum distance between a weed and a crop, so that any
weeds
exceeding this distance are ignored. This constraint may improve weed
removal/disposal
rate by avoiding removal of all possible weeds, and instead only focusing on
weeds that
may be most likely to affect crop growth.
[0056] Accordingly, the agricultural field can be mapped by either a binary
crop lane 550
and disposal lane 560 configuration (as shown in Fig. 5), or one where the
crop lane 550
corresponds to a minimum distance from the crop 510. The disposal lane 560 may
then
be yet further away from the bounds of the crop lane 550 (as shown in Fig. 6).
In the
configuration illustrated in Fig.6, the weeds 575 outside the crop lanes 550
are ignored by
the agricultural implement 200, while the weeds 520 removed from the crop
lanes 550
must be disposed of in the disposal lanes 560, and not in the intermediate
lanes 570
between the crop lanes 550 and disposal lanes 560. This may ensure, for
example, that
all weeds 520 are disposed of in a narrow corridor where they may later be
collected by a
tractor or any other suitable method/device.
Date Recue/Date Received 2022-02-14
12
[0057] The above constraints relate to weed disposal but may also be coupled
with
constraints related to weed removal to provide clear parameters for the
agricultural
implement to follow during weed removal and disposal as the vehicle 10 moves
in an
agricultural field.
[0058] With reference to Fig. 7, there is provided an example map 500 of an
agricultural
field. If weed removal consisted of removing a single weed, such as weed 520a,
the
simplest way to remove said weed from the crop lane 550 would be to evaluate
the
distance between the weed 520a and the two disposal lanes 560 adjacent to the
crop lane
550. The processor would then simply choose the closest disposal lane 560 and
guide the
agricultural implement along a straight horizontal line from the weed to the
disposal lane
560 and drop the weed in the closest point within the disposal lane 560. In a
possible
implementation, the processing module may be configured and adapted to control
the
implement for picking more than one weed at a time, such as when the weeds
form
clusters of weeds. In this case, the disposal process remains the same, that
is, the picked
weeds must be dropped in the disposal lanes, while preferably avoiding going
over and/or
near the crops.
[0059] With reference to Figs. 8 and 9, in a crop lane 550 where there are
multiple weeds
the processor should consider not only a single weed, but all the weeds that
need to be
removed across the crop lane 550. The algorithm may then determine an optimal
path that
.. would result in the shortest distance while visiting all the weeds and weed
disposal
locations present in the crop lane 550. As illustrated in Fig. 8, simply
drawing a horizontal
line from the first weed 520a to the nearest disposal lane 560 (while also
avoiding the
crop) and then attempting to reach the second weed 520 may result in a path
that is longer
than if the processor considered the location of the second weed 520b before
determining
which disposal lane 560 to dispose of the first weed 520a in, as illustrated
in Fig. 9. The
path to the disposal lane may additionally not simply be a horizontal line,
but diagonal.
[0060] An additional factor for the processor to consider is that the vehicle
10 is moving
through the crop lane 550, and so there is a finite amount of time whereby the
agricultural
implement 200 can remove a particular weed before it is out of range of the
agricultural
implement 200. Accordingly, the solution to removing all the weeds is not
purely abstract,
as in the travelling salesman problem but also constrained by practical
considerations
such as limited time to both remove and dispose of a weed, as well as the
computational
Date Recue/Date Received 2022-02-14
13
time required to calculate the optimal path. In one implementation, the
vehicle 10 may
accelerate, slow down, or stop along a crop lane 550 depending on the number
of weeds
or the time required to remove and dispose of weeds. In one implementation,
the speed
of the vehicle 10 may be constant. Accordingly, as the map 500 is updated with
movement
of the vehicle 10, the optimal path may be iteratively built and modified as
the agricultural
vehicle 10 travels through the agricultural field as new solutions become
possible.
[0061] In the travelling salesman problem, there is provided a number of
points that must
be visited by the hypothetical salesman, the solution algorithm to which
allows for all points
to be visited in the shortest distance or time. The problem is complicated to
solve
analytically due to the rapidly increasing complexity of the problem, which is
on the scale
of n! (with n representing the number of points). For example, the number of
possible paths
connecting a problem having 11 points is approximately 40 million, which is
approximately
35 million more possible paths than a problem with 10 points. There are a
number of
potential approaches or algorithms to the problem which obviate the need to
solve for each
of these possible paths then comparing all the results. One potential
travelling salesman
algorithm that does not require an analytical solution consists of simply
selecting the next
nearest point at every step, or the "nearest neighbour" algorithm. Such an
algorithm
reduces the complexity of the problem from n! to n2, a reduction in complexity
of several
orders of magnitude using the same examples of problems with 10 and 11 points.
Given
that zones or regions must be avoided between the picking and dropping
locations, to
avoid dropping soil onto crops for example, simply applying a travelling
salesman
algorithm may be insufficient. As such, a possible implementation of the
method comprises
applying a travelling salesman algorithm, which reduces travel distance to the
disposal
lanes, modified with a collision avoidance algorithm, which avoids travelling
over the crops
while disposing of the weeds.
[0062] In one implementation and in consideration of the above, the processor
may use a
"nearest neighbour" algorithm to determine the optimal path. In this
algorithm, the weed
nearest to the agricultural implement is selected, such as first weed 520a in
the
embodiment illustrated in Figs. 8 and 9, to start at for first picking and
disposal. The
agricultural implement then selects the second weed 520b for picking and
disposal by
determining the next nearest weed. The second weed 520b may then be picked
after the
first weed 520a has been disposed of in a disposal lane 560. The following
weed would
then be selected based on the nearest weed to the previous weed disposed of in
the
Date Recue/Date Received 2022-02-14
14
disposal lane, and the implement would visit every nearest weed step by step
until every
weed in the crop lane 550 has been removed. Alternatively, the processor may
consider
a plurality of potential paths depending on different starting weeds, then
choosing the
optimal path that may result in the least time or distance travelled for the
agricultural
implement to dispose of all the weeds. The "nearest neighbour" algorithm has
been
explained due to simplicity above, though any other algorithm for determining
the optimal
path may alternatively be used. In one implementation, the processor iterates
up to 3
potential solutions using a different starting weed and selects the optimal
solution based
on shortest total distance. In another implementation, the processor iterates
up to 6
potential solutions based on a different starting weed and selects the optimal
solution
based on shortest total distance. The number of iterations may be limited to
set an upper
limit on the use of computational resources and based on the time available
between two
picks. Other iteration values may also be selected. This possible "nearest
neighbor"
implementation can also be modified with a collision avoidance algorithm,
which avoids
travelling over the crops while disposing of the weeds.
[0063] With reference to Fig. 10, the path of the agricultural implement 200
should be as
short as possible. Each line between two points is a path segment representing
the path
of the agricultural implement. Each path segment corresponds to a linear
distance (straight
line) between the coordinates of a weed and the coordinates of a weed disposal
location.
If a straight line is drawn from the weed disposal location 530a to the second
weed 520b,
it would be the shortest distance between said locations, as shown in Fig. 10.
If a straight
line is drawn concatenating every weed to its respective weed disposal
position to
generate several concatenated path segments, an initial path corresponding to
the picking
and disposing trajectory is generated. The process of determining the picking
and
disposing trajectory may thus comprise a step of concatenating the different
path
segments, each path segment corresponding to a linear distance between one of
the weed
coordinates and disposal coordinates located within the disposal lanes.
[0064] With reference to Fig. 11, in one implementation each crop 510
comprises a crop
avoidance (or collision avoidance) zone 512. The crop avoidance zone 515
extends
outwardly from the crop 510 and may be defined by a field which pushes out any
point of
the path segment passing over the crop outwards. The field may, for example,
be
generated using a virtual potential field function in the algorithm. The
pushed-out points
may be connected by a series of discrete lines 514 representing the path of
the agricultural
Date Recue/Date Received 2022-02-14
15
implement 200. The discrete lines may then be smoothed to form one continuous
curve
516 from the first weed 520a to the second weed 520b, including the weed
disposal
location 530a, as illustrated in Fig. 12. The path comprising the concatenated
path
segments, modified to avoid the crop avoidance zones, can thus be smoothed.
The
agricultural implement may additionally be able to travel along a smooth
curve, such as
the continuous curve 516, faster than a number of straight sections, such as
the discrete
lines 514.
[0065] In a next step, it may also be found that the optimal path no longer
includes the
weed disposal location 530a simply being placed at the shortest horizontal
distance from
the first weed 520a. The weed disposal location 530a may be moved to
correspond to a
location along the continuous curve 516 which overlaps with the weed disposal
lane 560.
The algorithm may therefore adjust the positioning of the weed disposal
location 530a as
a result of the virtual potential field pathing. The resulting curve as
illustrated in Fig. 13
may be shorter, for example in comparison to the curve illustrated in Fig. 12.
In one
implementation, the coordinates of the weed disposal location 530a may be
optimized
using a gradient descent algorithm to produce the shortest path between the
two weeds
520a, 520b that passes through the disposal lane 560.
[0066] In one implementation, the processor may generate at least three
potential picking
and disposing trajectories comprising the smoothed concatenated path segments
with
optimized weed disposal locations, selecting the one having the shortest
distance or
representing the shortest amount of time for the implement to follow, as
described below.
[0067] The real constraints of the robot configuration, such as torque limits
of the
controller, may also be considered by the algorithm when evaluating potential
paths to
provide a speed of the agricultural implement along each path. For example,
although the
.. steps described above relate generally to a picking and disposing
trajectory based on
physical distance only, the processor can additionally, or in lieu of, provide
acceleration
and speed information to the controller so that acceleration and speed may be
determined
along each point in the picking and disposing trajectory. The processor may
then be able
to select a path that, in view of the dynamics of the robot or constraints of
the controller,
is the optimized path representing the shortest amount of time for the
implement to follow.
The determination of the picking and disposing trajectory can thus comprise a
step of
calculating a maximum speed of the implement based on the shape of the
trajectory and
Date Recue/Date Received 2022-02-14
16
based on constraints inherent to the controller controlling movement of the
implement,
such as the different motor drives constraints. Similarly, determination the
picking and
disposing trajectory can thus comprise a step of calculating accelerations and
decelerations based on limits of the controller. Constraints inherent to the
controller(s) can
include, for example, minimum and maximum acceleration and deceleration,
minimum
and maximum speeds, which can be translated into voltage ranges.
[0068] With reference to Figs. 14 and 15 there is shown two example solutions
comprising
two separate paths for the agricultural implement to follow, including
different weed
disposal locations, for removing all weeds in the crop lane 550. The solutions
may be
evaluated by the algorithm and optimized with respect to time, or distance
travelled, to find
the solution which is most efficient. The algorithm may be solved to N
iterations comprising
multiple solutions using varied orders of picking and disposing, as well as
varied disposal
locations, to find the most efficient solution. In one implementation, the
algorithm is solved
out to 3 iterations. In one implementation, the algorithm is solved to 6
iterations. The
number of iterations may be limited to set an upper limit on the use of
computational
resources. Other iteration values may also be selected. The method is thus
conceived
such that determining and/or updating the weed picking and disposing
trajectory can be
realized by the processor in less time than the time needed between two picks.
[0069] With reference to Fig. 16, there is provided an example flow chart
based on the
embodiments disclosed herein. In a first step, images are captured from a
camera (or
cameras), mounted onto the autonomous vehicle. In a second step, the
processing
module 400 of the vehicle then processes the images to generate a map 500 of
the
captured area which comprises coordinates of the weeds and crops. In a third
step,
disposal lanes 560 (i.e., positional information) are inputted to the map 500.
In a fourth
step, a disposal and picking trajectory that minimizes the time needed to pick
and dispose
of weeds is determined. In a fifth step, the agricultural implement follows
the trajectory to
pick and dispose of weeds.
[0070] Although reference and examples have been made to weed disposal in
accordance with one embodiment of the present disclosure, it is equally
envisaged that
other uses may be possible. For example, the agricultural implement may,
instead of
disposal weeds in a field, dispose of extra or immature crops. Crop thinning
is the process
of removing young crops from a field to make space for other, stronger crops.
For example,
Date Recue/Date Received 2022-02-14
17
carrots are often planted in a row in a field. It is sometimes necessary to
remove some
carrots from the row to allow other carrots to grow to full size. It may also
be necessary to
remove carrots in areas where growth has been less than desired, due to poor
sunlight or
other factors. The removed carrots may then be replanted elsewhere or disposed
of.
Accordingly, instead of identifying, removing and disposing of weeds from a
field, the
present disclosure may instead be directed to identifying, removing and
disposing of
immature or selected crops (i.e., unwanted plants), such as young carrots.
Date Recue/Date Received 2022-02-14
