Language selection

Search

Patent 2147602 Summary

Third-party information liability

Some of the information on this Web page has been provided by external sources. The Government of Canada is not responsible for the accuracy, reliability or currency of the information supplied by external sources. Users wishing to rely upon this information should consult directly with the source of the information. Content provided by external sources is not subject to official languages, privacy and accessibility requirements.

Claims and Abstract availability

Any discrepancies in the text and image of the Claims and Abstract are due to differing posting times. Text of the Claims and Abstract are posted:

  • At the time the application is open to public inspection;
  • At the time of issue of the patent (grant).
(12) Patent Application: (11) CA 2147602
(54) English Title: FISH SORTING MACHINE
(54) French Title: MACHINE POUR LE TRI DES POISSONS
Status: Dead
Bibliographic Data
(51) International Patent Classification (IPC):
  • B07C 5/342 (2006.01)
  • A22C 25/04 (2006.01)
  • B07C 5/10 (2006.01)
  • B07C 5/12 (2006.01)
(72) Inventors :
  • STRACHAN, NORVAL JAMES COLIN (United Kingdom)
(73) Owners :
  • THE MINISTER OF AGRICULTURE FISHERIES AND FOOD IN HER BRITANNIC MAJESTY' S GOVERNMENT OF THE UNITED KINGDOM OF GREAT BRITAIN AND NORTHEN IRELAND (United Kingdom)
(71) Applicants :
(74) Agent: FETHERSTONHAUGH & CO.
(74) Associate agent:
(45) Issued:
(86) PCT Filing Date: 1993-10-19
(87) Open to Public Inspection: 1994-05-11
Availability of licence: N/A
(25) Language of filing: English

Patent Cooperation Treaty (PCT): Yes
(86) PCT Filing Number: PCT/GB1993/002151
(87) International Publication Number: WO1994/009920
(85) National Entry: 1995-04-21

(30) Application Priority Data:
Application No. Country/Territory Date
9222338.7 United Kingdom 1992-10-23

Abstracts

English Abstract






A fish sortring apparatus comprising a conveyor belt (1) for carrying one or more rows of fish parallel to its direction of
travel, a means (4) for illuminating the fish, a colour video camera (3), a video memory having a number of storage areas connect-
ed to the video camera (3), a means for storing the camera derived image values of a fish from the one or more rows in a storage
area of the memory, a means for determining edge values of the image and storing them, a means for using the stored edge values
to generate shape descriptors the values of which are stored and compared with those already in the memory or the logic of a
computer processor (10) associated therewith, and a means for determining the colour and/or light intensity of predetermined
areas within the image periphery and storing and comparing values therefor with those stored in the memory. The descriptors are
subject to descriminant analysis whereby a signal is generated corresponding to the score derived from the analysis such that fish
on the conveyor are deflected into a desired reception area.


Claims

Note: Claims are shown in the official language in which they were submitted.




CLAIMS.

1. An apparatus for indicating the type of a fish comprising a means
for receiving light from the fish and generating an image of it in the
form of a set of values therefrom, a means for storing the set of
values in a computer memory, a means for assigning the values to areas
of the image of the fish, and a means for using these to determine and
indicate the type of the fish, characterised in that the apparatus
comprises a means for determining values indicative of the colour of
all or some of the areas of the fish and that the type of fish is
indicated by a means which compares the values indicative of colour by
area of the fish with predetermined such values stored in the memory
or a processor associated with it, these being characteristic of a
particular type of fish, and a means which designates the fish as that
type having stored values to which the determined values correspond.

2. An apparatus as claimed in claim 1 wherein the indication is used
to control a sorting mechanism for directing the fish to a
predetermined reception area associated with the type of fish with
which it is designated to most closely correspond.

3. An apparatus as claimed in claim 1 or claim 2 wherein the
orientation of the image of the fish is determined with respect to
which end is the front end of the fish and which is the back end
and/or which is the upper surface and which is the lower surface,
and the colour and shape descriptors are related to these.

4. An apparatus for indicating the type of a fish comprising

(a) a means for storing one or more sets of values indicative of
predetermined length, widths and areas, and the distribution of colour
in these areas, each set characteristic of an image of a particular
sort of fish;

21

(b) means for receiving the image of a fish, the type of which is to
be indicated, in a camera;

(c) means for receiving the image from the camera in the form of a set
of values and storing these in a computer memory means;

(d) means for determining boundary values of the image of the fish;

(e) means for determining values representing predetermined lengths,
widths and/or areas of the image and their ratios,

(f) means for determining values indicative of colour of predetermined
areas of the image,

(g) means for determining which end of the image is representative of
the front end and which is representative of the back end of the fish
and/or means for determining which is the upper and which is the lower
surface, and comparing the oriented values from (d) (e) and (f) with
corresponding values stored in the means (a) as characteristic of a
particular sort of fish;

(h) indicating means which indicates the type of fish to be that
having stored values corresponding with the determined values.

5. An apparatus as claimed in claim 4 wherein the indication means
controls a mechanisms which directs the fish to a predetermined
sorting reception area associated with the type of fish.

6. A sorting apparatus as claimed in any one of claims 1 to 4 wherein
the fish are presented to a camera and their ratios of predetermined
length, widths and/or areas, and the values indicative of colour in
all or some of the areas, as derived from the image produced by that
camera, are used to classify the fish, thus generating a control
signal which directs the fish to an appropriate destination.

22

7. An apparatus comprising a conveyor belt for carrying one or more
rows of fish, a means for illuminating the fish, a colour video
camera, a video memory having a number of storage areas connected to
the video camera, a means for storing the camera derived image values
of a fish from the one or more rows in a first storage area of the
memory, a means for determining the image size and orientation from
stored edge and/or colour values, a means for using the stored edge
values to generate shape descriptors the values of which are stored
and compared with those already in the memory or logic of a computer
processor associated therewith, and a means for determining the colour
and/or light intensity of predetermined areas within the image
periphery and storing and comparing values therefor with those stored
in the memory or logic.

8. An apparatus as claimed in any one of the preceding claims
comprising a colour video camera with red, green and blue signal
outputs connected to a computer memory whereby a pattern of pixels
corresponding to the image to be analysed can be produced, a processor
for generating the finite elements of aspect and area ratios and the
average red, green and blue values of selected parts of the image, the
processor being connected to the memory, a comparator for comparing
these elements with values previously generated from calibration of
the apparatus using a number of fish of known type and stored in
the memory or incorporating in the logic of a processor associated
therewith, and a means for generating a score indicative of the number
whether or not the fish is of a given type.

9. An apparatus as claimed in any one of the preceding claims
wherein the image of the fish is recorded by the camera under both
backlit and frontlit lighting conditions.

10. An apparatus as claimed in any one of the preceding claims
wherein signals from the red, green and blue (RGB) outputs from the
video camera are digitised to eight bits per colour using a frame

23

grabbing board and then stored on a computer workstation where they
are processed using algorithms.

11. An apparatus as claimed in any one of the preceding claims
wherein the orientation of the fish with respect to which end is the
head and which end is the tail is carried out by determining the width
of the image at each of two points, each placed between a
respective end and the mid-length of the image, and designating the
larger width to be indicative of the head.

12. An apparatus as claimed in claim 11 wherein the two points are
situated at a distance approximately one tenth of the total image
length from each end.

13. An apparatus as claimed in any one of the preceding claims
wherein the orientation of the fish with respect to which surface is
the upper surface and which surface the lower surface is carried out
by designating the lighter or less coloured surface the lower surface.

14. An apparatus as claimed in any one of the preceding claims
wherein the image values are used to generate aspect ratio (As) and
area ratio (Ar) shape descriptors.

15. An apparatus as claimed in claim 14 wherein those images having
aspect ratios of over 0.40 are indicated as those of flatfish and
those below 0.40 as roundfish.

16. An apparatus as claimed in claim 15 wherein the image of the
fish has an aspect ratio of over 0.40 and is used to generate a grid
constructed using vertical lines from the central line of symmetry.

17. An apparatus as claimed in claim 14 wherein the image of the fish
has an aspect ratio less than 0.40 and is used to generate a grid of
quadrilateral elements constructed from lines normal to its central
line of symmetry.


24

18. An apparatus as claimed in either of claims 16 or 17 wherein
the grid has a number of width values generated from the fishes nose
to its tail.

19. An apparatus as claimed in any one of the preceding claims
wherein the average R, G, B values of the image are determined for the
tail, the nose and for some or all of the elements of the shape grid,
having a computer processor which carries out multivariate analysis
processing using these values as descriptor values for determining the
species of the fish.

20. An apparatus as claimed in claim 19 wherein the width values are
used as descriptor values for multivariate analysis processing
wherein they are used to determine the species of the fish.

21. An apparatus as claimed in claim 19 or 20 wherein the linear
combination of the variables that best distinguishes between the
different species of fish to be sorted is installed in the logic of
the processor that carries out the multivariate analysis.

22. An apparatus according to any one of claims 19 to 21 wherein the
multivariate analysis is discriminant analysis.

23. An apparatus as claimed in any one of claims 19 to 22 wherein
discriminant functions are generated by introducing one variable at a
time wherein if the new function satisfies discriminant analysis
criterion then it is accepted and if not, it is rejected along with
the last variable introduced.

24. An apparatus as claimed in claim 23 wherein the classification
score for species group i, is given by a linear combination of Cij's:

jmax
Ci = .SIGMA. Cij Q(j) + Cio (6)
j=1



where i = 1,2 ... n (n = no of species of fish), Cio is a constant
and the Q(j)'s are the raw values of the variables.

24. An apparatus as claimed in any one of claims 21 to 23 wherein the
variables used comprise width shape descriptors, a length descriptor,
the R, G, B average values of the nose and front third shape grid
elements of the fish, the R, G, B average values of the middle third
shape grid elements of the fish, and the R, G, B average values of
the tail and tail third shape grid elements of the fish.

25. An apparatus as claimed in any one of claims 21 to 24 wherein
fish to be sorted for species are given n scores, one from each
classification function, and are sorted as the species whose function
had the highest score.

26. An apparatus as claimed in claim 25 adapted such that when the
species of a calibration set of fish inputted to the processor along
with the variables, the discriminant analysis algorithm produces the
Fisher classification coefficients, such that when the variables of
the test set of fish are introduced to the processor it can determine
the species of these fish using the Fisher linear discriminant
functions.

27. An apparatus as claimed in any one of the preceding claims
suitable for indicating type of and/or sorting desmersal fish
characterised in that it first sorts the fish images according to
aspect ratio (As), then according to area ratio (Ar), as defined
herein, then subject to multivariate analysis.

28. An apparatus as claimed in any one of claims 1 to 27
substantially as described in any one of the Examples 1, 2 or 3.

29. A method for operating an apparatus as claimed in any one of the
claims 1 to 27 substantially as described herein in any one
of Examples 1, 2 or 3.

Description

Note: Descriptions are shown in the official language in which they were submitted.


~vo 94/09920 2 1 ~ 7 6 0 ~ PCT/GB93/02151
-




FISH SORTING MACHINF.

The present invention relates to an apparatus for sorting flP~ible
food products, particularly fish, and a method for operating that
apparatus. Particularly provided is an apparatus capable of sorting
fish into species or groups thereof on the basis of shape and colour.

The provision of apparatus for automatically sorting fish by speri~s~
size and weight, for example on board fishing or factory vessels,
makes it possible to automate the collection of data on catch and
discards. This has important implications for management of fish
stocks and operation of the common fisheries policy.

The need for automatic sorting of fish by species has been discussed
by Tayama et al (Reito (Tokyo) Refrigeration Vol. 57 No. 661 (1982)
pp. 1146-1150; Shimdate, M. Mitsubishi Denki Hiko (Tokyo). Vol.
56 No. ~ (1982) pp. 44-48. ), Wagner et al (~eb~n! ittelindustrie
Vol.30 No.8 (1983) pp. 375-376.;T eben~ ittel;n~ trie Vol. 34 No.1
(1987) pp. 20-23. and Strachan et al ( Pattern Recognition Vol. 23
No. ~ (1990) pp. 539-544.; 'Computer Vision for the fish industry'
edited by Pau, L.F. and Olafsson, R., Marcel Decker (1992)).

These researchers have described the shape of fish by length
measurements, where length is defined as a horizontal straight line
from the tip of the head to the base of the tail; and by width
measurements where width is defined as the distance from the top to
the bottom of the body measured at a number of equidistant points
along the length. They found that they could sort 4 species of fish
with a reliability of 95% and 7 species with an accuracy of 90%. A
further suitable apparatus for carrying out simple fish sorting using
aspect ratio and area ratio is described in EP 0331390.

For a sorting ~chin~ to be used on board a fishing vessel, up to 30
species of fish may have to be sorted with a reliability of 99%. It

W O 94/09920 2 1 ~ 7 6 0 2 PCT/GB93/02151



is therefore necessary to use not only shape but also variables such
as colour or shA~in~ to provide additional sorting information.

Colour has been used for image analysis and pattern recognition, for
example to identify spray paint caps ( Berry, D.T. Pattern
Recognition Letters Vol. 6 (1987) pp. 69-75), to detect the colour
codes on resistors ( Bajon et al. 'Identification of multicoloured
objects using a vision module' Robot Vision and Sensory Control. 6th
Int. Conf. Paris, France. Vol. 10 (1986) pp. 21-30) and to guide
a robot to selectively pick up petri dishes (Massen et al Proc. Int.
Conf. Robot Vision and Sensory Controls (1988). pp. 115-122) ).
Colour has also been used to segment images into sets of uniform
colour regions ( Tominaga. Proc. 4th Int. Conf. on Pattern
Recognition, Cambridge, UK ( 1988) pp. 163-172; Celenk. 'An adaptive
~hine learning algorithm for colour image analysis and processing'
Int. Conf. on Manufacturing Systems and Technology, Cambridge, HA,
USA (1987) No. 4 pp. 403-412; Kl; nker et al ; SPIE Applications of
Artificial Intelligence. 937 (1988) pp. 229-244). and to detect
edges in images ( Nevatia. 'A colour edge detector' 3rd Int. Joint
Conf. on Pattern Recognition, Coronado, California, USA (1976) pp.
829-837).

The aim of the present invention is to provide an apparatus for
automated colour based sorting of irregular objects, eg. fish, which
is capable of being integrated with a shape descriptor system, eg
apparatus, into a system for sorting fish by species. An improved
shape descriptor system is also provided whereby the orientation of
the fish with respect to its front and back ends may be automatically
asessed and thus all other descriptors related to that.

In order to facilitate comparison of input descriptors with reference
descriptors in an automatic apparatus it is necessary to store the
latter in some form which can readily be referred to by automatic,
particularly electronic, means. Such reference descriptors are thus

~'O 94/09920 21 ~ ~ 6 ~ 2 PCT/GB93/02151



most conveniently stored as values in a computer memory or logic.

There are approximately 30 species of demersal and 5 of pelagic fish
landed in Scotland (Department of Agriculture and Fisheries for
Scotland 'Scottish Sea Fisheries Statistical Tables 1988'. Crown
Copyright (1989) 57 pp. ). A preferred apparatus according to the
invention sorts pelagic and demersal fish species and treats them
separately. This is convenient because they live in different parts
of the water column, demersal living on or near the seabed and pelagic
mainly in shoals near the surface, and hence they are ImlikPly to be
caught together.

The preferred apparatus of the present invention, and the method for
operating it, have been shown to be effective in experiments sorting 5
species of pelagic fish and 18 species of demersal fish; these latter
18 species representing more than 95% of all demersal fish landed in
Scotland. The invention offers a rel;Ahility of sorting the 18
species of demersal fish of 100% and of the 5 species of pelagic fish
of 98%. This presents a significant impro~ t over all other
results so far published, these quoting a sorting accuracy of only
90-95% for 4-7 species of fish.

The present invention provides an apparatus for indicating the type of
a fish comprising a means for receiving light from the fish and
generating an image of it in the form of a set of values therefrom, a
means for storing the set of values in a computer -~, a me_ns for
assigning the values to areas of the image, and a means for using
these to determine and indicate the type of the fish, characterised in
that the apparatus comprises a means for dete.- ining values indicative
of the colour of all or some of the areas of the fish and that the
type of fish is indicated by means which compares values indicative of
colour by area with predetermined such values stored in the memory or
a processor associated therewith, these being characteristic of a
particular type of fish, and a means which designates the fish as that

W O 94/09920 PCT/GB93/02151
21~7~02 4


type having stored values to which the determined values correspond.

Most advantageously the indication is used to direct the fish to a
predetermined reception area associated with the type of fish with
which it is designated to most closely correspond and the apparatus
thus is incorporated as part of an apparatus for automatically sorting
fish.

Most preferably the orientation of the fish with respect to which end
is that of the head and which is that of the tail is deteL- i n~ . This
is conveniently carried out by a novel method which determines the
width of the image at two points set a distance between each end and
the mid length of the fish image respectively, particularly about one
tenth of the total fish image length in from each end, as determined
in linear unbent form. Using this method the larger width measurement
is indicative of the head end.

rly, the orientation of a flat fish with respect to whether it
is top or belly side up may be assessed by interpreting colour as
indicative of the eye side surface and lack of colour as indicative of
the reverse side. With round fish such as cod, haddock, whiting and
saithe the upper half of the fish is always darker than the lower half.
Both head/tail and top/bottom orientation determination offer
significant advantage in identifying the fish by the present method.

More preferably the present invention comprises an apparatus for
indicating the type of a fish comprising

(a) a means for storing one or more sets of values indicative of
predetermined length, widths and areas, and the distribution of
colour and/or light intensity in these areas, each set characteristic
of an image of a particular type of fish

(b) means for receiving the image of a fish, the type of

~"0 94/09920 PCT/GB93/02151
-- 21 ~ 7602




which is to be indicated, in a camera;

(c) means for receiving the image from the camera in the form of a
set of values and storing these in a memory means;

(d) means for deteL ;n;ng boundary values of the image of the fish;

(e) means for deteL ;n;ng values representing predetermined lengths,
widths and/or areas of the image and their ratios,

(f) means for determining values indicative of colour of predetermined
areas of the image,

(g) means for determining which end of the image is representative of
the front end and which is representative of the back end of the fish
and/or means for deteL ;n;ng which is the upper and which is the lower
surface of the fish, and comparing the oriented values from (d) (e)
and (f) with correspon~;ng values stored in the means (a) as
characteristic of a particular sort of fish;

(h) indicating means which indicates the type of fish to be that
having stored values correspon~;ng with the detel ned values.

Most advantageously the indication (h) is used to direct fish to a
predetermined sorting area associated with their type.

The means for determining the front and back end preferably uses the
length and width data derived by (e) to carry out the method described
above based upon width measurements taken at each of two points, one
between each end and the mid-length, preferably at about one tenth of
the distance from each end. The means for deteL- ;n;ng top and bottom
surfaces preferably uses the values indicative of colour of
predetermined areas derived by (f) to carry out the method described
above whereby the darker or coloured surface is designated the top.

W O 94/09920 PCT/GB93/0215'
21~7602




Thus in a preferred embodiment the present invention provides a
sorting apparatus comprising an apparatus as described in either of
the embodiments above wherein the fish are presented to a camera, eg.
on a cor,veyor, and their ratios of predetermined length, widths and/or
areas, and values indicative of the colour of all or some of the
areas, as derived from the fish images produced by that camera, are
used to classify the fish, thus generating a control signal which
directs the fish to an app.opriate destination. Using preferred
'~'; ts of the present invention it is possible to indicate the
type of a fish when that fish may be presented in a variety of forms
due to its flexibility and defoL ~h; lity, ie. to indicate the type of
fish and preferably to direct the fish to an appropriate destination.

A particular form of apparatus of the present invention comprises a
conveyor belt for carrying one or more rows of fish, preferably in
rows parallel or perpendicular to its direction of travel, a means for
illuminating the fish, a colour video camera, a -ly having a number
of storage areas connected to the video camera output, a means for
storing the camera derived image values of the illuminated fish from
the one or more rows in a storage area of the I -Ly, a means for
deteL- ;ning the image size and orientation from stored edge values, a
means for using the stored edge values to generate shape descriptors
the values of which are stored and compared with those already in the
memory or the logic of a computer processor associated therewith, and
a means for deteL- n;nE the colour of predetermined areas within the
image periphery and storing and comparing values therefor with those
stored in the memory or the logic.

One suitable such means for deteL- n; nE shape descriptors and dividingthe image up into length, width and area values is described in EP
0331390, wherein values are stored and transferred between bit planes.

Typically apparatus of the present invention will comprise a colour
video camera with red, green and blue signal outputs connected to a

'O 94/09920 21 ~ 7 ~ 0 2 P ~ /GB93/02151



computer memory whereby a pattern of pixels correspon~ing to the image
to be analysed can be produced. This memory is connected to a
processor for generating the finite elements of aspect and area ratios
and the average red, green and blue values of selected parts of the
image. These elements, or descriptors, are compared with values
previously generated, eg. from calibration of the apparatus using a
number of fish of known type, ie. fish of known species, or built
into the computer logic, and a score generated indicative of whether
or not the fish is of a given type, eg. species.

It is preferred to determine the length, width and/or area values of
the fish by back-lighting it in the image receiving step. This is
conveniently achieved by use of a light source mounted below the
fish as it is presented to the means for receiving the image, ie.
the camera. Where a conveyor belt is used this is conveniently made
of a material which transmits at least a proportion of light emitted
from a source placed below it such that back-lighting can be achieved
thereby providing a distinct silhouette.

It is preferred to determine the colour values of the fish by
front-lighting the object in the image-receiving step. This is
particularly advantageously carried out using diffuse light generated
by directing light from one or more light sources onto a reflective
surface from which it passes onto the top of the object. Conveniently
the reflective surface is the inside a a housing in which the image
receiving means, eg. video camera, is mounted. Preferably the
reflective surface is of white colouration. The preferred method of
operating the apparatus of the invention calibrates the camera
outputs to no~ e them before operation.

It will be realised by those skilled in the art that signals generated
by the computer can readily be used to operate a series of deflectors
on a conveyor belt to deflect a fish to its determined sorting
destination where all fish of that type will be deflected to.

W O 94/09920 PCT/GB93/0215'
21il~ 602 8


TABLE 1: The groups of Aspect ratio and Area Ratio that are used
to classify fish into various groups according to the method of the
invention for use with the apparatus of the invention, including
the two main groups (Groups 1 and 2) requiring colour analysis

Fi.~h of Fi.~h of
All Fish A.~pect rnt~o Aren rntio
>0.40 >0.40 (Group 1 herein)
Angler fish Angler fish Dab
Catfish Dab Flounder
Cod Flounder Lemon Sole
Coley Lemon Sole Megrim
Dab Megrim Plaice
Flounder ~ Plaice Redfish
Haddock Redfish Sole
Hake Skate \ \ Witch
Lemon Sole Sole \ ~
Ling Witch \ 0.20-0.40
Megrim \ Angler fish
Plaice 0.20-0.40
Red mullet \ (Croup Z herein) ~
Redfish \ Catfish <0.20
Skate ~ Cod Skate
Sole Coley
Witch Haddock
Whiting Hake
Red mullet
Redfish
Whiting1 >0.80
< Z / Ling
Hake \ <0.80
Ling Hake

O 94/09920 PCT/GB93/02151


A method for operating the apparatus of the present invention is also
provided based upon use of multivariate analysis, particularly
discr;~;n~nt analysis and is that disclosed in the Examples described
below, although other multivariate data processing technoques such as
neural networking may equally be applied with success. The apparatus
and method of the present invention will now be illustrated with
reference the non-limiting examples, Examples l to 3 below, further
-'; ts falling within the scope of the claims will occur to those
skilled in the art in the light of these. Figures l to 8 are provided
to illustrate aspects of the Examples.

FIGURES

Figure l shows a shape grid constructed for a whiting in two different
positions, the grid being shown superimposed upon the fish silhouette.

Figure 2 shows a grid suitable for a flatfish as shown superimposed
upon the silhouette of a lemon sole (upper image) and a megrim (lower
image).

Figure 3 shows the position of one particular significant coloured
area (shown as a light area) in the grid for a flounder, a group l
fish by the method of Table l.

Figure 4 shows the position of some particular significant coloured
areas (shown as light areas) in the grids of catfish (upper silhouette)
and cod (lower silhouette), group 2 fish by the scheme of Table l.

Figure 5 shows the position of some particular significant coloured
areas (shown as light areas) in the grids of herring (upper
silhouette) and sprat (lower silhouette), pelagic fish.

Figure 6 shows a flow chart setting out the stages in determination of
the type of fish from their images, backlit and frontlit, as provided

W O 94/09920 P ~ /GB93/021''
214760~

by an embodiment of apparatus of the invention and a method of
operating it.

Figure 7 shows a diagrammatic representation of a perspective view of
a fish sorting apparatus of the present invention.

Figure 8 shows a diagrammatic representation of the arrangement
of the means for dete-- n;nF values indicative of colour of areas of
the object in a preferred embodiment of the invention and its
relation~hip to the other parts of the apparatus of Figure 7.

EXAMPT.T~ 1: App~r~tl-~ of the invention ~n~ ~ethod of ODer~ting it.

A fish sorting apparatus according to the present invention (see
Figure 7) was provided comprising quartz halogen lamps (4) GEC 300 W
power with a colour temperature of 2854 K, placed in a housing (2)
over a partially transparent conveyor belt (1). The interior of the
housing was matt white in colour such that the light from the lamps
(4) was reflected from its walls and ceiling to give the diffuse
lighting that is required to reduce the amount of specular reflection
from the glossy skin of the fish.

A video camera (3), Sony DXC 325PK three chip colour video camera, was
suspended from the top of the housing and before operation of the
apparatus the automatic gain control was switched off. The images of
the fish passed before the camera were taken both backlit using a
lightbox (5) placed between the runs of the conve~or to give the
silhouette, and frontlit to give the colour. Red, Green and Blue
values were fed from the camera to a computer arrangement (described
below and in Figure 8) which was used to determine the various values
from which descriminant analysis was used to determine fish type
(species). Outputs from the computer (9) were used to selectively
operate deflector elements (7) on a diverter conve~ur (6) such that
fish that had been typed were deflected into an a~ u~Liate one of

"O 94/09920 21~ 7 ~ 0 2 P ~ /GB93/02151



a number of species bins (8).

The colour values were obtained from the red, green and blue (RGB)
outputs from the video camera. These signals were then digitised to
eight bits per colour using a frame grabbing board (T ae~ne Technology
Inc. or Sprynt Colour Input Board) and then stored on a computer
workstation (486PC Sun 3-160C) where they were processed by an i860
CPU using algorithms written in the C pro~l- ne language.

Colour Calibration: The black (R(b), G(b) and B(b)) and white (R(w),
G(w) and B(w) camera output signal levels were obtained by lens
capping and use of a reference white tile with the camera aperture set
at f4 respectively. The red, green and blue (Rp, Gp, Bp) camera
outputs were then normalised according to the black and white levels,
using the following equations:


R = R~ - R(b) x 100% (l)
R(w) - R(b)

G = G~ - G(b) x 100% (2)
G(w) - G(b)

B = Bp - B(b) x 100% (3)
B(w) - B(b)

The calibration was done once a day using a Macbeth colour chart.

Simple ShAne Descri~tors. Two descriptors were used for crudely
sorting the fish. The first was the aspect ratio (As),


As = ~ (4)


W O 94/09920 214 7 6 0 2 PCT/GB93/02151



where w is the ~ width of the fish and l is the length of the
fish from its nose to the end of its tail.

The second was the area ratio (Ar)


Ar = - (5)
Ab




where Ar is the area of the front half of the fish, from the head to
the midpoint of the fish, and Ab is the area of the back half of the
fish, from the tail to the midpoint of the fish.

Position Reference Svstem An~ ShAne DescriDtors: Flatfish are fish
with large aspect ratios (As > 0.40) and do not bend very much. But
fish with small values (As < 0.40) bend and deform relatively easily
(Webb and Weihs, 'Fish Biomechanics', Praeger Puhlish~rs, New York,
USA. (1983)). To sort fish by species it is useful to analyse, for
example, how the width changes along its length and the colouration
across the fish. To do this a position reference coordinate system
should be estAhl;~hed which can acc ~dAte fish deformations and
bendlng .

Advantageously, the orientation of the fish with respect to which end
is that of the head and which is that of the tail was determined.
This was conveniently carried out by dete. ;n;ng the width of the
image at a distance of about one tenth of the total image length in
from each end. Using this method the larger measurement is indicative
of the head end. sir; 1 Arly, the orientation of a flat fish with
respect whether it was top or belly side up was assessed by
interpreting colour as indicative of the eye side surface and lack of
colour as indicative of the reverse side. With round fish such as
cod, haddock, whiting and saithe the upper half of the fish is always
darker than the lower half and this information is also usefully used

'~'~ 94/09920 21 ~ 7 6 ~ % P ~ /GB93/02151


in orienting fish images.

For esch fish image the computer processor was used to construct a
grid, typically consisting 36 quadrilateral elements as in Fig 1.
Strachan et al (J. Photographic Sci. 1990) have named this grid the
'shape grid' because it is derived from the shape of the fish. Shown
is the shape grid for a whiting in two different deformation states
showing that the grids bend with the fish.

For each fish type a shape grid was constructed as follows:

(i) The image of the fish silhouette was thresholded, made binary,
the principal axis was found and the fish silhouette oriented on it
horizontally (see Strachan: Pattern Recognition (1990) as above),

(ii) The width of the fish was determined at each point along its
length by dropping vertical lines from its top edge to its bottom edge
and the locus of the midpoints of these lines produced the central
line of the grid,

(iii) From the width profile of the fish along its length the tip of
the head and the narrowest point of the tail stalk were detected,

(iv) The central line was a~ o~imated at 10 segments of equal length
joining the tip of the head to the narrowest point of the tail stalk

(v) The normals of the central line were drawn from each of the
points joining these segments to both the upper and lower edge of
the fish image,

(vi) Each normal (both upper and lower) was split into two equal
segments, and

(vii) All of these segments were connected to produce the shape grid.

W O 94/09920 PCT/GB93/02151
2147~02
14

It can be seen that such a grid can be constructed to bend with the
fish and thus is suitably constructed by a computer from values in a
memory, ie. data in the form of pixels, representative of the image
of a bent fish. The bent grid may be transformed into a straight grid
or vice versa by processing with algorithms using method of Bookstein,
Transformations of quadrilaterals, tensor fields, and morphogenesis.
In P L Anton~ll; (Ed.) Mathematical Essays on Growth and the F - ~ellce
of Form. University of Alberta Press, Edmonton, Alberta (1985).

Such a grid is capable of overlapping itself and hence a different
sort of grid is required for fish with large aspect ratios, ie.
flatfish. The grid shown in Fig 2 is suitable for flatfish. It is
constructed using vertical lines instead of normals to the central
symmetry line. This grid will be poor at modelling the bending of
fish but as has been mentioned already flatfish bend very little and
hence this problem can be ignored.

From either type of shape grid a set of descriptors can be obtained to
describe the shape of the fish. These were the 10 widths of the fish
defined by the grids and the length of the fish from its nose to the
apex of its tail. These shape descriptors were used to sort the
fish by species.

Colour Descriptors: For the tail and the nose (which in the present
example consists of the front four shape grid elements) and for each
of the 36 other elements of the shape grid (or the simple shape grid),
the average R, G and B values are determined. This produces 114
variables (36 elements plus nose and tail with average R, G, B in each
= 38 x 3 = 114) which can be used to sort the fish by species.

Discriminant An~l ysis: Discriminant analysis (Nie et al 'Statistical
package for the social sciences, 2nd ed., McGraw-Hill, New York,
U.S.A. (1975). ) is used to process the shape descriptor and the
colour descriptor data derived from the image in the memory store,

~~'0 94/09920 21 ~ 7 6 0 2 PCT/GB93/02151


employing linear combinations of vsriables to distinguish between the
different species of fish.

The use of discriminant analysis requires the linesr combination of
the varisbles that best distinguishes between the different species of
fish to be found and used by the processor thst csrries out the
generation of the descriptors from the colour imsge. A stepwise
method wss used to generste discriminant functions by introducing one
vsriable at a time. If the new function satisfied the discriminant
analysis criterion then it wss sccepted. If not, it W8S rejected
along with the last variable introduced.

At the end of this process the resulting clsssification coefficients
ClJ (Fisher's linear discriminant functions) were obtained. Cl, the
classification score for species group i, is given by a linesr
combination of Ci~'s:

jmax
Cl = ~ ClJ Q(;) ~ C1O (6)
j=l

where i = 1,2 ... n (n = no of species of fish), C1O is a constant
and the Q(j)'s are the raw values of the variables.


The variables that were used in Examples l to 3 are:

l. The eleven shape descriptors (jmax = ll)

2. The R, G, B aversge vslues of the nose and front third shspe grid
elements of the fish (jmax = 39)

3. The R, G, ~ average values of the middle third shape grid elements
of the fish (jmax = 36)

W O 94/09920 2 1 4 7 S ~ ~ PCT/GB93/021~

16

4. The R, G, B average values of the tail and tail third shape grid
elements of the fish (jmax = 39)

The colour descriptors were split into three subgroups because in any
technique using multivariate analysis the analysis performs better if
the number of variables is reduced (Digby, P.G.N. and Kempton, R.A.
'Multivariate Analysis of Ecological Communities'. Ch~ ~n and HP11,
London, F.ngl Pnd ( 1987).

Since there were n species of fish, every new fish to be sorted for
species was given n scores, one from each classification function.
Fish were sorted as the species whose function had the highest score.

For the discriminant analysis the fish have to be split into two sets
(the calibration and the test set). The species of the calibration
set fish along with the variables is entered into the processor and
from these the discriminant analysis algorithm produces the Fisher
classification coefficients. The variables of the test set of fish
are then introduced to the processor and this determines the species
of these fish using the Fisher linear discriminant functions.

The arrangement of the processor with respect to the R, G, B input
from the video camera and its associated equipment is shown in Figure
8. The R, G, B analog inputs from the camera are fed to respective 8
bit analog to digital converters connected in turn to look up tables
and 2 megabytes of FIF0 store. This store is accessed by a 64 bit
data bus further connected to a video random access memory and a video
monitor, an 8 megabyte DRAM and the central processing unit (CPU).
The CPU has outputs which control the deflectors of the divertor col1veyor.

In experiments carried out using apparatus configured as described in
Example l and in Figures 5 to 7 there were at most 35 fish of each
species, hence where possible 20 fish of each species are used as the
calibration set. This is because the larger the calibration set the

~~~O 94/09920 ~ 1 ~ 7 S 0 2 P ~ /GB93/02151


better the discriminant analysis performs.

The aforesaid apparatus, with the processor pro l- cd to carry out
descriminant analyis as described was provided with fish of either
Demersal (Example 2) or Pelagic (Example 3) types, these not usually
requiring sorting together.

EXAMPLE 2: A~aratus for sorting Desmersal fish.

The apparatus was set up such that the computer processor first sorted
images according to aspect ratio As and then according to their area
ratio Ar (see Table l in introduction ). This identified angler fish,
skate, ling and hake. There remained two groups of fish which
required further sorting. Group l consisted mainly of flat fish and
some round fish which have large aspect ratios. Group 2 consisted
exclusively of round fish. 18 fish species were sorted in total.

For Group l the simple shape grid was generated for each of the
species of fish from which shape and colour descriptors were generated
(example position of colour varying area shown in Fig 3) and these
were subjected to discriminant analysis. For the calibration set the
discriminant analysis sorted all of the fish (135 fish) correctly ie.
lO0~ sorting reliability for the shape descriptors and the 3 sets of
colour descriptors. For the test set of fish the shape descriptors
had a sorting reliability of 94% (92 out of 98), the front third
colour descriptors had a sorting reliability of 95% (93 out ot 98),
the middle third colour descriptors had a sorting reliability of 95%
(93 out of 98) and the tail third colour descriptors had a sorting
reliability of 94% (92 out of 98). These four sets of results were
combined by simply classifying a fish as the one most preferred by
- adding the results together. In doing this the sorting reliability

W O 94/09920 214 7 6 0 2 PCT/GB93/02151

18

obtained was 100% (98 out of 98).

For the Group 2 fish the shape grid was used. From this the shape and
colour descriptors were obtained (examples of colour varying areas
shown in Fig 4) and these were then subjected to discriminant
analysis. For the calibration set the discriminant analysis sorted
all of the fish (117 fish) correctly, ie. a 100% sorting reliability
for the shape descriptors and the 3 sets of colour descriptors. For
the test set the shape descriptors had a sorting reliability of 90%
(69 out of 77), the front third colour descriptors had a sorting
reliability of 96% (74 out of 77), the middle third colour descriptors
had a sorting reliability of 100% (77 out of 77) and the tail third
colour descriptors had a sorting reliability of 100% (77 out of 77).
When these four sets of results were combined as before a sorting
reliability of 100% (77 out of 77) was obtained.

Hence the 18 species of demersal fish used in these experiments were
sorted with a reliability of 100% (133/133 test fish).

~xamvle ~: Ap~aratus for sorting Pel~gic fish.

The five main species of pelagic fish have similar values of Ar and
As therefore the procedure of Table 3 does not work on pelagic species.

Shape grid were generated for all of the pelagic fish and from these
the shape and colour descriptors were calculated (see examples of
position of colour varying areas Flg 5) and subjected to discriminant
analysis. For the calibration set the discriminant analysis sorted
all of the fish (80 fish) correctly ie a 100% sorting reliability for
the shape descriptors and the 3 (front, middle and tail) sets of
colour descriptors. For the test set of fish the shape descriptors
had a sorting reliability of 98% (55 out of 56), the front third
colour descriptors had a sorting reliability of 100% (56 out of 56),
the middle third colour descriptors had a sorting reliability of 88%

O 94/09920 21 ~ P ~ /GB93/02151

19

(49 out of 56) and the tail third shape descriptors had a sorting
reliability of 98% (55 out of 56). When these four sets of results
were combined (as in the previous section) a sorting reliability of
98% (55 out of 56) was obtained. The one fish which was not
classified correctly was a sprat and it was classified as 50%
probability of being sprat and 50% herring.

Representative Drawing
A single figure which represents the drawing illustrating the invention.
Administrative Status

For a clearer understanding of the status of the application/patent presented on this page, the site Disclaimer , as well as the definitions for Patent , Administrative Status , Maintenance Fee  and Payment History  should be consulted.

Administrative Status

Title Date
Forecasted Issue Date Unavailable
(86) PCT Filing Date 1993-10-19
(87) PCT Publication Date 1994-05-11
(85) National Entry 1995-04-21
Dead Application 1997-10-20

Abandonment History

Abandonment Date Reason Reinstatement Date
1996-10-21 FAILURE TO PAY APPLICATION MAINTENANCE FEE

Payment History

Fee Type Anniversary Year Due Date Amount Paid Paid Date
Application Fee $0.00 1995-04-21
Maintenance Fee - Application - New Act 2 1995-10-19 $100.00 1995-04-21
Registration of a document - section 124 $0.00 1995-11-09
Owners on Record

Note: Records showing the ownership history in alphabetical order.

Current Owners on Record
THE MINISTER OF AGRICULTURE FISHERIES AND FOOD IN HER BRITANNIC MAJESTY' S GOVERNMENT OF THE UNITED KINGDOM OF GREAT BRITAIN AND NORTHEN IRELAND
Past Owners on Record
STRACHAN, NORVAL JAMES COLIN
Past Owners that do not appear in the "Owners on Record" listing will appear in other documentation within the application.
Documents

To view selected files, please enter reCAPTCHA code :



To view images, click a link in the Document Description column. To download the documents, select one or more checkboxes in the first column and then click the "Download Selected in PDF format (Zip Archive)" or the "Download Selected as Single PDF" button.

List of published and non-published patent-specific documents on the CPD .

If you have any difficulty accessing content, you can call the Client Service Centre at 1-866-997-1936 or send them an e-mail at CIPO Client Service Centre.


Document
Description 
Date
(yyyy-mm-dd) 
Number of pages   Size of Image (KB) 
Cover Page 1995-08-14 1 18
Abstract 1994-05-11 1 54
Description 1994-05-11 19 722
Claims 1994-05-11 6 237
Drawings 1994-05-11 9 356
Representative Drawing 1998-02-12 1 6
International Preliminary Examination Report 1995-04-21 14 465
PCT Correspondence 1995-06-12 1 47
Office Letter 1995-06-02 1 22
Fees 1995-04-21 1 52