Note: Descriptions are shown in the official language in which they were submitted.
CA 02393145 2002-05-30
WO 01/45388 PCT/US00/34219
METHODS AND APPARATUS FOR IMAGE MANIPULATION
Background of the Invention
Field of the Invention
The invention relates to the acquisition, transmission, and processing of
digital images.
Description of the Related Art
Images have been acquired on various forms of light sensitive photographic
film for well over one hundred
years. Although a variety of advances in this technold~y have been made, the
basic principles used to acquire and
process such images has remained fundamentally unchanged for a long period of
time.
More recently, electronic image acquisition devices have been developed.
Rather than relying on film, an
array of pixels comprise light sensitive electronic elements. With these
elements, digital data is generated which
defines an acquired image in terms of one or more digital values associated
with each of the pixels of the array.
Applications of these devices include scanners and digital cameras. Although
these devices eliminate film and the
chemical development process, they require a large amount of memory for
storing image data. This increases the cost
of these devices, and limits the number of images that can be acquired in a
portable or hand-held device.
Summary of the Invention
The invention comprises image manipluation methods and systems as well as
devices for image capture and
transmission such as digital cameras. In one embodiment, a method of image
manipulation comprises acquiring one or
more digital images with a hand-held digital camera; transmitting them to a
network address substantially
simultaneously with the acquiring, and storing the digital images in a
database hosted by a network server associated
with the network address. In another embodiment, a method of image
manipulation comprises authorizing a portable
image acquisition device operator to access a portion of an image database and
receiving images from a portable
image acquisition device operated by the portable image acquisition device
operator automatically and substantially
concurrently with image acquisition. The images may then be stored in the
image database; and retrieved under the
direction of the portable image acquisition device operator after verifying
the authorization.
Image acquisition devices according to the invention may comprise image
acquisition circuitry having digital
image data as an output; a user actuated control for initiating image
acquisition, a wireless transmitter, and logic
circuitry coupled to the image acquisition circuitry, the user actuated
control, and the wireless transmitter, wherein
the logic circuitry is configured to route the digital image data to the
wireless transmitter in response to the user
actuated control. In another embodiment, a portable image acquisition device
comprises image data acquisition
circuitry, image data transmission circuitry; and a memory location having
stored therein a network address of a
network server to which image data is automatically forwarded.
Systems for image manipulation are also provided which may comprise a
plurality of portable image
acquisition devices, wherein each of the portable image acquisition devices
comprises a wireless transmitter. This
system may also comprise a computer system remote from the portable image
acquisition devices and in
CA 02393145 2002-05-30
WO 01/45388 PC'T/US00/34219
communication with the portable image acquisition devices by a communication
link accessible via the wireless
transmitters, wherein all of said plurality of portable image acquisition
devices are configured to transmit images to
the computer system substantially automatically as the images are acquired.
Brief Description of the Drawings
FIG. 1 is a block diagram of an image manipulation system in accordance with
one embodiment of the
invention.
FIG. 2 is a flow chart of one mode of operation of an image manipulation
system in one embodiment of the
invention.
FIG. 3 is a block diagram of the components of an image acquisition device in
accordance with one
embodiment of the invention which is suitable for use in the system of FIG. 1.
FIG. 4 is a block diagram of the components of an image acquisition device in
accordance with a second
embodiment of the invention which is suitable for use in the system of FIG. 1.
FIG. 5 is a perspective view of the image acquisition device of FIG. 4.
Detailed Descriution of the Preferred Embodiment
Embodiments of the invention will now be described with reference to the
accompanying Figures, wherein like
numerals refer to like elements throughout. The terminology used in the
description presented herein is not intended to be
interpreted in any limited or restrictive manner, simply because it is being
utilized in conjunction with a detailed description
of certain specific embodiments of the invention. Furthermore, embodiments of
the invention may include several novel
features, no single one of which is solely responsible for its desirable
attributes or which is essential to practicing the
inventions herein described.
Figure 1 illustrates a system for image manipulation which includes a
plurality of portable image acquisition
devices 12, 14, 16. The image acquisition devices 12, 14, 16 are devices that
produce digital data defining an image.
The image is typically a two-dimensional digital representation of a document,
thing, person, scenery, etc. or a
combination of these things which is proximate to the image acquisition
device.
In many embodiments, the image acquisition devices 12, 14, 16 are each
independently operated by
separate image acquisition device operators. As one specific example, the
image acquisition devices may be separate
digital cameras carried by separate vacationers or news reporters. As will be
explained below, this is one especially
advantageous application of the system of Figure 1. The image acquisition
devices may also be digital scanners.
Generally speaking, digital cameras focus a field of view onto a two-
dimensional pixel array using a lens system,
operating essentially as a conventional camera, but with a light sensitive
electronic pixel array instead of film. They
are typically hand-held, lightweight, and convenient to carry. A scanner may
also be light weight and portable,
although they are usually larger and heavier than a digital camera. Scanners
usually acquire an image by passing a
one-dimensional array of pixels across a two dimensional transparent surface
which has a document or other flat
object held against the transparent surface. Both digital cameras and scanners
are often owned, controlled, and
operated by individuals.
.2.
CA 02393145 2002-05-30
WO 01/45388 PCT/US00/34219
In another embodiment of the invention, the image acquisition devices 12, 14,
16, comprise surveillance
cameras. Surveillance cameras may also be lightweight and portable, although
once installed, they are typically not
moved frequently or carried by an individual on a regular basis. In this
embodiment, each camera will likely not be
independently operated by separate individuals, but will be part of a single
monitoring installation.
The image acquisition devices 12, 14, 16 advantageously include communication
circuitry for communicating
to a network 18. The connection to the network 18 may in some embodiments
comprise a wired connection, but in
especially advantageous embodiments of the invention, and as illustrated in
Figure 1, the communication circuitry
comprises wireless communication circuitry for communicating to the network 18
via a wireless communication
infrastructure. In this embodiment, the wireless communication infrastructure
be cellular, satellite, digital PCS, or any
other known or future developed wireless communication technology. In some
embodiments, the image acquisition
devices could comprise two separately housed components, one being hand-held,
and the other stationary. In this
case, the communication link to the network may comprise two parts, one link
from the handheld portion to the
stationary portion, and a second link from the stationary portion to the
network 18. Preferably, one or both of these
links is wireless, but this need not be the case in all embodiments of the
invention.
The network 18 may be any type of electronically connected group of computers
including, for instance, the
following networks: Internet, Intranet, Local Area Networks (LAN) or Wide Area
Networks (WANI. In addition, the
connectivity within the network 18 may be, for example, remote modem, Ethernet
(IEEE 802.31, Token Ring (IEEE
802.51, Fiber Distributed Datalink Interface (FDDIi, Asynchronous Transfer
Mode (ATM), or any other communication
protocol. The computing devices linked to the network 18 may be desktop,
server, portable, hand-held, set-top, or any
other desired type or configuration. In one advantageous embodiment, the
network 18 is the global wide area computer
network commonly referred to as "the Internet." This is an advantageous
implementation when the image acquisition
devices are independently owned and operated digital cameras or scanners. In a
surveillance camera embodiment, the
network 18 will usually be a private LAN.
Currently, many network communications within the global Internet are
performed using the TCPIIP format.
Transmission Control Protocol (TCPI is a transport layer protocol used to
provide a reliable, connection-oriented,
transport layer link among computer systems. The network layer provides
services to the transport layer. Using a
two-way handshaking scheme, TCP provides the mechanism for establishing,
maintaining, and terminating logical
connections among computer systems. TCP transport layer uses IP as its network
layer protocol. Internet Protocol
(IPI is a network layer protocol used by many corporations, governments, and
the Internet worldwide. IP network
layer supports many personal, technical and business applications, such as
electronic mail, electronic fund transfers,
medical records processing and similar data transfers. 1P is a connectionless
network layer protocol that performs
addressing, routing and control functions for transmitting and receiving
datagrams over a network.
Connected to the network 18 is a computer system 20 which receives image data
from the image
acquisition devices 12, 14, 16 via the network 18. The computer system 20 will
typically be a network server which
includes software for receiving image data and storing the image data in an
associated image database. As such, the
-3-
CA 02393145 2002-05-30
WO 01/45388 PCT/US00/34219
network server will be assigned a network address for identification during
communications between elements of the
network 18. Communications from other network connected devices such as the
image acquisition devices 12, 14,
16, to the computer system 20 will include the network address of the computer
system 20 as part of the
communication so that the computer system 20 is identified as the recipient of
the data. When the network 18 is the
global Internet, the network address is an IP address which may, at least in
part, route the image data to an e-mail
account, a web-site, or other Internet tool. The computer system 20 may also
host image processing software for
arithmetically and logically manipulating the image data received from the
plurality of image acquisition devices 12,
14, 16. This software may include pixel processing routines for equalization,
color correction, or any of a wide variety
of feature enhancement algorithms.
Also connected to the network 18 is another computer system 22. The computer
system 22 will typically
comprise a personal computer (PC), such as are available from International
Business Machines, Compaq Computer
Corporation, Sun Microsystems, and many others. These computer systems
normally include at least one
microprocessor, a display screen, a keyboard, and a storage medium such as a
hard disk. A printer is also
advantageously locally associated with the computer system 22. The computer
system 22 includes software for
communicating with the network server 20 via the network 18. When the network
comprises the Internet, such
software is commonly referred to as a web-browser. The e-mail account, web
site, or other tool may be available to
the web browser through a domain name (e.g. www.site.com) which maps to the IP
address of the network server 20.
The image acquisition devices, 12, 14, 16, the image storage and processing
computer 20, and the image
retrieval and printing computer 22 thus form a distributed image manipulation
system which has its components
connected through the network 18. This image manipulation system operates by
initially transmitting image data
acquired by the image acquisition devices 12, 14, 16 to the image storage and
processing computer system 20 via the
network 18, generally by routing the image data to a pre-defined network
address stored in the image acquisition
devices 12, 14, 16. After the image data is stored in the computer system 20,
it may be retrieved by the image
retrieval and printing computer system 22, where the images can be displayed
andlor printed.
In one advantageous embodiment, the image acquisition devices transmit image
data substantially as the
image data is acquired, and perform only minimal data storage and image
processing functions. This provides several
significant advantages. One is that the image acquisition devices 12, 14, 16
are less expensive because a large
amount of memory and computational power for image storage and processing need
not be provided. In addition,
image security is improved because the images are almost immediately stored on
the image database associated with
the image storage and processing computer system 20. This eliminates the risk
of losing the image acquisition device
or associated memory modules and thus losing a large number of acquired images
at the same time. The wireless
communication link connection to the network 18 is advantageous in these
embodiments because the camera,
scanner, etc. can be operated without being tied by wire or cable to any other
computer entities. It can be carried
around and used in a completely conventional manner as far as the user of the
device is concerned.
-4-
CA 02393145 2002-05-30
WO 01/45388 PCT/US00/34219
Preferably, image data transfer is fast enough to be able to transmit image
data at an average rate at least
as fast as image data is acquired, although it will be appreciated that
temporary differences in data acquisition and
transmission rates can be tolerated with internal data buffering as described
below. Accordingly, image data
compression is generally advantageous prior to transmission to reduce the
amount of data required to be sent with
each image. In some embodiments, a sequence of images may be acquired by the
image acquisition device in rapid
succession, such as 1, 2, 10, or even 100 images per second. In these
embodiments, the transmission data rate
should be able to accommodate these image sequences. Image compression
algorithms designed for motion pictures,
such as the well known MPEG standard, may be used in these applications to
provide for a high compression ratio.
Thus, in one advantageous embodiment, a central image repository is created
for hundreds or thousands of
individual digital camera users without any inconvenience or device handling
complications to the users themselves.
Each camera is pre-programmed to download image data as images are acquired.
Acquired images may be retrieved at
the user's convenience using their home computer. The users need never worry
about having sufficient memory to
acquire more images or worry about the whereabouts of disks, memory modules,
or other types of storage devices. If
desired, some cameras may be pre-programmed with image destinations other than
the image storage and processing
computer 20. For instance, some users may wish to program their cameras with
their own e-mail account, personal
web-site, or other Internet tool or location.
Referring now to Figure 2, one method of operation of a distributed image
manipulation system is illustrated.
First, at block 28, an image acquisition device operator, an individual
digital camera owner, for example, may be
authorized to access a portion of an image database associated with the image
storage and processing computer 20.
As the digital camera is used to acquire images, at block 30 the acquired
images are downloaded automatically and
substantially concurrently to the image storage and processing computer system
20, and at block 32 the images are
stored there in the database portion accessible to the image acquisition
device operator. As described above, the
digital camera operator may therefore take an unlimited amount of pictures
with the camera without purchasing,
carrying, securing, or in any way dealing with the memory required for image
storage. The risk of misplacing the
images and loss by theft or the like is also eliminated.
Upon returning home, the camera operator's home personal computer may comprise
the image retrieval and
printing computer system 22. By entering a selected domain name to a web
browser, the image storage and
processing computer 20 and its associated image database are accessed. At
block 34, by using a password or the
like, the authorization to access the portion of the database in which the
images are stored may be verified. After
verification, the user may be allowed to retrieve the images previously
acquired. The images may then be viewed,
specific desired image processing algorithms may be performed under the
direction of the operator, and the images
may be printed with a printer associated with the computer system 22.
Figure 3 illustrates the components of one embodiment of an image acquisition
device which may
advantageously be used in the system described above and shown in Figure 1.
Referring now to this Figure, the image
acquisition device 40 is controlled by a micro-controller 42 which includes
control and processing logic. Suitable
5-
CA 02393145 2002-05-30
WO 01/45388 PCT/US00/34219
micro-controllers are commercially available from, for example, Motorola
Corporation and a wide variety of other
companies. The micro-controller 42 is coupled to image acquisition circuitry
44, a data buffer 46, a wireless
transmitter 48, and a user control 50. The image acquisition circuitry will
typically include a one or two-dimensional
array of light sensitive elements, such as a CCD array or an array of
photodiodes. In operation, the image acquisition
circuitry 44 has digital image data as an output that is routed to the micro-
controller 42. The micro-controller then
routes the image data received from the image acquisition circuitry 44 to a
wireless transmitter 48 for transmission to
the network 18 of Figure 1. To reduce the amount of processing power required
to be part of the image acquisition
device 40, the micro-controller will generally not perform image analysis and
processing such as equalization, color
correction, or other logical and arithmetic pixel processing algorithms. In
advantageous embodiments, the data
manipulation performed by the micro-controller 42 andlor the wireless
transmitter 48 is limited to data compression,
data encryption, and packaging in appropriate format (such as TCPIIP) for
reception and handling by the wireless link
infrastructure and the network 18.
Advantageously, all of these functions may be initiated by a single simple
user control which is used to
initiate image acquisition. On a digital camera , for example, the shutter
release button may function as a single push
button control 50 which both initiates image acquisition as well as image
transmission with a single actuation. In this
embodiment, the beneficial aspects of the camera are made more user friendly,
because the entire transmission
process is automatic and transparent to the user of the device 40. As
described briefly above, in this embodiment the
image acquisition device is pre-programmed with the identification of the
server 20 to which the data will be sent by
storing the network address andlor other required data routing information in
the image acquisition device. Generally
this information is will be stored in EEPROM or other non-volatile program
memory internal to or associated with the
micro-controller 42 as part of the micro-controller programming that controls
data acquisition and transmission.
Because image data may be generated and transmitted at different rates, and
because image transmission
may be slightly delayed, an image data buffer 46 may be provided as a low
capacity temporary storage for image
data. This is especially useful if the user control 50 is also used to
initiate the establishment of the wireless link
through local wireless communication infrastructure (e.g. cellular telephone
service). In this case, acquired image data
may need to be stored temporarily in the buffer while the link is established.
Even if the link is established prior to
image acquisition, the rate of image data acquisition may be different from
the rate of image data transmission, and
so buffer storage 46 may also be needed during the process of transmission. To
reduce device cost, it is preferable to
minimize the capacity of this buffer, although of course buffer overflow
during image acquisition and transmission
should be avoided. In some embodiments, the buffer stores at most data for
three images. More buffer memory may
be required depending on the functionality of the image acquisition device.
The image acquisition device may also comprise a real-time clock 52 and a
Global Positioning System (GPS)
satellite signal receiver 54. Each of these devices 52, 54 may also be coupled
to the micro-controller 42, and image
acquisition time as well as the location of the device 40 when the image was
acquired may be associated with the
image data and transmitted to the computer system 20 as each image is
transmitted. Status information may also be
6-
CA 02393145 2002-05-30
WO 01/45388 PCT/US00/34219
associated with the image data for each image transmitted from the device 40.
This information may include, for
example, the ambient light level, battery charge level, andlor error
conditions that might be present such as
mechanical errors or buffer overflow.
In some embodiments, most notably the surveillance camera embodiment described
above, the device 40
may also be coupled to a renewable energy source such as a solar panel (not
shown). This allows for remote image
acquisition without a wired power source or batteries which require periodic
replacement.
In Figures 4 and 5, a second embodiment of an image acquisition device is
illustrated. Operation of the
device of Figure 4 is functionally similar to that described with reference to
Figure 3, however, in this embodiment, the
micro-controller 42 communicates with a removable circuit card 58 through a
disengagable connector 59. In this
embodiment, the wireless transmitter 48 is mounted on the removable card 58.
This is an advantageous configuration
because many commercially available digital cameras include removable circuit
cards on which is mounted the memory
which stores acquired images. The circuit card 58 of Figure 4 may be
compatible with these removable memory
cards, but will transmit the acquired images, rather than store them in on-
board memory.
An interface circuit 60 on the circuit card may mimic the interface of a
standard memory card so that the
micro-controller 42 operates in a manner identical to a conventional digital
camera. On board control logic 62 which is
pre-programmed with a network server destination for image data manages the
interface circuit 60 and the buffer 46
to implement wireless transfer of the image data as described above.
Figure 5 illustrates a digital camera embodiment incorporating these features.
The circuit card 58, which
may advantageously be in an industry standard PCMCIA format, fits into a
receptacle 68 provided in the housing 70 of
the camera. Electrical contacts 69 on one edge of the circuit card 58 couple
to the connector in the camera housing
70. With this embodiment, conventional cameras which are normally used with
removable memory cards may be used
in the image manipulation system herein described.
The foregoing description details certain emb odiments of the invention. It
will be appreciated, however, that no
matter how detailed the foregoing appears in text, the invention can be
practiced in many ways. As is also stated above,
it should be noted that the use of particular terminology when describing
certain features or aspects of the invention
should not be taken to imply that the terminology is being re-defined herein
to be restricted to including any specific
characteristics of the features or aspects of the invention with which that
terminology is associated. The scope of the
invention should therefore be construed in accordance with the appended claims
and any equivalents thereof.
7.
