Note: Descriptions are shown in the official language in which they were submitted.
~-~`
~ ~ 5t~
This invention relates to an apparatus and a method
which can be used to produce synthetic spelled spe~ch or
synthetic speech. More particularly, synthetic spelled speech
can be used as an aid for the blind to learn typing skills and
retain them; or as an aid for a blind programmer both to inter-
pret signals from a large computer and from a pocket calculator;
or as an aid for a person suffering the loss or absence of
speech to communicate with others: the same apparatus and :.
method can be used to produce speech from phoneme input/ and
this speech can be used in a reading machine for the blind or
to facilitate the learning of foreign languages.
Machines which can be used for audible reception -
of visual or other materials by the blind exist in a number of
different types or forms. One form, called the Optophone
generates buzz tones in response to the input of signals
generated by scanning printed material. A machine called the
Lexiphone produces musical tones modulated in frequency and ~ :
- amplitude. Another machine called the Optocon presents an
image of a character on a matrix of mechanical stimulators
: 20 which is read by tactile means.
For ease of learning as well as efficiency in use, ;
it would be preferable to have an input signal result in an
audible sound which is understandable in the language of the
user. However, such "talking machines" have been complex .
: and costly and a personal machi~e has, until now, not been
. practical. In the present invention, typing a letter of script
on an electric typewriter keyboard or inputting of other .
- information such as that resulting from electronic scanning of
~: .
a letter will result in the sound of that letter being produced.
While storage of letter sounds has previously been
; effected on a film-drum memory, this kind of letter storage
~.~
~ has the disadvantage of bulkiness, high cost, and ~i~ed
::'~ , ~
~as785~
duration of letter sounds. More versatility would be available
with the provision of a digital memory substitute.
Until now, however, it was considered that storage
of the letter sounds digitally would be prohibitively costly.
: .
. .
: . ;. :
.-., ' '.
,;',: ' ,;
: , . . : .
: : .
:,. , ,:
- la ~
: : ` ' .:
-
7~35~
As an estimate of the cost, consider that operatin~ at a speed ;
of 60 words per minute, with five letters to a word, a
letter would occupy 0.2 seconds on the avexage~ At a 12.5
kHz sampling rate using 9 bit samples and allowing 20% pause
time between letters, it requires 468,000 bits to store the
entire set of 26 letter sounds. With a me'mory costing about
one cent per bit, the memory alone would cost $4,680.00.
In the present invention, a substantial reduction
in memory is obtained by a particular structural arrangement
designed to synthesize the letter sounds af~er substantial
economies in stored information are effected. Memory for
storage of the required in~ormation is obtained using two
4,096 bit read only memories, which are available as semiconductor
integrated circuits.
The aforenoted economies were obtained through the
recognition that each of the letter sounds of the alphabet
can be perceived by extracting an extremely small segment
thereof and repeating that segment over and over again a
certain number of times. The numbers of repetition times are
variable for each letter sound. Thus the letter sound "t",
a percussive, need only be repeated once or twice, while the
vowel sound ~/ may be repeated 8 to 11 times.
~ Consequently, the application of the letter "a"
`; will result in the serial production of the two letter
sounds /~/ and /i/. Each of these sounds is termed a
phoneme; in this invention, each phoneme is fabricated by
repetitively generating sequential signals made up of only ~;;
portions of phoneme signals reproduced as noted above.
~ ~ Consequently, rather than requiring storage of the ;
; 30 entire letter sounds in memory, all that need be stored are
the basic phoneme portions, and numerical representations
of the number of repetitio~s required to form each basic
phoneme.
:` ' ' '
L~5 ~
Furthermore, the information concerning the phoneme
portion is stored digitally. It has been found that reduction
of memory space can be effected by storing, not as would be
expected, the amplitude and frequency signal information, but ~;
pulse difference amplitude data signals. Each pulse of the
pulse code modulated ou~put signal appearing for instance,
every 80 microseconds, is generated by a stored signal desig~
nating the difference between the prior pulse amplitude and the
next.
The entire spelled speech generator is controlled
by a microprocessor comprising a central processing unit,
which controls various memories, input and output port
circuits, etc. This specification is directed to the person
skilled in the art of utilization of such processors, and
consequently all of the detail concerning its operation, such
as the provision of clocks, strobe circuits, power supplies,
systems software, etc. will not be given, since it is
contemplated that the person wishing to fabricate this inventi~n
will consider the provision of such structures within his i~
ordinary skill. -
The advantages of the invention will be obtained by
the provision of a spelled speech generator comprising a central
processing unit (CPU), a first memory for storing signals repre-
senting execution programs for the CPU, a second memory, and an
address bus interconnecting the CPU and address inputs of the
memories. Means is provided including a digital-to-analog con- -
.~i; . . .
verter having an output from which an audio output signal is
- obtained. A data bus couples the input of the converter to the
output of the memory. Stored signals in the second memory `~
,.,j
represent prede~ermined portions of phonemes and of repetition ;
counts therefor. Means is provided for actuating ~he CPU to
serially address predetermined memory locations in the second
memory, whereby the phoneme-portion signals a~re
. :, .
~ ~ 3 ~
~ ~7 ~ ~r~
each repeated a number of times designated by the repetitive
count signals in a sequence and applied to the input of the
digital-to-analog converter to provide a complete phoneme signal.
In the preferred embodiment, the signals representative
of the portions of the phonemes are comprised of pulse
amplitude difference binary codes of pulse code modulated phoneme
portions.
The means for actuating the CPU can be comprised
of a teletypewriter, IBM Selectric* or other electric typewriter,
which may have to be modified slightly to input the correct
data signal in well known ASCII code to this invention.
It will be seen that the advantages of the invention
are obtained by the method of generating phonemes comprising
the steps of repeatedly providing a first signal representative
of a fraction of a phoneme in rapid succession a predetermined
number of times, and reproducing the repeated signal in an
audio translation circuit to provide an audio reproduction
of the repeated signal.
For use as a typing training aid for the blind, it is
desirable to be able to obtain a repetition of a last letter
; sound read out, or a precediny few words in case the operator
is distracted for a time. In another embodiment of this
invention, therefore, a memory is provided for retaining the
input signals corresponding to the last few words. The last
letter or the entire content of the memory typed in may be
read out serially as combinations of phonemes to provide the ;
last letter sound typed, or a series of letter sounds, of the
last few words typed.
In this case the invention includes a stacking
type temporary memory for storing a predetermined number of
.: .. :,
input signals. The stacking type memory is constantly
refreshed, and the oldest information is deleted as fresh
* Trade Mark
. . ~ `1~
~ 4-
~ . ~ . .. . ~- : . , - , .
t~ r;~
information en-ters. :
A more detailed explanation of the structure and `~
operation of the invention will be given below, and reference
is made to the following drawings, in which:
Figure 1 is a block diagram of the invention;
Figure 2 is a schematic diagram of part of the inven-
tion;
Figure 3 is a schematic diagram of the memory portion ~;
of the invention;
Figure 4 consisting of component drawings Figure 4A, ; ~ ~:
4B, and Figure 4C is an address map of the phoneme portion memory; :~
Figure 5, which appears on the same sheet as Figures 4A
and 4B, is a waveform diagram o a phoneme portion.
Turning first to Figure 1, a microproces~or, preferably
of the type 8080 manufactured by Intel Corporation, is provided,
shown as CPU 1. It has an output address bus 2, which is con~
nected to the input of read only memory (ROM) 3. A bidirectional :
data bus 4, also connected to CPU 1 is connected to the digital~
to-analog converter 5 via an audio output port circuit 6. The `~
;~ 20 output of the digital-to-analog converter is connected through
an audio outp~t amplifier to an unreferenced speaker, during ;~ :
operation.
Within the memory are stored representations of pre-
. determined portions of phonemes and of repetition counts there-
s for~ A signal input on the data bus 4 causes the CPU to address : :~
predetermined series of memory locations, whereby each pulse :`
- series making up the phoneme-portion signals can be sequenced
repeatedly the number of times designated by the repetitive `
count and applied to the input of the digital-to-analog conver- ~:
ter via the data bus, to provide a complete phoneme signal. : :
~uxtaposing phonemes then provide complete letter sounds.
As noted earlier, the signal representat:ive of the
portions of phonemes are preferably comprised of pulse .
_ 5 _
: , , , , ., . , , .,......... .. , . , . , . . .: . -
~s~s~ :
amplitude difference signals stored in the memory, for
numerical representation of serial segments of each phoneme-
portion making up the audio signal.
The data input bus 4 is connected to the data
input of the CPU. An input port cixcuit 7 provides a
buffer interface between a pheripheral piece of equipment,
and the data bus. A multiplicity of such port circuits can
be provided for different kinds of pheripheral equipment.
Memory 3 has its output connected to a memory data
bus 8, which i5 connected to the input of memory output port
circuit 9, which has its own output connected to data bus 4.
In operation, an input signal, such as will be `~
obtained from a teletypewriter, for instance in serial ASCII
code, is interfaced to input port 7 after being converted to
parallel coded ASCII. At an approprLate time/ the input
port 7 is strobed by the CPU, and the information therein
dumped on data bus 4, and is inputted to CPU 1. The CPU
:. ~
selects an address in the memory 3~corresponding to the ~
J:; .:
input code, and addresses the memory 3 via address bus 2.
The memory address contents will designate a
further memory location containing an iterative loop of 1;~
pulse difference code generation signals, as well as an ~;
index designating how many times the loop is to be iterated.
Under aontrol of the CPU, the signals stored at
each memory location of the iterative loop are serially read ~-
;
out on the memory data bus 8, while the CPU keeps track of
tha number of iterations by indexing a count designator each
time a loop is completed.
The s~gnals read out repetitive}y on memory data ~ ~ -
i~ 30 bus 8 are strobed through memory output port circuit 9 to data
bus 4, and through audio output port circuit 6 to the
digital-to-analog converter 5. Here the pulse amplitude
difference signals are converted to a pulse amplitucle modulated
;
-6- ~ ~
:, :
.: ~:. . .; . . . . . .
r
I ~ir~r7~
signal and smoothed to an audio signal. i
It should be noted that each time an iterative loop is
completed, a phoneme por-tion containing a complete predetermined
portion of the actual reproduced full phoneme is provided. By
repetitive concatenations of the phoneme portions, a pseudo~
complete but fully understandable phoneme is produced. -~
It should be pointed out that the completion of all `
iterations making up a phoneme need not signiy termination of
the operation. Completion of the phoneme-portion repetitions ~ ~ ~
may signify passage to a further memory address location for the '"", ' ~'''~! ' ,
~ ',', ~,'~',. 1 ', '
initiation of generation of a second or further phoneme. Again
there will be repetitive iterations of a Ioop of address loca- ,~
tions, reading out further pulse amplitude difference signals.
:,`. ,. ~:
In this way, a second phoneme will be produced. By the conjunc-
tion of the two phonemes, a complete letter sound is produced.
There will of course be required an additional ``
memory 41 connected between address bus 2 and the memory data
bus 8 for storage of the normal operation programs of the CPU,
and as well the memory 3 may be expanded by the provision of a ;~;
second memory 3a shown in Figure 3 connected similarly to the
first. A random-access-memory 42 stores information from the
input port 7. A memory select circuit 11 is provided to enable
each of the memories at a particular time to accept address in- -
i formation and to dump information on data bus 8.
:,. ., :
Where there is more than a single type or source of
input-information to be connected to data bus 4, an interrupt
port 12 is provided to designate to the CPU what type of informa-
~ tion or what source of information is providing the input data. -
"
The appropriate input port will as a result be strobed by the
CPU.
Preferably; both the input and output port circuits ~ ;~
are Intel type 8212 integrated circuits which are available
~ 7 ~ `~
. ' ~ ,~, ;- ,
~s~
as interface units with the Intel 8080 CP~. Also, preferably,
the memories 3 and 3a are Electronic Associates type EA4825
ROMs, and the program storage memory 41 is Intel PROM type
8702A.
A preferred embodiment of the input circuits will
now be described in detail, with reference to Figure 2.
The typewriter port 7 is shown, which accepts paralle~
ASCII type code for application to bus 4. Consequently
means is provided for ~ranslation of the input information ~;
from the electric typewriter to ASCII code. In this embodimen~r
this translation is obtained by the use of a converter chip
20, which preferably is a NATIONAL SEMI-CONDUCTOR "Selectric
To ASCII Converter" chip, type MM5230KP, which is available
as a 24 pin dual in-line package. Connected to the input
terminals to chip 20 are switches 21 which, for the Selectric
typewriter, provide inpu~s corresponding to the code which
determines rotation and tilt designations for the Selectric -
type ball. When a particular character is typed on the
- Selectric typewriter, one or more switches corresponding to
:. ,: . .
ball tilt designation is closed, and one or more switches
designating ball rotation are closed. The switches 21 designating
tilt are labelled in Figure 2 with a T prefix, and those
relating to rotation are prefixed by an R prefix. Upper
:: . ,: ,
case letters are designated by the switch labelled CASE.
~~ Pull-up resistors 22 connect individual ones of
the input lines of converter chip 20 to a positive potential;
consequently when a switch is closed, an input terminal Al - A~
to the converter chip is connected to ground, changing the
input from high positive level to a low level signal.
The output terminals Bl - B8 of converter chip 20
are connected lndividually through NAND gates 23 to AND
gates 24, which have their outputs connected to the :Lnput
terminals of typewriter input port circuit 7. ;~
s
! -
8S~
Assuming for a moment that AND gates 24 do not
exist, and simply pass a signal through, the typing of a
letter on the Selectric typewriter will cause the input of
converter chip 20 at the appropriate terminals to go to low
level, causing a conversion of the input code into ASCII,
The ASCII code is passed through NAND gates 23," and into
typewriter input port 7. When the port circuit 7 is strobed
at an appropriate time, the ASCII code is applied to bus 4,
from which it is received by the CPU 1. Further operation ` ~ -~
of the system has been described earlier.
It will be recognized that many additional designations -
~are required under normal operation of the typewriter. For
instance, there will be tabulation input signals, space bar :
input signals, ball return signals, etc. Each is designated
by an appropriate switch 25, each of which is a "special
character" switch. The common terminal of each of the `
., ,
switches is connected to a positive potential, and the
output terminals of the switches are each connected through
resistors to ground. Consequently, with the switches open,
; 20 each of the lines are held at ground potential. However,
closing a particular switch 25 will raise its output terminal to
a positive potential.
Each of the output terminals of switches 25 are
connec*ed through individual inverter amplifiers 26 to a
~- second input of AND gates 24. -~
,. :: 1,
~ Consequently, all of the ANiD gates 24 are held
-,
with one input operated (from the special character switches~
'~ under normal conditions, and every input generated from
I input switches 2L lS passed through gates 24 in a transparent `~
~.
~ 30 manner. However, when a special character switchi is closed,
. . , ~ .
an output on a particular line therefrom will be passed through
an inverter ,amplifier 26, and will cause a particular AND
gate 24 to pass the signal therethrough to port circuit 7.
g
~357~355
In addition, the output of each of the special
character switches 25 is connected to an input of ~OR gate
28, which is connected to a resistor-capacitor bounce filter
29, and via inverter amplifier 30 to the illpUt of NAND gate
31. The output of NAND gate 31 is connected to one of the ;
inputs of each of NAND gates 23 connected in parallel.
Consequently, in operation, when a special character switch
is depressed, after a delay to correct for contact bounce, the
code transmission path from converter chip 20 is blocked by the
inhibiting of gates 23 by the action of NAND gate 31. At the
same time, however, an input to typewriter port 7 is provided
through inverter amplifiers 26.
It will be seen that the above provides an input
coding structure for the typewriter input port 7. The
described technique for paralleling information into AND
gates 24 for special characters is to make up for the lack
of input capacity in converter chip 20. However, in the
,: ,. , .~
event a different type of input converter is available, or
in the case of the availability of proper parallel ASCII
code, the structure noted above need not be resorted to.
A further input from the switch labelled "gate",
grouped with the special character group of switches, is
applied to the input of NOR gate 32 to designate that a
valid input is present. A second input to NOR gate 32 is
provided from the output of bouncelfilter 29 (inverted in
inverter amplifier 30). NOR gate 32 is connected in a well-
known one shot multivibrator circuit 33, ~hich is connected
::: . . , :
in series with a further one shot multivibrator circuit 34,
to~provide a strobe pulse output at ter~inal 35. This pulse
will be used to strobe port circuit 7.
Interrupt port 12, as was described earlier,
provides information to the CPU 1 as to the type of in~ormation
" -10~
which is to be transmitted thereto. It is strobed in parallel
with input port 7. The details of how it is activated to
distinguish between serial ASCIX, parallel ASCII, or special
characters is well-known in the art, and consequently its
circuits will not be described in detail, since it will be ~i
assumed that parallel ASCII is provided. SufEice to say
that upon the reception of an indicator from the CPU that it
.,.~, ..
is ready to receive information, the presence of a strobe `~
pulse at the input of the interrupt port provides an indicator
on data bus 4 that normal ASCII information is to be applied
from typewriter port 7. Alternatively, it may signal on
data bus 4 that serial ASCII is to be providedj and thus that
the CPU should prepare itself for that type of input information.
: .
It should be noted that as an alternative to an `
electric typewriter input, a printed script readiny head
such as Scantrac I, available from Data Copy Corporation can
be used to apply input information. The Scantrac I is
comprised of a series of photocells disposed in such manner
that upon passing over a printed letter, a binary coded
output signal corresponding to the letter is provided. The ;;
code can be reduced, and can be converted to ASCII in a mini~
computer such as a Digital Equipment Corporation type PDP-8,
or through use of other electronic logic techniques. Suffice -
to say that while the technique for structurally reducing
the read script is not part of this invention, the application
: :.
~` of ASCII code derived therefrom to the present invention, to -~
provide a reading machine of normal books for the blind is
.. .. .
intended to be one of the applications of the present invention.
of course, it will be realized that the present
invention is not limited to the use of A5CII code, and the
structure can be provided to process any convenient input
code.
The general memory structure will now be described,
with reference to Figure 3. ;
Data bus 4 is shown as the group of lines connected
to input terminals D0-D7 of CPU 1. Connected thereto are
the output lines of typewriter input port 7 and interrupt
port 12, as well as the input lines of audlo output port 6,
described with reference to Figure 1. Also connected thereto
:~. . .'; '
are the output lines of memory output port 9.
The address bus 2 is designated by the individual
10 lines connected to terminals A0 - A15 of the CPU 1. Of ;-
course, other input and output terminals of the CPU 1 as are
normally required are also connected thereto, including
clock circuits, write enable circuits, reset circuits, etc.
Four of the address bus lines are connected to
memory selector circuit 11, which circuit simply decodes the
address of the selected memory and provides a single "chip
enable" CS signal at one of its output lines. Each of its
output lines are connected to an individual one of the CS ;~
(chip select) input of the individual memories to be activated.
The remaining address lines are connected through
inverters 40, to address inputs of the various memories.
Prom 41 is a semiconductor programmable read-only
memory connected to the address bus 2, and contains the
; operation programs for the CPU. This preferably is an ~.
- Intel type C1702A type memory. The CS input is connected to
the appropriate chip enable output line of memory selector
~ Memory 41A LS a similar type of memory, and is similarly
-~- connected with its address input to ~he address bus 2, the
output of inverters 40, and is connected similarly to Prom ~-
:~
41. Two memories 41 and 41A are used to carry the appropriate
numbers of general CPU operating instruction signals. The
~, . . . .
` -12-
.
,, :, ~ . ,~ ~
~ 7~
:
content of the instruction signals will not be descr1bed
herein, as they are considered to be within the ability o a
person skilled in the art, and do not specifically form the
substance of this invention.
A pair of phoneme information memories 3 and 3A ~-
are also connected with their address inputs connected to
bus 2. These memories are preferably read only memories
(ROMs) available from Electronics Associates as type EA4825.
Each can store over 4,000 binary bits.
A pair of random access memories 42 and 42A are
also connected with their address inputs to address bus 2.
These memories are stackable type memories for sequentially
storing input data, mentioned earlier, and as will be described
further later in this specification. It should be noted,
however, that they have data input terminals connected to
the data bus 4, for directly accessing ASCII code input
information. Consequently, as new information is applied
; to bus 4, the data stored in memories 42 and 42A is shifted
so that the earlier stored information is lost. It is
preferable that approximately 15 to 20 ASCII coded letters of
script be stored in memories 42 and 42A.
The output of all of the aforenoted memories are
connected to the memory data bus 8. For the particular ROMs
3 and 3~ selected, an inversion of the output is required prior
to application of the memory signals to the bus, shown
provided by inverter amplifiers 43. However, if a different
memory is chosen which does not require inversion, of course,
the inverter amplifiers 43 ~ill be delete~.
The data bus 8 is connected through memory output ;
port 9 to data bus 4.
The input terminals of audio output port 6, as was
noted earlier, are also connected to data bus 4. Its output
-13-
,. .
terminals are connected to digital-to-analog converter 5 as
was described with reference to Figure 1.
In operation, when a letter key on the Selectric
typewriter is depressed, an appropriate ASCII code, as
described with reference to Figure 2, is generated at the
output of typewriter input port circuit 7. At the sams
time, the interrupt input port 12 provides a signal via the ~; .
data bus that a certain kind of data is ava:ilable to the :~
CPU. The CPU accesses the appropriate translation program control
signals by addressing on address bus 2 a particular address
in programming Prom memories 41 or 41A, which provide operation
program information on memory data bus 8, which is applied
. to CPU 1. With the aid of the control program signals received
: from Prom 41 or 41A, the CPU translates the input ASCII code to an
address of information stored in memories 3 and 3A. The
address is applied to address bus 2, and the address in ~:
memory 3 or 3A accessed. The stored information at memory
3 or 3A is in response applied to memory data bus 8. This
information can be pulse code information usable by the
.. . ~: ~
audio output port 6 and the dlg1tal-to-analog converter 5,
or can be a designator to the CPU to select additional `~; .
memory locations in memories 3 and 3A.
Preferably, the information signal stored at the
.;
first memory address selected is both a further address and
~ a repetition index d1git. Recognizing this, the CPU accesses
:~ further program signals from memories 41 and 41A, to select : .
a phoneme portion resident memory locat1on at a different
address in memories 3 and 3A, and also to begin indexing the ~ :~
count of readout information each time an iterati~e loop of
.
~ ~30 phoneme-portion signals stored at that and a series of .. ~
: ~ .
addxesses has been read and completed.
This time, the information stored at the ne~ phoneme-
: -14- ~`
~357Bt~5 ~
portion resident m~mory location is pulse code difference
information which is applied to memory data bus 8, and
further applied through the memory output circuit 9 to data
bus 4. This inEormation is received at the input of audio
output port 6 and is applied to the digital-to-analog converter
5, whereupon a first audio output pulse, fabricating a small
portion of the output signal, is produced. ;
A second memory location is then addressed by the
CPU. The pulse code difference information stored at the
second memory location is applied as before to data bus 4 from
memory 3 or 3A, and is similarly treated by the audio output
port, digital-to-analog converter, etc., creating a second
element of the output signal.
This sequence is repeated until a terminator code
storéd at the end of the iterative loop has been reached.
At this point, the index number in the counter sequence in ~ -
the CPU 1 is indexed by 1, lndicating completion of a single (~ ~;
run through the iterative loop. A first phoneme-portion has
thus been produced.
Again, the first memory location in the iterative
loop is accessed under control of the program stored in
memories 41 and 4lA. Again, each memory location is accessed,
in sequence, through a second iterative loop which is
:
identical to the first~ This is continued for as many times
as it takes the CPU to index the.repetition number which was ; ~`
previou~sly read out to the cPu tô count from the number down
:~ ~ to 0, or from 0 up to that number (as desired~
It will be noted tha_ the pulse code signal read
out from each memory~location is simply that which can
~30 producé a small segment of the phoneme-portion. The completion
of an i~erative loop, completes only the phoneme portion,
and not a complete phoneme. It is at this point that a major
-15-
~71~
distinguishing feature between this invention and prior ar~
speech spelling machines becomes evident. In the present ~ -
invention, each run through an iterative loop repeats rapidly and
sequentially the same small portion of a phoneme. In the
prior art, a complete phoneme was stored, and reproduced once.
It was discovered by the present inventors that a complete
phoneme need not be read out for acceptably good acoustic
recognition of the phoneme, and that mereLy a small segment
thereof can be rapidly repeated a predetermined number of
times. It is because of this repetition that a substantial
saving in memory space can be obtained, with a subsequent
reduction in cost, allowing such machines to be much more
widely available than would otherwise be expected.
The time required to run through one complete
sequence of memory locations, prior to a following iteration,
is desirably about 80 microseconds. Depending on the letter, `
there may be a single, or up to, say, 20 iterations thereof
in order to provide a realistic sounding acoustic output
signal of a single complete phoneme. The number of iterations
per phoneme to be predetermined is subjective and will depend
on the esthetics of the reproduced sound to the ear of the builder
or his advisers.
In the case o a letter sound which requires more
than one phoneme, the terminator code signal stored at the
end of the iterative loop should designate a further memory
location which is accessed after the iterative loop repetition -
count has been completed. This further memory location will
designate to the CPU, in a similar manner to that described
; fbr the first, the first memory location of a further iterative
loop, as well as a corresponding index designating a repetition
count.
.:, ,
-16_
~ ' ' .
r - ~ ,
~35'7~3~S
The output signal is reproduced as described above,
and the two, conjointive fabricated phonemes are read out as
a sequence of phonemes which sound to the listener as an
alphabetical letter.
Needless to say, the above sequence happens very
rapidly, and the depression of a key on the typewriter
produces an audible output sound of the letter what appears ~;
to be immediately.
Clearly, this invention is useful to teach blind
people the correct keys to type for the learning of typing. `
It has been found that users rapidly obtain a speed of 50 to
70 words per minute using this machine. After some time,
the blind person need not utilize the acoustic output, and
the work product will be a typed page of script. `
For stenographic duties, it is often desirable to
recall the previously typed letter, or the previously typed
few words, assuming the user has either forgotten the last
typed letter or has left the machine for a time.
To recall the last letter or few words, memories `
~ 42 and 42A are used. As was noted earlier, these Intel type
8101 random access memories are stacked with input information,
and have a mode of operation of first in~ormation in, first
~; information out.
Upon depression of an appropriate special typewriter
:, - l`'
. character, as was noted earlier, the CPU addresses the
appropriate memory location in memories 42 and 42A, and a l~
corresponding output signal of the data stored at the address is , ;
applied to memory data bus 8 in ASCII code (appearing to be
a new data input). This data is passed through the memory
output port 9 and is applied to the input to the CPU 1 on data
bus 4, as if it had been a new data signal received from the `~
typewriter input port 7. The CPU 1, receiving this data signal,
-17-
-. '
. ~ .
lf~ffSff78f~f~ :
operates on lt in a similar manner as if it had received new
data, and accesses memories 3 and 3A to provide an acoustic out-
put signal made up of repetitive concatenated phoneme portions,
as described earlier.
- If a predetermined special purpose input character
is applied as data input from the typewriter, only the last
stored ASCII letter code at a first memory location in memories
42 and 42A is applied to the memory data bus. If a dif~erent
special purpose input character is applied as data input, the
memory 42 and 42A addresses are accessed sequentially, from first
in to last in, in order that the complete, spelled words last
typed in would be acoustically reproduced. The memories 42 and `-
42A are of course accessed in a well known manner by the chip
enable outputs o~ memory selector 11, as well as by designation ;~
of the appropriate addresses applied from CPU 1 on address bus 2.
Now that the general structure and operation of the
machine has been described, the internal memory architecture of
memories 3 and 3A will be described. --
The tables in Figures 4A and 4B relate to Memory II ,
~, . :.:
and Memory I, respectively, referred to below and show in the ~;
first column the memory address (location), in the second column
the name of the information stored thereat, and in a thlrd column
the octal coded designator signals. Of course, while numbers will
be re~erred toj it is intended that the binary designation signals
thereof be the actual signals which are stored in a well known
., ."~ :, .
manner.
; Taking as an example of operation of the desire to
access and read out the phoneme relating to the letter A, the
ASCII code the letter A will designate memory location 1 in
30 Memory I. Opposite memory location 1 in the table ~le letter A
will be seen, and in the octal column will be found the desig-
nator 034. The CPU should add the numeral 4,000 to the octal
. . .
- designator, indicating a memory location in
- 18 -
.
` 1():~ 71~5.~
Memory II of 4034, the location of further stored information
signals.
Turning to the Memory II first column, and reading
down the memory location (address) list for 4,034, it will
be seen that the designator ~ corresponds to two phonemes
E and I. A block of three addresses 4,034, 4,035, and 4,036 ~
are allotted to the letter A, since two phonemes are involved, ;
plus a terminator designation. The octal number stored at ~
. . .
the first memory location is 032. The CPU breaks up this
- 10 number to 03 and 2. The numeral 2 designates the number of
repetitions for the iterative loop, by setting how far a
counter loop in the CPU must count to reach zero. The
number 03 has the number 4,000 added to it as before, and ~
the Memory II is accessed at memory location 4,003. ~ ;
At memory address 4,003, which corresponds to the
; sound EI, the octal number 012 is stored. The number 012 in
octal corresponds to the binary digits 00001010. The binary
number should be shifted to the left 4 places ~i.e multiplied
by 2 = 16) to result in 10100000, which equals the octal
numeral 240. This number designates the first memory location, ;
in Memory I, which holds the pulse difference code information.
Turning to the two column address map portion
"
shown as in Figure 4C, address locations a40 to 300 contain
all the pulse difference code information to reproduce the
phoneme portion.
Taking the memory location 240, for example,
stored thereat is the numeral 022, in binary signal notation.
The corresponding octal number is 00010010, which contains
information for two sequential pulse difference code points,
30 0001 and 0010. The two numbers are read out of memory in
sequence to the data bus, and sequentially transIatecl by the
digital analog converter into two serial elements of an
audio output signal.
-19
~ ,t~355
The memory is then indexed to address loca~ions
241, 242, etc., and sequentially each of the octal coded binary
number signals stored thereat are read ou-t to create the
complete phoneme portion. When memory loca~lon 300 is
accessed, the number 167 is stored thereat, which is a
terminator signal, indicating that the sequence has been
completed. Running through the entire aforenoted sequence
takes approximately 80 microseconds, as noted earlier.
The counter in the CPU is then indexed once, and
10 memory location 240 is again accessed, and the entire sequence ~ -~
is repeated. Repetitions of the loop are continued unti:L
the counter in the CPU completes the number of loops indicated
by the repetition index. The phoneme /o~ has thus been completed. -
Once the proper number of iterative loops have
been completed, the second memory location 4035 in memory II
is accessed. The octal number stored thereat is 065, which
is broken up to 06 and 5, 5 indicating the number of repetitions
of the loop, and the number 6 to be operated upon in a
similar fashion as the 03 operated upon for the earlier
phoneme portion-
After completing the required number of repetitionsof the second loop, the phoneme /~/ has been acoustically reproduced,~
resulting in the se~uence /Q~ /i/ for the letter A. The
terminator number 377 is then accessed at Memory II location
4036~ This indicates completion of the production of the ~;
entire letter sound t and the return of the entire system to
a stable condition, awaiting input of a further ASCII encoded
letJer from the electric typewriter or other input device. `
It should be understood that~the number stored for the -
3~ n~mber of repetitions of the phoneme-portion which was
indicated above does not itself designate the speci~ic
number of times of repetition. It is preferred, for instance,
-20- -~
~ .
:~ .
~(3S78~5
that if the number 0 is stored,.15 repetitions of the loop should
be made, the number 1 to designate 13 repetitions, the
number 2:11 repetitions, the number 3:9 repetitions etc.
and the number 7, one repetition. In this respect, a minus
configured designator can be provided to the CPU, whereby
addition to zero can be effected.
Accordingly, the reader will understand that the
invention provides a method for repeatedly providing a first
signal which is representative of only a fraction of a
phoneme a predetermined number of times, and of reproducing
the repeated signal in an audio translation circuit to ~ ;
provide an audio reproduction of the repeated signal.
Furthermore, other portions of phonemes can be similarly
reproduced in series with the first to provide a complete
letter sound.
Experimental results with the described system has
confirmed that most blind subjects can learn to recognize 26
synthesized letter sounds after a short period of training,
and they could read the spelled sentences between 65 and 75
words per minute with an intelligibility score of about 85
correct. A bandwidth of about 6 kHz was about the minimum
bandwidth for reasonable correctness.
Since the letter sounds of the alphabet contain
many redundant phonemes (i.e~ letters Br C, D, E, A, etc.),
to minimize the number of samples required to represent
letter sounds, memory space can be minimized by eliminatlng
the redundancy. A list of 18 basic phonemes was selected in
such a way t'hat distinct sounds of the whole set of 26
letters of the alphabet can be generated by concantentation
of these basic phonemes. A list of these phonemes is as
follows:~
-21-
~ 7~
Consonants /b/ ~s/ /d/ /dz/ /k/ /p/ /t/ /v/ /w/
Vowels, Liquld and
Nasals /i/ /e/ /E/ /~/ /0/ /U/ /Q/ /m/ /n/
In addition, the following is a list of the letter
sounds and their concatenated basic phonemes is fQund below:
Letter ~ B C D E F G H I
Sound ei bi si di i EV dzi eidz ~E
Letter J K L M N O P Q R
Sound dzei kei E~ Em En OU pi kiu a
Letter S T U V W X Y Z ;~
Sound ES ti iu vi d~biu Eks w~Ee sEd `;~
A tremendous amount of memory conservation is thus
obtained. For spoken letter sound having duration of 4.1 `
seconds the typical amount of memory space which would be
required under ordinary conditions would be about 460 kilobits.
By the use of the present invention, however, the amount of
storage required due to the technique of storing only pulse
difference code information for only a small segment of the -~
' basic phoneme (which is of course repeated a designated ~-
; 20 number of times), was about 20 kilobits, clearly a significant ~ -
reduction from 460 kilobits.
' ,''~" :: '~
.. , ' ,. ":'
. ~:
"~
~ . . .
K - 22 -
`~
~ ,7~,~5
' In addition to the 26 spelled speech letter
sounds above, the same set of 18 phonemes can be used
to generate the punctuation sounds and the sounds for
~he numbers as listed below:
Numbers Pronouhced as Co~ents
. (approximately) .
. . . .
1 One w u n . .
2 Tw~ t u . .
3 Three t v 1 llspedl ..
4 Four . v o u Initial v in place o~ ~ . .
Fi~e . v a ~ e v l~itial v in place of f
6 . Six 8 i k s . -~
7 Seven ~ c ~ n .
8 Eight e i t . . .
. 9 Nine . ~ e n . .
. O Zero 8 i W O lisped and lnitial
. . . . ........ . conso~ant inaxact
. - . -.. ,. __ .. _ I .
. . _ - ~ 'I ::
Pumctuation . . . . .
I exclaim eksk~eim . . .
quotR . t~outs . .
it . number . ~aliba . .
, S dollar daQa rather short
X percent pUS~t ' . .
m~er~ ~nd - and . .
' apos (trophe) apos . . . .
~ left.~bracket) Q~Yt . .
) right(bracket) w~et . . .
* star . ~ta . . .
- dash dad more like "da~"
~- equ~l3 ikwu~s - ;; . .
@ at . ~t ~ . . .
backwards bakwds . : . -: ~ :
. t up twards) ap . . . - .
. ~ Tab . ~b . . . .
left square (bracket~ ~cvtskw~a Barrack square ::
right square (brac~et) W~etskwa Sergent ~ajor accent ~:
le~s ~than) Q8 , ,
larger Qad a ~ .
questio~ kwstun . ~:.
' comma ' , kama ... .. . . ::
. (full) stop ~tap . . . . . ~: :
over ova . . :-
: - . , ' pst pst ~ . . :
~ ' D ~ ' ' pst . . ,
;~: I slasX s~ad . . .
spacebar ~pe~ 9 .
Speech itself can be generated by concatenating --~
a larger number of phonemes. A complete set o~ 43 phonemes ;.
are listed in Table 1 and Table 2 appended hereto.
- 22a -
~,.
.' ~ .
~o57~355 ~
The total memory space required would be 53.6 kb (corresponding
to 4b/sample).
Figure 5 shows a waveform of one phoneme portion. ~ -
This phoneme portion will be repeated as ~any times as the
repetition index designates.
It should be noted that the waveform is made up of ,
small dots, each of which is spaced along the horizontal ~ -
axis in time 80 microseconds. The difference in amplitude
of each dot from the previous one corresponds to an amplitude '-
difference specified by the difference code stored in the
Memory I, as described earlier. Each sequential 2 dots will
.
.
~;!
:.', ' :,~:` i:
1 .. ~.
' .: ;::
- 22b -
. ; ~ .
7b~5~
be determined by the two numbers stored in each of the
aforenoted memory locations. Each of the read only memories
as described above store 2,000 eight bit bytes for a total
of 16,000 bits per memory. With two memories, 32,000 bits ~ ;
can be stored, more than the total memory required.
With the relatively small amounl: of memory required,
and with the availability of semiconductor chip central `~
processing units, it is believed that this invention can be
used widely in a variety of environments. Institutions of
the blind are now able to rapidly teach blind people communication
by typing, allowing the avenue of office work to be opened
up to the blind, and as well, with the previously noted photo-
electric reader, a blind person can read ordinary printed
books, and need not be confined to Braille. Individual
letters will be spelled out, or speech can be generated.
It will be now understood by someone skilled in
the art that various alternative structures to the preferred
... .
~ embodiment described above may be fabricated, within the
: bounds of the structural and method concepts described. All
.:
are considered to be within the scope of this invention, as
. defined in the appended claims.
~ ,. . .
-,
:.:
:, . .-
. .;,~
. ~ . ,
.: ::
:: -
- 23 -
: . ~
~' '',.
. ~ ,,
7B~S ~ ~
. 'TABLE 1 CONSON~NTS ~ `
Consonant L ~ number Example Tran~i~nt or Fricative? ' .
of samples
¦b~ 328 bean T
¦c¦ 511 cheap T
¦d¦ 2~9 dean T `
¦f¦ 512 e~ F
Igl 330 ~ain T
¦h¦ 896 ~eap F :
l~l 617 ~aw T
¦k¦ 768 cane T
: 10 ¦mt¦- ¦m¦ 190 ~ay T ~Initial position only)
IPI 400 pen T ;
l-l 960 sheep F :
¦r¦ 509 ray F
¦8¦ 504 saw F
¦t¦ 384 ten T
: ¦v¦ 960 : river F
¦w¦ - 449 weak' T
; ¦Zl : ~ 512 leisure F
732 ~!~wn F
. :20 ¦z¦ 511 bu~z F
¦0i 966 thin F
439 . the~
. ~let ' T (Initial position Only)
~el ~ Inl 220 ~ow ~ ~ T ~Initial position only~
: r ~
. . ~ :', ' '
'~,: :
;,' , ~ ~; ::
."': " ~'~'
-: - 24 - .'. ~
': . `` ~ . ,':'~'
7~S~
TABLE 2 VOWELS AND T~IE NASAL GROUP ¦m¦¦n ¦ ¦~ ¦ AND THE LATEP~L
Duration OI a
L' ~ samples per single cycle
: Name voice-bar period Repetition Example (milliseconds)
24 sane
lil 62 24 seep 5.1
: ¦a ¦ 62 24 slgh (extracted from
vowel-glide)
InI 59 16 pencil ~ ~:
lol 63 32 cope
; ¦u¦ 60 32 shoe
; I I 63 24 ~aw 5.2
¦aeI 60 24 pan
¦l 54 24 pen 4.5
¦I¦ 59 24 plp : ~
~¦ 54 24 pun
~j 57 24 put
. .
~ I 60 ,~1~ surprise ~
. . ~ . .
: ¦au¦ 58 24 town (extracted from . :
vowel glide)
~ il 63 24 poise (extracted from
:~ vowel glide) .
¦1I 55 24 pal ` ~j~
~r~ 56 24 purr
l ¦aer¦ 62 32 father -designated ¦ ~ ¦ in
,~), first list ..
67 24 sing
;~ I I 58 32 aim :
''1
~'~`'. ' .
~ :,
. ' ' ' ~
:
