Learning the Meaning of Verbs

Experiment 1

Purpose: Test how adult learners generalize to different verb-concepts from a small number of stimulus-utterance pairs 
Procedure: 50 adult subjects from the MIT community will perform a psycholinguistic experiment at a location of their choosing within the MIT campus. Each subject, in the experimenter's presence, will use a computer to access a URL (http://web.mit.edu/niyogi/www/expt/expt.htm) which explains the task and requests their informed consent.

In accordance with MIT's COUHES, this URL explains:
  • Subjects participation is voluntary and may be withdrawn at any time without prejudice
  • Subjects can decline answering any question
  • Subjects can ask the experimenter questions at any time
  • All information collected will be anonymous and confidential and will protect the identity of all participants
  • The length of the experiment will be approximately 30 minutes
  • In the unlikely event of an injury, subjects are told their medical and legal options, taken verbatim from the COUHES web site.
Upon obtaining their informed consent in writing, the experiment will begin. Subjects will use the URL to access up to 20 MPEG stimuli over the web. Stimuli will be viewed at normal viewing distance. Each stimuli will be approximately 60-90 seconds long, and consist of 4 training stimuli of 1 or more persons interacting with 1 or more ordinary objects (table, balloon, box, pot, etc.) and 3 similar test stimuli. Superimposed of the training stimuli are audio clips of short English sentences (e.g. "He is gorping the balloon") containing a novel verb (e.g. "gorping"). Superimposed over the test stimuli are Yes-No. Subjects will be requested to view each of the stimuli and record their Yes-No responses written down on answer sheet provided to subject (a printed version of web page).
No audiotaping or videotaping will be used. All consent forms and data will be secured by the principal investigator.
Training:
For a particular novel verb, subjects are shown 3 training stimuli (see Stimuli below) paired with utterances of one of the following forms:
(a) Transitive: "X is Ving Y", supporting causative or unergative verb syntax
(b) Intransitive: "Y is Ving", supporting causative or unaccusative verb syntax
All stimuli have only one interval of stuff happening (no "result"), so there is no temporal uncertainty
Some stimuli have two "accidental" discernable actions going on (e.g. rub-lower1 shows rubbing and lowering of a balloon) while others are "generic" (e.g. lower1 has only lowering)
Test:
Subjects are shown 4 test stimuli and asked either (a) "Is X Ving Y?" or (b) "Is Y Ving?" in the same 'frame' as the training stimuli.(No generalization across syntactic frames tested).
Test stimuli are either generic examples of what the training examples have in common (lifting), a more general concept (holding/moving) or are superficially similar to one or more of the training stimuli (rubbing)
Hypotheses: (1) Learners will use what the training examples have in common, rather than similarity to the training examples, to arrive at a verb-concept mapping used in judging test stimuli
Predictions in line with this hypothesis are shown in yellow below
(2) Learners will use syntax to guide their semantic concepts.
Given the same training stimuli but different utterance forms (see 1/2,3/4, 9/10), learners fit different conceptual bits. The unaccusative/causative concept learning problem will involve a smaller # of bits than for the unergative concept learning problem
Subjects Pilot on 10 adults, max 30 mins per subject. 
Results Results are summarized as % of subjects who classified test stimuli as falling under verb concept inferred from training examples.
A perfect fit to our hypothesis would have 100% - 1, 0% - 0 for our predictions
 
Stimuli
Training
Test
Predictions
#
 Novel Verb Train Syntax Training 1 Training 2 Training 3 Test Syntax Test 1 Test 2 Test 3 Test 4 Most Likely Concept Pred 1 Pred 2 Pred 3 Pred 4
1 gorping Transitive rub-lower1 rub-lower2 rub-lower3 Transitive lower2 rubontable1 rub-raise2 pat-lower2 rub, lower 0 1 1 0
2 pilking Intrans rub-lower1 rub-lower2 rub-lower3 Intrans lower2 rubontable1 rub-raise2 pat-lower2 lower 1 0 0 1
3 karbing Transitive rubontable2 rub-lower3 rub-raise1 Transitive lower1 rubinmidair2 rub-lower2 pat-lower4 rub 0 1 1 0
4 prigging Intrans rubontable2 rub-lower3 rub-raise1 Intrans lower1 rubinmidair2 rub-lower2 pat-lower4 - 0 0 0 0
5 jilfing Transitive rub-lower1 pat-lower1 pat-lower2 Transitive lower3 pat-inmidair rub-lower3 pat-lower3 lower 1 0 1 1
6 rilting Transitive lower1 lower2 lower3 Transitive lower4 raise-hold1 moveforward pat-lower1 lower 1 0 0 1
7 toving Transitive rubinmidair1 rubinmidair2 rubontable1 Transitive rubontable2 hold-midair pat-inmidair rub-raise1 rub 1 0 1 0
8 vabing Transitive lower1 raise hold-midair Transitive hold-midair2 lower1 raise-hold2 touch1 hold 1 0 1 0
9 gimming Transitive lower2 raise raise-hold2 Transitive hold-midair raise-hold3 lower5 touch1 hold, move 1 1 0 0
10 goving Intrans lower2 raise raise-hold2 Intrans hold-midair raise-hold3 lower5 touch1 move 0 1 1 0
11 rebing Transitive roll1 roll2 rollback1 Transitive roll3 rotateinmidair1 flipover2 spinning roll 1 0 0 1
12 yarging Transitive roll4 rotateinmidair1 rotateontable Transitive rollback3 rotateinmidair4 flipover2 spinning rotate 1 1 0 1
13 serbing Transitive raise rotateinmidair5 spin2 Transitive lower1 rollback3 touching1 rest move 1 1 0 0

 

Experiment 2

Procedure: Same as above, but now we introduce temporal uncertainty.  In Expt 1 stimuli have 1 interval, in Expt 2 stimuli have 2-3 distinct intervals (+1-2 instant).
The verb could be referring to the first interval, the second interval, both intervals, the transition out of the first interval, or into the second interval -- the learner doesn't know.
A rational learner, given this uncertainty, would use all the available evidence -- with 1 example, all concepts may be equally supported (or learners could have priors on which concepts should be preferred).With more than 1 example, however, a rational learner could let each interval (or combination of interval/instants) vote for each hypothesis -- the hypothesis most voted for would be most likely concept.
Training:
Shown below, but now 4 examples are provided so that the "target" concept is the first interval in half the training stimuli, the second interval in the other half. Training stimuli order is randomized
Test:
Same as above, test stimuli order randomized from below.Test 1 is "most likely" concept, test 2 and 3 superficially similar to training, test 4 is "potpourri" to balance
 
Stimuli
 
Training
Test
Predictions
Order
# Novel Verb Train Syntax Training 1 Training 2 Training 3 Training 4 Test Syntax Test 1 Test 2 Test 3 Test 4 Most Likely Concept Pred 1 Pred 2 Pred 3 Pred 4 Train Test
14 farbing Transitive lift-drop(-bounce) grab-lift raise-lower grab-lift(-drop) Transitive raise grab(-lift) (hold-flip-hold)-drop pat-lift1 lift y n n y 1234 4321
15 barping Transitive hold-lower(-raise-lower hold-flip(-hold) (hold-)flip-hold(-drop) Transitive hold-midair lower1 flipping rollpatforward hold y y n n 2341 3214
16 parging Transitive grab-flip(-fall) (grab-)flip-grab grab-lift (lift-drop-)bounce-grab Transitive grab-lift flipping raise catch grab y n n y 3412 2143
17 beeling Transitive removefrombox-rest throw-remove removepushintopot rolltobox-remove Transitive removefrompot throw pushontable removefrombox-rest2 remove y n n y 4123 1432
18 yeeping Transitive rolltobox-remove grab-flip(-fall) rotating-drop hold-flip(-hold) Transitive rotateinmidair4 removefrombox grab(-lift) roll4 rotate y n n y 1234 4321

Stimuli Classification

Below are the MPEG filenames and the consistency of their contents with some English verbs.
 
Legend rub pat lift/raise lower fall hold rest touch roll rotate/flip spin move/go bounce catch grab drop hit pop throw fling punch push/pull put remove shake slide squeeze
Generic Representative Example rubinmidair1 pat-inmidair raise lower1 fall1 hold-midair rest touch1 roll1 rotateinmidair1 spin2 moveforward bouncing catch grab-lift rotating-drop hitting1 pop throw flinging punch-fall push-letgo putinbox removefrombox shaking slide2 squeeze-midair
Accidental Example rubinmidair2 patontable1 raise-hold1 lower2 fall2 hold-midair2 rest2 touching1 roll2 rotateinmidair2 spinning moving bouncing2 throwup-catch grab-flip-fall grab-lift-drop hitting3 throw2 flinging2 push-letgo2 putinbox2 removefromboxtop shaking2 slideback squeeze-midair2
Generic, Temporally Uncertain rubontable1 patontable2 raise-hold2 lower3 fall3 hold-ontable fallrest roll3 rotateinmidair3 spinonbox blow-move bouncing3 throwing-catching grab-flip-grab hold-flip-hold-drop   throw3 propel push-letgo3 putinpot removefrompot shaking3 slidebox
rubontable2 pat-lift1 raise-hold3 lower4 fall4 hold-flip-hold rest-blow-move roll4 rotateinmidair4 spin-catch moving-forward-and-backward bouncing4 throwing-catching2 grab-lift-drop throw4   push-letgo4 putonbox removefrompot2 shaking4 slidein-andout-ofbox
rub-lift1 pat-lift2 pat-lift1 lower5 fall5 hold-flip-hold-drop roll5 rotateinmidair5 bouncing5 throwing-catching3 throwing-catching pushletgo5 putonbox2 removepushintopot
rub-lower1 pat-lower1 pat-lift2 pat-lower1 fall6 hold-lower-raise-lower rollback1 rotateinmidair6 bouncing6 throwing-catching4 throwing-catching2 pushontable rolltobox-remove
rub-lower2 pat-lower2 rub-lift1 pat-lower2 fall7 rollback2 rotateinmidair7 bouncing7 throwing-catching5 throwing-catching3 pushontable2 throwinpot-remove
rub-lower3 pat-lower3 rub-raise1 pat-lower3 fallrest rollback3 rotating-around throw-bounce throwing-catching4 push-ontable3 throw-remove
rub-raise1 pat-lower4 rub-raise2 pat-lower4 grab-flip-fall rollforward1 rotating-drop throwing-catching5 push-ontable4 throw-remove2
rub-raise2 pat-raise1 lift-drop-bounce rub-lower1 punch-fall rollpatforward rotateontable throwatpot-miss   push-holding removefrombox-rest
pat-raise2 lift-drop-bounce-grab rub-lower2 rolltobox-remove rotateontable2 throw-bounce pullontable removefrombox-rest2
rollpatforward lift-dropbouncegrab2 rub-lower3 roll-atpot rotateontable throwinbox push-pull
grab-lift raise-lower rollintobox2 flipover2 throwinpot-remove removepushintopot
grab-lift-drop flipover3 throwintobox pushletgotwice
lift-drop1 flipping throwintopot pushfromboxontotable
raise-lower grab-flip-fall throw-remove pushfromboxontotable2
hold-lower-raise-lower grab-flip-grab throw-remove2 pushfromboxtopintopot
hold-flip-hold throwup-catch
hold-flip-hold-drop