Computación y señales sociales
-
Upload
universitat-de-barcelona -
Category
Technology
-
view
1.112 -
download
0
description
Transcript of Computación y señales sociales
Bla, bla, bla
Computación y señales sociales
Jordi Vitrià
Universitat de BarcelonaDepartament de Matemàtica Aplicada i Anàlisi
&&Centre de Visió per Computador
Computación y señales socialesPág. 2
Computación y señales socialesPág. 3
Computación y señales socialesPág. 4
Index
A new look to human behavior
A new opportunity for machine perceptionA new opportunity for machine perception
Social Signals ProcessingSocial Signals Processing
Computación y señales socialesPág. 5
A new look to human behavior
Racism and racial segregation.
The thin‐slice theoryy
FROM TWO APPARENTLY NON‐CONNECTEDFROM TWO APPARENTLY NON CONNECTED TOPICS TO A NEW COMPUTING PARADIGM…
Computación y señales socialesPág. 6
Thomas C. Schelling,2005 Economy Nobel Laureate
Computación y señales socialesPág. 7
Nobel Prize winner Thomas C. Schelling in a 1971article applied game theory to explain on aarticle applied game theory to explain, on ahypothetical level, how racially mixed neighborhoodscould quickly become segregated even if the peoplecould quickly become segregated, even if the peopleinvolved are not very segregationist.
Computación y señales socialesPág. 8
Computación y señales socialesPág. 9
THE BIRTH OF SOCIAL PHYSICS/NETWORK SCIENCE
Schelling’s game implies that social outcomes needn’t, l i fl i b i hat least in some cases, reflect in any obvious way the
desires or intentions, habits or attitudes of anyone t llat all.
Hi i li i i i h i l h i hHis implicit point is that social happenings can have simple origins and are subject to laws not unlike those of physics (complexity, network analysis, small world effects, power laws, etc.)
Computación y señales socialesPág. 10
Computación y señales socialesPág. 11
Nalini Ambady & the Thin Slice Theory
Computación y señales socialesPág. 12
THE EXPERIMENT
In a 1993 study published in the Journal of Personality and SocialP h l (V l 64 N 3) A b d d ll id t dPsychology (Vol. 64, No. 3), Ambady and a colleague videotaped13 graduate teaching fellows as they taught their classes.
She then took three random 10‐second clips from each tape,combined them into one 30‐second clip for each teacher andcombined them into one 30‐second clip for each teacher andshowed the silent clips to students who did not know theteachers. Students assigned a rate to each teacher.g
Then, Ambady correlated that rating with the teachers' end‐of‐
THIN SLICE of BEHAVIOR
, y gsemester evaluations from actual students.
Computación y señales socialesPág. 13
THE RESULTTHE RESULT
"We were shocked at how high the correlationgwas," she says. It was 0.76. In social psychologyanything above 0.6 is considered very strong.y g y g
These results have been re‐validated in several experimentsand extended to other tasks and environments.
Computación y señales socialesPág. 14
THIN SLICING and PERCEPTION
What is important is not what we say but h b h d i h i ihow we behave during the interaction.
Voice Face Body
Characteristics
Pitch
Expressions
Frowning
Head Postures
Head NodLoudnessSpeaking rateTurn taking…
gSmilingLip‐PoutTense‐Mouth…
…
Head NodHead ShakeHead Tilt‐back…
Body Gestures
Gaze
Who is she looking at?Gaze dynamics
y
Shoulder shrugForward leanHand behind head…
Conjugate lateral eye movementFast blink…
Computación y señales socialesPág. 15
THIN SLICING THEORY
Thin slices have been shown to have predictivevalidity in a number of different contexts:validity in a number of different contexts:
• teaching (teacher expectancy on studentsteaching (teacher expectancy on students, teacher bias, teacher effectiveness), • learning (student achievement),learning (student achievement), • job performance (telephone operators, sales managers, management consultants),managers, management consultants), • employment interviews, • health care (doctors effectiveness, patienthealth care (doctors effectiveness, patient satisfaction), etc.
Computación y señales socialesPág. 16
THIN SLICING THEORY
One of the most impressive examples of thin slicesof data predicting important, long‐termconsequences is marital research conducted byGottman and his colleagues.
Carrère and Gottman (1999) were able to predictmarital outcomes over a six year period based onhuman social signaling micro‐coding over just thefirst 3 minutes of a marital conflict (i.e., a thin sliceof expressive behavior).
Computación y señales socialesPág. 17
• Importance interest and other social attitudes are reflected in• Importance, interest and other social attitudes are reflected in speaking style and body language.• People automatically perceive this information and can use it p y pto accurately predict behavior (even being unaware of it!).
Computación y señales socialesPág. 18
Computación y señales socialesPág. 19
Computación y señales socialesPág. 20
Are we taking about emotions?Are we taking about emotions?
Computación y señales socialesPág. 21
Paul Ekman
Are we taking about emotions?
Computación y señales socialesPág. 22
P. Ekman is the most well‐known advocate of this approach, which is basedroughly on the theory that people perceive others’ emotions throughstereotyped displays of facial expression, tone of voice, etc.
The simplicity and perceptual grounding of this theory has recently given riseThe simplicity and perceptual grounding of this theory has recently given riseto considerable interest in the computational literature .
Are we taking about emotions?
Computación y señales socialesPág. 23
Computación y señales socialesPág. 24
Conclusions
1. Social psychology, behavioral economics and otheri l i l i h h hexperimental social sciences have shown that some
interesting human behaviors are not directly related toour conscious experience but are driven by ratherour conscious experience, but are driven by rather“simple” unconscious decision mechanisms.
2 Moreover some of these decision mechanisms are based2. Moreover, some of these decision mechanisms are basedon the (unconscious) perception of social signals that are(unconsciously) emitted by all of us when interacting with( y) y gothers.
3 These two approaches defy the classical “cognitive3. These two approaches defy the classical cognitivescience” approach for understanding human behavior.
Computación y señales socialesPág. 25
Conclusions
A fundamental assumption of cognitive science is thatthe individual is the correct unit of analysis forunderstanding human intelligence.
These approaches present evidence that the socialnetworks containing the individuals are an importantadditional unit of analysis, and that this `networkintelligence’ is significantly mediated by non‐linguisticprocesses.
Computación y señales socialesPág. 26
And what about computer perception of the world?
A new opportunity for machineA new opportunity for machine perceptionp p
Computación y señales socialesPág. 27
The Original Sin of Computer Vision
Computer Vision models are based/inspired in
The Original Sin of Computer Vision
Computer Vision models are based/inspired incognitive models from the beginning (Marr & thetheory of reconstruction)theory of reconstruction).
This models are based on the individual and do notThis models are based on the individual, and do notconsider the social world in which we live.
People is considered just an another “object” that canbe identified tracked etcbe identified, tracked, etc.
Computación y señales socialesPág. 28
One legend in AI: In 1967 Marvin Minsky assigned a graduate student (Gerald Sussman) to the problem of computer vision, believing that the problem could be largely solved within one summer.
Computación y señales socialesPág. 29
The summer project was not successful, but its approach to solve the problem has not changed a lot from then….
Definition:
As a scientific discipline, computer vision is concerned with thetheory and technology for building artificial systems that obtaintheory and technology for building artificial systems that obtaininformation from images. The image data can take many forms,such as a video sequence, views from multiple cameras, or multi‐di i l d t f di ldimensional data from a medical scanner.
obtain information from images =
physical word description
Computación y señales socialesPág. 30
From David Marr's book: Vision 1982From David Marr s book: Vision, 1982.
Computación y señales socialesPág. 31
What about understanding people?
THE CANONICAL VIEW
1 Th i t d f t th t1. There is a great need for computer programs that can describe and predict people activities from video,
2. This is difficult to do, because it is hard to identify and2. This is difficult to do, because it is hard to identify and track people in video sequences, because we have no common vocabulary for describing what people are doing, and because the interpretation of what people are doing depends very strongly on context.
That’s true, but this is not the whole truth: there is also a lack of appropriate models for understanding pp p g
people and their social world.
Computación y señales socialesPág. 32
Artificial H man Perception is foc sed onArtificial Human Perception is focused on:
Identity, gender, ethnicity, age, etc.EmotionsActivities
Gestures (for interaction)Gestures (for interaction)
But this model of human behavior is not complete…
Computación y señales socialesPág. 33
From: A.Vinciarelli, M.Pantic, H.Boulard, Social signal processing: Survey of an emerging domain, Image and Vision Computing, Volume 27, Issue 12, November 2009, Pages 1743‐1759
Computación y señales socialesPág. 34
Social Signals
Voice Face Body
Characteristics
PitchLoudnessSpeaking rate
Expressions
FrowningSmilingLip‐PoutTense Mouth
Head Postures
Head NodHead ShakeHead Tilt‐back
… Tense‐Mouth…
Gaze
……
Body Gestures
Shoulder shrugForward lean
Who is she looking at?Gaze dynamicsConjugate lateral eye movementFast blink…
Forward leanHand behind head…
Computación y señales socialesPág. 35
What we know about social signaling?
It’s short term: People need about 30+ seconds of observation to infer information from it.
It’s important: there are some real social contexts where human decisions are mainly (but not exclusively) basedhuman decisions are mainly (but not exclusively) based on the content conveyed by this specific, non consciouscommunication channel between humans.
It’s a collective game: Information is mainly encoded in h d ( d b lthe interaction dynamics (voice and gestures = nonverbal cues).
Turn‐taking, emphasys, Arms, body, face, eyes...
Computación y señales socialesPág. 36
We can built social signal detectors by analyzing speech andbody movementsbody movements.
This signals can be aggregated in communicative patterns(some of them were proposed by Charles Darwin!)(some of them were proposed by Charles Darwin!)
1 Influence patterns (speaking pitch patterns of turn taking1. Influence patterns (speaking pitch, patterns of turn taking, etc.) It is an indicator of dominance.
2. Activity patterns (energy expended in the communication) It . Activity patterns (energy expended in the communication) Itis an indicator of interest and excitement.
3. Mimicry patterns (reflexive copying of smiles, interjections, y p py g jhead nodding) It is an indicator of empathy.
4. Consistency (emphasys and timing). It is an indicator of determination (+) and openess (‐).
Computación y señales socialesPág. 37
F iFunctions
1. Forming impressions.
2. Expressing emotions.2. Expressing emotions.
3. Sending relational messages (power, persuasion,dominance deception )dominance, deception…)
4. Managing interactions.
5. Etc.
Computación y señales socialesPág. 38
Social cues Example social behaviours Tech.
Emotion Personality Status Dominance Persuasion Regulation Rapport Speech analysis Computer vision Biometry
Physical appearance√ √ √ √Height √ √ √ √
Attractiveness √ √ √ √ √ √ √
Body shape √ √ √ √
Gesture and posture
Hand gestures √ √ √ √ √ √ √
Posture √ √ √ √ √ √ √ √ √Walking √ √ √ √ √
Face and eyes behaviourFace and eyes behaviourFacial expressions √ √ √ √ √ √ √ √ √
Gaze behaviour √ √ √ √ √ √ √ √
Focus of attention √ √ √ √ √ √ √ √
Vocal behaviourProsody √ √ √ √ √ √Turn taking √ √ √ √ √ √ √
Vocal outbursts √ √ √ √ √ √ √√ √ √ √ √ √ √
Silence √ √ √ √
Space and environmentDistance √ √ √ √ √ √
SeatingSeatingarrangement √ √ √ √
From: A.Vinciarelli, M.Pantic, H.Boulard, Social signal processing: Survey of an emerging domain, Image and Vision Computing, Volume 27, Issue 12, November 2009, Pages 1743‐1759
Computación y señales socialesPág. 39
Social Signal Processing
Computación y señales socialesPág. 40
H ?How?
1. Processing real‐time video/audio and detecting andanalyzing social signals.
2. Building rich descriptions from these signals (socialmessages).g )
3. Understanding syntax and semantics of socialmessages (social language) by mining examples andmessages (social language) by mining examples andlearning to predict.
Computación y señales socialesPág. 41
Example: Reading FacesExample: Reading Faces
• Demographics: Identity, age, gender…
• Emotional ExpressionsEmotional Expressions
• Lip Reading
• ?
Computación y señales socialesPág. 42
Reading Faces
People automatically evaluate faces on multiple trait dimensions and thesemultiple trait dimensions, and these evaluations predict important social outcomes, ranging from electoral success to sentencing decisionsto sentencing decisions.
Aggressiveness, dominance, confidence, attractiveness, trustworthiness,attractiveness, trustworthiness, competence, etc.
Computación y señales socialesPág. 43
DominanceDominance
Computación y señales socialesPág. 44
CompetenceCompetence
Computación y señales socialesPág. 45
Computación y señales socialesPág. 46
Body Behavior + Non Verbal SpeechBody Behavior + Non Verbal Speech
(MIT)
1. Speed dating prediction.
2 Call center customer2. Call center customer satisfaction.
3 “One minute elevator3. One‐minute elevator pitch” assessment.
4 Depression monitoring4. Depression monitoring.
5. Etc.
Computación y señales socialesPág. 47
In these scenarios, we can get anwe can get an outcome
prediction with a level of accuracylevel of accuracy between 80 and
90%
Computación y señales socialesPág. 48
Organization Dynamics:
Other experiments show that it ispossible to identify:p y
•Connectors and central peoplel kin a social network
•The boss in an organization•The leader of a team•The leader of a team•The outcome of negotiations•The degree of persuasiveness in•The degree of persuasiveness inspeech•Group affiliations
Automatically captured group dynamicsp
Computación y señales socialesPág. 49
Long term behavior analysis
Computación y señales socialesPág. 50
Long term behavior analysis
Computación y señales socialesPág. 51
Agitation in ICU
Computación y señales socialesPág. 52
Dominance in computer mediated discussions
Data Blogging heads New gg g
York Times data base (http://video.nytimes.com/)
Experiments Observers inquiry Observers inquiry Manual test Automatic testSi l Signals Head nodding Speaking time (A/V)
S f l Successfulinterruptions
Floor grabbingi l i
52
Gesticulation
Computación y señales socialesPág. 53
Social Signals and Reality ShowsCan we predict who will be fired?Can we predict who will be fired?
:
The Apprentice is a reality television show that originated in the UnitedStates on NBC. Billed as "The Ultimate Job Interview," the show depicted agroup of 15‐18 businessmen and ‐women competing in an elimination‐stylecompetition for a one‐year, $250,000 job of running one of host andexecutive producer Donald Trump's companies.
Computación y señales socialesPág. 54
Conclusions• Social signal processing can contribute to the understanding of the
role of non‐verbal behaviour during social interactions (a new toolrole of non‐verbal behaviour during social interactions (a new toolfor social sciences).
• The range of application areas for socially aware systems toucheson many aspects of computing, and as computing becomes morebi i i ll f i i i h bjubiquitous, practically every aspect of interaction with objects,
and the environment, as well as human‐human interaction willmake use of these techniques (situated communication supportq ( pptools, human‐computer interaction tools, intelligent tutoringsystems, telecommunication facilities, intelligent multimediaser ers health monitorin s stems intelli ent robots )servers, health monitoring systems, intelligent robots )
Bla, bla, bla
Muchas Gracias