Textual Entailment. Arindam Bhattacharya. M.Tech, Computer Science Indian Institute of Technology, Bombay. November 9, 2011

Transcription

1 Textual Entailment Arindam Bhattacharya M.Tech, Computer Science Indian Institute of Technology, Bombay November 9, 2011 Arindam (IITB) Textual Entailment November 9, / 59

2 Outline 1 Introduction Definition Entailment Triggers Role of Knowledge RTE Challenges Resources 2 General Strategy 3 Lexical Approach 4 Machine Learning Approach 5 Graphical Approach 6 Deep Semantic Approach Text Entailment using UNL Arindam (IITB) Textual Entailment November 9, / 59

3 Outline 1 Introduction Definition Entailment Triggers Role of Knowledge RTE Challenges Resources 2 General Strategy 3 Lexical Approach 4 Machine Learning Approach 5 Graphical Approach 6 Deep Semantic Approach Text Entailment using UNL Arindam (IITB) Textual Entailment November 9, / 59

4 Outline 1 Introduction Definition Entailment Triggers Role of Knowledge RTE Challenges Resources 2 General Strategy 3 Lexical Approach 4 Machine Learning Approach 5 Graphical Approach 6 Deep Semantic Approach Text Entailment using UNL Arindam (IITB) Textual Entailment November 9, / 59

5 Definition Classical Definition A text t entails a hypothesis h if h is true in every circumstance (possible world) in which t is true. Strict Entailment! Doesn t account for real world uncertainties. Example: T: Ram was born and brought up in Maharashtra. H: Ram can speak Marathi. Applied Definition t entails h (t h) if humans reading t will infer that h is most likely true. Arindam (IITB) Textual Entailment November 9, / 59

6 Definition Classical Definition A text t entails a hypothesis h if h is true in every circumstance (possible world) in which t is true. Strict Entailment! Doesn t account for real world uncertainties. Example: T: Ram was born and brought up in Maharashtra. H: Ram can speak Marathi. Applied Definition t entails h (t h) if humans reading t will infer that h is most likely true. Arindam (IITB) Textual Entailment November 9, / 59

7 Probabilistic Interpretation Applied definition sounds good. But doesn t sound concrete of mathematical. Probabilistic interpretation t probabilistically entails h if: P(h is true t) > P(h is true) P(h is true t) is called Entailment Confidence. Arindam (IITB) Textual Entailment November 9, / 59

8 Probabilistic Interpretation Applied definition sounds good. But doesn t sound concrete of mathematical. Probabilistic interpretation t probabilistically entails h if: P(h is true t) > P(h is true) P(h is true t) is called Entailment Confidence. Arindam (IITB) Textual Entailment November 9, / 59

9 Goal Figure: Textual Entailment Arindam (IITB) Textual Entailment November 9, / 59

10 Outline 1 Introduction Definition Entailment Triggers Role of Knowledge RTE Challenges Resources 2 General Strategy 3 Lexical Approach 4 Machine Learning Approach 5 Graphical Approach 6 Deep Semantic Approach Text Entailment using UNL Arindam (IITB) Textual Entailment November 9, / 59

11 Entailment Triggers Triggers are linguistic features that affect entailment [?]. Here are some examples to show how these various factors affect entailment. Synonymy: Very common form of entailment trigger, where a word is replaced by its synonym. T: World War I began in H: World War I started in Arindam (IITB) Textual Entailment November 9, / 59

12 Entailment Triggers Hypernymy/Hyponymy: Certain concept can be either generalized or specialized, leading to entailment. T: Reptiles have scale. H: Snakes have scale. (Specialization or Hyponymy) T: Beckham plays football. H: Beckham plays a game. (Generalization or Hypernymy) Arindam (IITB) Textual Entailment November 9, / 59

13 Entailment Triggers Co-reference: One of the main sources for text entailment. Especially with long text containing paragraphs! T: Barrack Obama came to India. The American President had a meeting with Manmahon Singh. H: Barrack Obama had a meeting with Manmahon Singh. Arindam (IITB) Textual Entailment November 9, / 59

14 Entailment Triggers Modality/Polarity/Factive: Plays critical role in entailment as they affect the degree of reliability on the remaining sentence. Especially troublesome for lexical approaches. Modality denotes possibility or necessity and sometimes may lead to wrong entailment. e.g. may, can, shall, must etc. are modality triggers. T: The government may approve the anti-corruption bill. H: The government approved the anti-corruption bill. Arindam (IITB) Textual Entailment November 9, / 59

15 Entailment Triggers Polarity determines whether the fact asserted or its negation is going to occur. e.g. not, never, deny etc. are polarity triggers. T: The watchman denied that he was sleeping. H: The watchman was sleeping. Factivity deals with presupposition. It states a fact assuming another has occurred. e.g. realize, regret etc. are factivity triggers. T: Martha regrets eating John s homemade cake. H: Martha ate John s homemade cake. Arindam (IITB) Textual Entailment November 9, / 59

16 Entailment Triggers Passivization: In some case one of the text or hypothesis was is in active while the other is in passive. Subject and object of the main verb gets reversed. Can only be handled by assigning semantic roles to each entity. T: Yahoo bought Overture. H: Overture was bought by Yahoo. Arindam (IITB) Textual Entailment November 9, / 59

17 Entailment Triggers Dropping or Inserting Adjunct: Adding or dropping adjuncts affect entailment based on which of T or H is modified, and the polarity. T: Bob was running quickly. H: Bob was running. T: Carl was eating. H: Carl was eating slowly. [Incorrect entailment] T: Alice was not driving. H: Alice was not driving fast. T: Derek was not writing properly. H: Derek was not writing. [Incorrect entailment] Arindam (IITB) Textual Entailment November 9, / 59

18 Entailment Triggers Protocols: Some common conventions such as mentioning birth-death year may trigger entailment. T: Charles de Gaulle, , French general and statesman, was the first president of the Fifth Republic. H: Charles de Gaulle died in Arindam (IITB) Textual Entailment November 9, / 59

19 Entailment Triggers Numerals: In some cases, certain level of numeric calculation affects entailment. T: 3 men and 2 women were found dead in the apartment. H: 5 people were found dead in an apparent. Arindam (IITB) Textual Entailment November 9, / 59

20 Outline 1 Introduction Definition Entailment Triggers Role of Knowledge RTE Challenges Resources 2 General Strategy 3 Lexical Approach 4 Machine Learning Approach 5 Graphical Approach 6 Deep Semantic Approach Text Entailment using UNL Arindam (IITB) Textual Entailment November 9, / 59

21 Role of Knowledge Background knowledge is crucial in entailment as in any AI application! Example T: President of Russia visited Paris. H: President of Russia visited France. B: Paris is situated in France. Background knowledge B alone should not entail the hypothesis H and text T must contain necessary information (may not be sufficient). (T B) = H but B H Arindam (IITB) Textual Entailment November 9, / 59

22 Role of Knowledge Background knowledge is crucial in entailment as in any AI application! Example T: President of Russia visited Paris. H: President of Russia visited France. B: Paris is situated in France. Background knowledge B alone should not entail the hypothesis H and text T must contain necessary information (may not be sufficient). (T B) = H but B H Arindam (IITB) Textual Entailment November 9, / 59

23 Role of Knowledge Background knowledge is crucial in entailment as in any AI application! Example T: President of Russia visited Paris. H: President of Russia visited France. B: Paris is situated in France. Background knowledge B alone should not entail the hypothesis H and text T must contain necessary information (may not be sufficient). (T B) = H but B H Arindam (IITB) Textual Entailment November 9, / 59

24 Outline 1 Introduction Definition Entailment Triggers Role of Knowledge RTE Challenges Resources 2 General Strategy 3 Lexical Approach 4 Machine Learning Approach 5 Graphical Approach 6 Deep Semantic Approach Text Entailment using UNL Arindam (IITB) Textual Entailment November 9, / 59

25 Recognizing Textual Entailment Challenges Goal The recognizing textual entailment is an attempt to promote an abstract generic task that captures major semantic inference needs across applications. Held every year starting RTE - 1,2 and 3 organized by PASCAL 1. Organized by Text Analysis Conference (TAC) since then. Shifted focus to real world applications since RTE-5 (2009) rather than T-H pair entailment recognition. 1 Pattern Analysis, Statistical Modeling and Computational Learning Arindam (IITB) Textual Entailment November 9, / 59

26 RTE Main Task: Given a corpus and a set of candidate sentences retrieved by Lucene from that corpus, RTE systems are required to identify all the sentences from among the candidate sentences that entail a given Hypothesis. each topic contains two sets of documents ( A and B ) Corpus is the set A and H is a sentence taken from B Arindam (IITB) Textual Entailment November 9, / 59

27 Outline 1 Introduction Definition Entailment Triggers Role of Knowledge RTE Challenges Resources 2 General Strategy 3 Lexical Approach 4 Machine Learning Approach 5 Graphical Approach 6 Deep Semantic Approach Text Entailment using UNL Arindam (IITB) Textual Entailment November 9, / 59

28 Resources used for Textual Entailment Resource Type Author Brief Description WordNet Verbnet Roget s Thesaurus Lexical DB Lexical DB Univer- Princeton sity University of Colorado, Boulder DB of nouns, verbs, adjectives and adverbs Lexicon for English verbs organized into classes extending Levin (1993) classes through refinement and addition of subclasses to achieve syntactic and semantic coherence among members of a class. Thesaurus Peter Mark Roget Roget s Thesaurus is a widely-used English thesaurus. The electronic edition (version 1.02) is made available by University of Chicago. Table: Knowledge Resources Arindam (IITB) Textual Entailment November 9, / 59

29 Resources used for Textual Entailment Resource Type Author Brief Description DIRT Paraphrase Collection Collection of paraphrases TEASE Collection Collection of Entailment Rules University of Alberta Bar-Ilan University Table: Knowledge Resources DIRT (Discovery of Inference Rules from Text) knowledge collection of paraphrases from over a 1GB set of newspaper text. Output of the TEASE algorithm. Collection of several entailment templates from web resources. Arindam (IITB) Textual Entailment November 9, / 59

30 Outline 1 Introduction Definition Entailment Triggers Role of Knowledge RTE Challenges Resources 2 General Strategy 3 Lexical Approach 4 Machine Learning Approach 5 Graphical Approach 6 Deep Semantic Approach Text Entailment using UNL Arindam (IITB) Textual Entailment November 9, / 59

31 Sub-tasks Recognizing textual entailment requires various sub-tasks such as: Phrasal Verb Recognition Named Entity Recognition Semantic Role Labeling An example illustrates the need for these tasks Arindam (IITB) Textual Entailment November 9, / 59

32 Example Arindam (IITB) Textual Entailment November 9, / 59

33 General Strategy A general two-step strategy involves: 1 Representation of the information into a form that can be used by the entailment algorithm 2 Entailment Recognition Algorithm that matches the text T along with knowledge B with hypothesis H Arindam (IITB) Textual Entailment November 9, / 59

34 Representation Logical Semantic Syntactic Lexical Raw Text T Re-representation φ(t ) Figure: Various Representations The complexity of representation increases as we go higher. Arindam (IITB) Textual Entailment November 9, / 59

35 Entailment Recognition Text φ(t ) φ(h) Hypothesis e? Y/N φ(b) Knowledge Base Figure: General Strategy e checks if the degree of subsumption of φ(h) with φ(t ) and φ(b) is over a certain threshold e Arindam (IITB) Textual Entailment November 9, / 59

36 Outline 1 Introduction Definition Entailment Triggers Role of Knowledge RTE Challenges Resources 2 General Strategy 3 Lexical Approach 4 Machine Learning Approach 5 Graphical Approach 6 Deep Semantic Approach Text Entailment using UNL Arindam (IITB) Textual Entailment November 9, / 59

37 Lexical Approaches Shallow Approaches: operate on surface level Carries out some basic preprocessing Does not compute elaborate representations Make the entailment decision solely based on the lexical evidences Arindam (IITB) Textual Entailment November 9, / 59

38 Preprocessing Surface preprocessing includes: tokenization stemming/lemmatization identifying the stop words Some systems does a bit deeper preprocessing such as: Phrasal Verb Recognition e.g. take off, put on Idiom processing e.g. A Picture Paints a Thousand Words Named Entity Recognition and Normalization Arindam (IITB) Textual Entailment November 9, / 59

39 Representation Lexical approaches use on of following representation Bag-of-words: Both T and H are represented as a set of words. n-grams: Sequence of n tokens are grouped together. Bag of words is an extreme case of n-gram, with n=1. Example: The fixed routine of a bedtime story before sleeping has a relaxing effect. Bag-of words: The, fixed, routine, of, a, bedtime, story, before, sleeping, has, relaxing, effect Bigram model (n-gram with n=2): The fixed, fixed routine, routine of, of a, a bedtime, bedtime story, story before, before sleeping, sleeping has, has a, a relaxing, relaxing effect Arindam (IITB) Textual Entailment November 9, / 59

40 Example: LLM Algorithm Local Lexical Matching (LLM) is a lexical approach for text entailment that uses bag of words representation INPUT: Text T and Hypothesis H. OUTPUT: The matching score. for all word in T and H do if word in stopwordlist then remove word; end if if no words left in T or H then return 0; end if end for numbermatched = 0; for all word W T in T do Lemma T = Lemmatize(W T ); for all word W H in H do Lemma H = Lemmatize(W H ); if LexicalCompare(Lemma H, Lemma T ) then numbermatched + +; end if end for end for Figure: LLM Algorithm Arindam (IITB) Textual Entailment November 9, / 59

41 LexicalCompare The LexicalCompare() procedure is checks similarity with help of WordNet. if Lemma H == Lemma T then return TRUE; end if if HypernymDistance(W H, W T ) d Hyp then return TRUE; end if if MeronymDistance(W H, W T ) d Mer then return TRUE; end if if MemberOfDistance(W H, W T ) d Mem then return TRUE; end if if SynonymOf(W H, W T ) then return TRUE; end if Figure: Lexical Compare Procedure Arindam (IITB) Textual Entailment November 9, / 59

42 Outline 1 Introduction Definition Entailment Triggers Role of Knowledge RTE Challenges Resources 2 General Strategy 3 Lexical Approach 4 Machine Learning Approach 5 Graphical Approach 6 Deep Semantic Approach Text Entailment using UNL Arindam (IITB) Textual Entailment November 9, / 59

43 Textual Entailment as a Classification Task Figure: Text Entailment as a Classification Task Arindam (IITB) Textual Entailment November 9, / 59

44 Feature Space What could be a possible feature space? Most important decision! Distance Features Features of some distance between T and H. Entailment Triggers Features that triggers entailment (or non-entailment) Syntactic Feature Syntax of T-H pair modeled to exploit rewrite rules. Arindam (IITB) Textual Entailment November 9, / 59

45 Distance Features Possible Features Number of words in common. Longest common subsequnce. Longest common syntactic subtree. Requires representation of T and H as Bag-of words or n-grams Syntactic representation Semantic Representation Arindam (IITB) Textual Entailment November 9, / 59

46 Distance Features Possible Features Number of words in common. Longest common subsequnce. Longest common syntactic subtree. Requires representation of T and H as Bag-of words or n-grams Syntactic representation Semantic Representation Arindam (IITB) Textual Entailment November 9, / 59

47 Distance Features For example: T: At the end of the year, all solid companies pay dividends. H: At the end of the year, all solid insurance companies pay dividends. The above example, possible feature, value pair could be WordsInCommon, 11 or LongestSubsequence, 8. Arindam (IITB) Textual Entailment November 9, / 59

48 Entailment Triggers [?] Capture presence of linguistic features that triggers entailment. Example T: The government may approve the anti-corruption bill. H: The government approved the anti-corruption bill. A feature, value pair could be modal, 1 Arindam (IITB) Textual Entailment November 9, / 59

49 Exploiting Re-write rules How the rewrite rules are exploited is illustrated by following example. Consider the the pair: T: Loki was killed by Thor. H: Loki died. Using the syntactic pair features we can learn rules such as: Figure: Exploiting rewrite rules Arindam (IITB) Textual Entailment November 9, / 59

50 Outline 1 Introduction Definition Entailment Triggers Role of Knowledge RTE Challenges Resources 2 General Strategy 3 Lexical Approach 4 Machine Learning Approach 5 Graphical Approach 6 Deep Semantic Approach Text Entailment using UNL Arindam (IITB) Textual Entailment November 9, / 59

51 Textual Entailment as Graph Matching Convert hypothesis and text into graphs. Either syntactic or semantic. Measure similarity. Similarity score gives entailment. Arindam (IITB) Textual Entailment November 9, / 59

52 Different from Classical Graph Matching! Scoring is not symmetric. Node similarity can not be reduced to label level (i.e token level). Consideration of linguistically motivated graph transformation (nominalization, passivization). Arindam (IITB) Textual Entailment November 9, / 59

53 Text to Graph Generation of dependency graph using a dependency parser Graph edges are labeled by hand made rules (e.g. subj, amod) Applying certain enhancements Arindam (IITB) Textual Entailment November 9, / 59

54 Enhancements to Dependency Graph Collapse Collocations and Named-Entities. Collocations: [blow] [off] [blow off] Named entities: [Micheal] [Jackson] [Micheal Jackson] Dependency Folding so that certain dependencies (such as modifying prepositions become labels). Skeleton -[in]-> cupboard. Arindam (IITB) Textual Entailment November 9, / 59

55 Enhancements to Dependency Graph Semantic Role Labeling. Arcs are labeled with Propbank style semantic roles. This helps to create links between words which share a deep semantic relation not evident in the surface syntax. e.g. Pakistan got independence in [1947] Temporal. Co-reference Links Using a co-reference resolution tagger, coref links are added throughout the graph. In the case of multiple sentence texts, it is our only link in the graph between entities in the two sentences. Arindam (IITB) Textual Entailment November 9, / 59

56 Entailment Model Entailment model determines the matching cost between graphs of T and H. The final cost is a linear combination of cost of matching vertices and edges. Cost = α VertexCost + (1 α) EdgeCost Arindam (IITB) Textual Entailment November 9, / 59

57 Additional Checks Certain additional checks can be applied to the system to improve its performance [?]. They are listed below. Negation Check: Check if there is a negation in a sentence. Example, T:Clinton s book is not a bestseller. H:Clinton s book is a bestseller. Factive Check: Non-factive verbs (claim, think, charged, etc.) in contrast to factive verbs (know, regret, etc.) have sentential complements which do not represent true propositions. T:Clonaid claims to have cloned 13 babies worldwide. H:Clonaid has cloned 13 babies. Arindam (IITB) Textual Entailment November 9, / 59

58 Additional Checks Superlative Check: invert the typical monotonicity of entailment. Example, T: The Osaka World Trade Center is the tallest building in Western Japan. H: The Osaka World Trade Center is the tallest building in Japan. Antonym Check: It is observed that the WordNet::Similarity measures gave high similarity to antonyms. Explicit check of whether a matching involved antonyms is done and unless one of the vertices had a negation modifier, its rejected. Arindam (IITB) Textual Entailment November 9, / 59

59 An Example T: In 1994, Amazon.com was founded by Jeff Bezos. H: Bezos established a company. VC = ( )/3 = 0.13 EC = 0 (isomorphic edges) Cost = (0.55) (0.13) + (0.45) (0) = (let α = 0.55) Arindam (IITB) Textual Entailment November 9, / 59

60 Outline 1 Introduction Definition Entailment Triggers Role of Knowledge RTE Challenges Resources 2 General Strategy 3 Lexical Approach 4 Machine Learning Approach 5 Graphical Approach 6 Deep Semantic Approach Text Entailment using UNL Arindam (IITB) Textual Entailment November 9, / 59

61 What is UNL? tool for representing text in terms of semantic relation between different entities consist of Universal Words (UW), Relations and Attributes Arindam (IITB) Textual Entailment November 9, / 59

62 Example Google goes public. agt obj Google(icl>organization) public(aoj>thing, ant>private) agt(go(icl>do, equ>travel, Google(icl>organization)) obj(go(icl>do, equ>travel, public(aoj>thing, ant>private)) Arindam (IITB) Textual Entailment November 9, / 59

63 Outline 1 Introduction Definition Entailment Triggers Role of Knowledge RTE Challenges Resources 2 General Strategy 3 Lexical Approach 4 Machine Learning Approach 5 Graphical Approach 6 Deep Semantic Approach Text Entailment using UNL Arindam (IITB) Textual Entailment November 9, / 59

64 The Approach [CS : Lecture 29, Prasad Pradip Joshi] Represent both text and hypothesis in their UNL form and do analysis on the UNL expressions List of atomic facts (predicates) emerging from the UNL graph of the hypothesis statement must be a subset (either explicitly or implicitly) of the atomic facts emerging from the UNL graph of the text statement The algorithm has two main parts: Extending the set of atomic truths of the text graph based on those which are present. (referred to as growth-rules) Carrying out the matching of the atomic facts in the hypothesis and the text graph (referred to as matching-rules) Arindam (IITB) Textual Entailment November 9, / 59

65 Illustration [CS : Lecture 29, Prasad Pradip Joshi] Text: Manmohan Singh along with president George Bush signed a letter in Hypothesis: Bush signed a document. Text representation agt(sign@entry@past,manmohan Singh) cag(sign@entry@past,president) nam(president,george Bush) obj(sign@entry@past,letter@indef) tim(sign@entry@past,2006) aoj(president,george Bush) cag(sign@entry@past,george Bush) Hypothesis Representation agt(sign@entry@past,bush) obj(sign@entry@past,document@indef) tim(sign@entry@past,2006) Arindam (IITB) Textual Entailment November 9, / 59

66 Illustration [CS : Lecture 29, Prasad Pradip Joshi] Text: Manmohan Singh along with president George Bush signed a letter in Hypothesis: Bush signed a document. Text representation agt(sign@entry@past,manmohan Singh) cag(sign@entry@past,president) nam(president,george Bush) obj(sign@entry@past,letter@indef) tim(sign@entry@past,2006) aoj(president,george Bush) cag(sign@entry@past,george Bush) Hypothesis Representation agt(sign@entry@past,bush) obj(sign@entry@past,document@indef) tim(sign@entry@past,2006) Arindam (IITB) Textual Entailment November 9, / 59

67 Illustration [CS : Lecture 29, Prasad Pradip Joshi] Text: Manmohan Singh along with president George Bush signed a letter in Hypothesis: Bush signed a document. Text representation agt(sign@entry@past,manmohan Singh) cag(sign@entry@past,president) nam(president,george Bush) obj(sign@entry@past,letter@indef) tim(sign@entry@past,2006) aoj(president,george Bush) cag(sign@entry@past,george Bush) Hypothesis Representation agt(sign@entry@past,bush) obj(sign@entry@past,document@indef) tim(sign@entry@past,2006) Arindam (IITB) Textual Entailment November 9, / 59

68 Illustration [CS : Lecture 29, Prasad Pradip Joshi] Text: Manmohan Singh along with president George Bush signed a letter in Hypothesis: Bush signed a document. Text representation agt(sign@entry@past,manmohan Singh) cag(sign@entry@past,president) nam(president,george Bush) obj(sign@entry@past,letter@indef) tim(sign@entry@past,2006) aoj(president,george Bush) cag(sign@entry@past,george Bush) Hypothesis Representation agt(sign@entry@past,bush) obj(sign@entry@past,document@indef) tim(sign@entry@past,2006) Arindam (IITB) Textual Entailment November 9, / 59

69 Illustration [CS : Lecture 29, Prasad Pradip Joshi] Text: Manmohan Singh along with president George Bush signed a letter in Hypothesis: Bush signed a document. Text representation agt(sign@entry@past,manmohan Singh) cag(sign@entry@past,president) nam(president,george Bush) obj(sign@entry@past,letter@indef) tim(sign@entry@past,2006) aoj(president,george Bush) cag(sign@entry@past,george Bush) Hypothesis Representation agt(sign@entry@past,bush) obj(sign@entry@past,document@indef) tim(sign@entry@past,2006) Arindam (IITB) Textual Entailment November 9, / 59

70 Results On the training set, (200 pairs of gold standard UNL from RTE and FRACAS) the precision value stands at 96.55% and the recall stands at 95.72% Using UNL enconvertor (70.1% accurate), on phenomenon studied FRACAS (100 pairs), precision is 63.04% and recall is 60.1% On complete FRACAS dataset, precision 60.1% and recall 46% Arindam (IITB) Textual Entailment November 9, / 59

71 Growth Rule [CS : Lecture 29, Prasad Pradip Joshi] pos-mod rule: Presence of pos(a,b) add mod(a,b) Navy of India Indian Navy plc closure: Presence of plc(a,b) and plc(b,c) leads to the addition of plc(a,c) Paris is capital of France. France is in Europe. Paris is in Europe. Introduction of words based on UNL relations and attributes: finish or over Relations: plc located pos belongs to or owned by Arindam (IITB) Textual Entailment November 9, / 59

72 Growth Rules [Maheshwari, 2009] Figure: Growth Rules Arindam (IITB) Textual Entailment November 9, / 59

73 Matching Rules [CS : Lecture 29, Prasad Pradip Joshi] Two types Matching the UNL relations (predicate names) Look up whether a relation belongs to the same family as other e.g. agt(agent),cag(co-agent),aoj(attribute of object) Matching the argument part. A narrowing edit of thing pointed to by aoj A broadening edit of thing pointed to by obj Arindam (IITB) Textual Entailment November 9, / 59

74 Universal Words Representation UNL: <UW> := < integer > (<POS><WORDNETID>) Natural Language: <UW> := < root > [< suffix > ] Example Concept: a piece of furniture with tableware for a meal laid out on it UNL Representation: NL Representation : table(icl>furniture) Arindam (IITB) Textual Entailment November 9, / 59

75 Relations labeled arcs connecting a node to another node in a UNL graph correspond to two-place semantic predicates holding between two Universal Words used to describe semantic dependencies between syntactic constituents organized in a hierarchy where lower nodes subsume upper nodes Arindam (IITB) Textual Entailment November 9, / 59

76 Relations Example Bob slept = agt(slept,bob) Alice died = obj(died,alice) John believes in Mary = aoj(believes,john) John worked while Peter talked = coo(worked,talked) Arindam (IITB) Textual Entailment November 9, / 59

77 Attributes Syntax arcs linking a node to itself In opposition to relations, they correspond to one-place predicates used to represent information conveyed by natural language grammatical categories (such as tense, mood, aspect, number, etc) <attribute> <attribute-name> := <character>+ Arindam (IITB) Textual Entailment November 9, / 59

78 Pair Features Example T: At the end of the year, all solid companies pay dividends. H: At the end of the year, all solid insurance companies pay dividends. Possible feature pairs: Bag of words: Text Hypothesis end T year T end H year H solid T solid H We can learn: T implies H as when T contains end T does not imply H when H contains end Totally useless??? Arindam (IITB) Textual Entailment November 9, / 59

79 Pair Features Example T: At the end of the year, all solid companies pay dividends. H: At the end of the year, all solid insurance companies pay dividends. Possible feature pairs: Bag of words: Text Hypothesis end T year T end H year H solid T solid H We can learn: T implies H as when T contains end T does not imply H when H contains end Totally useless??? Arindam (IITB) Textual Entailment November 9, / 59

80 Pair Features Example T: At the end of the year, all solid companies pay dividends. H: At the end of the year, all solid insurance companies pay dividends. Possible feature pairs: Bag of words: Text Hypothesis end T year T end H year H solid T solid H We can learn: T implies H as when T contains end T does not imply H when H contains end Totally useless??? Arindam (IITB) Textual Entailment November 9, / 59

81 Effectively using Pair Feature Space [?] Example T : At the end of the year, all solid companies pay dividends. H 1 : At the end of the year, all solid insurance companies pay dividends. H 2 : At the end of the year, all solid companies pay cash dividends. Distance feature will plot < T, H 1 > and < T, H 2 > to be same points. We need a space that considers the content and the structure of textual entailment examples. Arindam (IITB) Textual Entailment November 9, / 59

82 Effectively using Pair Feature Space [?] Example T : At the end of the year, all solid companies pay dividends. H 1 : At the end of the year, all solid insurance companies pay dividends. H 2 : At the end of the year, all solid companies pay cash dividends. Distance feature will plot < T, H 1 > and < T, H 2 > to be same points. We need a space that considers the content and the structure of textual entailment examples. Arindam (IITB) Textual Entailment November 9, / 59

83 Effectively using Pair Feature Space [?] Example T : At the end of the year, all solid companies pay dividends. H 1 : At the end of the year, all solid insurance companies pay dividends. H 2 : At the end of the year, all solid companies pay cash dividends. Distance feature will plot < T, H 1 > and < T, H 2 > to be same points. We need a space that considers the content and the structure of textual entailment examples. Arindam (IITB) Textual Entailment November 9, / 59

84 Effectively using Pair Feature Space [?] Example T : At the end of the year, all solid companies pay dividends. H 1 : At the end of the year, all solid insurance companies pay dividends. H 2 : At the end of the year, all solid companies pay cash dividends. Distance feature will plot < T, H 1 > and < T, H 2 > to be same points. We need a space that considers the content and the structure of textual entailment examples. Arindam (IITB) Textual Entailment November 9, / 59

85 Effectively using Pair Feature Space [?] Example T : At the end of the year, all solid companies pay dividends. H 1 : At the end of the year, all solid insurance companies pay dividends. H 2 : At the end of the year, all solid companies pay cash dividends. Distance feature will plot < T, H 1 > and < T, H 2 > to be same points. We need a space that considers the content and the structure of textual entailment examples. Arindam (IITB) Textual Entailment November 9, / 59

86 Kernel Trick Syntactic pair feature space. Cross Pair Similarity K(< T, H >, < T, H >) = K(< T, T >) + K(< H, H >) defining the distance K(P 1, P 2 ) instead of features as max (K T (t(h, c), t(h, i)) + K T (t(t, c), t(t, i))) c C Makes Pair Feature look useful. Arindam (IITB) Textual Entailment November 9, / 59