Knowledge Graph Random Wall mple positive contexts on Sim_edge by sliding window Knowledge Difficulty KEWE Base Context Sim_edge Sampling negative contexts on Dissim_edge Dissim_edge...... Sentences Sliding window Hybrid two types Text on sentences Text of contexts Corpus Context KEWE Figure 1:Illustration of the knowledge-enriched word embedding models.KEWE is based on the difficulty context,while KEWEh is based on both the difficulty and text contexts. the other is the hybrid word embedding model(KEWE)which compensates the domain knowledge with text corpus 3.1 The Knowledge-only Word Embedding Model (KEWE) In the classic word embedding models,such as C&W,CBOW,and Skip-Gram,the context of a word is represented by its surrounding words in the text corpus.Levy and Goldberg (2014a)have incorporated the syntactic context from the dependency parse-trees and found that the trained word embedding could capture more functional and less topical similarity.For readability assessment,reading difficulty other than function or topic becomes more important.Hence,we introduce a kind of difficulty context,and try to learn a difficulty-focusing word embedding,which leads to KEWEk.In the following,we describe this model in three steps:domain knowledge extraction,knowledge graph construction,and graph-based word embedding learning.The former two steps focus on modeling the relationship among words on reading difficulty,and the final step on deriving the difficulty context and learning the word embedding 3.1.1 Domain Knowledge Extraction To model the relationship among words on reading difficulty,we first introduce how to extract the knowl- edge on word-level difficulty from different perspectives.Specifically,we consider three types of word- level difficulty:acquisition difficulty,usage difficulty,and structure difficulty. Acquisition difficulty.Word acquisition refers to the temporal stage at which children learn the meaning of new words.Researchers have shown that the information on word acquisition is useful for readability assessment (Kidwell et al.,2009;Schumacher et al.,2016).Generally,the words acquired at primary school are easier than those acquired at high school.We call the reading difficulty reflected by word acquisition as the acquisition difficulty.Formally,given a word w,its acquisition difficulty is described by a distribution K over the age-of-acquisition (AoA)(Kidwell et al.,2009). Since the rating on AoA is an unsolved problem in cognitive science(Brysbaert and Biemiller,2016) and not available for many languages,we explore extra materials to describe the acquisition difficulty.In particular,we collect three kinds of knowledge teaching materials,i.e.,in-class teaching material,extra- curricular teaching material,and proficiency test material.These materials are arranged as lists of words, each of which contains words learned in the same time period and hence corresponds to a certain level of acquisition difficulty.For example,given a lists of words,we can defineKR,whereK1 if a word w belongs to the list i,andK=0otherwise. Usage difficulty.Researchers used to count the usage frequency to measure the difficulty of words (Dale and Chall,1948),which can separate the words which are frequently used from those rarely used. We call the difficulty reflected by usage preference as the usage difficulty.Formally,given a word w,its usage difficulty is described by a distribution K over the usage preferences. We provide two ways to measure the usage difficulty.One way is estimating the level of words' 368368 KEWEk Knowledge Base Sentences gggggg Text Corpus KEWEh Knowledge Graph Dissim_edge Sim_edge Difficulty Context Sampling negative contexts on Dissim_edge Sample positive contexts by sliding window Paths ggg Random Walk on Sim_edge Text Context Sliding window on sentences Hybrid two types of contexts Figure 1: Illustration of the knowledge-enriched word embedding models. KEWEk is based on the difficulty context, while KEWEh is based on both the difficulty and text contexts. the other is the hybrid word embedding model (KEWEh) which compensates the domain knowledge with text corpus. 3.1 The Knowledge-only Word Embedding Model (KEWEk) In the classic word embedding models, such as C&W, CBOW, and Skip-Gram, the context of a word is represented by its surrounding words in the text corpus. Levy and Goldberg (2014a) have incorporated the syntactic context from the dependency parse-trees and found that the trained word embedding could capture more functional and less topical similarity. For readability assessment, reading difficulty other than function or topic becomes more important. Hence, we introduce a kind of difficulty context, and try to learn a difficulty-focusing word embedding, which leads to KEWEk. In the following, we describe this model in three steps: domain knowledge extraction, knowledge graph construction, and graph-based word embedding learning. The former two steps focus on modeling the relationship among words on reading difficulty, and the final step on deriving the difficulty context and learning the word embedding. 3.1.1 Domain Knowledge Extraction To model the relationship among words on reading difficulty, we first introduce how to extract the knowl￾edge on word-level difficulty from different perspectives. Specifically, we consider three types of word￾level difficulty: acquisition difficulty, usage difficulty, and structure difficulty. Acquisition difficulty. Word acquisition refers to the temporal stage at which children learn the meaning of new words. Researchers have shown that the information on word acquisition is useful for readability assessment (Kidwell et al., 2009; Schumacher et al., 2016). Generally, the words acquired at primary school are easier than those acquired at high school. We call the reading difficulty reflected by word acquisition as the acquisition difficulty. Formally, given a word w, its acquisition difficulty is described by a distribution KA w over the age-of-acquisition (AoA) (Kidwell et al., 2009). Since the rating on AoA is an unsolved problem in cognitive science (Brysbaert and Biemiller, 2016) and not available for many languages, we explore extra materials to describe the acquisition difficulty. In particular, we collect three kinds of knowledge teaching materials, i.e., in-class teaching material, extra￾curricular teaching material, and proficiency test material. These materials are arranged as lists of words, each of which contains words learned in the same time period and hence corresponds to a certain level of acquisition difficulty. For example, given a lists of words, we can define KA w ∈ R a , where KA w,i = 1 if a word w belongs to the list i, and KA w,i = 0 otherwise. Usage difficulty. Researchers used to count the usage frequency to measure the difficulty of words (Dale and Chall, 1948), which can separate the words which are frequently used from those rarely used. We call the difficulty reflected by usage preference as the usage difficulty. Formally, given a word w, its usage difficulty is described by a distribution KU w over the usage preferences. We provide two ways to measure the usage difficulty. One way is estimating the level of words’