Intelligence comes from the Latin word intelligere, which means “to understand.” The ability to comprehend is central to intelligence and a prerequisite to reasoning and using acquired knowledge to affect your environment which is what we, humans, have been doing with a lot of success for over 100,000 years. Given the complexity of processing human language, it is not surprising that in the past several decades of research in Natural Language Processing (NLP), we have happily assumed that linguistic input can be understood by anyone speaking the language. In other words, NLP is a field of AI that strives to make machines capable of understanding language like a human, tacitly assuming that the “human” is a constant: any speaker of English can understand any input spoken or written in English. This tacit assumption has served us well both from a theoretical and a practical perspective.

The advent of the internet has slowly but dramatically changed the landscape. Twenty years ago there were hardly any school-aged children consuming information from the internet. There was hardly any need for out of the workforce adults to try to reskill or upskill in order to be able to have a job. To give an example, most of the Wikipedia articles are beyond the level of understanding of elementary school students and most of the scholarly articles are beyond the level of understanding of even college graduates. Most of Coursera or other MOOCs initially designed to provide free education opportunities of high quality to underserved people around the world ended up being attended by people who already had Master’s and PhD degrees. Using NLP to enable access to web content to an unprecedented range of learners opens new and exciting lines of AI research with the potential of enormous societal impact.

Current Research

At the core of my research interests is identifying robust representations of content in unstructured data that can be combined with personalized representations of content consumed by learners.

In the course of the past seven decades, several models of document/content representations have been proposed ranging from early set theoretic models that represent documents as sets of words, to algebraic models that represent documents as vectors and matrices and probabilistic models, e.g., language models, that treat language understanding as a probabilistic problem. It is interesting to evaluate how these models perform when we introduce learner models as a factor. In other words, how can we model and extract meaning from text to allow and evaluate if this extracted meaning can, indeed, be processed and understood by individual learners?

It is well-known, yet poorly modeled, that a learner’s ability to extract information from texts depends not only on their ability to decode written language but, also, their level of interest and familiarity with the content of texts. The more familiar a learner is with a topic, the more topic related content words and concepts the student will know, and the easier it is for the reader to make sense of texts. To address the problem of analyzing linguistic meaning that we extract from unstructured data with respect to the learner’s ability to comprehend it, we need to be able to a) analyze the quality and complexity of unstructured content in real time web search and b) create a model of the learner that represents his or her current and prior knowledge in order to predict his or her ability to comprehend a previously unseen document, if any, on the topic of interest.

This line of research involves addressing the following questions.

1. Text analysis and readability
   1. How do we collect, represent, and level the semantic/thematic content of unstructured data?
   2. How do we detect new topics as they emerge?
2. Learner model
   1. How do we capture automatically the reading level of a learner from sparse and unstructured data?
   2. How do we represent and update a learner’s knowledge model?
   3. How do we learn a learner’s model from incomplete and unstructured data?
3. Text Simplification
   1. How can we simplify content for different reading abilities?
   2. How can we make technical content accessible to the wide variety of learner backgrounds?
   3. How do we determine what concepts are prerequisites to understanding a new concept?
4. Information retrieval
   1. How do we evaluate the quality and reliability of web content?
   2. How do we determine if prior information consumed by the learner provides the required background to understand novel content?

Text Analysis and Readability

The problem of analyzing the reading level of a document with the intent of evaluating its suitability for the reading ability of the intended reader dates back to ‘20s. In the 50s and 60s, several readability formulas were proposed, which to this day they are the go-to formulas for practical applications. In the 80s, research efforts focused was on creating models that reflect insights from cognition, and in the beginning of 2000s NLP and Machine Learning approaches gained ground, including models with semantic and discourse features. Despite the extensive efforts, the state of the art never reached accuracy beyond 70% for specific domains.

Just like earlier work on readability, recent work using NLP and Machine Learning to evaluate text difficulty is focused on evaluating linguistic content regardless of the learner. I posit that in order to have a breakthrough in this area, it is critical to develop a model of the learner. In my research approach, we address the above problem by a) building global readability and theme models as a first step and b) personalizing the readability prediction based on a dynamic model of learner’s world knowledge. We have shown that this approach is both scientifically innovative and it addresses the practical implementation challenges of building a robust smart search engine for K-12 teachers and students that returns results based on a user query and filters them by reading level and theme in real-time search (choosito.com).

Text Simplification

It is not the first time in the history of scientific discovery that looking into solutions for practical problems can lead us to novel ways of thinking about an old problem. The whopping number of people currently missing from the workforce combined with a declining rate of workforce growth is making efficient workforce training an acute necessity. A major challenge in current workforce development and upskilling is that employees or potential employees come from very diverse backgrounds. Most training courses, tutorials, and other materials available for training assume a knowledge background that most of the employees do not have. Despite initial hopes that MOOCs would enable quality learning for people who need training to get into the workforce, a significant number of people attending MOOCs have already Master’s and Doctorate Degrees. Trying to address this problem with traditional text simplification approaches has revealed some fundamental shortcomings. Splitting sentences into smaller chunks or replacing difficult words with easier synonyms is not sufficient in this context. What synonym can you offer for “gradient descent” that will help the learner know what it is?

This basic insight has led us to recasting text simplification as an information retrieval problem. In collaboration with Chris Callison Burch, we proposed a hybrid solution: a) improve traditional text simplification techniques such as word substitution and sentence rewriting, b) introduce a novel information retrieval approach based on identifying concepts and prerequisite concepts critical for the comprehensibility of advanced documents, and c) retrieve semantically resources that expose these critical concepts at the appropriate reading level. At the core of this research is the identification of terms and concepts that are predicted to be challenging for a specific learner, critically taking into account his or her background knowledge and experience with the target content or skill (or lack thereof).

This new line of research is aspiring to make significant contributions to contribute to our understanding of the strengths and limitations of highly influential NLP frameworks such as BERT, TransformerXL, OpenAI’s GPT-3, XLNet, ERNIE2.0, RoBERTa etc., on novel classification tasks that require capturing much deeper understanding of lexical semantics and discourse structure.

Antelogue

is a pronoun resolution system that uses natural language techniques to process dialogues and identify co-referring relations between pronouns and their antecedents in the dialogue. Antelogue is a computationally efficient system which uses linguistically rich resources to achieve high precision resolution. The current version of Antelogue is specifically designed to process dialogues from the popular TV series 'Lost'. It achieves 93% accuracy for first, second and third person pronouns. Plural pronouns are not handled yet.

The PENN Discourse Treebank

To-date the largest discourse annotated corpus. PDTB 1.0 was released in 2006 and contained annotations of discourse connectives (explicit and implicit) and their arguments. PDTB 2.0, released in January 2008, is enriched with annotations of speaker attribution and the senses of connectives.

Automated Evaluation of Coherence

In student essays implements a centering-based algorithm to identify topic discontinuities in students essays. Statistical analysis of the performance of the algorithm on a corpus of student essays shows that the topic discontinuity model can improve the performance of e-rater, the automated essay scoring system developed at ETS. Related publications link coming.