ChatBots are outgrowths of Interactive Voice Response (IVR) systems, familiar dialog platforms that have powered phone interactions with customer support for decades. ChatBots can handle workflows where the dialogue can be standardized, but not open-ended interactions.
Virtual assistant-style conversational agents (e.g. perhaps Siri or Alexa) attempt to resolve any user question whatsoever. This is known as the ‘Open World Approach’. However, consumers’ expectations are frequently low; if a virtual assistant misinterprets a command, the consumer can rephrase it until the virtual assistant gets it right. Or, they can give up and simply place a pizza order for themselves! This is one reason why consumer-facing virtual assistants have not been widely deployed for corporate training or customer support use cases – they are primarily intended for household rather than business use.
Recent developments in Natural Language Processing (NLP) are taking both ChatBots and Virtual Assistants beyond keywords. The ‘meaning’ of a word is reduced to its proximity to other words in a vector space.
Word embeddings represent a major advancement on keyword-driven systems by looking at the neighbors of words to assess meaning and context. Words surrounding a given word or phrase can be read by the nodes of the neural network in either direction, as opposed to traditional recurrent neural network (RNN)-based approaches that only read sentences from left to right. Transformer neural networks can thus predict a missing word in a sentence by evaluating many examples of the words that surround it in other sentences.
However, not all context exists within language itself – instead, language exists in the real world, used by human beings for specific purposes. This means that statistical models of context can never reconstruct the physical world. Thanks to deep learning, methods now exist for automating the labor of logically representing the meaning of language.
At Soffos, we started by deploying transformer models to handle the problem of finding an answer to a user’s question in the closed world of an enterprise knowledge silo, and found that transformer architectures are capable of providing sophisticated answers to users’ open-ended questions in cases where the system must produce an inference, not just pattern-matching terms in questions against sentences in documents. By investing in R&D on the challenging problem of logically parsing language, we can ensure that our system architecture will be deeply proprietary and will solve challenges that are currently viewed as almost impossible.
However, the outputs of transformer neural networks are not the end of the line in Soffos’ processing of meaning. Instead, his conversational capabilities are accomplished via an end-to-end semantic refinement pipeline that includes the following stages:
Contextually and semantically aware lexical modeling (e.g. word embeddings and Wordnet);
Syntactical modeling (custom dependency and constituency parsing frameworks, “deep semantic” approaches to syntax parsing like case grammar, and custom syntax parsing into first-order logical predicates);
Logical modeling (prepositional connectors between predicates, quantification, modality, recognition of named constants, model-checking and theorem proving, support for traditional logical inference);
Ontological modeling (cause/effect, temporal relations, ‘common sense’ facts such as humans have only one left hand and one right hand, property-based and relational models of important entities, etc.)
In these ways, Soffos understands document meaning not at the keyword level, but at the level of the logically atomic unit of linguistic meaning: the clause. Thus word-associations are parsed not simply as proximity in a vector space, but as n-ary relations obtained in a first-order logical framework.
Relations are combined into a ‘knowledge graph’. This architecture represents a modified ‘closed-world’ approach to the question-answering problem. Many enterprise knowledge graphs are manually developed at enormous expense, but Soffos automates the process through sophisticated lexical, syntactic, and neural processing frameworks for recognizing named entities and their relations in a document. Thus we are investing in developing our own linguistic and logical annotation methods for scoring Soffos’ answering and reasoning abilities.
In short, the technology we are developing is pushing the boundaries of current thinking to an entirely new level, far beyond that attempted by computational linguists designing ‘closed world’ decision tree chatbots or ‘open world’ general assistants. This is what sets our technology aside, and our initial patents have been filed with Soffos’ autonomous nature in mind.