In 2022, ELIZA, an early natural language processing (NLP) system developed in 1966, won a Peabody Award for demonstrating that software could be used to create empathy. Over 50 years later, human language technologies have evolved significantly beyond the basic pattern-matching and substitution methodologies that powered ELIZA. As we enter the new age of ChatGP, generative AI, and large language models (LLMs), here’s a quick primer on the key components — NLP, NLU (natural language understanding), and NLG (natural language generation), of NLP systems.
What is NLP?
NLP is an interdisciplinary field that combines multiple techniques from linguistics, computer science, AI, and statistics to enable machines to understand, interpret, and generate human language.
The earliest language models were rule-based systems that were extremely limited in scalability and adaptability. The field soon shifted towards data-driven statistical models that used probability estimates to predict the sequences of words. Though this approach was more powerful than its predecessor, it still had limitations in terms of scaling across large sequences and capturing long-range dependencies. The advent of recurrent neural networks (RNNs) helped address several of these limitations but it would take the emergence of transformer models in 2017 to bring NLP into the age of LLMs. The transformer model introduced a new architecture based on attention mechanisms. Unlike sequential models like RNNs, transformers are capable of processing all words in an input sentence in parallel. More importantly, the concept of attention allows them to model long-term dependencies even over long sequences. Transformer-based LLMs trained on huge volumes of data can autonomously predict the next contextually relevant token in a sentence with an exceptionally high degree of accuracy.
In recent years, domain-specific biomedical language models have helped augment and expand the capabilities and scope of ontology-driven bioNLP applications in biomedical research. These domain-specific models have evolved from non-contextual models, such as BioWordVec, BioSentVec, etc., to masked language models, such as BioBERT, BioELECTRA, etc., and to generative language models, such as BioGPT and BioMedLM.
Knowledge-Enhanced biomedical language models have proven to be more effective at knowledge-intensive BioNLP tasks than generic LLMs. In 2020, researchers created the Biomedical Language Understanding and Reasoning Benchmark (BLURB), a comprehensive benchmark and leaderboard to accelerate the development of biomedical NLP.
NLP = NLU + NLG + NLQ
NLP is a field of artificial intelligence (AI) that focuses on the interaction between human language and machines. It employs a constantly expanding range of techniques, such as tokenization, lemmatization, syntactic parsing, semantic analysis, and machine translation, to extract meaning from unstructured natural languages and to facilitate more natural, bidirectional communication between humans and machines.
SOURCE: TechTarget
Modern NLP systems are powered by three distinct natural language technologies (NLT), NLP, NLU, and NLG. It takes a combination of all these technologies to convert unstructured data into actionable information that can drive insights, decisions, and actions. According to Gartner ’s Hype Cycle for NLTs, there has been increasing adoption of a fourth category called natural language query (NLQ). So, here’s a quick dive into NLU, NLG, and NLQ.
NLU
While NLP converts unstructured language into structured machine-readable data, NLU helps bridge the gap between human language and machine comprehension by enabling machines to understand the meaning, context, sentiment, and intent behind the human language. NLU systems process human language across three broad linguistic levels: a syntactical level to understand language based on grammar and syntax, a semantic level to extract meaning, and a pragmatic level to decipher context and intent.
These systems leverage several advanced techniques, including semantic analysis, named entity recognition, relation extraction and coreference resolution, to assign structure, rules, and logic to language to enable machines to get a human-level comprehension of natural languages. The challenge is to evolve from pipeline models, where each task is performed separately, to blended models that can combine critical bioNLP tasks, such as biomedical named entity recognition (BioNER) and biomedical relation extraction (BioRE), into one unified framework.
NLG
Where NLU focuses on transforming complex human languages into machine-understandable information, NLG, another subset of NLP, involves interpreting complex machine-readable data in natural human-like language. This typically involves a six-stage process flow that includes content analysis, data interpretation, information structuring, sentence aggregation, grammatical structuring, and language presentation. NLG systems generate understandable and relevant narratives from large volumes of structured and unstructured machine data and present them as natural language outputs, thereby simplifying and accelerating the transfer of knowledge between machines and humans.
To explain the NLP-NLU-NLG synergies in extremely simple terms, NLP converts language into structured data, NLU provides the syntactic, semantic, grammatical, and contextual comprehension of that data and NLG generates natural language responses based on data.
NLQ
The increasing sophistication of modern language technologies has renewed research interest in natural language interfaces like NLQ that allow even non-technical users to search, interact, and extract insights from data using everyday language. Most NLQ systems feature both NLU and NLG modules. The NLU module extracts and classifies the utterances, keywords, and phrases in the input query, in order to understand the intent behind the database search. NLG becomes part of the solution when the results pertaining to the query are generated as written or spoken natural language.
NLQ tools are broadly categorized as either search-based or guided NLQ. The search-based approach uses a free text search bar for typing queries which are then matched to information in different databases. A key limitation of this approach is that it requires users to have enough information about the data to frame the right questions. The guided approach to NLQ addresses this limitation by adding capabilities that proactively guide users to structure their data questions using modeled questions, autocomplete suggestions, and other relevant filters and options.
Augmenting life sciences research with NLP
At BioStrand, our mission is to enable an authentic systems biology approach to life sciences research, and natural language technologies play a central role in achieving that mission. Our LENSai Complex Intelligence Technology platform leverages the power of our HYFT® framework to organize the entire biosphere as a multidimensional network of 660 million data objects. Our proprietary bioNLP framework then integrates unstructured data from text-based information sources to enrich the structured sequence data and metadata in the biosphere. The platform also leverages the latest development in LLMs to bridge the gap between syntax (sequences) and semantics (functions). For instance, the use of retrieval-augmented generation (RAG) models enables the platform to scale beyond the typical limitations of LLM, such as knowledge cutoff and hallucinations, and provide the up-to-date contextual reference required for biomedical NLP applications.
With the LENSai, researchers can now choose to launch their research by searching for a specific biological sequence. Or they may search in the scientific literature with a general exploratory hypothesis related to a particular biological domain, phenomenon, or function. In either case, our unique technological framework returns all connected sequence-structure-text information that is ready for further in-depth exploration and AI analysis. By combining the power of HYFT®, NLP, and LLMs, we have created a unique platform that facilitates the integrated analysis of all life sciences data. Thanks to our unique retrieval-augmented multimodal approach, now we can overcome the limitations of LLMs such as hallucinations and limited knowledge.
Stay tuned for hearing more in our next blog.
Subscribe to our blog:
