Understanding Heuristics-based NLP in simpler terms

Heuristics-based NLP

In this article we'd start to understand how the implementation of NLP has been solved gradually. The three ways to solve NLP are heuristic, machine learning and deep learning-based NLP. We will start by understanding how we can solve NLP with a heuristic-based approach. Before we head to the NLP aspect of problem solving, let us try to understand an effective approach to problem solving.

Understanding Problem-Solving in NLP:

Then we'll be able to chart out a path to solve database problems more effectively. There are fundamentally three steps to consider while solving a database problem. The first is understanding existing rules. While solving problems, it is essential that we understand the fundamentals. Most tasks that we are trying to automate or do better with the help of data are already being done in a certain way.

Understanding the existing rules and validating it with experts gives us a second good foundation to build on top. This leads to good fundamental understanding that is validated by a subject matter expert. Second is match existing system. Now the problem being solved for is already being done in a manner that is considered acceptable. If you can build a system to match or come close to it, how it is already being done, it's a win situation.

Once we are successful in matching the existing system we can then proceed to the next step. And the last is improve the system. This step involves working towards bettering the problem solving to the point where it becomes world class. This would involve steps such as customizing solution to solve for multiple use cases, industries, and scenarios. It would also involve working towards covering all possible edge cases to solve the problem completely.

A good example here can be the way Tesla is approaching self-driving. In the initial years, they spent time gathering data and understanding user behavior. Then they rolled out advanced driving assist to match the existing system. The next step would be to work towards bettering the self-driving model to drive safer than the human being.

Now that we have understood how to solve a problem, let's dive deeper into how it can be done for natural language processing. Like the approach discussed in the previous article, early attempts at designing NLP systems were based on a rule system that were developed by people with domain expertise. For example, the words that were positive or negative were marked from the dictionary, and this was used to decide if a certain text had a positive or a negative sentiment. So what is a heuristic-based approach? A heuristic-based approach to a problem is defined as a practical method to problem solving. This is not guaranteed to be optimal, perfect or rational. But it is sufficient to get the job done.

Heuristic Techniques in NLP:

Keeping this in mind, let's understand some of the heuristic-based approaches to NLP, starting with knowledge base. Beside the use of dictionaries and thesaurus, there are more elaborate ways of storing knowledge to aid NLP. There is more to language than the word itself, such as synonyms, hyponyms, meronyms and etc.

We will go deeper into understanding this in the coming videos. An example of knowledge base is WordNet which is a database of words and relationship between them. There are also larger common sense world knowledge databases such as Open Mind Common Sense. Next, let's look into Regex. Regex or regular expressions are great tool for text analysis and building a heuristic rule-based system. Regular expression or a Regex can be used to match and find strings, substrings or patterns in a text.

For instance, in this example, this indicates that the text should contain any character other than the ones which are in the bracket, such as i, *, &, 2, or @ sign. This can be supremely useful and can be customized for multi-domain use case such as finding topics, extracting email IDs, and other such entities. Even now, rules in heuristic play a significant role across NLP lifecycle.

Summary:

Fact is heuristic-based NLP models are great way to build an effective first version. The additional benefit is that they also help you get a better understanding of the problem. In the next following video, we'll understand a little bit more about machine learning-based approaches to NLP.

Comments

Post a Comment

Popular posts from this blog

What are some ethical implications of Large Language models?

Image
Ethical consideration in LLMs In this article, we're going to address the ethical challenges surrounding the usage of large language models, including potential biases and privacy issues.  Bias in AI system: AI systems, particularly, will deliver biased results. There's no other way to get around it. Search engine technology, for example. It's not neutral, processes a large amount of data, and prioritizes results based on clicks and user preferences. So, a search engine can become essentially an echo chamber. And you can see that it will uphold certain biases of the real world, potentially because looking at the clicks and the user preferences of a majority of users using that search engine. And this is how we can come up with prejudices and stereotypes that are furthered online because of these systems that are in place. So, when we talk about bias in AI, it could be something like as simple as a gender bias, right? So, gender biases should be avoided, but it can't sho...
Read more

Introduction to the fine tuning in Large Language Models

Image
Fine tuning in LLM In this article, we're going to delve into the techniques used to adapt a generic pre-trained model for a specific task or application. Now, pre-trained language models can do really impressive stuff just immediately off the shelf, including things like text generation, summarization, and even providing levels of code. But LLMs are not one-size-fits-all as you can imagine and maybe I want to specialize it even further after the model has been trained. I'd like to make it really good at answering questions based on certain topics such as medicine or travel or code or something. This is what the idea of fine-tuning comes in.  Methods used in Fine tuning: Fine-tuning a large language model refers to the process of taking a pre-trained language model or a model that's already been trained on a large dataset and further training it on a smaller task-specific dataset. Now this process will allow the model to adapt and specialize its capability. And if we take t...
Read more

Understanding the basic architecture of LLMs

Image
Architecture of LLMs In this article, we're going to discover the key components in general architecture that make up typical large language models.  Introduction to LLM Architecture: Now, large language models will have things like the transformer neural networks at their very underlying pieces of it. Remember, these transformers are just a large number of parameters are attached to this transformer model and these LLMs are able to then understand and generate accurate responses because it's using this type of neural network at its core. In fact, what you're going to see is that these LLMs are referred to as foundation models and it's all it means is a foundation model is so large and impactful that it serves as its own class of artificial intelligence so that we can start experimenting with these types. Transformer Model Structure: It's just a way for us to classify these models that are being used today. Now, what exactly is a transformer model? We'll get to ...
Read more