What is reinforcement learning in AI?

reinforcement learning

The another kind of machine learning model that we'll discuss today is reinforcement learning. Unlike what we've been discussing in our last few articles, reinforcement learning is not a deep learning technique, it's a different subfield of machine learning. Reinforcement learning is a machine learning paradigm where an agent learns to make decisions by interacting with the environment, with the objective of maximizing cumulative rewards through the process of trial and error.

How Reinforcement learning actually works?

You can kind of imagine that this is how life works, you don't know what actions are going to make you successful, and what actions are going to cause you to fail. You just learn by interacting with the environment, by performing the actions, seeing what the results are, and then tweaking your actions accordingly. This is the basis of reinforcement learning. There are two main actors or components in a reinforcement learning model, the first is an agent, the agent is the learner or the decision maker that interacts with the environment and makes decisions. The agent seeks to maximize a cumulative reward by choosing the right actions.

What actions might an agent perform? Well, that depends on where exactly you are using reinforcement learning. For example, let's say the agent is some kind of robot, and the objective of the robot is to climb up a hill, the actions might be to step left, step right, step forward and step backward. The environment is where the agent operates, everything the agent interacts with, and everything that the agent learns from, the environment gives feedback to the agent, it's the external system or world.

The environment of the respective ML model provides feedback to the agent/model in the form of rewards or punishment and state transitions. A reward is when the agent has performed the right step that takes the agent closer to its goal. A punishment could be something bad that happens when the agent makes a wrong decision. The agent learns from the feedback that the environment gives it and tries to modify its behavior accordingly. Here are the two main components of a reinforcement learning algorithm the agent and environment and you already know what they do. At any point in time, the environment presents to the agent the state of the world currently that can be thought of as the state St.


The state can be thought of as a representation of the current situation or configuration of the environment, which contains all of the relevant information needed to make decisions, these states can be continuous or discrete. The agent's job is to use all of the information that it has gathered so far from previous interactions with the environment, then look at the current context and then take some action A. The action A is a set of possible choices or decisions the agent can make at each time step. So at time step t, let's say the agent goes ahead and decides to perform action At.

If the agent is a robot climbing a hill, let's say the agent decides to step sideways. Every action that the agent chooses will have a consequence, and every action affects and changes the state of the environment. So, the state of the environment at the next time step will be St+1. Every action that an agent takes results in a reward from the environment. The environment rewards the agent for each action, the size of the reward can be large or small, and it can even be negative, so it may not be a reward, it may be a punishment for taking this action.


For example, your hill climbing robot, if the robot gets further up the hill, that's a reward. If the robot falls down a pit, that's a punishment. The agent's objective is to maximize the rewards collected, and this reward indicates how desirable the action is for the agent. If the reward for a particular action is high, the agent will tend to repeat that action over and over again.

The objective of the agent is to maximize the cumulative rewards that it collects, not the reward at a particular step, but the total rewards as it moves towards its goal. Reinforcement learning thus operates on a feedback loop- every action made by an agent results in a reward, and this process repeats over and over again. The agent will of course, try to perform those actions that maximizes its cumulative rewards.

How exactly an agent makes decisions is governed by a policy- the policy determines the decision making process of the agent, this policy is not predefined, this is what evolves over time. As the agent moves around in the environment, the agent tries to learn the best policy to maximize the expected cumulative rewards earned. Just like in real life, the agent has to balance an important trade off.

Exploration vs exploitation:

Exploration vs. Exploitation. Exploration refers to the strategy of selecting actions that the agent has not yet tried or has limited experience with. The goal of exploration is to gather more information about the environment and discover potentially better actions that improve the agent's understandings of how actions affect rewards. At the same time, it is possible that exploration leads to worse results.

The agent also needs to use exploitation, where the agent selects actions that it knows will yield the highest expected cumulative reward based on its current knowledge or estimates. These actions are chosen because they are known to be the best options according to the agent's current policy. It's important for the agent to strike the right balance between exploration and exploitation. This is the critical challenge in reinforcement learning, the agent needs to explore enough to gather valuable information while exploiting its current knowledge to maximize rewards.

Comments

Post a Comment

Popular posts from this blog

What are some ethical implications of Large Language models?

Image
Ethical consideration in LLMs In this article, we're going to address the ethical challenges surrounding the usage of large language models, including potential biases and privacy issues.  Bias in AI system: AI systems, particularly, will deliver biased results. There's no other way to get around it. Search engine technology, for example. It's not neutral, processes a large amount of data, and prioritizes results based on clicks and user preferences. So, a search engine can become essentially an echo chamber. And you can see that it will uphold certain biases of the real world, potentially because looking at the clicks and the user preferences of a majority of users using that search engine. And this is how we can come up with prejudices and stereotypes that are furthered online because of these systems that are in place. So, when we talk about bias in AI, it could be something like as simple as a gender bias, right? So, gender biases should be avoided, but it can't sho...
Read more

Introduction to the fine tuning in Large Language Models

Image
Fine tuning in LLM In this article, we're going to delve into the techniques used to adapt a generic pre-trained model for a specific task or application. Now, pre-trained language models can do really impressive stuff just immediately off the shelf, including things like text generation, summarization, and even providing levels of code. But LLMs are not one-size-fits-all as you can imagine and maybe I want to specialize it even further after the model has been trained. I'd like to make it really good at answering questions based on certain topics such as medicine or travel or code or something. This is what the idea of fine-tuning comes in.  Methods used in Fine tuning: Fine-tuning a large language model refers to the process of taking a pre-trained language model or a model that's already been trained on a large dataset and further training it on a smaller task-specific dataset. Now this process will allow the model to adapt and specialize its capability. And if we take t...
Read more

Understanding the basic architecture of LLMs

Image
Architecture of LLMs In this article, we're going to discover the key components in general architecture that make up typical large language models.  Introduction to LLM Architecture: Now, large language models will have things like the transformer neural networks at their very underlying pieces of it. Remember, these transformers are just a large number of parameters are attached to this transformer model and these LLMs are able to then understand and generate accurate responses because it's using this type of neural network at its core. In fact, what you're going to see is that these LLMs are referred to as foundation models and it's all it means is a foundation model is so large and impactful that it serves as its own class of artificial intelligence so that we can start experimenting with these types. Transformer Model Structure: It's just a way for us to classify these models that are being used today. Now, what exactly is a transformer model? We'll get to ...
Read more