Plug your Django application logging directly into AWS CloudWatch

Python 3.7 + Django 2.2.6 + AWS CloudWatch + watchtower

It is convenient for people to use Django to build their websites, and it is easy to debug locally. However, after launching your applications on AWS, without tools, you should ssh AWS to read the error messages, or even you don’t know what happens.

Therefore, if you want to track information, you can use some useful tools such as Sentry, and Amazon CloudWatch.

Sentry

Sentry provides open-source and hosted error monitoring that helps all software teams discover, triage, and prioritize errors in real-time. It is easy to add in Django settings.py and track error in scripts. If you want to use Sentry, you can read its document.

image source: https://sentry.io/welcome/

Amazon CloudWatch

If you want to use AWS tools, CloudWatch meets your needs. According to the website, CloudWatch is a monitoring and observability service. You can use CloudWatch to detect anomalous behavior in your environments, set alarms, visualize logs and metrics side by side, take automated actions, troubleshoot issues, and discover insights to keep your applications running smoothly.

image source: https://aws.amazon.com/cloudwatch/

In this article, we will explain how to plug your Django application logging directly into Amazon CloudWatch.

Use Watchtower

watchtower is an open source log handler for Amazon Web Services CloudWatch Logs. It is easy for you to send your logs to AWS.

Repository: https://github.com/kislyuk/watchtower

Ok, let’s start! Install the package.

pip install watchtower

At first, I followed the README.md to write the Django settings.py. However, it did not work on my mac, so I changed to the following code:

There are 2 ways to connect to your AWS. One is to create a role and another one is to paste your AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_REGION_NAME to the settings.py. If you don’t have a AWS account, please sign up first.

In this article, we will use the latter way, because the project is small and it is easy to debug. However, if you manage several user accounts, I recommend you to create a role. If you don’t know where to find your key and secret, you can refer to the paragraph below — ‘How can I get my AWS Access Key ID, AWS Secret Access Key?’. If you do have that, just jump to ‘Add logger and see your logs at the CloudWatch’

How can I get my AWS Access Key ID, AWS Secret Access Key?

1. Log in your AWS account, click the ‘My Security Credentials’ and click ‘Access Keys’. If you already have one, just copy it, if not, create an IAM user.

find Access key or create an IAM user

2. After clicking ‘creating an IAM user’, you can click ‘Add user’ and follow the steps to create a new user. Note, you should add ‘CloudWatchLogsFullAccess’ to your policies. Without this policy, your user can not push log to CloudWatch.

create an IAM user

add CloudWatchLogsFullAccess to your policies

3. After that you can copy your AWS Access Key ID and your AWS Secret Access Key!

Create a IAM user successfully

Add logger and see your logs at the CloudWatch

After setting up, you can import logging in <your app>.views.py to track information. For example:

<your_app>/views.py
import logging

logger = logging.getLogger('watchtower-logger')

def add(a, b):
    try:
         return int(a) + int(b)
    except:
         logger.error('test!') # it will be sent to the CloudWatch

# Call add function with invalid parameters.
# Therefore, the error message will be sent to the CloudWatch.
add(1, 'test')

Just running your application, and visit the AWS CloudWatch, clicking Logs, and your will find your Log Group created by Django!

Congratulation! you can see the test log in the Log Stream!

kislyuk/watchtower