server: NGINX
monitoring tool: NGINX Amplify
monitoring tool: ngxtop
monitoring tool: access_log
Necessity of Monitoring a Server
As a professional, it’s essential you monitor the status of your tools. If you’re building a bridge, you check for cracks; if you’re running a server, you monitor logs. It’s important. If people come to your website and may get turned away because of some mis-configuration or server fault. In this case, there is no point in having a website at all if all the traffic is getting sent away. Further, if your website is working extremely slow, then you’re also going to have visitors very quickly going to find another resource on the internet.
So, we know that it is very important to monitor the status of your web server. I will provide some of the tools that I use in this post.
Warning: Making your server run fast and cleanly is addictive, once you begin down this road, it is like a rabbit hole and you will not return the same (lazy web administrator).
NGINX Amplify
By far, one of the easiest to use and elaborate monitoring tool is Amplify by NGINX (they may still be in Beta, but beta is a great thing when the developers respond immediately to issues). Amplify goes into much greater detail then I will here about how to install and configure Amplify on the first of three configuration blogs: Setting Up NGINX Amplify in 10 Minutes. The default metrics that show up on your Amplify Dashboard are very good for monitoring your server, more so now that they’ve done some UI changes, but it’s worth the time to also go into the Alerts and Custom Dashboards sections to see if they are giving some suggestions to change your server configuration (NGINX itself), as well as custom graphical metrics.
Stub Status must be turned on for some more back end information, so don’t miss that in the configuration step, but more importantly I feel is setting up the access log configuration acutely.
They have put together three full posts on how to configure your Amplify here:
- Setting Up NGINX Amplify in 10 Minutes
- Improving Server Configuration with NGINX Amplify Reports
- Using NGINX Amplify Custom Dashboards and Filters for Better NGINX Monitoring
NGINX Access and Error Log
NGINX’s built in access_log and error_log directive should most definitely be turned on. This provides all kinds of essential information regarding your NGINX server. Without this critical information, you’re kind of shooting in the dark when it comes to troubleshooting your server errors.
Using the access log, you can also set the log format, this can be customized to give some detailed information about what’s happening on your server in regards to error codes, request URI, referrer, etc.
You set the access_log format inside the nginx.conf file (typically located in etc/nginx/nginx.conf on a Ubuntu deployment), but you can also set the access_log parameters inside each virtual server configuration block so that you can have a unique log files for each virtual servers (typically located in etc/nginx/sites-available/siteName). Virtual servers are usually used if you have multiple websites hosted by the single NGINX server.
A caveat that should be noted is that if you define access_log within each unique virtual server, Amplify can provide a breakdown of each websites access_logs.
For a full list of access_log log_format descriptors, see NGINX Metrics & Metadata documentation.
note: I’m still working on ways to customize my access_log file results so I can better identify exact URL requests for 4XX and 5XX errors, so if you have a resource or some tips, I’d love to hear them below.
NGXTOP
From the command line within your server you can run ngxtop to get very detailed breakdown of what’s happening on your server in real time. There are multiple commands that you can customize, so do have a look at the GitHub documentation to get a better idea of what you can do with this powerful tool. This is more of a real-time monitoring tool (as the Ubuntu top command is, too).
Python Script
I say Python script because Python is quickly becoming my favorite programming language, but you could probably use any language you like to parse the access_log files for basically anything you’re looking for. If one was so inclined, they could even turn this into an hourly cron job that sent results to you via email.
Working on this code now. 🙂
Hope some of this helps, and I’d love to hear if you have any other suggestions that are essential for you and your server administration!