From 43b0708769eb61392050045b881f8e6ba39c5b66 Mon Sep 17 00:00:00 2001 From: Mitja Felicijan Date: Fri, 26 May 2023 00:40:40 +0200 Subject: Massive update to posts, archetypes Added a archetypes for creating notes and posts so it auto-populates fields. Fixed existing posts so they align with the rule of 80 columns now. --- content/posts/2021-01-25-goaccess.md | 90 ++++++++++++++++++++++++++---------- 1 file changed, 66 insertions(+), 24 deletions(-) (limited to 'content/posts/2021-01-25-goaccess.md') diff --git a/content/posts/2021-01-25-goaccess.md b/content/posts/2021-01-25-goaccess.md index e8fcee3..efbd175 100644 --- a/content/posts/2021-01-25-goaccess.md +++ b/content/posts/2021-01-25-goaccess.md @@ -1,28 +1,42 @@ --- title: Using GoAccess with Nginx to replace Google Analytics url: using-goaccess-with-nginx-to-replace-google-analytics.html -date: 2021-01-25 +date: 2021-01-25T12:00:00+02:00 draft: false --- ## Introduction -I know! You cannot simply replace Google Analytics with parsing access logs and displaying a couple of charts. But to be honest, I actually never used Google Analytics to the fullest extent and was usually interested in seeing page hits and which pages were visited most often. +I know! You cannot simply replace Google Analytics with parsing access logs and +displaying a couple of charts. But to be honest, I actually never used Google +Analytics to the fullest extent and was usually interested in seeing page +hits and which pages were visited most often. -I recently moved my blog from Firebase to a VPS and also decided to remove Google Analytics tracking code from the site since its quite malicious and tracks users across other pages also and is creating a profile of a user, and I've had it. But I also need some insight of what is happening on a server and which content is being read the most etc. +I recently moved my blog from Firebase to a VPS and also decided to remove +Google Analytics tracking code from the site since its quite malicious and +tracks users across other pages also and is creating a profile of a user, and +I've had it. But I also need some insight of what is happening on a server +and which content is being read the most etc. I have looked at many existing solutions like: + - [Umami](https://umami.is/) - [Freshlytics](https://github.com/sheshbabu/freshlytics) - [Matomo](https://matomo.org/) -But the more I looked at them the more I noticed that I am replacing one evil with another one. Don't get me wrong. Some of these solutions are absolutely fantastic but would require installation of databases and something like PHP or Node. And I was not ready to put those things on my fresh server. Also having Docker installed is out of the question. +But the more I looked at them the more I noticed that I am replacing one evil +with another one. Don't get me wrong. Some of these solutions are absolutely +fantastic but would require installation of databases and something like PHP +or Node. And I was not ready to put those things on my fresh server. Also +having Docker installed is out of the question. ## Opting for log parsing -So, I defaulted to parsing already existing logs and generating HTML reports from this data. +So, I defaulted to parsing already existing logs and generating HTML reports +from this data. -I found this amazing software [GoAccess](https://goaccess.io/) which provides all the functionalities I need, and it's a single binary. Written in Go. +I found this amazing software [GoAccess](https://goaccess.io/) which provides +all the functionalities I need, and it's a single binary. Written in Go. GoAccess can be used in two different modes. @@ -32,11 +46,15 @@ GoAccess can be used in two different modes. ![GoAccess HTML](/assets/goaccess/goaccess-dash-html.png)
Running in a browser
-I, however, need this to run in a browser. So, the second option is the way to go. The Idea is to periodically run cronjob and export this report into a folder that gets then server by Nginx behind a Basic authentication. +I, however, need this to run in a browser. So, the second option is the way +to go. The Idea is to periodically run cronjob and export this report into a +folder that gets then server by Nginx behind a Basic authentication. ## Getting Nginx ready -I choose Ubuntu on [DigitalOcean](https://www.digitalocean.com/). First I installed [Nginx](https://nginx.org/en/), and [Letsencrypt](https://letsencrypt.org/getting-started/) certbot and all the necessary dependencies. +I choose Ubuntu on [DigitalOcean](https://www.digitalocean.com/). First I +installed [Nginx](https://nginx.org/en/), and [Letsencrypt](https://letsencrypt.org/getting-started/) +certbot and all the necessary dependencies. ```sh # log in as root user @@ -49,7 +67,8 @@ apt update && apt upgrade -y apt install nginx certbot python3-certbot-nginx apache2-utils ``` -After all this is installed we can create a new configuration for a statistics. Stats will be available at `stats.domain.com`. +After all this is installed we can create a new configuration for a statistics. +Stats will be available at `stats.domain.com`. ```sh # creates directory where html will be hosted @@ -71,20 +90,27 @@ server { } ``` -Now we check if the configuration is ok. We can do this with `nginx -t`. If all is ok, we can restart Nginx with `service nginx restart`. +Now we check if the configuration is ok. We can do this with `nginx -t`. If +all is ok, we can restart Nginx with `service nginx restart`. -After all that you should add A record for this domain that points to IP of a droplet. +After all that you should add A record for this domain that points to IP of +a droplet. -Before enabling SSL you should test if DNS records have propagated with `curl stats.domain.com`. +Before enabling SSL you should test if DNS records have propagated with +`curl stats.domain.com`. -Now, it's time to provision TLS certificate. To achieve this, you execute command `certbot --nginx`. Follow the wizard and when you are asked about redirection always choose 2 (always redirect to HTTPS). +Now, it's time to provision TLS certificate. To achieve this, you execute command +`certbot --nginx`. Follow the wizard and when you are asked about redirection +always choose 2 (always redirect to HTTPS). -When this is done you can visit https://stats.domain.com and you should get 404 not found error which is correct. +When this is done you can visit https://stats.domain.com and you should get 404 +not found error which is correct. ## Getting GoAccess ready -If you are using Debian like system GoAccess should be available in repository. Otherwise refer to the official website. +If you are using Debian like system GoAccess should be available in repository. +Otherwise refer to the official website. ```sh apt install goaccess @@ -122,11 +148,19 @@ goaccess \ rm /var/log/nginx/access-all.log ``` -Because after a while nginx creates multiple files with access logs we use [`zcat`](https://linux.die.net/man/1/zcat) to extract Gziped contents and create a file that has all the access logs. After this file is used we delete it. +Because after a while nginx creates multiple files with access logs we use +[`zcat`](https://linux.die.net/man/1/zcat) to extract Gziped contents and +create a file that has all the access logs. After this file is used we +delete it. -If you want to exclude your home IP's result look at the `--exclude-ip` option in script and instead of `0.0.0.0` add your own home IP address. You can find your home IP by executing `curl ifconfig.me` from your local machine and NOT from the droplet. +If you want to exclude your home IP's result look at the `--exclude-ip` option +in script and instead of `0.0.0.0` add your own home IP address. You can find +your home IP by executing `curl ifconfig.me` from your local machine and NOT +from the droplet. -Test the script by executing `sh /var/www/html/stats.domain.com/generate-stats.sh` and then checking `https://stats.domain.com`. If you can see stats instead of 404 than you are set. +Test the script by executing `sh /var/www/html/stats.domain.com/generate-stats.sh` +and then checking `https://stats.domain.com`. If you can see stats instead of +404 than you are set. It's time to add this script to cron with `cron -e`. @@ -136,13 +170,17 @@ It's time to add this script to cron with `cron -e`. ## Securing with Basic authentication -You probably don't want stats to be publicly available, so we should create a user and a password for Basic authentication. +You probably don't want stats to be publicly available, so we should create a +user and a password for Basic authentication. First we create a password for a user `stats` with `htpasswd -c /etc/nginx/.htpasswd stats`. -Now we update config file with `nano /etc/nginx/sites-available/stats.domain.com`. You probably noticed that the file looks a bit different from before. This is because `certbot` added additional rules for SSL. +Now we update config file with `nano /etc/nginx/sites-available/stats.domain.com`. +You probably noticed that the file looks a bit different from before. This is +because `certbot` added additional rules for SSL. -Your location portion the config file should now look like. You should add `auth_basic` and `auth_basic_user_file` lines to the file. +Your location portion the config file should now look like. You should add +`auth_basic` and `auth_basic_user_file` lines to the file. ```nginx location / { @@ -152,8 +190,12 @@ location / { } ``` -Test if config is still ok with `nginx -t` and if it is you can restart Nginx with `service nginx restart`. +Test if config is still ok with `nginx -t` and if it is you can restart +Nginx with `service nginx restart`. + +If you now visit `https://stats.domain.com` you should be prompted for username +and password. If not, try reopening your browser. -If you now visit `https://stats.domain.com` you should be prompted for username and password. If not, try reopening your browser. +That is all. You now have analytics for your server that gets refreshed every +10 minutes. -That is all. You now have analytics for your server that gets refreshed every 10 minutes. -- cgit v1.2.3