diff options
Diffstat (limited to 'content/posts/2021-01-25-goaccess.md')
| -rw-r--r-- | content/posts/2021-01-25-goaccess.md | 90 |
1 files changed, 66 insertions, 24 deletions
diff --git a/content/posts/2021-01-25-goaccess.md b/content/posts/2021-01-25-goaccess.md index e8fcee3..efbd175 100644 --- a/content/posts/2021-01-25-goaccess.md +++ b/content/posts/2021-01-25-goaccess.md | |||
| @@ -1,28 +1,42 @@ | |||
| 1 | --- | 1 | --- |
| 2 | title: Using GoAccess with Nginx to replace Google Analytics | 2 | title: Using GoAccess with Nginx to replace Google Analytics |
| 3 | url: using-goaccess-with-nginx-to-replace-google-analytics.html | 3 | url: using-goaccess-with-nginx-to-replace-google-analytics.html |
| 4 | date: 2021-01-25 | 4 | date: 2021-01-25T12:00:00+02:00 |
| 5 | draft: false | 5 | draft: false |
| 6 | --- | 6 | --- |
| 7 | 7 | ||
| 8 | ## Introduction | 8 | ## Introduction |
| 9 | 9 | ||
| 10 | I know! You cannot simply replace Google Analytics with parsing access logs and displaying a couple of charts. But to be honest, I actually never used Google Analytics to the fullest extent and was usually interested in seeing page hits and which pages were visited most often. | 10 | I know! You cannot simply replace Google Analytics with parsing access logs and |
| 11 | displaying a couple of charts. But to be honest, I actually never used Google | ||
| 12 | Analytics to the fullest extent and was usually interested in seeing page | ||
| 13 | hits and which pages were visited most often. | ||
| 11 | 14 | ||
| 12 | I recently moved my blog from Firebase to a VPS and also decided to remove Google Analytics tracking code from the site since its quite malicious and tracks users across other pages also and is creating a profile of a user, and I've had it. But I also need some insight of what is happening on a server and which content is being read the most etc. | 15 | I recently moved my blog from Firebase to a VPS and also decided to remove |
| 16 | Google Analytics tracking code from the site since its quite malicious and | ||
| 17 | tracks users across other pages also and is creating a profile of a user, and | ||
| 18 | I've had it. But I also need some insight of what is happening on a server | ||
| 19 | and which content is being read the most etc. | ||
| 13 | 20 | ||
| 14 | I have looked at many existing solutions like: | 21 | I have looked at many existing solutions like: |
| 22 | |||
| 15 | - [Umami](https://umami.is/) | 23 | - [Umami](https://umami.is/) |
| 16 | - [Freshlytics](https://github.com/sheshbabu/freshlytics) | 24 | - [Freshlytics](https://github.com/sheshbabu/freshlytics) |
| 17 | - [Matomo](https://matomo.org/) | 25 | - [Matomo](https://matomo.org/) |
| 18 | 26 | ||
| 19 | But the more I looked at them the more I noticed that I am replacing one evil with another one. Don't get me wrong. Some of these solutions are absolutely fantastic but would require installation of databases and something like PHP or Node. And I was not ready to put those things on my fresh server. Also having Docker installed is out of the question. | 27 | But the more I looked at them the more I noticed that I am replacing one evil |
| 28 | with another one. Don't get me wrong. Some of these solutions are absolutely | ||
| 29 | fantastic but would require installation of databases and something like PHP | ||
| 30 | or Node. And I was not ready to put those things on my fresh server. Also | ||
| 31 | having Docker installed is out of the question. | ||
| 20 | 32 | ||
| 21 | ## Opting for log parsing | 33 | ## Opting for log parsing |
| 22 | 34 | ||
| 23 | So, I defaulted to parsing already existing logs and generating HTML reports from this data. | 35 | So, I defaulted to parsing already existing logs and generating HTML reports |
| 36 | from this data. | ||
| 24 | 37 | ||
| 25 | I found this amazing software [GoAccess](https://goaccess.io/) which provides all the functionalities I need, and it's a single binary. Written in Go. | 38 | I found this amazing software [GoAccess](https://goaccess.io/) which provides |
| 39 | all the functionalities I need, and it's a single binary. Written in Go. | ||
| 26 | 40 | ||
| 27 | GoAccess can be used in two different modes. | 41 | GoAccess can be used in two different modes. |
| 28 | 42 | ||
| @@ -32,11 +46,15 @@ GoAccess can be used in two different modes. | |||
| 32 |  | 46 |  |
| 33 | <center><i>Running in a browser</i></center> | 47 | <center><i>Running in a browser</i></center> |
| 34 | 48 | ||
| 35 | I, however, need this to run in a browser. So, the second option is the way to go. The Idea is to periodically run cronjob and export this report into a folder that gets then server by Nginx behind a Basic authentication. | 49 | I, however, need this to run in a browser. So, the second option is the way |
| 50 | to go. The Idea is to periodically run cronjob and export this report into a | ||
| 51 | folder that gets then server by Nginx behind a Basic authentication. | ||
| 36 | 52 | ||
| 37 | ## Getting Nginx ready | 53 | ## Getting Nginx ready |
| 38 | 54 | ||
| 39 | I choose Ubuntu on [DigitalOcean](https://www.digitalocean.com/). First I installed [Nginx](https://nginx.org/en/), and [Letsencrypt](https://letsencrypt.org/getting-started/) certbot and all the necessary dependencies. | 55 | I choose Ubuntu on [DigitalOcean](https://www.digitalocean.com/). First I |
| 56 | installed [Nginx](https://nginx.org/en/), and [Letsencrypt](https://letsencrypt.org/getting-started/) | ||
| 57 | certbot and all the necessary dependencies. | ||
| 40 | 58 | ||
| 41 | ```sh | 59 | ```sh |
| 42 | # log in as root user | 60 | # log in as root user |
| @@ -49,7 +67,8 @@ apt update && apt upgrade -y | |||
| 49 | apt install nginx certbot python3-certbot-nginx apache2-utils | 67 | apt install nginx certbot python3-certbot-nginx apache2-utils |
| 50 | ``` | 68 | ``` |
| 51 | 69 | ||
| 52 | After all this is installed we can create a new configuration for a statistics. Stats will be available at `stats.domain.com`. | 70 | After all this is installed we can create a new configuration for a statistics. |
| 71 | Stats will be available at `stats.domain.com`. | ||
| 53 | 72 | ||
| 54 | ```sh | 73 | ```sh |
| 55 | # creates directory where html will be hosted | 74 | # creates directory where html will be hosted |
| @@ -71,20 +90,27 @@ server { | |||
| 71 | } | 90 | } |
| 72 | ``` | 91 | ``` |
| 73 | 92 | ||
| 74 | Now we check if the configuration is ok. We can do this with `nginx -t`. If all is ok, we can restart Nginx with `service nginx restart`. | 93 | Now we check if the configuration is ok. We can do this with `nginx -t`. If |
| 94 | all is ok, we can restart Nginx with `service nginx restart`. | ||
| 75 | 95 | ||
| 76 | After all that you should add A record for this domain that points to IP of a droplet. | 96 | After all that you should add A record for this domain that points to IP of |
| 97 | a droplet. | ||
| 77 | 98 | ||
| 78 | Before enabling SSL you should test if DNS records have propagated with `curl stats.domain.com`. | 99 | Before enabling SSL you should test if DNS records have propagated with |
| 100 | `curl stats.domain.com`. | ||
| 79 | 101 | ||
| 80 | Now, it's time to provision TLS certificate. To achieve this, you execute command `certbot --nginx`. Follow the wizard and when you are asked about redirection always choose 2 (always redirect to HTTPS). | 102 | Now, it's time to provision TLS certificate. To achieve this, you execute command |
| 103 | `certbot --nginx`. Follow the wizard and when you are asked about redirection | ||
| 104 | always choose 2 (always redirect to HTTPS). | ||
| 81 | 105 | ||
| 82 | When this is done you can visit https://stats.domain.com and you should get 404 not found error which is correct. | 106 | When this is done you can visit https://stats.domain.com and you should get 404 |
| 107 | not found error which is correct. | ||
| 83 | 108 | ||
| 84 | 109 | ||
| 85 | ## Getting GoAccess ready | 110 | ## Getting GoAccess ready |
| 86 | 111 | ||
| 87 | If you are using Debian like system GoAccess should be available in repository. Otherwise refer to the official website. | 112 | If you are using Debian like system GoAccess should be available in repository. |
| 113 | Otherwise refer to the official website. | ||
| 88 | 114 | ||
| 89 | ```sh | 115 | ```sh |
| 90 | apt install goaccess | 116 | apt install goaccess |
| @@ -122,11 +148,19 @@ goaccess \ | |||
| 122 | rm /var/log/nginx/access-all.log | 148 | rm /var/log/nginx/access-all.log |
| 123 | ``` | 149 | ``` |
| 124 | 150 | ||
| 125 | Because after a while nginx creates multiple files with access logs we use [`zcat`](https://linux.die.net/man/1/zcat) to extract Gziped contents and create a file that has all the access logs. After this file is used we delete it. | 151 | Because after a while nginx creates multiple files with access logs we use |
| 152 | [`zcat`](https://linux.die.net/man/1/zcat) to extract Gziped contents and | ||
| 153 | create a file that has all the access logs. After this file is used we | ||
| 154 | delete it. | ||
| 126 | 155 | ||
| 127 | If you want to exclude your home IP's result look at the `--exclude-ip` option in script and instead of `0.0.0.0` add your own home IP address. You can find your home IP by executing `curl ifconfig.me` from your local machine and NOT from the droplet. | 156 | If you want to exclude your home IP's result look at the `--exclude-ip` option |
| 157 | in script and instead of `0.0.0.0` add your own home IP address. You can find | ||
| 158 | your home IP by executing `curl ifconfig.me` from your local machine and NOT | ||
| 159 | from the droplet. | ||
| 128 | 160 | ||
| 129 | Test the script by executing `sh /var/www/html/stats.domain.com/generate-stats.sh` and then checking `https://stats.domain.com`. If you can see stats instead of 404 than you are set. | 161 | Test the script by executing `sh /var/www/html/stats.domain.com/generate-stats.sh` |
| 162 | and then checking `https://stats.domain.com`. If you can see stats instead of | ||
| 163 | 404 than you are set. | ||
| 130 | 164 | ||
| 131 | It's time to add this script to cron with `cron -e`. | 165 | It's time to add this script to cron with `cron -e`. |
| 132 | 166 | ||
| @@ -136,13 +170,17 @@ It's time to add this script to cron with `cron -e`. | |||
| 136 | 170 | ||
| 137 | ## Securing with Basic authentication | 171 | ## Securing with Basic authentication |
| 138 | 172 | ||
| 139 | You probably don't want stats to be publicly available, so we should create a user and a password for Basic authentication. | 173 | You probably don't want stats to be publicly available, so we should create a |
| 174 | user and a password for Basic authentication. | ||
| 140 | 175 | ||
| 141 | First we create a password for a user `stats` with `htpasswd -c /etc/nginx/.htpasswd stats`. | 176 | First we create a password for a user `stats` with `htpasswd -c /etc/nginx/.htpasswd stats`. |
| 142 | 177 | ||
| 143 | Now we update config file with `nano /etc/nginx/sites-available/stats.domain.com`. You probably noticed that the file looks a bit different from before. This is because `certbot` added additional rules for SSL. | 178 | Now we update config file with `nano /etc/nginx/sites-available/stats.domain.com`. |
| 179 | You probably noticed that the file looks a bit different from before. This is | ||
| 180 | because `certbot` added additional rules for SSL. | ||
| 144 | 181 | ||
| 145 | Your location portion the config file should now look like. You should add `auth_basic` and `auth_basic_user_file` lines to the file. | 182 | Your location portion the config file should now look like. You should add |
| 183 | `auth_basic` and `auth_basic_user_file` lines to the file. | ||
| 146 | 184 | ||
| 147 | ```nginx | 185 | ```nginx |
| 148 | location / { | 186 | location / { |
| @@ -152,8 +190,12 @@ location / { | |||
| 152 | } | 190 | } |
| 153 | ``` | 191 | ``` |
| 154 | 192 | ||
| 155 | Test if config is still ok with `nginx -t` and if it is you can restart Nginx with `service nginx restart`. | 193 | Test if config is still ok with `nginx -t` and if it is you can restart |
| 194 | Nginx with `service nginx restart`. | ||
| 195 | |||
| 196 | If you now visit `https://stats.domain.com` you should be prompted for username | ||
| 197 | and password. If not, try reopening your browser. | ||
| 156 | 198 | ||
| 157 | If you now visit `https://stats.domain.com` you should be prompted for username and password. If not, try reopening your browser. | 199 | That is all. You now have analytics for your server that gets refreshed every |
| 200 | 10 minutes. | ||
| 158 | 201 | ||
| 159 | That is all. You now have analytics for your server that gets refreshed every 10 minutes. | ||
