aboutsummaryrefslogtreecommitdiff
path: root/content/posts/2021-01-25-goaccess.md
diff options
context:
space:
mode:
authorMitja Felicijan <mitja.felicijan@gmail.com>2023-05-26 00:40:40 +0200
committerMitja Felicijan <mitja.felicijan@gmail.com>2023-05-26 00:40:40 +0200
commit43b0708769eb61392050045b881f8e6ba39c5b66 (patch)
tree3939579a13b8325325d5ebb8e05324a41ed78a6d /content/posts/2021-01-25-goaccess.md
parent49e7e7d555a6cd9810d81561fa3e98e3d64502be (diff)
downloadmitjafelicijan.com-43b0708769eb61392050045b881f8e6ba39c5b66.tar.gz
Massive update to posts, archetypes
Added a archetypes for creating notes and posts so it auto-populates fields. Fixed existing posts so they align with the rule of 80 columns now.
Diffstat (limited to 'content/posts/2021-01-25-goaccess.md')
-rw-r--r--content/posts/2021-01-25-goaccess.md90
1 files changed, 66 insertions, 24 deletions
diff --git a/content/posts/2021-01-25-goaccess.md b/content/posts/2021-01-25-goaccess.md
index e8fcee3..efbd175 100644
--- a/content/posts/2021-01-25-goaccess.md
+++ b/content/posts/2021-01-25-goaccess.md
@@ -1,28 +1,42 @@
1--- 1---
2title: Using GoAccess with Nginx to replace Google Analytics 2title: Using GoAccess with Nginx to replace Google Analytics
3url: using-goaccess-with-nginx-to-replace-google-analytics.html 3url: using-goaccess-with-nginx-to-replace-google-analytics.html
4date: 2021-01-25 4date: 2021-01-25T12:00:00+02:00
5draft: false 5draft: false
6--- 6---
7 7
8## Introduction 8## Introduction
9 9
10I know! You cannot simply replace Google Analytics with parsing access logs and displaying a couple of charts. But to be honest, I actually never used Google Analytics to the fullest extent and was usually interested in seeing page hits and which pages were visited most often. 10I know! You cannot simply replace Google Analytics with parsing access logs and
11displaying a couple of charts. But to be honest, I actually never used Google
12Analytics to the fullest extent and was usually interested in seeing page
13hits and which pages were visited most often.
11 14
12I recently moved my blog from Firebase to a VPS and also decided to remove Google Analytics tracking code from the site since its quite malicious and tracks users across other pages also and is creating a profile of a user, and I've had it. But I also need some insight of what is happening on a server and which content is being read the most etc. 15I recently moved my blog from Firebase to a VPS and also decided to remove
16Google Analytics tracking code from the site since its quite malicious and
17tracks users across other pages also and is creating a profile of a user, and
18I've had it. But I also need some insight of what is happening on a server
19and which content is being read the most etc.
13 20
14I have looked at many existing solutions like: 21I have looked at many existing solutions like:
22
15- [Umami](https://umami.is/) 23- [Umami](https://umami.is/)
16- [Freshlytics](https://github.com/sheshbabu/freshlytics) 24- [Freshlytics](https://github.com/sheshbabu/freshlytics)
17- [Matomo](https://matomo.org/) 25- [Matomo](https://matomo.org/)
18 26
19But the more I looked at them the more I noticed that I am replacing one evil with another one. Don't get me wrong. Some of these solutions are absolutely fantastic but would require installation of databases and something like PHP or Node. And I was not ready to put those things on my fresh server. Also having Docker installed is out of the question. 27But the more I looked at them the more I noticed that I am replacing one evil
28with another one. Don't get me wrong. Some of these solutions are absolutely
29fantastic but would require installation of databases and something like PHP
30or Node. And I was not ready to put those things on my fresh server. Also
31having Docker installed is out of the question.
20 32
21## Opting for log parsing 33## Opting for log parsing
22 34
23So, I defaulted to parsing already existing logs and generating HTML reports from this data. 35So, I defaulted to parsing already existing logs and generating HTML reports
36from this data.
24 37
25I found this amazing software [GoAccess](https://goaccess.io/) which provides all the functionalities I need, and it's a single binary. Written in Go. 38I found this amazing software [GoAccess](https://goaccess.io/) which provides
39all the functionalities I need, and it's a single binary. Written in Go.
26 40
27GoAccess can be used in two different modes. 41GoAccess can be used in two different modes.
28 42
@@ -32,11 +46,15 @@ GoAccess can be used in two different modes.
32![GoAccess HTML](/assets/goaccess/goaccess-dash-html.png) 46![GoAccess HTML](/assets/goaccess/goaccess-dash-html.png)
33<center><i>Running in a browser</i></center> 47<center><i>Running in a browser</i></center>
34 48
35I, however, need this to run in a browser. So, the second option is the way to go. The Idea is to periodically run cronjob and export this report into a folder that gets then server by Nginx behind a Basic authentication. 49I, however, need this to run in a browser. So, the second option is the way
50to go. The Idea is to periodically run cronjob and export this report into a
51folder that gets then server by Nginx behind a Basic authentication.
36 52
37## Getting Nginx ready 53## Getting Nginx ready
38 54
39I choose Ubuntu on [DigitalOcean](https://www.digitalocean.com/). First I installed [Nginx](https://nginx.org/en/), and [Letsencrypt](https://letsencrypt.org/getting-started/) certbot and all the necessary dependencies. 55I choose Ubuntu on [DigitalOcean](https://www.digitalocean.com/). First I
56installed [Nginx](https://nginx.org/en/), and [Letsencrypt](https://letsencrypt.org/getting-started/)
57certbot and all the necessary dependencies.
40 58
41```sh 59```sh
42# log in as root user 60# log in as root user
@@ -49,7 +67,8 @@ apt update && apt upgrade -y
49apt install nginx certbot python3-certbot-nginx apache2-utils 67apt install nginx certbot python3-certbot-nginx apache2-utils
50``` 68```
51 69
52After all this is installed we can create a new configuration for a statistics. Stats will be available at `stats.domain.com`. 70After all this is installed we can create a new configuration for a statistics.
71Stats will be available at `stats.domain.com`.
53 72
54```sh 73```sh
55# creates directory where html will be hosted 74# creates directory where html will be hosted
@@ -71,20 +90,27 @@ server {
71} 90}
72``` 91```
73 92
74Now we check if the configuration is ok. We can do this with `nginx -t`. If all is ok, we can restart Nginx with `service nginx restart`. 93Now we check if the configuration is ok. We can do this with `nginx -t`. If
94all is ok, we can restart Nginx with `service nginx restart`.
75 95
76After all that you should add A record for this domain that points to IP of a droplet. 96After all that you should add A record for this domain that points to IP of
97a droplet.
77 98
78Before enabling SSL you should test if DNS records have propagated with `curl stats.domain.com`. 99Before enabling SSL you should test if DNS records have propagated with
100`curl stats.domain.com`.
79 101
80Now, it's time to provision TLS certificate. To achieve this, you execute command `certbot --nginx`. Follow the wizard and when you are asked about redirection always choose 2 (always redirect to HTTPS). 102Now, it's time to provision TLS certificate. To achieve this, you execute command
103`certbot --nginx`. Follow the wizard and when you are asked about redirection
104always choose 2 (always redirect to HTTPS).
81 105
82When this is done you can visit https://stats.domain.com and you should get 404 not found error which is correct. 106When this is done you can visit https://stats.domain.com and you should get 404
107not found error which is correct.
83 108
84 109
85## Getting GoAccess ready 110## Getting GoAccess ready
86 111
87If you are using Debian like system GoAccess should be available in repository. Otherwise refer to the official website. 112If you are using Debian like system GoAccess should be available in repository.
113Otherwise refer to the official website.
88 114
89```sh 115```sh
90apt install goaccess 116apt install goaccess
@@ -122,11 +148,19 @@ goaccess \
122rm /var/log/nginx/access-all.log 148rm /var/log/nginx/access-all.log
123``` 149```
124 150
125Because after a while nginx creates multiple files with access logs we use [`zcat`](https://linux.die.net/man/1/zcat) to extract Gziped contents and create a file that has all the access logs. After this file is used we delete it. 151Because after a while nginx creates multiple files with access logs we use
152[`zcat`](https://linux.die.net/man/1/zcat) to extract Gziped contents and
153create a file that has all the access logs. After this file is used we
154delete it.
126 155
127If you want to exclude your home IP's result look at the `--exclude-ip` option in script and instead of `0.0.0.0` add your own home IP address. You can find your home IP by executing `curl ifconfig.me` from your local machine and NOT from the droplet. 156If you want to exclude your home IP's result look at the `--exclude-ip` option
157in script and instead of `0.0.0.0` add your own home IP address. You can find
158your home IP by executing `curl ifconfig.me` from your local machine and NOT
159from the droplet.
128 160
129Test the script by executing `sh /var/www/html/stats.domain.com/generate-stats.sh` and then checking `https://stats.domain.com`. If you can see stats instead of 404 than you are set. 161Test the script by executing `sh /var/www/html/stats.domain.com/generate-stats.sh`
162and then checking `https://stats.domain.com`. If you can see stats instead of
163404 than you are set.
130 164
131It's time to add this script to cron with `cron -e`. 165It's time to add this script to cron with `cron -e`.
132 166
@@ -136,13 +170,17 @@ It's time to add this script to cron with `cron -e`.
136 170
137## Securing with Basic authentication 171## Securing with Basic authentication
138 172
139You probably don't want stats to be publicly available, so we should create a user and a password for Basic authentication. 173You probably don't want stats to be publicly available, so we should create a
174user and a password for Basic authentication.
140 175
141First we create a password for a user `stats` with `htpasswd -c /etc/nginx/.htpasswd stats`. 176First we create a password for a user `stats` with `htpasswd -c /etc/nginx/.htpasswd stats`.
142 177
143Now we update config file with `nano /etc/nginx/sites-available/stats.domain.com`. You probably noticed that the file looks a bit different from before. This is because `certbot` added additional rules for SSL. 178Now we update config file with `nano /etc/nginx/sites-available/stats.domain.com`.
179You probably noticed that the file looks a bit different from before. This is
180because `certbot` added additional rules for SSL.
144 181
145Your location portion the config file should now look like. You should add `auth_basic` and `auth_basic_user_file` lines to the file. 182Your location portion the config file should now look like. You should add
183`auth_basic` and `auth_basic_user_file` lines to the file.
146 184
147```nginx 185```nginx
148location / { 186location / {
@@ -152,8 +190,12 @@ location / {
152} 190}
153``` 191```
154 192
155Test if config is still ok with `nginx -t` and if it is you can restart Nginx with `service nginx restart`. 193Test if config is still ok with `nginx -t` and if it is you can restart
194Nginx with `service nginx restart`.
195
196If you now visit `https://stats.domain.com` you should be prompted for username
197and password. If not, try reopening your browser.
156 198
157If you now visit `https://stats.domain.com` you should be prompted for username and password. If not, try reopening your browser. 199That is all. You now have analytics for your server that gets refreshed every
20010 minutes.
158 201
159That is all. You now have analytics for your server that gets refreshed every 10 minutes.