aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
-rw-r--r--assets/goaccess/goaccess-dash-html.pngbin0 -> 16129 bytes
-rw-r--r--assets/goaccess/goaccess-dash-term.pngbin0 -> 9188 bytes
-rw-r--r--posts/2021-01-24-replacing-dropbox-with-s3.md (renamed from posts/2021-01-25-replacing-dropbox-with-s3.md)2
-rw-r--r--posts/2021-01-25-goaccess.md156
4 files changed, 157 insertions, 1 deletions
diff --git a/assets/goaccess/goaccess-dash-html.png b/assets/goaccess/goaccess-dash-html.png
new file mode 100644
index 0000000..917d959
--- /dev/null
+++ b/assets/goaccess/goaccess-dash-html.png
Binary files differ
diff --git a/assets/goaccess/goaccess-dash-term.png b/assets/goaccess/goaccess-dash-term.png
new file mode 100644
index 0000000..e3f6357
--- /dev/null
+++ b/assets/goaccess/goaccess-dash-term.png
Binary files differ
diff --git a/posts/2021-01-25-replacing-dropbox-with-s3.md b/posts/2021-01-24-replacing-dropbox-with-s3.md
index 7d137bc..f3714b6 100644
--- a/posts/2021-01-25-replacing-dropbox-with-s3.md
+++ b/posts/2021-01-24-replacing-dropbox-with-s3.md
@@ -3,7 +3,7 @@ Title: Replacing Dropbox in favor of DigitalOcean spaces
3Description: Replacing Dropbox in favor of DigitalOcean spaces 3Description: Replacing Dropbox in favor of DigitalOcean spaces
4Slug: replacing-dropbox-in-favor-of-digitalocean-spaces 4Slug: replacing-dropbox-in-favor-of-digitalocean-spaces
5Listing: true 5Listing: true
6Created: 2021, January 25 6Created: 2021, January 24
7Tags: [] 7Tags: []
8--- 8---
9 9
diff --git a/posts/2021-01-25-goaccess.md b/posts/2021-01-25-goaccess.md
new file mode 100644
index 0000000..c72430a
--- /dev/null
+++ b/posts/2021-01-25-goaccess.md
@@ -0,0 +1,156 @@
1---
2Title: Using GoAccess with Nginx to replace Google Analytics
3Description: Using GoAccess with Nginx to replace Google Analytics
4Slug: using-goaccess-with-nginx-to-replace-google-analytics
5Listing: true
6Created: 2021, January 25
7Tags: []
8---
9
101. [Opting for log parsing](#opting-for-log-parsing)
112. [Getting Nginx ready](#getting-nginx-ready)
123. [Getting GoAccess ready](#getting-goaccess-ready)
134. [Securing with Basic authentication](#securing-with-basic-authentication)
14
15I know! You cannot simply replace Google Analytics with parsing access logs and displaying a couple of charts. But to be honest, I actually never used Google Analytics to the fullest extent and was usually interested in seeing page hits and which pages were visited most often.
16
17I recently moved my blog from Firebase to a VPS and also decided to remove Google Analytics tracking code from the site since its quite malicious and tracks users across other pages also and is creating a profile of a user, and I've had it. But I also need some insight of what is happening on a server and which content is being read the most etc.
18
19I have looked at many existing solutions like:
20- [Umami](https://umami.is/)
21- [Freshlytics](https://github.com/sheshbabu/freshlytics)
22- [Matomo](https://matomo.org/)
23
24But the more I looked at them the more I noticed that I am replacing one evil with another one. Don't get me wrong. Some of these solutions are absolutely fantastic but would require installation of databases and something like PHP or Node. And I was not ready to put those things on my fresh server. Also having Docker installed is out of the question.
25
26## Opting for log parsing
27
28So, I defaulted to parsing already existing logs and generating HTML reports from this data.
29
30I found this amazing software [GoAccess](https://goaccess.io/) which provides all the functionalities I need, and it's a single binary. Written in Go.
31
32GoAccess can be used in two different modes.
33
34![GoAccess Terminal](/assets/goaccess/goaccess-dash-term.png)
35<center><i>Running in a terminal</i></center>
36
37![GoAccess HTML](/assets/goaccess/goaccess-dash-html.png)
38<center><i>Running in a browser</i></center>
39
40I, however, need this to run in a browser. So, the second option is the way to go. The Idea is to periodically run cronjob and export this report into a folder that gets then server by Nginx behind a Basic authentication.
41
42## Getting Nginx ready
43
44I choose Ubuntu on [DigitalOcean](https://www.digitalocean.com/). First I installed [Nginx](https://nginx.org/en/), and [Letsencrypt](https://letsencrypt.org/getting-started/) certbot and all the necessary dependencies.
45
46```sh
47# log in as root user
48sudo su -
49
50# first let's update the system
51apt update && apt upgrade -y
52
53# let's install
54apt install nginx certbot python3-certbot-nginx apache2-utils
55```
56
57After all this is installed we can create a new configuration for a statistics. Stats will be available at `stats.domain.com`.
58
59```sh
60# creates directory where html will be hosted
61mkdir -p /var/www/html/stats.domain.com
62
63cp /etc/nginx/sites-available/default /etc/nginx/sites-available/stats.domain.com
64nano /etc/nginx/sites-available/stats.domain.com
65```
66
67```nginx
68server {
69 root /var/www/html/stats.domain.com;
70 server_name stats.domain.com;
71
72 index index.html;
73 location / {
74 try_files $uri $uri/ =404;
75 }
76}
77```
78
79Now we check if the configuration is ok. We can do this with `nginx -t`. If all is ok, we can restart Nginx with `service nginx restart`.
80
81After all that you should add A record for this domain that points to IP of a droplet.
82
83Before enabling SSL you should test if DNS records have propagated with `curl stats.domain.com`.
84
85Now, it's time to provision TLS certificate. To achieve this, you execute command `certbot --nginx`. Follow the wizard and when you are asked about redirection always choose 2 (always redirect to HTTPS).
86
87When this is done you can visit https://stats.domain.com and you should get 404 not found error which is correct.
88
89
90## Getting GoAccess ready
91
92If you are using Debian like system GoAccess should be available in repository. Otherwise refer to the official website.
93
94```sh
95apt install goaccess
96```
97
98Now we create a shell script that will be executed every 10 minutes.
99
100```sh
101nano /var/www/html/stats.stats.com/generate-stats.sh
102```
103
104Contents of this file should look like this.
105
106```sh
107#!/bin/sh
108
109zcat -f /var/log/nginx/access.log* > /var/log/nginx/access-all.log
110
111goaccess \
112 --log-file=/var/log/nginx/access-all.log \
113 --log-format=COMBINED \
114 --exclude-ip=0.0.0.0 \
115 --ignore-crawlers \
116 --real-os \
117 --output=/var/www/html/stats.domain.com/index.html
118
119rm /var/log/nginx/access-all.log
120```
121
122Because after a while nginx creates multiple files with access logs we use [`zcat`](https://linux.die.net/man/1/zcat) to extract Gziped contents and create a file that has all the access logs. After this file is used we delete it.
123
124If you want to exclude your home IP's result look at the `--exclude-ip` option in script and instead of `0.0.0.0` add your own home IP address. You can find your home IP by executing `curl ifconfig.me` from your local machine and NOT from the droplet.
125
126Test the script by executing `sh /var/www/html/stats.domain.com/generate-stats.sh` and then checking `https://stats.domain.com`. If you can see stats instead of 404 than you are set.
127
128It's time to add this script to cron with `cron -e`.
129
130```go
131*/10 * * * * sh /var/www/html/stats.domain.com/generate-stats.sh
132```
133
134## Securing with Basic authentication
135
136You probably don't want stats to be publicly available, so we should create a user and a password for Basic authentication.
137
138First we create a password for a user `stats` with `htpasswd -c /etc/nginx/.htpasswd stats`.
139
140Now we update config file with `nano /etc/nginx/sites-available/stats.domain.com`. You probably noticed that the file looks a bit different from before. This is because `certbot` added additional rules for SSL.
141
142Your location portion the config file should now look like. You should add `auth_basic` and `auth_basic_user_file` lines to the file.
143
144```go
145location / {
146 try_files $uri $uri/ =404;
147 auth_basic "Private Property";
148 auth_basic_user_file /etc/nginx/.htpasswd;
149}
150```
151
152Test if config is still ok with `nginx -t` and if it is you can restart Nginx with `service nginx restart`.
153
154If you now visit `https://stats.domain.com` you should be prompted for username and password. If not, try reopening your browser.
155
156That is all. You now have analytics for your server that gets refreshed every 10 minutes.