aboutsummaryrefslogtreecommitdiff
path: root/public/using-goaccess-with-nginx-to-replace-google-analytics.html
diff options
context:
space:
mode:
authorMitja Felicijan <m@mitjafelicijan.com>2023-07-08 23:25:41 +0200
committerMitja Felicijan <m@mitjafelicijan.com>2023-07-08 23:25:41 +0200
commitcd6644ea4ddc78597934ab0ef5ba50e3c3daa927 (patch)
tree03de331a8db6386dfd6fa75155bfbcea6b4feaf3 /public/using-goaccess-with-nginx-to-replace-google-analytics.html
parent84ed124529ffeee1590295b8de3a8faf51848680 (diff)
downloadmitjafelicijan.com-cd6644ea4ddc78597934ab0ef5ba50e3c3daa927.tar.gz
Moved to a simpler SSG
Diffstat (limited to 'public/using-goaccess-with-nginx-to-replace-google-analytics.html')
-rwxr-xr-xpublic/using-goaccess-with-nginx-to-replace-google-analytics.html96
1 files changed, 96 insertions, 0 deletions
diff --git a/public/using-goaccess-with-nginx-to-replace-google-analytics.html b/public/using-goaccess-with-nginx-to-replace-google-analytics.html
new file mode 100755
index 0000000..9424885
--- /dev/null
+++ b/public/using-goaccess-with-nginx-to-replace-google-analytics.html
@@ -0,0 +1,96 @@
1<!doctype html><html lang=en-us><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><link href="data:image/x-icon;base64,AAABAAEAEBAAAAEAIABoBAAAFgAAACgAAAAQAAAAIAAAAAEAIAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAL69vf8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAv76+/8LBwQkAAAAAAAAAAAAAAAC+vb3/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAL+9vf/Bv78JAAAAAAAAAAAAAAAAu7q6/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAC7ubr/vr29CAAAAAAAAAAAy8nJAZ6foP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAnqGj/6GipAoAAAAAHLjU/xcXHf/BwsL/I8XY/yPK3v8XGiD/IbjL/yPF2f8XGiD/Fxkf/yLF2f8gnK3/Fxog/62ztv8fwNf/FRcd/x271v8mz93/GRsi/xkXHf8p097/GiIp/xobIv8p0t3/KdPe/xocIv8fYmr/KNPe/xoZH/8aHCL/J87c/xy81/8VFxz/IsPZ/8zS0/8XGiD/Ir/R/yPH2/8XGiD/Fxkf/yPH2/8dd4T/GBog/yPJ3f8jyNr/uru9/xcUGv8cudb/EhITDKi5vRKlvMP/RUpOERwcHRAdOj4QHTk8EBwdHRAdNTgQHTo/EBwcHRAcHB0QSGduEKW4vf+koqQfHzg+EBqz0ewSFRv7EyMr/xq51vsTERb7ExUb+xq41fsau9j7ExUb+xiPp/sZudb7ExUb+xMVG/sZuNX/GKvI/BIUGfMdvdn/IrfL/xcaIP8n1eb/J9Dh/xkcIf8ZGR7/J8/f/xxCSv8ZGyH/J9Dg/ybQ4P8ZHCL/FSQs/yPK3/8UExj/GE1b/ybS5P8ZGB7/Ghwj/ynW5P8p2Ob/Ghwi/yWrtv8p1eH/Ghwi/xocIv8p1uT/J8XT/xkcIv8m1un/Hb7d/xUYH/8hzOr/HtHu/xcaIf8XGB//I8vi/xgxOv8XGSD/I8rg/yPK4P8XGiD/GUFL/yPP6f8SERj/Fhkh/x3A4f8AAAAAJ2f9/ydr//8mZPH/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAlYu38J2v//ydo/f8AAAAAAAAAAAd8/fkFqf//Iob8sAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMY39awWr//8FfP3/AAAAAAAAAAAFm/7/SfD//wR+/f8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAOB/f9B7v//BaX+/wAAAAAAAAAAQ878SAyZ/v9n1v4KAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADu9v8DDJb+/z3N/XgAAAAA3/sAAN/7AADf+wAA3/sAAAAAAAAAAAAAAAAAAN/7AAAAAAAAAAAAAAAAAAAAAAAAj/EAAI/5AACP8QAA3/sAAA==" rel=icon type=image/x-icon><title>Using GoAccess with Nginx to replace Google Analytics</title><meta name=description content="IntroductionI know!"><link rel=alternate type=application/rss+xml title="Mitja Felicijan's posts" href=https://mitjafelicijan.com/index.xml><link rel=alternate type=application/rss+xml title="Mitja Felicijan's notes" href=https://mitjafelicijan.com/notes.xml><style>body{padding:1rem;max-width:760px;background:#fff;font-family:times new roman,Times,serif;line-height:1.35rem}hr{margin-block-start:1.5rem}h1,h2,h3{line-height:initial}footer{margin-block-start:3rem}table{max-width:100%;border-collapse:separate;border-spacing:2px;border:1px solid #000;border-left:1px solid #999;border-top:1px solid #999}blockquote{font-style:italic}table thead{background:#eee}td,th{border:1px solid #000;padding:4px;border-right:1px solid #999;border-bottom:1px solid #999;text-align:left}pre{text-wrap:nowrap;overflow-x:auto;margin-block-start:1.5rem;margin-block-end:1.5rem;padding:.5rem 0;border-top:1px solid #000;border-bottom:1px solid #000}pre code{line-height:1.3em}pre,code,pre *,code *{font-family:monospace;font-size:initial!important}img,video,audio{max-width:100%}header{display:flex;flex-direction:row;gap:3rem}nav{display:flex;gap:.75rem}.pstatus-orange{background:gold}.pstatus-green{background:#9acd32}.pstatus-red{background:#cd5c5c}@media only screen and (max-width:600px){header{flex-direction:column;gap:1rem}a{word-wrap:break-word}}</style><header><nav class=main><a href=/>Home</a>
2<a href=https://git.mitjafelicijan.com/ target=_blank>Git</a>
3<a href=https://files.mitjafelicijan.com/ target=_blank>Files</a>
4<a href=/mitjafelicijan.pgp.pub.txt target=_blank>PGP</a>
5<a href=/curriculum-vitae.html>CV</a>
6<a href=/index.xml target=_blank>RSS</a></nav></header><main><div><h1>Using GoAccess with Nginx to replace Google Analytics</h1><p>Jan 25, 2021<div><h2 id=introduction>Introduction</h2><p>I know! You cannot simply replace Google Analytics with parsing access logs and
7displaying a couple of charts. But to be honest, I actually never used Google
8Analytics to the fullest extent and was usually interested in seeing page hits
9and which pages were visited most often.<p>I recently moved my blog from Firebase to a VPS and also decided to remove
10Google Analytics tracking code from the site since its quite malicious and
11tracks users across other pages also and is creating a profile of a user, and
12I've had it. But I also need some insight of what is happening on a server and
13which content is being read the most etc.<p>I have looked at many existing solutions like:<ul><li><a href=https://umami.is/>Umami</a><li><a href=https://github.com/sheshbabu/freshlytics>Freshlytics</a><li><a href=https://matomo.org/>Matomo</a></ul><p>But the more I looked at them the more I noticed that I am replacing one evil
14with another one. Don't get me wrong. Some of these solutions are absolutely
15fantastic but would require installation of databases and something like PHP or
16Node. And I was not ready to put those things on my fresh server. Also having
17Docker installed is out of the question.<h2 id=opting-for-log-parsing>Opting for log parsing</h2><p>So, I defaulted to parsing already existing logs and generating HTML reports
18from this data.<p>I found this amazing software <a href=https://goaccess.io/>GoAccess</a> which provides
19all the functionalities I need, and it's a single binary. Written in Go.<p>GoAccess can be used in two different modes.<p><img src=/assets/goaccess/goaccess-dash-term.png alt="GoAccess Terminal"></p><center><i>Running in a terminal</i></center><p><img src=/assets/goaccess/goaccess-dash-html.png alt="GoAccess HTML"></p><center><i>Running in a browser</i></center><p>I, however, need this to run in a browser. So, the second option is the way to
20go. The Idea is to periodically run cronjob and export this report into a folder
21that gets then server by Nginx behind a Basic authentication.<h2 id=getting-nginx-ready>Getting Nginx ready</h2><p>I choose Ubuntu on <a href=https://www.digitalocean.com/>DigitalOcean</a>. First I
22installed <a href=https://nginx.org/en/>Nginx</a>, and
23<a href=https://letsencrypt.org/getting-started/>Letsencrypt</a> certbot and all the
24necessary dependencies.<pre tabindex=0 style=background-color:#fff><code><span style=display:flex><span><span style=color:green># log in as root user</span>
25</span></span><span style=display:flex><span>sudo su -
26</span></span><span style=display:flex><span>
27</span></span><span style=display:flex><span><span style=color:green># first let&#39;s update the system</span>
28</span></span><span style=display:flex><span>apt update &amp;&amp; apt upgrade -y
29</span></span><span style=display:flex><span>
30</span></span><span style=display:flex><span><span style=color:green># let&#39;s install</span>
31</span></span><span style=display:flex><span>apt install nginx certbot python3-certbot-nginx apache2-utils
32</span></span></code></pre><p>After all this is installed we can create a new configuration for a statistics.
33Stats will be available at <code>stats.domain.com</code>.<pre tabindex=0 style=background-color:#fff><code><span style=display:flex><span><span style=color:green># creates directory where html will be hosted</span>
34</span></span><span style=display:flex><span>mkdir -p /var/www/html/stats.domain.com
35</span></span><span style=display:flex><span>
36</span></span><span style=display:flex><span>cp /etc/nginx/sites-available/default /etc/nginx/sites-available/stats.domain.com
37</span></span><span style=display:flex><span>nano /etc/nginx/sites-available/stats.domain.com
38</span></span></code></pre><pre tabindex=0 style=background-color:#fff><code><span style=display:flex><span><span style=color:#00f>server</span> {
39</span></span><span style=display:flex><span> <span style=color:#00f>root</span> <span style=color:#a31515>/var/www/html/stats.domain.com</span>;
40</span></span><span style=display:flex><span> <span style=color:#00f>server_name</span> <span style=color:#a31515>stats.domain.com</span>;
41</span></span><span style=display:flex><span>
42</span></span><span style=display:flex><span> <span style=color:#00f>index</span> <span style=color:#a31515>index.html</span>;
43</span></span><span style=display:flex><span> <span style=color:#00f>location</span> <span style=color:#a31515>/</span> {
44</span></span><span style=display:flex><span> <span style=color:#00f>try_files</span> $uri $uri/ =404;
45</span></span><span style=display:flex><span> }
46</span></span><span style=display:flex><span>}
47</span></span></code></pre><p>Now we check if the configuration is ok. We can do this with <code>nginx -t</code>. If all
48is ok, we can restart Nginx with <code>service nginx restart</code>.<p>After all that you should add A record for this domain that points to IP of a
49droplet.<p>Before enabling SSL you should test if DNS records have propagated with <code>curl stats.domain.com</code>.<p>Now, it's time to provision TLS certificate. To achieve this, you execute
50command <code>certbot --nginx</code>. Follow the wizard and when you are asked about
51redirection always choose 2 (always redirect to HTTPS).<p>When this is done you can visit <a href=https://stats.domain.com>https://stats.domain.com</a> and you should get 404
52not found error which is correct.<h2 id=getting-goaccess-ready>Getting GoAccess ready</h2><p>If you are using Debian like system GoAccess should be available in repository.
53Otherwise refer to the official website.<pre tabindex=0 style=background-color:#fff><code><span style=display:flex><span>apt install goaccess
54</span></span></code></pre><p>To enable Geo location we also need one additiona thing.<pre tabindex=0 style=background-color:#fff><code><span style=display:flex><span>cd /var/www/html/stats.stats.com
55</span></span><span style=display:flex><span>wget https://github.com/P3TERX/GeoLite.mmdb/raw/download/GeoLite2-City.mmdb
56</span></span></code></pre><p>Now we create a shell script that will be executed every 10 minutes.<pre tabindex=0 style=background-color:#fff><code><span style=display:flex><span>nano /var/www/html/stats.domain.com/generate-stats.sh
57</span></span></code></pre><p>Contents of this file should look like this.<pre tabindex=0 style=background-color:#fff><code><span style=display:flex><span><span style=color:#00f>#!/bin/sh
58</span></span></span><span style=display:flex><span><span style=color:#00f></span>
59</span></span><span style=display:flex><span>zcat -f /var/log/nginx/access.log* &gt; /var/log/nginx/access-all.log
60</span></span><span style=display:flex><span>
61</span></span><span style=display:flex><span>goaccess <span style=color:#a31515>\
62</span></span></span><span style=display:flex><span><span style=color:#a31515></span> --log-file=/var/log/nginx/access-all.log <span style=color:#a31515>\
63</span></span></span><span style=display:flex><span><span style=color:#a31515></span> --log-format=COMBINED <span style=color:#a31515>\
64</span></span></span><span style=display:flex><span><span style=color:#a31515></span> --exclude-ip=0.0.0.0 <span style=color:#a31515>\
65</span></span></span><span style=display:flex><span><span style=color:#a31515></span> --geoip-database=/var/www/html/stats.domain.com/GeoLite2-City.mmdb <span style=color:#a31515>\
66</span></span></span><span style=display:flex><span><span style=color:#a31515></span> --ignore-crawlers <span style=color:#a31515>\
67</span></span></span><span style=display:flex><span><span style=color:#a31515></span> --real-os <span style=color:#a31515>\
68</span></span></span><span style=display:flex><span><span style=color:#a31515></span> --output=/var/www/html/stats.domain.com/index.html
69</span></span><span style=display:flex><span>
70</span></span><span style=display:flex><span>rm /var/log/nginx/access-all.log
71</span></span></code></pre><p>Because after a while nginx creates multiple files with access logs we use
72<a href=https://linux.die.net/man/1/zcat><code>zcat</code></a> to extract Gziped contents and create
73a file that has all the access logs. After this file is used we delete it.<p>If you want to exclude your home IP's result look at the <code>--exclude-ip</code> option
74in script and instead of <code>0.0.0.0</code> add your own home IP address. You can find
75your home IP by executing <code>curl ifconfig.me</code> from your local machine and NOT
76from the droplet.<p>Test the script by executing <code>sh /var/www/html/stats.domain.com/generate-stats.sh</code> and then checking
77<code>https://stats.domain.com</code>. If you can see stats instead of 404 than you are
78set.<p>It's time to add this script to cron with <code>cron -e</code>.<pre tabindex=0 style=background-color:#fff><code><span style=display:flex><span>*/10 * * * * sh /<span style=color:#00f>var</span>/www/html/stats.domain.com/generate-stats.sh
79</span></span></code></pre><h2 id=securing-with-basic-authentication>Securing with Basic authentication</h2><p>You probably don't want stats to be publicly available, so we should create a
80user and a password for Basic authentication.<p>First we create a password for a user <code>stats</code> with <code>htpasswd -c /etc/nginx/.htpasswd stats</code>.<p>Now we update config file with <code>nano /etc/nginx/sites-available/stats.domain.com</code>. You probably noticed that the
81file looks a bit different from before. This is because <code>certbot</code> added
82additional rules for SSL.<p>Your location portion the config file should now look like. You should add
83<code>auth_basic</code> and <code>auth_basic_user_file</code> lines to the file.<pre tabindex=0 style=background-color:#fff><code><span style=display:flex><span><span style=color:#00f>location</span> <span style=color:#a31515>/</span> {
84</span></span><span style=display:flex><span> <span style=color:#00f>try_files</span> $uri $uri/ =404;
85</span></span><span style=display:flex><span> <span style=color:#00f>auth_basic</span> <span style=color:#a31515>&#34;Private</span> <span style=color:#a31515>Property&#34;</span>;
86</span></span><span style=display:flex><span> <span style=color:#00f>auth_basic_user_file</span> <span style=color:#a31515>/etc/nginx/.htpasswd</span>;
87</span></span><span style=display:flex><span>}
88</span></span></code></pre><p>Test if config is still ok with <code>nginx -t</code> and if it is you can restart Nginx
89with <code>service nginx restart</code>.<p>If you now visit <code>https://stats.domain.com</code> you should be prompted for username
90and password. If not, try reopening your browser.<p>That is all. You now have analytics for your server that gets refreshed every 10
91minutes.</div></div></main><footer><hr><div><h3>Want to comment or have something to add?</h3>You can write me an email at
92<a href=mailto:m@mitjafelicijan.com>m@mitjafelicijan.com</a> or catch up
93with me
94<a href=https://telegram.me/mitjafelicijan target=_blank>on Telegram</a>.</div><hr><p>This website does not track you. Content is made available under
95the <a href=https://creativecommons.org/licenses/by/4.0/ target=_blank rel=noreferrer>CC BY 4.0 license</a> unless specified
96otherwise. Blog feed is available as <a href=/index.xml target=_blank>RSS feed</a>.</footer><script src=https://cdn.usefathom.com/script.js data-site=XHQARKXP defer></script> \ No newline at end of file
generated by cgit v1.2.3 (git 2.46.0) at 2026-05-01 01:50:41 +0000