aboutsummaryrefslogtreecommitdiff
path: root/public/using-goaccess-with-nginx-to-replace-google-analytics.html
diff options
context:
space:
mode:
Diffstat (limited to 'public/using-goaccess-with-nginx-to-replace-google-analytics.html')
-rwxr-xr-xpublic/using-goaccess-with-nginx-to-replace-google-analytics.html129
1 files changed, 0 insertions, 129 deletions
diff --git a/public/using-goaccess-with-nginx-to-replace-google-analytics.html b/public/using-goaccess-with-nginx-to-replace-google-analytics.html
deleted file mode 100755
index 08ceb9d..0000000
--- a/public/using-goaccess-with-nginx-to-replace-google-analytics.html
+++ /dev/null
@@ -1,129 +0,0 @@
1<!doctype html><html lang=en-us><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><meta name=generator content="JBMAFP - github.com/mitjafelicijan/jbmafp"><link href="data:image/x-icon;base64,AAABAAEAEBAAAAEAIABoBAAAFgAAACgAAAAQAAAAIAAAAAEAIAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAL69vf8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAv76+/8LBwQkAAAAAAAAAAAAAAAC+vb3/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAL+9vf/Bv78JAAAAAAAAAAAAAAAAu7q6/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAC7ubr/vr29CAAAAAAAAAAAy8nJAZ6foP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAnqGj/6GipAoAAAAAHLjU/xcXHf/BwsL/I8XY/yPK3v8XGiD/IbjL/yPF2f8XGiD/Fxkf/yLF2f8gnK3/Fxog/62ztv8fwNf/FRcd/x271v8mz93/GRsi/xkXHf8p097/GiIp/xobIv8p0t3/KdPe/xocIv8fYmr/KNPe/xoZH/8aHCL/J87c/xy81/8VFxz/IsPZ/8zS0/8XGiD/Ir/R/yPH2/8XGiD/Fxkf/yPH2/8dd4T/GBog/yPJ3f8jyNr/uru9/xcUGv8cudb/EhITDKi5vRKlvMP/RUpOERwcHRAdOj4QHTk8EBwdHRAdNTgQHTo/EBwcHRAcHB0QSGduEKW4vf+koqQfHzg+EBqz0ewSFRv7EyMr/xq51vsTERb7ExUb+xq41fsau9j7ExUb+xiPp/sZudb7ExUb+xMVG/sZuNX/GKvI/BIUGfMdvdn/IrfL/xcaIP8n1eb/J9Dh/xkcIf8ZGR7/J8/f/xxCSv8ZGyH/J9Dg/ybQ4P8ZHCL/FSQs/yPK3/8UExj/GE1b/ybS5P8ZGB7/Ghwj/ynW5P8p2Ob/Ghwi/yWrtv8p1eH/Ghwi/xocIv8p1uT/J8XT/xkcIv8m1un/Hb7d/xUYH/8hzOr/HtHu/xcaIf8XGB//I8vi/xgxOv8XGSD/I8rg/yPK4P8XGiD/GUFL/yPP6f8SERj/Fhkh/x3A4f8AAAAAJ2f9/ydr//8mZPH/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAlYu38J2v//ydo/f8AAAAAAAAAAAd8/fkFqf//Iob8sAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMY39awWr//8FfP3/AAAAAAAAAAAFm/7/SfD//wR+/f8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAOB/f9B7v//BaX+/wAAAAAAAAAAQ878SAyZ/v9n1v4KAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADu9v8DDJb+/z3N/XgAAAAA3/sAAN/7AADf+wAA3/sAAAAAAAAAAAAAAAAAAN/7AAAAAAAAAAAAAAAAAAAAAAAAj/EAAI/5AACP8QAA3/sAAA==" rel=icon type=image/x-icon><title>Using GoAccess with Nginx to replace Google Analytics</title><meta name=description content="IntroductionI know!"><meta name=author content="Mitja Felicijan"><link rel=alternate type=application/rss+xml title="Mitja Felicijan's posts" href=https://mitjafelicijan.com/index.xml><link rel=alternate type=application/rss+xml title="Mitja Felicijan's notes" href=https://mitjafelicijan.com/notes.xml><style>:root{--border-color:gainsboro;--border-size:2px;--link-color:blue;--bg-color:#eee}*::selection{background:var(--link-color);color:#fff}*::-moz-selection{background:var(--link-color);color:#fff}*::-webkit-selection{background:var(--link-color);color:#fff}body{padding:2.5rem;max-width:1900px;background:#fff;font-family:sans-serif;line-height:1.35rem;font-size:16px}hr{border:0;border-bottom:var(--border-size)solid var(--border-color);margin-block-start:1.5rem}a{color:var(--link-color);text-decoration:none}h1,h2,h3{line-height:initial}h1{font-size:xx-large}footer{margin-block-start:2rem}cap{text-transform:capitalize}blockquote{font-style:italic}table{max-width:100%;border:var(--border-size)solid var(--border-color);border-collapse:separate;border-spacing:0}table thead tr th{border-bottom:var(--border-size)solid var(--border-color);text-align:left}table th,table td{padding:.5em .8em}ul.list li{padding:.2em 0}ul{line-height:1.35em}pre{text-wrap:nowrap;overflow-x:auto;padding:0 1em;border:var(--border-size)solid var(--border-color)}code{padding:0 3px;font-size:14px;border:0;background:var(--bg-color)}pre code{line-height:1.3em;background:#fff}pre,code,pre *,code *{font-family:monospace}figure{margin-inline-start:0;margin-inline-end:0}figcaption{width:800px;max-width:100%;text-align:center}figcaption p{margin:.3em 0 1.5em;font-style:italic}img,video,audio{width:800px;max-width:100%;border:var(--border-size)solid var(--border-color);padding:.5em}header nav{display:flex;gap:.9rem}article iframe{margin:0!important}audio::-webkit-media-controls-enclosure{border-radius:0}@media only screen and (max-width:600px){body{padding:.5em;word-wrap:break-word}header nav{gap:.7rem}header nav .hob{display:none}a{word-wrap:break-word}img,video,audio{padding:0}}</style><header><nav class=main itemscope itemtype=http://schema.org/SiteNavigationElement role=navigation aria-label="Main navigation"><a href=/>Home</a>
2<a href=/#posts>Posts</a>
3<a href=/#notes>Notes</a>
4<a href=/#sideprojects class=hob>Side Projects</a>
5<a href=/vault.html>Vault</a>
6<a href=https://github.com/mitjafelicijan target=_blank>Code</a>
7<a href=/mitjafelicijan.pgp.pub.txt target=_blank class=hob>PGP</a>
8<a href=/curriculum-vitae.html>CV</a>
9<a href=/index.xml target=_blank class=hob>RSS</a></nav></header><main role=main><article itemtype=http://schema.org/Article><h1 itemtype=headline>Using GoAccess with Nginx to replace Google Analytics</h1><p><cap>post</cap>, Jan 25, 2021 on <a href=https://mitjafelicijan.com>Mitja Felicijan's blog</a><div><h2 id=introduction>Introduction</h2><p>I know! You cannot simply replace Google Analytics with parsing access logs and
10displaying a couple of charts. But to be honest, I actually never used Google
11Analytics to the fullest extent and was usually interested in seeing page hits
12and which pages were visited most often.<p>I recently moved my blog from Firebase to a VPS and also decided to remove
13Google Analytics tracking code from the site since its quite malicious and
14tracks users across other pages also and is creating a profile of a user, and
15I've had it. But I also need some insight of what is happening on a server and
16which content is being read the most etc.<p>I have looked at many existing solutions like:<ul><li><a href=https://umami.is/>Umami</a><li><a href=https://github.com/sheshbabu/freshlytics>Freshlytics</a><li><a href=https://matomo.org/>Matomo</a></ul><p>But the more I looked at them the more I noticed that I am replacing one evil
17with another one. Don't get me wrong. Some of these solutions are absolutely
18fantastic but would require installation of databases and something like PHP or
19Node. And I was not ready to put those things on my fresh server. Also having
20Docker installed is out of the question.<h2 id=opting-for-log-parsing>Opting for log parsing</h2><p>So, I defaulted to parsing already existing logs and generating HTML reports
21from this data.<p>I found this amazing software <a href=https://goaccess.io/>GoAccess</a> which provides
22all the functionalities I need, and it's a single binary. Written in Go.<p>GoAccess can be used in two different modes.<figure><img src=/posts/goaccess/goaccess-dash-term.png alt="GoAccess Terminal"></figure><p><em>Running in a terminal</em><figure><img src=/posts/goaccess/goaccess-dash-html.png alt="GoAccess HTML"></figure><p><em>Running in a browser</em><p>I, however, need this to run in a browser. So, the second option is the way to
23go. The Idea is to periodically run cronjob and export this report into a folder
24that gets then server by Nginx behind a Basic authentication.<h2 id=getting-nginx-ready>Getting Nginx ready</h2><p>I choose Ubuntu on <a href=https://www.digitalocean.com/>DigitalOcean</a>. First I
25installed <a href=https://nginx.org/en/>Nginx</a>, and
26<a href=https://letsencrypt.org/getting-started/>Letsencrypt</a> certbot and all the
27necessary dependencies.<pre tabindex=0 style=background-color:#fff><code><span style=display:flex><span><span style=color:green># log in as root user</span>
28</span></span><span style=display:flex><span>sudo su -
29</span></span><span style=display:flex><span>
30</span></span><span style=display:flex><span><span style=color:green># first let&#39;s update the system</span>
31</span></span><span style=display:flex><span>apt update &amp;&amp; apt upgrade -y
32</span></span><span style=display:flex><span>
33</span></span><span style=display:flex><span><span style=color:green># let&#39;s install</span>
34</span></span><span style=display:flex><span>apt install nginx certbot python3-certbot-nginx apache2-utils
35</span></span></code></pre><p>After all this is installed we can create a new configuration for a statistics.
36Stats will be available at <code>stats.domain.com</code>.<pre tabindex=0 style=background-color:#fff><code><span style=display:flex><span><span style=color:green># creates directory where html will be hosted</span>
37</span></span><span style=display:flex><span>mkdir -p /var/www/html/stats.domain.com
38</span></span><span style=display:flex><span>
39</span></span><span style=display:flex><span>cp /etc/nginx/sites-available/default /etc/nginx/sites-available/stats.domain.com
40</span></span><span style=display:flex><span>nano /etc/nginx/sites-available/stats.domain.com
41</span></span></code></pre><pre tabindex=0 style=background-color:#fff><code><span style=display:flex><span><span style=color:#00f>server</span> {
42</span></span><span style=display:flex><span> <span style=color:#00f>root</span> <span style=color:#a31515>/var/www/html/stats.domain.com</span>;
43</span></span><span style=display:flex><span> <span style=color:#00f>server_name</span> <span style=color:#a31515>stats.domain.com</span>;
44</span></span><span style=display:flex><span>
45</span></span><span style=display:flex><span> <span style=color:#00f>index</span> <span style=color:#a31515>index.html</span>;
46</span></span><span style=display:flex><span> <span style=color:#00f>location</span> <span style=color:#a31515>/</span> {
47</span></span><span style=display:flex><span> <span style=color:#00f>try_files</span> $uri $uri/ =404;
48</span></span><span style=display:flex><span> }
49</span></span><span style=display:flex><span>}
50</span></span></code></pre><p>Now we check if the configuration is ok. We can do this with <code>nginx -t</code>. If all
51is ok, we can restart Nginx with <code>service nginx restart</code>.<p>After all that you should add A record for this domain that points to IP of a
52droplet.<p>Before enabling SSL you should test if DNS records have propagated with <code>curl stats.domain.com</code>.<p>Now, it's time to provision TLS certificate. To achieve this, you execute
53command <code>certbot --nginx</code>. Follow the wizard and when you are asked about
54redirection always choose 2 (always redirect to HTTPS).<p>When this is done you can visit <a href=https://stats.domain.com>https://stats.domain.com</a> and you should get 404
55not found error which is correct.<h2 id=getting-goaccess-ready>Getting GoAccess ready</h2><p>If you are using Debian like system GoAccess should be available in repository.
56Otherwise refer to the official website.<pre tabindex=0 style=background-color:#fff><code><span style=display:flex><span>apt install goaccess
57</span></span></code></pre><p>To enable Geo location we also need one additiona thing.<pre tabindex=0 style=background-color:#fff><code><span style=display:flex><span>cd /var/www/html/stats.stats.com
58</span></span><span style=display:flex><span>wget https://github.com/P3TERX/GeoLite.mmdb/raw/download/GeoLite2-City.mmdb
59</span></span></code></pre><p>Now we create a shell script that will be executed every 10 minutes.<pre tabindex=0 style=background-color:#fff><code><span style=display:flex><span>nano /var/www/html/stats.domain.com/generate-stats.sh
60</span></span></code></pre><p>Contents of this file should look like this.<pre tabindex=0 style=background-color:#fff><code><span style=display:flex><span><span style=color:#00f>#!/bin/sh
61</span></span></span><span style=display:flex><span><span style=color:#00f></span>
62</span></span><span style=display:flex><span>zcat -f /var/log/nginx/access.log* &gt; /var/log/nginx/access-all.log
63</span></span><span style=display:flex><span>
64</span></span><span style=display:flex><span>goaccess <span style=color:#a31515>\
65</span></span></span><span style=display:flex><span><span style=color:#a31515></span> --log-file=/var/log/nginx/access-all.log <span style=color:#a31515>\
66</span></span></span><span style=display:flex><span><span style=color:#a31515></span> --log-format=COMBINED <span style=color:#a31515>\
67</span></span></span><span style=display:flex><span><span style=color:#a31515></span> --exclude-ip=0.0.0.0 <span style=color:#a31515>\
68</span></span></span><span style=display:flex><span><span style=color:#a31515></span> --geoip-database=/var/www/html/stats.domain.com/GeoLite2-City.mmdb <span style=color:#a31515>\
69</span></span></span><span style=display:flex><span><span style=color:#a31515></span> --ignore-crawlers <span style=color:#a31515>\
70</span></span></span><span style=display:flex><span><span style=color:#a31515></span> --real-os <span style=color:#a31515>\
71</span></span></span><span style=display:flex><span><span style=color:#a31515></span> --output=/var/www/html/stats.domain.com/index.html
72</span></span><span style=display:flex><span>
73</span></span><span style=display:flex><span>rm /var/log/nginx/access-all.log
74</span></span></code></pre><p>Because after a while nginx creates multiple files with access logs we use
75<a href=https://linux.die.net/man/1/zcat><code>zcat</code></a> to extract Gziped contents and create
76a file that has all the access logs. After this file is used we delete it.<p>If you want to exclude your home IP's result look at the <code>--exclude-ip</code> option
77in script and instead of <code>0.0.0.0</code> add your own home IP address. You can find
78your home IP by executing <code>curl ifconfig.me</code> from your local machine and NOT
79from the droplet.<p>Test the script by executing <code>sh /var/www/html/stats.domain.com/generate-stats.sh</code> and then checking
80<code>https://stats.domain.com</code>. If you can see stats instead of 404 than you are
81set.<p>It's time to add this script to cron with <code>cron -e</code>.<pre tabindex=0 style=background-color:#fff><code><span style=display:flex><span>*/10 * * * * sh /<span style=color:#00f>var</span>/www/html/stats.domain.com/generate-stats.sh
82</span></span></code></pre><h2 id=securing-with-basic-authentication>Securing with Basic authentication</h2><p>You probably don't want stats to be publicly available, so we should create a
83user and a password for Basic authentication.<p>First we create a password for a user <code>stats</code> with <code>htpasswd -c /etc/nginx/.htpasswd stats</code>.<p>Now we update config file with <code>nano /etc/nginx/sites-available/stats.domain.com</code>. You probably noticed that the
84file looks a bit different from before. This is because <code>certbot</code> added
85additional rules for SSL.<p>Your location portion the config file should now look like. You should add
86<code>auth_basic</code> and <code>auth_basic_user_file</code> lines to the file.<pre tabindex=0 style=background-color:#fff><code><span style=display:flex><span><span style=color:#00f>location</span> <span style=color:#a31515>/</span> {
87</span></span><span style=display:flex><span> <span style=color:#00f>try_files</span> $uri $uri/ =404;
88</span></span><span style=display:flex><span> <span style=color:#00f>auth_basic</span> <span style=color:#a31515>&#34;Private</span> <span style=color:#a31515>Property&#34;</span>;
89</span></span><span style=display:flex><span> <span style=color:#00f>auth_basic_user_file</span> <span style=color:#a31515>/etc/nginx/.htpasswd</span>;
90</span></span><span style=display:flex><span>}
91</span></span></code></pre><p>Test if config is still ok with <code>nginx -t</code> and if it is you can restart Nginx
92with <code>service nginx restart</code>.<p>If you now visit <code>https://stats.domain.com</code> you should be prompted for username
93and password. If not, try reopening your browser.<p>That is all. You now have analytics for your server that gets refreshed every 10
94minutes.</div></article></main><section><hr><h2>Posts from blogs I follow around the net</h2><ul><li><a href=https://utcc.utoronto.ca/~cks/space/blog/linux/NFSv4ServerLockClients target=_blank rel=noopener>Finding which NFSv4 client owns a lock on a Linux NFS(v4) server</a> — <a href=https://utcc.utoronto.ca/~cks/space/blog/>Chris's Wiki :: blog</a><div>A while back I wrote an entry about finding which NFS client owns
95a lock on a Linux NFS server, which turned
96out to be specific to NFS v3 (which I really should have seen coming,
97since it involved NLM and lockd). Finding the NFS v4 client that
98owns a lock is, depending on your perspective, either simpl…<li><a href=http://www.landley.net/notes-2023.html#28-10-2023 target=_blank rel=noopener>October 28, 2023</a> — <a href=http://www.landley.net/notes-2023.html>Rob Landley's Blog Thing for 2023</a><div>Oh good grief, two of my least favorite licensing people, Larry Rosen
99and Bradley Kuhn, are interacting on the OSI's license-discuss
100list where the're doing
101bad computer history and insisting that a guy Larry Rosen
102coincidentally interviewed for a book years ago is clearly the origin of
103somethin…<li><a href="http://offbeatpursuit.com:80/blog/?id=25" target=_blank rel=noopener>A fix by any other name</a> — <a href=http://offbeatpursuit.com:80/blog/>WLOG - blog</a><div>tags:
104i2c, plan9
105Another month, another file system.
106Well, if you can’t fix it in software, fix it in hardware (looking at
107you, bme680, we’re not
108done yet). The show must go on, as they say, and I would like my
109experiments to go on.
110So a “new” addition to the environmental sensor family connected to
111the h…<li><a href=https://mirzapandzo.com/next-image-url-parameter-is-valid-but-upstream-response-is-invalid target=_blank rel=noopener>Next/Image "url" parameter is valid but upstream response is invalid</a> — <a href=https://mirzapandzo.com/>Mirza Pandzo's Blog</a><div>Getting "url" parameter is valid but upstream response is invalid error with Next/Image on WSL2<li><a href=https://drewdevault.com/2023/10/13/Going-off-script.html target=_blank rel=noopener>Going off-script</a> — <a href=https://drewdevault.com>Drew DeVault's blog</a><div>There is a phenomenon in society which I find quite bizarre. Upon our entry to
112this mortal coil, we are endowed with self-awareness, agency, and free will.
113Each of the 8 billion members of this human race represents a unique person, a
114unique worldview, and a unique agency. Yet, many of us have the sam…<li><a href=https://szymonkaliski.com/writing/2023-10-02-building-a-diy-pen-plotter/ target=_blank rel=noopener>Building a DIY Pen Plotter</a> — <a href=http://github.com/dylang/node-rss>Szymon Kaliski</a><div>This article documents my learnings from designing and building a DIY Pen Plotter during the summer of 2023.
115My ultimate goal is to build my…<li><a href=https://neil.computer/notes/chart-of-accounts-for-startups-and-saas-companies/ target=_blank rel=noopener>Chart of Accounts for Startups and SaaS Companies</a> — <a href=https://neil.computer/>Neil Panchal</a><div>Accounting is fundamental to starting a business. You need to have a basic understanding of accounting principles and essential bookkeeping. I had to learn it. There was no choice. For filing taxes, your CPA is going to ask you for an Income Statement (also known as P/L statement). If<li><a href=https://journal.valeriansaliou.name/deploy-a-nomad-cluster-on-alpine-linux-with-vultr/ target=_blank rel=noopener>Deploy a Nomad Cluster on Alpine Linux with Vultr</a> — <a href=https://journal.valeriansaliou.name/>Valerian Saliou</a><div>After spending countless hours trying to understand how to deploy my apps on Kubernetes for the first time to host Mirage, an AI API service that I run, I ended up making myself a promise that the next app I work on would be using a more productive & simpler<li><a href=https://jcs.org/2023/10/25/wifi_da target=_blank rel=noopener>BlueSCSI Wi-Fi Desk Accessory 1.0 Released</a> — <a href=https://jcs.org/>joshua stein</a><div>BlueSCSI Wi-Fi Desk Accessory
1161.0 has been released:
117wifi_da-1.0.sit
118(StuffIt 3 archive)
119SHA256: ccfc9d27dd5da7412d10cef73b81119a1fec3848e4d1d88ff652a07ffdc6a69aSHA1: ff124972f202ceda6d7fa4788110a67ccda6a13a
120This is the initial public release of my BlueSCSI Wi-Fi Desk Accessory for
121classic MacOS.<li><a href=https://michael.stapelberg.ch/posts/2023-10-25-my-all-flash-zfs-network-storage-build/ target=_blank rel=noopener>My 2023 all-flash ZFS NAS (Network Storage) build</a> — <a href=https://michael.stapelberg.ch/>Michael Stapelbergs Website</a><div>For over 10 years now, I run two self-built NAS (Network Storage) devices which serve media (currently via Jellyfin) and run daily backups of all my PCs and servers.
122In this article, I describe my goals, which hardware I picked for my new build (and why) and how I set it up.
123Design Goals
124I use my netw…</ul><p>Generated with <a href=https://git.sr.ht/~sircmpwn/openring target=_blank rel=noopener>openring</a>.</section><footer><hr><p><big><strong>Want to comment or have something to add?</strong></big><p>You can write me an email
125at <a href=mailto:mitja.felicijan@gmail.com>mitja.felicijan@gmail.com</a> or
126catch up with me <a href=https://telegram.me/mitjafelicijan target=_blank>on Telegram</a>.<hr><p>This website does not track you. Content is made available under the <a href=https://creativecommons.org/licenses/by/4.0/ target=_blank rel=noreferrer>CC BY 4.0 license</a> unless
127specified otherwise. Blog is also available as <a href=/index.xml target=_blank>RSS feed</a>.</footer><script>
128 window.va = window.va || function () { (window.vaq = window.vaq || []).push(arguments); };
129 </script><script defer src=/_vercel/insights/script.js></script> \ No newline at end of file
generated by cgit v1.2.3 (git 2.46.0) at 2026-04-30 21:39:42 +0000