1---
  2title: "Bringing all of my projects together under one umbrella"
  3url: bringing-all-of-my-projects-together-under-one-umbrella.html
  4date: 2023-07-01T18:49:07+02:00
  5type: post
  6draft: true
  7---
  8
  9## What is the issue anyway?
 10
 11Over the years, I have accumulated a bunch of virtual servers on my
 12[DigitalOcean](https://www.digitalocean.com/) account for small experimental
 13projects I dabble in. And this has resulted in quite a bill. I mean, I wouldn't
 14care if these projects were actually being used. But there were just being there
 15unused and wasting resources. Which makes this an unnecessary burden for me.
 16
 17Most of them are just small HTML pages that have an endpoint or two to read data
 18from or to, and for that reason I wrote servers left and right. To be honest,
 19all of those things could have been done with [CGI
 20scripts](https://en.wikipedia.org/wiki/Common_Gateway_Interface) and that would
 21have been more than enough.
 22
 23Recently, I decided to stop language hopping and focus on a simpler stack which
 24includes C, Go and Lua. And I can accomplish all the things I am interested in.
 25
 26## Finding a web server replacement
 27
 28Usually I had [Nginx](https://nginx.org/en/) in front of these small web servers
 29and I had to manage SSL certificates and all that jazz. I am bored with these
 30things. I don't want to manage any of this bullshit anymore.
 31
 32So the logical move forward was to find a solid alternative for this. I have
 33ended up on [Caddy server](https://caddyserver.com/). I've used it in the past
 34but kind of forgotten about it. What I really like about it is an ease of use
 35and a bunch of out of the box functionalities that come with it.
 36
 37These are the _pitch_ points from their website:
 38
 39- **Secure by Default**: Caddy is the only web server that uses HTTPS by
 40  default. A hardened TLS stack with modern protocols preserves privacy and
 41  exposes MITM attacks.
 42- **Config API**: As its primary mode of configuration, Caddy's REST API makes
 43  it easy to automate and integrate with your apps.
 44- **No Dependencies**: Because Caddy is written in Go, its binaries are entirely
 45  self-contained and run on every platform, including containers without libc.
 46- **Modular Stack**: Take back control over your compute edge. Caddy can be
 47  extended with everything you need using plugins.
 48
 49I had just a few requirements:
 50
 51- Automatic SSL
 52- Static file server
 53- Basic authentication
 54- CGI script support
 55
 56And the vanilla version does all of it, but CGI scripts. But that can easily be
 57fixed with their modular approach. You can do this on their website and build a
 58custom version of the server, or do it with Docker.
 59
 60This is a `Dockerfile` I used to build a custom server.
 61
 62```Dockerfile
 63FROM caddy:builder AS builder
 64
 65RUN xcaddy build \
 66  --with github.com/aksdb/caddy-cgi
 67
 68FROM caddy:latest
 69RUN apk add --no-cache nano
 70
 71COPY --from=builder /usr/bin/caddy /usr/bin/caddy
 72```
 73
 74## Getting rid of all the unnecessary virtual machines
 75
 76The next step was to get a handle on the number of virtual servers I have all
 77over the place.
 78
 79I decided to move all the projects and services into two main VMs:
 80
 81- personal server (still Nginx)
 82  - git server
 83  - static file server
 84  - personal blog
 85- projects server (Caddy server)
 86  - personal experiments
 87  - other projects
 88
 89I will focus on projects' server in this post since it's more interesting.
 90
 91## Testing CGI scripts
 92
 93The first thing I tested was how CGI scripts work under Caddy. This is
 94particularly import to me because almost all of my experiments and mini projects
 95need this to work.
 96
 97To configure Caddy server, you must provide the server with a configuration
 98file. By default, it's called `Caaddyfile`.
 99
100```caddyfile
101{
102  order cgi before respond
103}
104
105examples.mitjafelicijan.com {
106  cgi /bash-test /opt/projects/examples/bash-test.sh
107  cgi /tcl-test /opt/projects/examples/tcl-test.tcl
108  cgi /lua-test /opt/projects/examples/lua-test.lua
109  cgi /python-test /opt/projects/examples/python-test.py
110
111  root * /opt/projects/examples
112  file_server
113}
114```
115
116- The order is very important. Make sure that `order cgi before respond` is at
117  the top of the configuration file.
118- Also, when you run with Caddy v2, make sure you provide `adapter` argument
119  like this `/usr/bin/caddy run --watch --environ --config /etc/caddy/Caddyfile
120  --adapter caddyfile`. Otherwise, Caddy will try to use a different format for
121  config file.
122
123I did a small batch of tests with [Bash](https://www.gnu.org/software/bash/),
124[Tcl](https://www.tcl-lang.org/), [Lua](https://www.lua.org/) and
125[Python](https://www.python.org/). Here is a cheat sheet if you need it.
126
127Let's get Bash out of the way first.
128
129```bash
130#!/usr/bin/bash
131
132printf "Content-type: text/plain\n\n"
133
134printf "Hello from Bash\n\n"
135printf "PATH_INFO     [%s]\n" $PATH_INFO
136printf "QUERY_STRING  [%s]\n" $QUERY_STRING
137printf "\n"
138
139for i in {0..9..1}; do
140  printf "> %s\n" $i
141done
142
143exit 0
144```
145
146This one is for Tcl script.
147
148```tcl
149#!/usr/bin/tclsh
150
151puts "Content-type: text/plain\n"
152
153puts "Hello from Tcl\n"
154puts "PATH_INFO     \[$env(PATH_INFO)\]"
155puts "QUERY_STRING  \[$env(QUERY_STRING)\]"
156puts ""
157
158for {set i 0} {$i < 10} {incr i} {
159  puts "> $i"
160}
161```
162
163And for all you Python enjoyers.
164
165```python
166#!/usr/bin/python3
167
168import os
169
170print("Content-type: text/plain\n")
171
172print("Hello from Python\n")
173print("PATH_INFO     [{}]".format(os.environ['PATH_INFO']))
174print("QUERY_STRING  [{}]".format(os.environ['QUERY_STRING']))
175print("")
176
177for i in range(10):
178  print("> {}".format(i))
179```
180
181And for the final example, Lua.
182
183```lua
184#!/usr/bin/lua
185
186print("Content-type: text/plain\n")
187
188print("Hello from Lua\n")
189print(string.format("PATH_INFO     [%s]", os.getenv("PATH_INFO")))
190print(string.format("QUERY_STRING  [%s]", os.getenv("QUERY_STRING")))
191print()
192
193for i = 0, 9 do
194  print(string.format("> %d", i))
195end
196```
197
198## Basic authentication
199
200One thing was also to have an option for some sort of authentication, and
201something like [Basic access
202authentication](https://en.wikipedia.org/wiki/Basic_access_authentication) would
203be more than enough.
204
205Thankfully, Caddy supports this out of the box already. Below is an updated
206example.
207
208```Caddyfile
209{
210    order cgi before respond
211}
212
213examples.mitjafelicijan.com {
214  cgi /bash-test /opt/projects/examples/bash-test.sh
215  cgi /tcl-test /opt/projects/examples/tcl-test.tcl
216  cgi /lua-test /opt/projects/examples/lua-test.lua
217  cgi /python-test /opt/projects/examples/python-test.py
218
219  root * /opt/projects/examples
220  file_server
221
222  basicauth * {
223    bob $2a$14$/wCgaf9oMnmQa20txB76u.nI1AldGMBT/1J7fXCfgOiRShwz/JOkK
224  }
225}
226```
227
228`basicauth *` matches everything under this domain/sub-domain and protects it
229with Basic Authentication.
230
231- `bob` is the username
232- `hash` is the password
233
234To generate these passwords, execute `caddy hash-password` and this will prompt
235you to insert a password twice and spit out a hashed password that you can put
236in your configuration file.
237
238Restart the server and you are ready to go.
239
240## Making Caddy a service with systemd
241
242After the tests were successful, I copied `caddy` to `/usr/bin/caddy` and copied
243`Caddyfile` to `/etc/caddy/Caddyfile`.
244
245Now off to the systemd. Each systemd service requires you to create a service
246file.
247
248- I created a `/etc/systemd/system/caddy.service` and put the following content
249  in the file.
250
251```systemd
252[Unit]
253Description=Caddy
254Documentation=https://caddyserver.com/docs/
255After=network.target network-online.target
256Requires=network-online.target
257
258[Service]
259Type=notify
260User=root
261Group=root
262ExecStart=/usr/bin/caddy run --environ --config /etc/caddy/Caddyfile --adapter caddyfile
263ExecReload=/usr/bin/caddy reload --config /etc/caddy/Caddyfile --force --adapter caddyfile
264TimeoutStopSec=5s
265LimitNOFILE=1048576
266LimitNPROC=512
267PrivateTmp=true
268ProtectSystem=full
269AmbientCapabilities=CAP_NET_ADMIN CAP_NET_BIND_SERVICE
270
271[Install]
272WantedBy=multi-user.target
273```
274
275- You might need to reload systemd with `systemctl daemon-reload`.
276- Then I enabled the service with `systemctl enable caddy.service`.
277- And then I started the service with `systemctl start caddy.service`.
278
279This was about all that I needed to do to get it running. Now I can easily add
280new subdomains and domains to the main configuration file and be done with
281it. No manual Let's Encrypt shenanigans needed.