aboutsummaryrefslogtreecommitdiff
path: root/public/digitalocean-spaces-to-sync-between-computers.html
diff options
context:
space:
mode:
Diffstat (limited to 'public/digitalocean-spaces-to-sync-between-computers.html')
-rwxr-xr-xpublic/digitalocean-spaces-to-sync-between-computers.html98
1 files changed, 0 insertions, 98 deletions
diff --git a/public/digitalocean-spaces-to-sync-between-computers.html b/public/digitalocean-spaces-to-sync-between-computers.html
deleted file mode 100755
index 33ce9da..0000000
--- a/public/digitalocean-spaces-to-sync-between-computers.html
+++ /dev/null
@@ -1,98 +0,0 @@
1<!doctype html><html lang=en-us><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><meta name=generator content="JBMAFP - github.com/mitjafelicijan/jbmafp"><link href="data:image/x-icon;base64,AAABAAEAEBAAAAEAIABoBAAAFgAAACgAAAAQAAAAIAAAAAEAIAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAL69vf8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAv76+/8LBwQkAAAAAAAAAAAAAAAC+vb3/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAL+9vf/Bv78JAAAAAAAAAAAAAAAAu7q6/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAC7ubr/vr29CAAAAAAAAAAAy8nJAZ6foP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAnqGj/6GipAoAAAAAHLjU/xcXHf/BwsL/I8XY/yPK3v8XGiD/IbjL/yPF2f8XGiD/Fxkf/yLF2f8gnK3/Fxog/62ztv8fwNf/FRcd/x271v8mz93/GRsi/xkXHf8p097/GiIp/xobIv8p0t3/KdPe/xocIv8fYmr/KNPe/xoZH/8aHCL/J87c/xy81/8VFxz/IsPZ/8zS0/8XGiD/Ir/R/yPH2/8XGiD/Fxkf/yPH2/8dd4T/GBog/yPJ3f8jyNr/uru9/xcUGv8cudb/EhITDKi5vRKlvMP/RUpOERwcHRAdOj4QHTk8EBwdHRAdNTgQHTo/EBwcHRAcHB0QSGduEKW4vf+koqQfHzg+EBqz0ewSFRv7EyMr/xq51vsTERb7ExUb+xq41fsau9j7ExUb+xiPp/sZudb7ExUb+xMVG/sZuNX/GKvI/BIUGfMdvdn/IrfL/xcaIP8n1eb/J9Dh/xkcIf8ZGR7/J8/f/xxCSv8ZGyH/J9Dg/ybQ4P8ZHCL/FSQs/yPK3/8UExj/GE1b/ybS5P8ZGB7/Ghwj/ynW5P8p2Ob/Ghwi/yWrtv8p1eH/Ghwi/xocIv8p1uT/J8XT/xkcIv8m1un/Hb7d/xUYH/8hzOr/HtHu/xcaIf8XGB//I8vi/xgxOv8XGSD/I8rg/yPK4P8XGiD/GUFL/yPP6f8SERj/Fhkh/x3A4f8AAAAAJ2f9/ydr//8mZPH/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAlYu38J2v//ydo/f8AAAAAAAAAAAd8/fkFqf//Iob8sAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMY39awWr//8FfP3/AAAAAAAAAAAFm/7/SfD//wR+/f8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAOB/f9B7v//BaX+/wAAAAAAAAAAQ878SAyZ/v9n1v4KAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADu9v8DDJb+/z3N/XgAAAAA3/sAAN/7AADf+wAA3/sAAAAAAAAAAAAAAAAAAN/7AAAAAAAAAAAAAAAAAAAAAAAAj/EAAI/5AACP8QAA3/sAAA==" rel=icon type=image/x-icon><title>Using Digitalocean Spaces to sync between computers</title><meta name=description content="I&amp;#39;ve been using Dropbox for probably 10+ yearsnow and I-ve became so used to it that it runs in the background that I don&amp;#39;teven imagine a world without it."><meta name=author content="Mitja Felicijan"><link rel=alternate type=application/rss+xml title="Mitja Felicijan's posts" href=https://mitjafelicijan.com/index.xml><link rel=alternate type=application/rss+xml title="Mitja Felicijan's notes" href=https://mitjafelicijan.com/notes.xml><style>:root{--border-color:gainsboro;--border-size:2px;--link-color:blue;--bg-color:#eee}*::selection{background:var(--link-color);color:#fff}*::-moz-selection{background:var(--link-color);color:#fff}*::-webkit-selection{background:var(--link-color);color:#fff}body{padding:2.5rem;max-width:1900px;background:#fff;font-family:sans-serif;line-height:1.35rem;font-size:16px}hr{border:0;border-bottom:var(--border-size)solid var(--border-color);margin-block-start:1.5rem}a{color:var(--link-color);text-decoration:none}h1,h2,h3{line-height:initial}h1{font-size:xx-large}footer{margin-block-start:2rem}cap{text-transform:capitalize}blockquote{font-style:italic}table{max-width:100%;border:var(--border-size)solid var(--border-color);border-collapse:separate;border-spacing:0}table thead tr th{border-bottom:var(--border-size)solid var(--border-color);text-align:left}table th,table td{padding:.5em .8em}ul.list li{padding:.2em 0}ul{line-height:1.35em}pre{text-wrap:nowrap;overflow-x:auto;padding:0 1em;border:var(--border-size)solid var(--border-color)}code{padding:0 3px;font-size:14px;border:0;background:var(--bg-color)}pre code{line-height:1.3em;background:#fff}pre,code,pre *,code *{font-family:monospace}figure{margin-inline-start:0;margin-inline-end:0}figcaption{width:800px;max-width:100%;text-align:center}figcaption p{margin:.3em 0 1.5em;font-style:italic}img,video,audio{width:800px;max-width:100%;border:var(--border-size)solid var(--border-color);padding:.5em}header nav{display:flex;gap:.9rem}article iframe{margin:0!important}audio::-webkit-media-controls-enclosure{border-radius:0}@media only screen and (max-width:600px){body{padding:.5em;word-wrap:break-word}header nav{gap:.7rem}header nav .hob{display:none}a{word-wrap:break-word}img,video,audio{padding:0}}</style><header><nav class=main itemscope itemtype=http://schema.org/SiteNavigationElement role=navigation aria-label="Main navigation"><a href=/>Home</a>
2<a href=/#posts>Posts</a>
3<a href=/#notes>Notes</a>
4<a href=/#sideprojects class=hob>Side Projects</a>
5<a href=/vault.html>Vault</a>
6<a href=https://github.com/mitjafelicijan target=_blank>Code</a>
7<a href=/mitjafelicijan.pgp.pub.txt target=_blank class=hob>PGP</a>
8<a href=/curriculum-vitae.html>CV</a>
9<a href=/index.xml target=_blank class=hob>RSS</a></nav></header><main role=main><article itemtype=http://schema.org/Article><h1 itemtype=headline>Using Digitalocean Spaces to sync between computers</h1><p><cap>post</cap>, Sep 9, 2020 on <a href=https://mitjafelicijan.com>Mitja Felicijan's blog</a><div><p>I've been using <a href=https://www.dropbox.com/>Dropbox</a> for probably <strong>10+ years</strong>
10now and I-ve became so used to it that it runs in the background that I don't
11even imagine a world without it. But it's not without problems.<p>At first I had problems with <code>.venv</code> environments for Python and the only
12solution for excluding synchronization for this folder was to manually exclude a
13specific folder which is not really scalable. FYI, my whole project folder is
14synced on <a href=https://www.dropbox.com/>Dropbox</a>. This of course introduced a lot
15of syncing of files and folders that are not needed or even break things on
16other machines. In the case of <strong>Python</strong>, I couldn't use that on my second
17machine. I needed to delete <code>.venv</code> folder and pip it again which synced files
18again to the main machine. This was very frustrating. <strong>Nodejs</strong> handles this
19much nicer and I can just run the scripts without deleting <code>node_modules</code> again
20and reinstalling. However, <code>node_modules</code> is a beast of its own. It creates so
21many files that OS has a problem counting them when you check the folder
22contents for size.<p>I wanted something similar to Dropbox. I could without the instant syncing but
23it would need to be fast and had the option for me to exclude folders like
24<code>node_modules, .venv, .git</code> and folders like that.<p>I went on a hunt for an alternative to <a href=https://www.dropbox.com/>Dropbox</a>
25and found:<ul><li><a href=https://tresorit.com/>Tresorit</a><li><a href=https://sync.com>Sync.com</a><li><a href=https://www.box.com/>Box</a></ul><p>You know, the usual list of suspects. I didn't include <a href=https://drive.google.com>Google
26drive</a> or <a href=https://onedrive.live.com/>One drive</a>
27since they are even more draconian than Dropbox.<blockquote><p>All this does not stem from me being paranoid but recently these companies
28have became more and more aggressive and they keep violating our privacy when
29they share our data with 3rd party services. It is getting out of control.</blockquote><p>So, my main problem was still there. No way of excluding a specific folder from
30syncing. And before we go into "<em>But you have git, isn't that enough?</em>", I must
31say, that many of the files (PDFs, spreadsheets, etc) I have in a <code>git</code> repo
32don't get pushed upstream to Git and I still want to have them synced across my
33computers.<p>I initially wanted to use <a href=https://linux.die.net/man/1/rsync>rsync</a> but I would
34need to then have a remote VPS or transfer between my computers directly. I
35wanted a solution where all my files could be accessible to me without my
36machine.<blockquote><p><strong>WARNING: This solution will cost you money!</strong> DigitalOcean Spaces are $5 per
37month and there are some bandwidth limitations and if you go beyond that you get
38billed additionally.</blockquote><p>Then I remembered that I could use something like
39<a href=https://en.wikipedia.org/wiki/Amazon_S3>S3</a> since it has versioning and is
40fully managed. I didn't want to go down the AWS rabbit hole with this so I
41choose <a href=https://www.digitalocean.com/products/spaces/>DigitalOcean Spaces</a>.<p>Then I needed a command-line tool to sync between source and target. I found
42this nice tool <a href=https://s3tools.org/s3cmd>s3cmd</a> and it is in the Ubuntu
43repositories.<pre tabindex=0 style=background-color:#fff><code><span style=display:flex><span>sudo apt install s3cmd
44</span></span></code></pre><p>After installation will I create a new Space bucket on DigitalOcean. Remember
45the zone you will choose because you will need it when you will configure
46<code>s3cmd</code>.<p>Then I visited <a href=https://cloud.digitalocean.com/account/api/tokens>Digitalocean Applications &
47API</a> and generated <strong>Spaces
48access keys</strong>. Save both key and secret somewhere safe because when you will
49leave the page secret will not be available anymore to you and you will need to
50re-generate it.<pre tabindex=0 style=background-color:#fff><code><span style=display:flex><span><span style=color:green># enter your key and secret and correct endpoint</span>
51</span></span><span style=display:flex><span><span style=color:green># my endpoint is ams3.digitaloceanspaces.com because</span>
52</span></span><span style=display:flex><span><span style=color:green># I created my bucket in Amsterdam regiin</span>
53</span></span><span style=display:flex><span>s3cmd --configure
54</span></span></code></pre><p>After that I played around with options for <code>s3cmd</code> and got to the following
55command.<pre tabindex=0 style=background-color:#fff><code><span style=display:flex><span><span style=color:green># I executed this command from my projects folder</span>
56</span></span><span style=display:flex><span>cd projects
57</span></span><span style=display:flex><span>s3cmd sync --delete-removed --exclude <span style=color:#a31515>&#39;node_modules/*&#39;</span> --exclude <span style=color:#a31515>&#39;.git/*&#39;</span> --exclude <span style=color:#a31515>&#39;.venv/*&#39;</span> ./ s3://my-bucket-name/projects/
58</span></span></code></pre><p>When syncing int he other direction you will need to change the order of the
59<code>SOURCE</code> and <code>TARGET</code> to <code>s3://my-bucket-name/projects/</code> and <code>./</code>.<blockquote><p>Be sure that all the paths have trailing slash so that sync knows that this
60are directories.</blockquote><p>I am planning to implement some sort of a <code>.ignore</code> file that will enable me to
61have a project-specific exclude options.<p>I am currently running this every hour as a cronjob which is perfectly fine for
62now when I am testing how this whole thing works and how it all will turn out.<p>I have also created a small Gnome extension which is still very unstable, but
63when/if this whole experiment pays of I will share on Github.</div></article></main><section><hr><h2>Posts from blogs I follow around the net</h2><ul><li><a href=https://utcc.utoronto.ca/~cks/space/blog/linux/NFSv4ServerLockClients target=_blank rel=noopener>Finding which NFSv4 client owns a lock on a Linux NFS(v4) server</a> — <a href=https://utcc.utoronto.ca/~cks/space/blog/>Chris's Wiki :: blog</a><div>A while back I wrote an entry about finding which NFS client owns
64a lock on a Linux NFS server, which turned
65out to be specific to NFS v3 (which I really should have seen coming,
66since it involved NLM and lockd). Finding the NFS v4 client that
67owns a lock is, depending on your perspective, either simpl…<li><a href=http://www.landley.net/notes-2023.html#28-10-2023 target=_blank rel=noopener>October 28, 2023</a> — <a href=http://www.landley.net/notes-2023.html>Rob Landley's Blog Thing for 2023</a><div>Oh good grief, two of my least favorite licensing people, Larry Rosen
68and Bradley Kuhn, are interacting on the OSI's license-discuss
69list where the're doing
70bad computer history and insisting that a guy Larry Rosen
71coincidentally interviewed for a book years ago is clearly the origin of
72somethin…<li><a href="http://offbeatpursuit.com:80/blog/?id=25" target=_blank rel=noopener>A fix by any other name</a> — <a href=http://offbeatpursuit.com:80/blog/>WLOG - blog</a><div>tags:
73i2c, plan9
74Another month, another file system.
75Well, if you can’t fix it in software, fix it in hardware (looking at
76you, bme680, we’re not
77done yet). The show must go on, as they say, and I would like my
78experiments to go on.
79So a “new” addition to the environmental sensor family connected to
80the h…<li><a href=https://mirzapandzo.com/next-image-url-parameter-is-valid-but-upstream-response-is-invalid target=_blank rel=noopener>Next/Image "url" parameter is valid but upstream response is invalid</a> — <a href=https://mirzapandzo.com/>Mirza Pandzo's Blog</a><div>Getting "url" parameter is valid but upstream response is invalid error with Next/Image on WSL2<li><a href=https://drewdevault.com/2023/10/13/Going-off-script.html target=_blank rel=noopener>Going off-script</a> — <a href=https://drewdevault.com>Drew DeVault's blog</a><div>There is a phenomenon in society which I find quite bizarre. Upon our entry to
81this mortal coil, we are endowed with self-awareness, agency, and free will.
82Each of the 8 billion members of this human race represents a unique person, a
83unique worldview, and a unique agency. Yet, many of us have the sam…<li><a href=https://szymonkaliski.com/writing/2023-10-02-building-a-diy-pen-plotter/ target=_blank rel=noopener>Building a DIY Pen Plotter</a> — <a href=http://github.com/dylang/node-rss>Szymon Kaliski</a><div>This article documents my learnings from designing and building a DIY Pen Plotter during the summer of 2023.
84My ultimate goal is to build my…<li><a href=https://neil.computer/notes/chart-of-accounts-for-startups-and-saas-companies/ target=_blank rel=noopener>Chart of Accounts for Startups and SaaS Companies</a> — <a href=https://neil.computer/>Neil Panchal</a><div>Accounting is fundamental to starting a business. You need to have a basic understanding of accounting principles and essential bookkeeping. I had to learn it. There was no choice. For filing taxes, your CPA is going to ask you for an Income Statement (also known as P/L statement). If<li><a href=https://journal.valeriansaliou.name/deploy-a-nomad-cluster-on-alpine-linux-with-vultr/ target=_blank rel=noopener>Deploy a Nomad Cluster on Alpine Linux with Vultr</a> — <a href=https://journal.valeriansaliou.name/>Valerian Saliou</a><div>After spending countless hours trying to understand how to deploy my apps on Kubernetes for the first time to host Mirage, an AI API service that I run, I ended up making myself a promise that the next app I work on would be using a more productive & simpler<li><a href=https://jcs.org/2023/10/25/wifi_da target=_blank rel=noopener>BlueSCSI Wi-Fi Desk Accessory 1.0 Released</a> — <a href=https://jcs.org/>joshua stein</a><div>BlueSCSI Wi-Fi Desk Accessory
851.0 has been released:
86wifi_da-1.0.sit
87(StuffIt 3 archive)
88SHA256: ccfc9d27dd5da7412d10cef73b81119a1fec3848e4d1d88ff652a07ffdc6a69aSHA1: ff124972f202ceda6d7fa4788110a67ccda6a13a
89This is the initial public release of my BlueSCSI Wi-Fi Desk Accessory for
90classic MacOS.<li><a href=https://michael.stapelberg.ch/posts/2023-10-25-my-all-flash-zfs-network-storage-build/ target=_blank rel=noopener>My 2023 all-flash ZFS NAS (Network Storage) build</a> — <a href=https://michael.stapelberg.ch/>Michael Stapelbergs Website</a><div>For over 10 years now, I run two self-built NAS (Network Storage) devices which serve media (currently via Jellyfin) and run daily backups of all my PCs and servers.
91In this article, I describe my goals, which hardware I picked for my new build (and why) and how I set it up.
92Design Goals
93I use my netw…</ul><p>Generated with <a href=https://git.sr.ht/~sircmpwn/openring target=_blank rel=noopener>openring</a>.</section><footer><hr><p><big><strong>Want to comment or have something to add?</strong></big><p>You can write me an email
94at <a href=mailto:mitja.felicijan@gmail.com>mitja.felicijan@gmail.com</a> or
95catch up with me <a href=https://telegram.me/mitjafelicijan target=_blank>on Telegram</a>.<hr><p>This website does not track you. Content is made available under the <a href=https://creativecommons.org/licenses/by/4.0/ target=_blank rel=noreferrer>CC BY 4.0 license</a> unless
96specified otherwise. Blog is also available as <a href=/index.xml target=_blank>RSS feed</a>.</footer><script>
97 window.va = window.va || function () { (window.vaq = window.vaq || []).push(arguments); };
98 </script><script defer src=/_vercel/insights/script.js></script> \ No newline at end of file
generated by cgit v1.2.3 (git 2.46.0) at 2026-04-30 20:28:01 +0000