diff options
Diffstat (limited to 'public/digitalocean-spaces-to-sync-between-computers.html')
| -rwxr-xr-x | public/digitalocean-spaces-to-sync-between-computers.html | 65 |
1 files changed, 0 insertions, 65 deletions
diff --git a/public/digitalocean-spaces-to-sync-between-computers.html b/public/digitalocean-spaces-to-sync-between-computers.html deleted file mode 100755 index 04b7708..0000000 --- a/public/digitalocean-spaces-to-sync-between-computers.html +++ /dev/null | |||
| @@ -1,65 +0,0 @@ | |||
| 1 | <!doctype html><html lang=en-us><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><link href="data:image/x-icon;base64,AAABAAEAEBAAAAEAIABoBAAAFgAAACgAAAAQAAAAIAAAAAEAIAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAL69vf8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAv76+/8LBwQkAAAAAAAAAAAAAAAC+vb3/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAL+9vf/Bv78JAAAAAAAAAAAAAAAAu7q6/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAC7ubr/vr29CAAAAAAAAAAAy8nJAZ6foP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAnqGj/6GipAoAAAAAHLjU/xcXHf/BwsL/I8XY/yPK3v8XGiD/IbjL/yPF2f8XGiD/Fxkf/yLF2f8gnK3/Fxog/62ztv8fwNf/FRcd/x271v8mz93/GRsi/xkXHf8p097/GiIp/xobIv8p0t3/KdPe/xocIv8fYmr/KNPe/xoZH/8aHCL/J87c/xy81/8VFxz/IsPZ/8zS0/8XGiD/Ir/R/yPH2/8XGiD/Fxkf/yPH2/8dd4T/GBog/yPJ3f8jyNr/uru9/xcUGv8cudb/EhITDKi5vRKlvMP/RUpOERwcHRAdOj4QHTk8EBwdHRAdNTgQHTo/EBwcHRAcHB0QSGduEKW4vf+koqQfHzg+EBqz0ewSFRv7EyMr/xq51vsTERb7ExUb+xq41fsau9j7ExUb+xiPp/sZudb7ExUb+xMVG/sZuNX/GKvI/BIUGfMdvdn/IrfL/xcaIP8n1eb/J9Dh/xkcIf8ZGR7/J8/f/xxCSv8ZGyH/J9Dg/ybQ4P8ZHCL/FSQs/yPK3/8UExj/GE1b/ybS5P8ZGB7/Ghwj/ynW5P8p2Ob/Ghwi/yWrtv8p1eH/Ghwi/xocIv8p1uT/J8XT/xkcIv8m1un/Hb7d/xUYH/8hzOr/HtHu/xcaIf8XGB//I8vi/xgxOv8XGSD/I8rg/yPK4P8XGiD/GUFL/yPP6f8SERj/Fhkh/x3A4f8AAAAAJ2f9/ydr//8mZPH/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAlYu38J2v//ydo/f8AAAAAAAAAAAd8/fkFqf//Iob8sAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMY39awWr//8FfP3/AAAAAAAAAAAFm/7/SfD//wR+/f8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAOB/f9B7v//BaX+/wAAAAAAAAAAQ878SAyZ/v9n1v4KAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADu9v8DDJb+/z3N/XgAAAAA3/sAAN/7AADf+wAA3/sAAAAAAAAAAAAAAAAAAN/7AAAAAAAAAAAAAAAAAAAAAAAAj/EAAI/5AACP8QAA3/sAAA==" rel=icon type=image/x-icon><title>Using Digitalocean Spaces to sync between computers</title><meta name=description content="I&#39;ve been using Dropbox for probably 10+ yearsnow and I-ve became so used to it that it runs in the background that I don&#39;teven imagine a world without it."><link rel=alternate type=application/rss+xml title="Mitja Felicijan's posts" href=https://mitjafelicijan.com/index.xml><link rel=alternate type=application/rss+xml title="Mitja Felicijan's notes" href=https://mitjafelicijan.com/notes.xml><style>body{padding:1rem;max-width:760px;background:#fff;font-family:times new roman,Times,serif;line-height:1.35rem}hr{margin-block-start:1.5rem}h1,h2,h3{line-height:initial}footer{margin-block-start:3rem}table{max-width:100%;border-collapse:separate;border-spacing:2px;border:1px solid #000;border-left:1px solid #999;border-top:1px solid #999}blockquote{font-style:italic}table thead{background:#eee}td,th{border:1px solid #000;padding:4px;border-right:1px solid #999;border-bottom:1px solid #999;text-align:left}pre{text-wrap:nowrap;overflow-x:auto;margin-block-start:1.5rem;margin-block-end:1.5rem;padding:.5rem 0;border-top:1px solid #000;border-bottom:1px solid #000}pre code{line-height:1.3em}pre,code,pre *,code *{font-family:monospace;font-size:initial!important}img,video,audio{max-width:100%}header{display:flex;flex-direction:row;gap:3rem}nav{display:flex;gap:.75rem}.pstatus-orange{background:gold}.pstatus-green{background:#9acd32}.pstatus-red{background:#cd5c5c}@media only screen and (max-width:600px){header{flex-direction:column;gap:1rem}a{word-wrap:break-word}}</style><header><nav class=main><a href=/>Home</a> | ||
| 2 | <a href=https://git.mitjafelicijan.com/ target=_blank>Git</a> | ||
| 3 | <a href=https://files.mitjafelicijan.com/ target=_blank>Files</a> | ||
| 4 | <a href=/mitjafelicijan.pgp.pub.txt target=_blank>PGP</a> | ||
| 5 | <a href=/curriculum-vitae.html>CV</a> | ||
| 6 | <a href=/index.xml target=_blank>RSS</a></nav></header><main><div><h1>Using Digitalocean Spaces to sync between computers</h1><p>Sep 9, 2020<div><p>I've been using <a href=https://www.dropbox.com/>Dropbox</a> for probably <strong>10+ years</strong> | ||
| 7 | now and I-ve became so used to it that it runs in the background that I don't | ||
| 8 | even imagine a world without it. But it's not without problems.<p>At first I had problems with <code>.venv</code> environments for Python and the only | ||
| 9 | solution for excluding synchronization for this folder was to manually exclude a | ||
| 10 | specific folder which is not really scalable. FYI, my whole project folder is | ||
| 11 | synced on <a href=https://www.dropbox.com/>Dropbox</a>. This of course introduced a lot | ||
| 12 | of syncing of files and folders that are not needed or even break things on | ||
| 13 | other machines. In the case of <strong>Python</strong>, I couldn't use that on my second | ||
| 14 | machine. I needed to delete <code>.venv</code> folder and pip it again which synced files | ||
| 15 | again to the main machine. This was very frustrating. <strong>Nodejs</strong> handles this | ||
| 16 | much nicer and I can just run the scripts without deleting <code>node_modules</code> again | ||
| 17 | and reinstalling. However, <code>node_modules</code> is a beast of its own. It creates so | ||
| 18 | many files that OS has a problem counting them when you check the folder | ||
| 19 | contents for size.<p>I wanted something similar to Dropbox. I could without the instant syncing but | ||
| 20 | it would need to be fast and had the option for me to exclude folders like | ||
| 21 | <code>node_modules, .venv, .git</code> and folders like that.<p>I went on a hunt for an alternative to <a href=https://www.dropbox.com/>Dropbox</a> | ||
| 22 | and found:<ul><li><a href=https://tresorit.com/>Tresorit</a><li><a href=https://sync.com>Sync.com</a><li><a href=https://www.box.com/>Box</a></ul><p>You know, the usual list of suspects. I didn't include <a href=https://drive.google.com>Google | ||
| 23 | drive</a> or <a href=https://onedrive.live.com/>One drive</a> | ||
| 24 | since they are even more draconian than Dropbox.<blockquote><p>All this does not stem from me being paranoid but recently these companies | ||
| 25 | have became more and more aggressive and they keep violating our privacy when | ||
| 26 | they share our data with 3rd party services. It is getting out of control.</blockquote><p>So, my main problem was still there. No way of excluding a specific folder from | ||
| 27 | syncing. And before we go into "<em>But you have git, isn't that enough?</em>", I must | ||
| 28 | say, that many of the files (PDFs, spreadsheets, etc) I have in a <code>git</code> repo | ||
| 29 | don't get pushed upstream to Git and I still want to have them synced across my | ||
| 30 | computers.<p>I initially wanted to use <a href=https://linux.die.net/man/1/rsync>rsync</a> but I would | ||
| 31 | need to then have a remote VPS or transfer between my computers directly. I | ||
| 32 | wanted a solution where all my files could be accessible to me without my | ||
| 33 | machine.<blockquote><p><strong>WARNING: This solution will cost you money!</strong> DigitalOcean Spaces are $5 per | ||
| 34 | month and there are some bandwidth limitations and if you go beyond that you get | ||
| 35 | billed additionally.</blockquote><p>Then I remembered that I could use something like | ||
| 36 | <a href=https://en.wikipedia.org/wiki/Amazon_S3>S3</a> since it has versioning and is | ||
| 37 | fully managed. I didn't want to go down the AWS rabbit hole with this so I | ||
| 38 | choose <a href=https://www.digitalocean.com/products/spaces/>DigitalOcean Spaces</a>.<p>Then I needed a command-line tool to sync between source and target. I found | ||
| 39 | this nice tool <a href=https://s3tools.org/s3cmd>s3cmd</a> and it is in the Ubuntu | ||
| 40 | repositories.<pre tabindex=0 style=background-color:#fff><code><span style=display:flex><span>sudo apt install s3cmd | ||
| 41 | </span></span></code></pre><p>After installation will I create a new Space bucket on DigitalOcean. Remember | ||
| 42 | the zone you will choose because you will need it when you will configure | ||
| 43 | <code>s3cmd</code>.<p>Then I visited <a href=https://cloud.digitalocean.com/account/api/tokens>Digitalocean Applications & | ||
| 44 | API</a> and generated <strong>Spaces | ||
| 45 | access keys</strong>. Save both key and secret somewhere safe because when you will | ||
| 46 | leave the page secret will not be available anymore to you and you will need to | ||
| 47 | re-generate it.<pre tabindex=0 style=background-color:#fff><code><span style=display:flex><span><span style=color:green># enter your key and secret and correct endpoint</span> | ||
| 48 | </span></span><span style=display:flex><span><span style=color:green># my endpoint is ams3.digitaloceanspaces.com because</span> | ||
| 49 | </span></span><span style=display:flex><span><span style=color:green># I created my bucket in Amsterdam regiin</span> | ||
| 50 | </span></span><span style=display:flex><span>s3cmd --configure | ||
| 51 | </span></span></code></pre><p>After that I played around with options for <code>s3cmd</code> and got to the following | ||
| 52 | command.<pre tabindex=0 style=background-color:#fff><code><span style=display:flex><span><span style=color:green># I executed this command from my projects folder</span> | ||
| 53 | </span></span><span style=display:flex><span>cd projects | ||
| 54 | </span></span><span style=display:flex><span>s3cmd sync --delete-removed --exclude <span style=color:#a31515>'node_modules/*'</span> --exclude <span style=color:#a31515>'.git/*'</span> --exclude <span style=color:#a31515>'.venv/*'</span> ./ s3://my-bucket-name/projects/ | ||
| 55 | </span></span></code></pre><p>When syncing int he other direction you will need to change the order of the | ||
| 56 | <code>SOURCE</code> and <code>TARGET</code> to <code>s3://my-bucket-name/projects/</code> and <code>./</code>.<blockquote><p>Be sure that all the paths have trailing slash so that sync knows that this | ||
| 57 | are directories.</blockquote><p>I am planning to implement some sort of a <code>.ignore</code> file that will enable me to | ||
| 58 | have a project-specific exclude options.<p>I am currently running this every hour as a cronjob which is perfectly fine for | ||
| 59 | now when I am testing how this whole thing works and how it all will turn out.<p>I have also created a small Gnome extension which is still very unstable, but | ||
| 60 | when/if this whole experiment pays of I will share on Github.</div></div></main><footer><hr><div><h3>Want to comment or have something to add?</h3>You can write me an email at | ||
| 61 | <a href=mailto:m@mitjafelicijan.com>m@mitjafelicijan.com</a> or catch up | ||
| 62 | with me | ||
| 63 | <a href=https://telegram.me/mitjafelicijan target=_blank>on Telegram</a>.</div><hr><p>This website does not track you. Content is made available under | ||
| 64 | the <a href=https://creativecommons.org/licenses/by/4.0/ target=_blank rel=noreferrer>CC BY 4.0 license</a> unless specified | ||
| 65 | otherwise. Blog feed is available as <a href=/index.xml target=_blank>RSS feed</a>.</footer><script src=https://cdn.usefathom.com/script.js data-site=XHQARKXP defer></script> \ No newline at end of file | ||