From 8697555125c57ae64a0c9b78514b4aac4fd523de Mon Sep 17 00:00:00 2001 From: Mitja Felicijan Date: Tue, 27 Jun 2023 14:50:20 +0200 Subject: Massive formatting and added figcaption --- content/posts/2020-09-09-digitalocean-sync.md | 83 ++++++++++++++------------- 1 file changed, 42 insertions(+), 41 deletions(-) (limited to 'content/posts/2020-09-09-digitalocean-sync.md') diff --git a/content/posts/2020-09-09-digitalocean-sync.md b/content/posts/2020-09-09-digitalocean-sync.md index 2eeea2e..aa3cce4 100644 --- a/content/posts/2020-09-09-digitalocean-sync.md +++ b/content/posts/2020-09-09-digitalocean-sync.md @@ -5,25 +5,25 @@ date: 2020-09-09T12:00:00+02:00 draft: false --- -I've been using [Dropbox](https://www.dropbox.com/) for probably **10+ years** -now and I-ve became so used to it that it runs in the background that I don't +I've been using [Dropbox](https://www.dropbox.com/) for probably **10+ years** +now and I-ve became so used to it that it runs in the background that I don't even imagine a world without it. But it's not without problems. -At first I had problems with `.venv` environments for Python and the only -solution for excluding synchronization for this folder was to manually exclude -a specific folder which is not really scalable. FYI, my whole project folder is -synced on [Dropbox](https://www.dropbox.com/). This of course introduced a lot -of syncing of files and folders that are not needed or even break things on -other machines. In the case of **Python**, I couldn't use that on my second -machine. I needed to delete `.venv` folder and pip it again which synced files +At first I had problems with `.venv` environments for Python and the only +solution for excluding synchronization for this folder was to manually exclude a +specific folder which is not really scalable. FYI, my whole project folder is +synced on [Dropbox](https://www.dropbox.com/). This of course introduced a lot +of syncing of files and folders that are not needed or even break things on +other machines. In the case of **Python**, I couldn't use that on my second +machine. I needed to delete `.venv` folder and pip it again which synced files again to the main machine. This was very frustrating. **Nodejs** handles this much nicer and I can just run the scripts without deleting `node_modules` again and reinstalling. However, `node_modules` is a beast of its own. It creates so -many files that OS has a problem counting them when you check the folder +many files that OS has a problem counting them when you check the folder contents for size. -I wanted something similar to Dropbox. I could without the instant syncing -but it would need to be fast and had the option for me to exclude folders like +I wanted something similar to Dropbox. I could without the instant syncing but +it would need to be fast and had the option for me to exclude folders like `node_modules, .venv, .git` and folders like that. I went on a hunt for an alternative to [Dropbox](https://www.dropbox.com/) @@ -33,48 +33,51 @@ and found: - [Sync.com](https://sync.com) - [Box](https://www.box.com/) -You know, the usual list of suspects. I didn't include [Google drive](https://drive.google.com) -or [One drive](https://onedrive.live.com/) since they are even more draconian -than Dropbox. +You know, the usual list of suspects. I didn't include [Google +drive](https://drive.google.com) or [One drive](https://onedrive.live.com/) +since they are even more draconian than Dropbox. > All this does not stem from me being paranoid but recently these companies > have became more and more aggressive and they keep violating our privacy when > they share our data with 3rd party services. It is getting out of control. -So, my main problem was still there. No way of excluding a specific folder -from syncing. And before we go into "*But you have git, isn't that enough?*", -I must say, that many of the files (PDFs, spreadsheets, etc) I have in a `git` -repo don't get pushed upstream to Git and I still want to have them synced -across my computers. +So, my main problem was still there. No way of excluding a specific folder from +syncing. And before we go into "*But you have git, isn't that enough?*", I must +say, that many of the files (PDFs, spreadsheets, etc) I have in a `git` repo +don't get pushed upstream to Git and I still want to have them synced across my +computers. -I initially wanted to use [rsync](https://linux.die.net/man/1/rsync) but I -would need to then have a remote VPS or transfer between my computers directly. -I wanted a solution where all my files could be accessible to me without my +I initially wanted to use [rsync](https://linux.die.net/man/1/rsync) but I would +need to then have a remote VPS or transfer between my computers directly. I +wanted a solution where all my files could be accessible to me without my machine. -> **WARNING: This solution will cost you money!** DigitalOcean Spaces are $5 -per month and there are some bandwidth limitations and if you go beyond that -you get billed additionally. +> **WARNING: This solution will cost you money!** DigitalOcean Spaces are $5 per +month and there are some bandwidth limitations and if you go beyond that you get +billed additionally. -Then I remembered that I could use something like [S3](https://en.wikipedia.org/wiki/Amazon_S3) -since it has versioning and is fully managed. I didn't want to go down the AWS -rabbit hole with this so I choose [DigitalOcean Spaces](https://www.digitalocean.com/products/spaces/). +Then I remembered that I could use something like +[S3](https://en.wikipedia.org/wiki/Amazon_S3) since it has versioning and is +fully managed. I didn't want to go down the AWS rabbit hole with this so I +choose [DigitalOcean Spaces](https://www.digitalocean.com/products/spaces/). -Then I needed a command-line tool to sync between source and target. I found -this nice tool [s3cmd](https://s3tools.org/s3cmd) and it is in the Ubuntu +Then I needed a command-line tool to sync between source and target. I found +this nice tool [s3cmd](https://s3tools.org/s3cmd) and it is in the Ubuntu repositories. ```bash sudo apt install s3cmd ``` -After installation will I create a new Space bucket on DigitalOcean. Remember -the zone you will choose because you will need it when you will configure `s3cmd`. +After installation will I create a new Space bucket on DigitalOcean. Remember +the zone you will choose because you will need it when you will configure +`s3cmd`. -Then I visited [Digitalocean Applications & API](https://cloud.digitalocean.com/account/api/tokens) -and generated **Spaces access keys**. Save both key and secret somewhere safe -because when you will leave the page secret will not be available anymore to -you and you will need to re-generate it. +Then I visited [Digitalocean Applications & +API](https://cloud.digitalocean.com/account/api/tokens) and generated **Spaces +access keys**. Save both key and secret somewhere safe because when you will +leave the page secret will not be available anymore to you and you will need to +re-generate it. ```bash # enter your key and secret and correct endpoint @@ -101,10 +104,8 @@ When syncing int he other direction you will need to change the order of the I am planning to implement some sort of a `.ignore` file that will enable me to have a project-specific exclude options. -I am currently running this every hour as a cronjob which is perfectly fine -for now when I am testing how this whole thing works and how it all will turn -out. +I am currently running this every hour as a cronjob which is perfectly fine for +now when I am testing how this whole thing works and how it all will turn out. I have also created a small Gnome extension which is still very unstable, but when/if this whole experiment pays of I will share on Github. - -- cgit v1.2.3