aboutsummaryrefslogtreecommitdiff
path: root/_posts
diff options
context:
space:
mode:
Diffstat (limited to '_posts')
-rw-r--r--_posts/2015-11-10-software-development-pitfalls.md14
-rw-r--r--_posts/2017-01-12-gce-aws-docker-and-why-i-choose-classic-vms-and-digitalocean.md7
-rw-r--r--_posts/2017-03-07-golang-profiling-simplified.md9
-rw-r--r--_posts/2017-04-17-what-i-ve-learned-developing-ad-server.md9
-rw-r--r--_posts/2017-04-21-profiling-python-web-applications-with-visual-tools.md6
-rw-r--r--_posts/2017-08-11-simple-iot-application.md13
-rw-r--r--_posts/2018-01-16-using-digitalocean-spaces-object-storage-with-fuse.md21
7 files changed, 73 insertions, 6 deletions
diff --git a/_posts/2015-11-10-software-development-pitfalls.md b/_posts/2015-11-10-software-development-pitfalls.md
index 862ae08..69aa77d 100644
--- a/_posts/2015-11-10-software-development-pitfalls.md
+++ b/_posts/2015-11-10-software-development-pitfalls.md
@@ -4,6 +4,20 @@ title: Software development and my favorite pitfalls
4description: Couple of observations regarding project management. 4description: Couple of observations regarding project management.
5--- 5---
6 6
7**Table of content**
8
9- [Initial thoughts](#initial-thoughts)
10- [Ping emails](#ping-emails)
11- [Everybody is a project manager](#everybody-is-a-project-manager)
12- [We are never wrong](#we-are-never-wrong)
13- [Micromanaging](#micromanaging)
14- [Human contact - no need for it!](#human-contact---no-need-for-it)
15- [MVP is killing innovation](#mvp-is-killing-innovation)
16- [Pressure wasteland](#pressure-wasteland)
17- [Conclusion](#conclusion)
18
19## Initial thoughts
20
7Over the years I had privilege to work on some very excited projects both in software development field and also in electronics field and every experience taught me some invaluable lessons about how NOT TO approach development. And through this post I will try to point out some of the absurd outdated techniques I find the most annoying and damaging during a development cycle. There will be swearing because this topic really gets on my nerves and I never coherently tried to explain them in writing. So if I get heated up please bare with me :) 21Over the years I had privilege to work on some very excited projects both in software development field and also in electronics field and every experience taught me some invaluable lessons about how NOT TO approach development. And through this post I will try to point out some of the absurd outdated techniques I find the most annoying and damaging during a development cycle. There will be swearing because this topic really gets on my nerves and I never coherently tried to explain them in writing. So if I get heated up please bare with me :)
8 22
9As new methods of project management are emerging, underlaying processes still stay old and outdated. This is mainly because we as people are unable to completely shift away from this approaches. 23As new methods of project management are emerging, underlaying processes still stay old and outdated. This is mainly because we as people are unable to completely shift away from this approaches.
diff --git a/_posts/2017-01-12-gce-aws-docker-and-why-i-choose-classic-vms-and-digitalocean.md b/_posts/2017-01-12-gce-aws-docker-and-why-i-choose-classic-vms-and-digitalocean.md
index 3f202a8..17714b1 100644
--- a/_posts/2017-01-12-gce-aws-docker-and-why-i-choose-classic-vms-and-digitalocean.md
+++ b/_posts/2017-01-12-gce-aws-docker-and-why-i-choose-classic-vms-and-digitalocean.md
@@ -4,6 +4,13 @@ title: GCE, AWS, Docker and why I choose classic VM’s and DigitalOcean for my
4description: Reasons why I choose DigitalOcean for my project 4description: Reasons why I choose DigitalOcean for my project
5--- 5---
6 6
7**Table of content**
8
9- [Docker tools and complexity that comes with it](#docker-tools-and-complexity-that-comes-with-it)
10- [Lack of real life examples of Docker in action](#lack-of-real-life-examples-of-docker-in-action)
11- [Ease of deployment](#ease-of-deployment)
12- [Where to go from here](#where-to-go-from-here)
13
7I have been developing a product for the past few months and one of product’s requirement is the ability to automatically scale quickly on system’s demand. 14I have been developing a product for the past few months and one of product’s requirement is the ability to automatically scale quickly on system’s demand.
8 15
9As most of you probably know system design is much more important then actual code that will drive the product. And this was my main concern when developing this product. I have read anything I could get my hands on about Docker as it was hyped so much in media for the past two years. At a first glance Docker was ideal fit for this platform. But then as I started to seriously experiment with it and developing around it several problems occurred. Well, it would be unfair to call them problems but lets say drawbacks when developing rapidly. 16As most of you probably know system design is much more important then actual code that will drive the product. And this was my main concern when developing this product. I have read anything I could get my hands on about Docker as it was hyped so much in media for the past two years. At a first glance Docker was ideal fit for this platform. But then as I started to seriously experiment with it and developing around it several problems occurred. Well, it would be unfair to call them problems but lets say drawbacks when developing rapidly.
diff --git a/_posts/2017-03-07-golang-profiling-simplified.md b/_posts/2017-03-07-golang-profiling-simplified.md
index f74c7b2..5b9f6ed 100644
--- a/_posts/2017-03-07-golang-profiling-simplified.md
+++ b/_posts/2017-03-07-golang-profiling-simplified.md
@@ -4,6 +4,15 @@ title: Golang profiling simplified
4description: Golang profiling made easy 4description: Golang profiling made easy
5--- 5---
6 6
7**Table of content**
8
9- [Where are my pprof files?](#where-are-my-pprof-files)
10- [Why is my cpu profile empty?](#why-is-my-cpu-profile-empty)
11- [Profiling](#profiling)
12 - [Memory profiling](#memory-profiling)
13 - [CPU profiling](#cpu-profiling)
14 - [Generating profiling reports](#generating-profiling-reports)
15
7Many posts have been written regarding profiling in Golang and I haven’t found proper tutorial regarding this. Almost all of them are missing some part of important information and it gets pretty frustrating when you have a deadline and are not finding simple distilled solution. 16Many posts have been written regarding profiling in Golang and I haven’t found proper tutorial regarding this. Almost all of them are missing some part of important information and it gets pretty frustrating when you have a deadline and are not finding simple distilled solution.
8 17
9Nevertheless, after searching and experimenting I have found a solution that works for me and probably should also for you. 18Nevertheless, after searching and experimenting I have found a solution that works for me and probably should also for you.
diff --git a/_posts/2017-04-17-what-i-ve-learned-developing-ad-server.md b/_posts/2017-04-17-what-i-ve-learned-developing-ad-server.md
index 772fb98..5de4cf5 100644
--- a/_posts/2017-04-17-what-i-ve-learned-developing-ad-server.md
+++ b/_posts/2017-04-17-what-i-ve-learned-developing-ad-server.md
@@ -4,6 +4,15 @@ title: What I've learned developing ad server
4description: Lessons I learned developing contextual ad server 4description: Lessons I learned developing contextual ad server
5--- 5---
6 6
7**Table of content**
8
9- [Aggregate everything](#aggregate-everything)
10- [Measure everything](#measure-everything)
11- [Cache control is your friend](#cache-control-is-your-friend)
12- [Learn NGINX](#learn-nginx)
13- [Use Redis/Memcached](#use-redismemcached)
14- [Conclusion](#conclusion)
15
7For the past year and half I have been developing native advertising server that contextually matches ads and displays them in different template forms on variety of websites. This project grew from serving thousands of ads per day to millions. 16For the past year and half I have been developing native advertising server that contextually matches ads and displays them in different template forms on variety of websites. This project grew from serving thousands of ads per day to millions.
8 17
9The system is made from couple of core components: 18The system is made from couple of core components:
diff --git a/_posts/2017-04-21-profiling-python-web-applications-with-visual-tools.md b/_posts/2017-04-21-profiling-python-web-applications-with-visual-tools.md
index 5bbfe48..ca5dbdd 100644
--- a/_posts/2017-04-21-profiling-python-web-applications-with-visual-tools.md
+++ b/_posts/2017-04-21-profiling-python-web-applications-with-visual-tools.md
@@ -4,6 +4,12 @@ title: Profiling Python web applications with visual tools
4description: Missing link when debugging and profiling python web applications 4description: Missing link when debugging and profiling python web applications
5--- 5---
6 6
7**Table of content**
8
9- [Simple web-service](#simple-web-service)
10- [Visualize profile](#visualize-profile)
11 - [Update 2017-04-22](#update-2017-04-22)
12
7I have been profiling my software with KCachegrind for a long time now and I was missing this option when I am developing API's or other web services. I always knew that this is possible but never really took the time and dive into it. 13I have been profiling my software with KCachegrind for a long time now and I was missing this option when I am developing API's or other web services. I always knew that this is possible but never really took the time and dive into it.
8 14
9Before we begin there are some requirements. We will need to: 15Before we begin there are some requirements. We will need to:
diff --git a/_posts/2017-08-11-simple-iot-application.md b/_posts/2017-08-11-simple-iot-application.md
index f2bcabc..5774b66 100644
--- a/_posts/2017-08-11-simple-iot-application.md
+++ b/_posts/2017-08-11-simple-iot-application.md
@@ -4,6 +4,19 @@ title: Simple IOT application supported by real-time monitoring and data history
4description: Develop simple IOT application with Arduino MKR1000 and Python 4description: Develop simple IOT application with Arduino MKR1000 and Python
5--- 5---
6 6
7**Table of content**
8
9- [Initial thoughts](#initial-thoughts)
10- [Simple Python API](#simple-python-api)
11 - [Basic web application](#basic-web-application)
12 - [Web application security](#web-application-security)
13 - [Simple API for writing data-points](#simple-api-for-writing-data-points)
14- [Sending data to API with Arduino MKR1000](#sending-data-to-api-with-arduino-mkr1000)
15- [Data visualization](#data-visualization)
16- [Conclusion](#conclusion)
17
18## Initial thoughts
19
7I have been developing these kind of application for the better part of my last 5 years and people keep asking me how to approach developing such application and I will give a try explaining it here. 20I have been developing these kind of application for the better part of my last 5 years and people keep asking me how to approach developing such application and I will give a try explaining it here.
8 21
9IOT applications are really no different than any other kind of applications. We have data that needs to be collected and visualized in some form of tables or charts. The main difference here is that most of the times these data is collected by some kind of device foreign to developer that mainly operates in web domain. But fear not, it's not that different than writing some JavaScript. 22IOT applications are really no different than any other kind of applications. We have data that needs to be collected and visualized in some form of tables or charts. The main difference here is that most of the times these data is collected by some kind of device foreign to developer that mainly operates in web domain. But fear not, it's not that different than writing some JavaScript.
diff --git a/_posts/2018-01-16-using-digitalocean-spaces-object-storage-with-fuse.md b/_posts/2018-01-16-using-digitalocean-spaces-object-storage-with-fuse.md
index 7442956..6b89e2b 100644
--- a/_posts/2018-01-16-using-digitalocean-spaces-object-storage-with-fuse.md
+++ b/_posts/2018-01-16-using-digitalocean-spaces-object-storage-with-fuse.md
@@ -4,17 +4,26 @@ title: Using DigitalOcean Spaces Object Storage with FUSE
4description: Using DigitalOcean Spaces Object Storage with FUSE 4description: Using DigitalOcean Spaces Object Storage with FUSE
5--- 5---
6 6
7**Table of content**
8
9- [Is it possible to use them as a mounted drive with FUSE?](#is-it-possible-to-use-them-as-a-mounted-drive-with-fuse)
10- [Will the performance degrade over time and over different sizes of objects?](#will-the-performance-degrade-over-time-and-over-different-sizes-of-objects)
11 - [Measurement experiment 1: File copy](#measurement-experiment-1-file-copy)
12 - [Measurement experiment 2: SQLite performanse](#measurement-experiment-2-sqlite-performanse)
13- [Can storage be mounted on multiple machines at the same time and be writable?](#can-storage-be-mounted-on-multiple-machines-at-the-same-time-and-be-writable)
14- [Observations and conslusion](#observations-and-conslusion)
15
7Couple of months ago [DigitalOcean](https://www.digitalocean.com) introduced new product called [Spaces](https://blog.digitalocean.com/introducing-spaces-object-storage/) which is Object Storage very similar to Amazon's S3. This really peaked my interest, because this was something I was missing and even the thought of going over the internet for such functionality was in no interest to me. Also in fashion with their previous pricing this also is very cheap and pricing page is a no-brainer compared to AWS or GCE. [Prices are clearly and precisely defined and outlined](https://www.digitalocean.com/pricing/). You must love them for that :) 16Couple of months ago [DigitalOcean](https://www.digitalocean.com) introduced new product called [Spaces](https://blog.digitalocean.com/introducing-spaces-object-storage/) which is Object Storage very similar to Amazon's S3. This really peaked my interest, because this was something I was missing and even the thought of going over the internet for such functionality was in no interest to me. Also in fashion with their previous pricing this also is very cheap and pricing page is a no-brainer compared to AWS or GCE. [Prices are clearly and precisely defined and outlined](https://www.digitalocean.com/pricing/). You must love them for that :)
8 17
9### Initial requirements 18### Initial requirements
10 19
11* [Is it possible to use them as a mounted drive with FUSE?](#fuse) (tl;dr YES) 20* Is it possible to use them as a mounted drive with FUSE? (tl;dr YES)
12* [Will the performance degrade over time and over different sizes of objects?](#performanse) (tl;dr NO&YES) 21* Will the performance degrade over time and over different sizes of objects? (tl;dr NO&YES)
13* [Can storage be mounted on multiple machines at the same time and be writable?](#multiuser) (tl;dr YES) 22* Can storage be mounted on multiple machines at the same time and be writable? (tl;dr YES)
14 23
15> Let me be clear. This scripts I use are made just for benchmarking and are not intended to be used in real-life situations. Besides that, I am looking into using this approaches but adding caching service in front of it and then dumping everything as an object to storage. This could potentially be some interesting post of itself. But in case you would need real-time data without eventual consistency please take this scripts as they are: not usable in such situations. 24> Let me be clear. This scripts I use are made just for benchmarking and are not intended to be used in real-life situations. Besides that, I am looking into using this approaches but adding caching service in front of it and then dumping everything as an object to storage. This could potentially be some interesting post of itself. But in case you would need real-time data without eventual consistency please take this scripts as they are: not usable in such situations.
16 25
17## <a name="fuse"></a>Is it possible to use them as a mounted drive with FUSE? 26## Is it possible to use them as a mounted drive with FUSE?
18 27
19Well, actually they can be used in such manor. Because they are similar to [AWS S3](https://aws.amazon.com/s3/) many tools are available and you can find many articles and [Stackoverflow items](https://stackoverflow.com/search?q=s3+fuse). 28Well, actually they can be used in such manor. Because they are similar to [AWS S3](https://aws.amazon.com/s3/) many tools are available and you can find many articles and [Stackoverflow items](https://stackoverflow.com/search?q=s3+fuse).
20 29
@@ -67,7 +76,7 @@ Additional information on FUSE:
67* [Github project page for s3fs](https://github.com/s3fs-fuse/s3fs-fuse) 76* [Github project page for s3fs](https://github.com/s3fs-fuse/s3fs-fuse)
68* [FUSE - Filesystem in Userspace](https://en.wikipedia.org/wiki/Filesystem_in_Userspace) 77* [FUSE - Filesystem in Userspace](https://en.wikipedia.org/wiki/Filesystem_in_Userspace)
69 78
70## <a name="performanse"></a>Will the performance degrade over time and over different sizes of objects? 79## Will the performance degrade over time and over different sizes of objects?
71 80
72For this task I didn't want to just read and write text files or uploading images. I actually wanted to figure out if using something like SQlite is viable in this case. 81For this task I didn't want to just read and write text files or uploading images. I actually wanted to figure out if using something like SQlite is viable in this case.
73 82
@@ -249,7 +258,7 @@ You can download [raw result here](/files/sqlite-benchmarks.tsv). And again, the
249})(); 258})();
250</script> 259</script>
251 260
252## <a name="multiuser"></a>Can storage be mounted on multiple machines at the same time and be writable? 261## Can storage be mounted on multiple machines at the same time and be writable?
253 262
254Well, this one didn't take long to test. And the answer is **YES**. I mounted space on both machines and measured same performance on both machines. But because file is downloaded before write and then uploaded on complete there could potentially be problems is another process is trying to access the same file. 263Well, this one didn't take long to test. And the answer is **YES**. I mounted space on both machines and measured same performance on both machines. But because file is downloaded before write and then uploaded on complete there could potentially be problems is another process is trying to access the same file.
255 264