diff options
Diffstat (limited to '_posts')
7 files changed, 73 insertions, 6 deletions
diff --git a/_posts/2015-11-10-software-development-pitfalls.md b/_posts/2015-11-10-software-development-pitfalls.md index 862ae08..69aa77d 100644 --- a/_posts/2015-11-10-software-development-pitfalls.md +++ b/_posts/2015-11-10-software-development-pitfalls.md | |||
| @@ -4,6 +4,20 @@ title: Software development and my favorite pitfalls | |||
| 4 | description: Couple of observations regarding project management. | 4 | description: Couple of observations regarding project management. |
| 5 | --- | 5 | --- |
| 6 | 6 | ||
| 7 | **Table of content** | ||
| 8 | |||
| 9 | - [Initial thoughts](#initial-thoughts) | ||
| 10 | - [Ping emails](#ping-emails) | ||
| 11 | - [Everybody is a project manager](#everybody-is-a-project-manager) | ||
| 12 | - [We are never wrong](#we-are-never-wrong) | ||
| 13 | - [Micromanaging](#micromanaging) | ||
| 14 | - [Human contact - no need for it!](#human-contact---no-need-for-it) | ||
| 15 | - [MVP is killing innovation](#mvp-is-killing-innovation) | ||
| 16 | - [Pressure wasteland](#pressure-wasteland) | ||
| 17 | - [Conclusion](#conclusion) | ||
| 18 | |||
| 19 | ## Initial thoughts | ||
| 20 | |||
| 7 | Over the years I had privilege to work on some very excited projects both in software development field and also in electronics field and every experience taught me some invaluable lessons about how NOT TO approach development. And through this post I will try to point out some of the absurd outdated techniques I find the most annoying and damaging during a development cycle. There will be swearing because this topic really gets on my nerves and I never coherently tried to explain them in writing. So if I get heated up please bare with me :) | 21 | Over the years I had privilege to work on some very excited projects both in software development field and also in electronics field and every experience taught me some invaluable lessons about how NOT TO approach development. And through this post I will try to point out some of the absurd outdated techniques I find the most annoying and damaging during a development cycle. There will be swearing because this topic really gets on my nerves and I never coherently tried to explain them in writing. So if I get heated up please bare with me :) |
| 8 | 22 | ||
| 9 | As new methods of project management are emerging, underlaying processes still stay old and outdated. This is mainly because we as people are unable to completely shift away from this approaches. | 23 | As new methods of project management are emerging, underlaying processes still stay old and outdated. This is mainly because we as people are unable to completely shift away from this approaches. |
diff --git a/_posts/2017-01-12-gce-aws-docker-and-why-i-choose-classic-vms-and-digitalocean.md b/_posts/2017-01-12-gce-aws-docker-and-why-i-choose-classic-vms-and-digitalocean.md index 3f202a8..17714b1 100644 --- a/_posts/2017-01-12-gce-aws-docker-and-why-i-choose-classic-vms-and-digitalocean.md +++ b/_posts/2017-01-12-gce-aws-docker-and-why-i-choose-classic-vms-and-digitalocean.md | |||
| @@ -4,6 +4,13 @@ title: GCE, AWS, Docker and why I choose classic VM’s and DigitalOcean for my | |||
| 4 | description: Reasons why I choose DigitalOcean for my project | 4 | description: Reasons why I choose DigitalOcean for my project |
| 5 | --- | 5 | --- |
| 6 | 6 | ||
| 7 | **Table of content** | ||
| 8 | |||
| 9 | - [Docker tools and complexity that comes with it](#docker-tools-and-complexity-that-comes-with-it) | ||
| 10 | - [Lack of real life examples of Docker in action](#lack-of-real-life-examples-of-docker-in-action) | ||
| 11 | - [Ease of deployment](#ease-of-deployment) | ||
| 12 | - [Where to go from here](#where-to-go-from-here) | ||
| 13 | |||
| 7 | I have been developing a product for the past few months and one of product’s requirement is the ability to automatically scale quickly on system’s demand. | 14 | I have been developing a product for the past few months and one of product’s requirement is the ability to automatically scale quickly on system’s demand. |
| 8 | 15 | ||
| 9 | As most of you probably know system design is much more important then actual code that will drive the product. And this was my main concern when developing this product. I have read anything I could get my hands on about Docker as it was hyped so much in media for the past two years. At a first glance Docker was ideal fit for this platform. But then as I started to seriously experiment with it and developing around it several problems occurred. Well, it would be unfair to call them problems but lets say drawbacks when developing rapidly. | 16 | As most of you probably know system design is much more important then actual code that will drive the product. And this was my main concern when developing this product. I have read anything I could get my hands on about Docker as it was hyped so much in media for the past two years. At a first glance Docker was ideal fit for this platform. But then as I started to seriously experiment with it and developing around it several problems occurred. Well, it would be unfair to call them problems but lets say drawbacks when developing rapidly. |
diff --git a/_posts/2017-03-07-golang-profiling-simplified.md b/_posts/2017-03-07-golang-profiling-simplified.md index f74c7b2..5b9f6ed 100644 --- a/_posts/2017-03-07-golang-profiling-simplified.md +++ b/_posts/2017-03-07-golang-profiling-simplified.md | |||
| @@ -4,6 +4,15 @@ title: Golang profiling simplified | |||
| 4 | description: Golang profiling made easy | 4 | description: Golang profiling made easy |
| 5 | --- | 5 | --- |
| 6 | 6 | ||
| 7 | **Table of content** | ||
| 8 | |||
| 9 | - [Where are my pprof files?](#where-are-my-pprof-files) | ||
| 10 | - [Why is my cpu profile empty?](#why-is-my-cpu-profile-empty) | ||
| 11 | - [Profiling](#profiling) | ||
| 12 | - [Memory profiling](#memory-profiling) | ||
| 13 | - [CPU profiling](#cpu-profiling) | ||
| 14 | - [Generating profiling reports](#generating-profiling-reports) | ||
| 15 | |||
| 7 | Many posts have been written regarding profiling in Golang and I haven’t found proper tutorial regarding this. Almost all of them are missing some part of important information and it gets pretty frustrating when you have a deadline and are not finding simple distilled solution. | 16 | Many posts have been written regarding profiling in Golang and I haven’t found proper tutorial regarding this. Almost all of them are missing some part of important information and it gets pretty frustrating when you have a deadline and are not finding simple distilled solution. |
| 8 | 17 | ||
| 9 | Nevertheless, after searching and experimenting I have found a solution that works for me and probably should also for you. | 18 | Nevertheless, after searching and experimenting I have found a solution that works for me and probably should also for you. |
diff --git a/_posts/2017-04-17-what-i-ve-learned-developing-ad-server.md b/_posts/2017-04-17-what-i-ve-learned-developing-ad-server.md index 772fb98..5de4cf5 100644 --- a/_posts/2017-04-17-what-i-ve-learned-developing-ad-server.md +++ b/_posts/2017-04-17-what-i-ve-learned-developing-ad-server.md | |||
| @@ -4,6 +4,15 @@ title: What I've learned developing ad server | |||
| 4 | description: Lessons I learned developing contextual ad server | 4 | description: Lessons I learned developing contextual ad server |
| 5 | --- | 5 | --- |
| 6 | 6 | ||
| 7 | **Table of content** | ||
| 8 | |||
| 9 | - [Aggregate everything](#aggregate-everything) | ||
| 10 | - [Measure everything](#measure-everything) | ||
| 11 | - [Cache control is your friend](#cache-control-is-your-friend) | ||
| 12 | - [Learn NGINX](#learn-nginx) | ||
| 13 | - [Use Redis/Memcached](#use-redismemcached) | ||
| 14 | - [Conclusion](#conclusion) | ||
| 15 | |||
| 7 | For the past year and half I have been developing native advertising server that contextually matches ads and displays them in different template forms on variety of websites. This project grew from serving thousands of ads per day to millions. | 16 | For the past year and half I have been developing native advertising server that contextually matches ads and displays them in different template forms on variety of websites. This project grew from serving thousands of ads per day to millions. |
| 8 | 17 | ||
| 9 | The system is made from couple of core components: | 18 | The system is made from couple of core components: |
diff --git a/_posts/2017-04-21-profiling-python-web-applications-with-visual-tools.md b/_posts/2017-04-21-profiling-python-web-applications-with-visual-tools.md index 5bbfe48..ca5dbdd 100644 --- a/_posts/2017-04-21-profiling-python-web-applications-with-visual-tools.md +++ b/_posts/2017-04-21-profiling-python-web-applications-with-visual-tools.md | |||
| @@ -4,6 +4,12 @@ title: Profiling Python web applications with visual tools | |||
| 4 | description: Missing link when debugging and profiling python web applications | 4 | description: Missing link when debugging and profiling python web applications |
| 5 | --- | 5 | --- |
| 6 | 6 | ||
| 7 | **Table of content** | ||
| 8 | |||
| 9 | - [Simple web-service](#simple-web-service) | ||
| 10 | - [Visualize profile](#visualize-profile) | ||
| 11 | - [Update 2017-04-22](#update-2017-04-22) | ||
| 12 | |||
| 7 | I have been profiling my software with KCachegrind for a long time now and I was missing this option when I am developing API's or other web services. I always knew that this is possible but never really took the time and dive into it. | 13 | I have been profiling my software with KCachegrind for a long time now and I was missing this option when I am developing API's or other web services. I always knew that this is possible but never really took the time and dive into it. |
| 8 | 14 | ||
| 9 | Before we begin there are some requirements. We will need to: | 15 | Before we begin there are some requirements. We will need to: |
diff --git a/_posts/2017-08-11-simple-iot-application.md b/_posts/2017-08-11-simple-iot-application.md index f2bcabc..5774b66 100644 --- a/_posts/2017-08-11-simple-iot-application.md +++ b/_posts/2017-08-11-simple-iot-application.md | |||
| @@ -4,6 +4,19 @@ title: Simple IOT application supported by real-time monitoring and data history | |||
| 4 | description: Develop simple IOT application with Arduino MKR1000 and Python | 4 | description: Develop simple IOT application with Arduino MKR1000 and Python |
| 5 | --- | 5 | --- |
| 6 | 6 | ||
| 7 | **Table of content** | ||
| 8 | |||
| 9 | - [Initial thoughts](#initial-thoughts) | ||
| 10 | - [Simple Python API](#simple-python-api) | ||
| 11 | - [Basic web application](#basic-web-application) | ||
| 12 | - [Web application security](#web-application-security) | ||
| 13 | - [Simple API for writing data-points](#simple-api-for-writing-data-points) | ||
| 14 | - [Sending data to API with Arduino MKR1000](#sending-data-to-api-with-arduino-mkr1000) | ||
| 15 | - [Data visualization](#data-visualization) | ||
| 16 | - [Conclusion](#conclusion) | ||
| 17 | |||
| 18 | ## Initial thoughts | ||
| 19 | |||
| 7 | I have been developing these kind of application for the better part of my last 5 years and people keep asking me how to approach developing such application and I will give a try explaining it here. | 20 | I have been developing these kind of application for the better part of my last 5 years and people keep asking me how to approach developing such application and I will give a try explaining it here. |
| 8 | 21 | ||
| 9 | IOT applications are really no different than any other kind of applications. We have data that needs to be collected and visualized in some form of tables or charts. The main difference here is that most of the times these data is collected by some kind of device foreign to developer that mainly operates in web domain. But fear not, it's not that different than writing some JavaScript. | 22 | IOT applications are really no different than any other kind of applications. We have data that needs to be collected and visualized in some form of tables or charts. The main difference here is that most of the times these data is collected by some kind of device foreign to developer that mainly operates in web domain. But fear not, it's not that different than writing some JavaScript. |
diff --git a/_posts/2018-01-16-using-digitalocean-spaces-object-storage-with-fuse.md b/_posts/2018-01-16-using-digitalocean-spaces-object-storage-with-fuse.md index 7442956..6b89e2b 100644 --- a/_posts/2018-01-16-using-digitalocean-spaces-object-storage-with-fuse.md +++ b/_posts/2018-01-16-using-digitalocean-spaces-object-storage-with-fuse.md | |||
| @@ -4,17 +4,26 @@ title: Using DigitalOcean Spaces Object Storage with FUSE | |||
| 4 | description: Using DigitalOcean Spaces Object Storage with FUSE | 4 | description: Using DigitalOcean Spaces Object Storage with FUSE |
| 5 | --- | 5 | --- |
| 6 | 6 | ||
| 7 | **Table of content** | ||
| 8 | |||
| 9 | - [Is it possible to use them as a mounted drive with FUSE?](#is-it-possible-to-use-them-as-a-mounted-drive-with-fuse) | ||
| 10 | - [Will the performance degrade over time and over different sizes of objects?](#will-the-performance-degrade-over-time-and-over-different-sizes-of-objects) | ||
| 11 | - [Measurement experiment 1: File copy](#measurement-experiment-1-file-copy) | ||
| 12 | - [Measurement experiment 2: SQLite performanse](#measurement-experiment-2-sqlite-performanse) | ||
| 13 | - [Can storage be mounted on multiple machines at the same time and be writable?](#can-storage-be-mounted-on-multiple-machines-at-the-same-time-and-be-writable) | ||
| 14 | - [Observations and conslusion](#observations-and-conslusion) | ||
| 15 | |||
| 7 | Couple of months ago [DigitalOcean](https://www.digitalocean.com) introduced new product called [Spaces](https://blog.digitalocean.com/introducing-spaces-object-storage/) which is Object Storage very similar to Amazon's S3. This really peaked my interest, because this was something I was missing and even the thought of going over the internet for such functionality was in no interest to me. Also in fashion with their previous pricing this also is very cheap and pricing page is a no-brainer compared to AWS or GCE. [Prices are clearly and precisely defined and outlined](https://www.digitalocean.com/pricing/). You must love them for that :) | 16 | Couple of months ago [DigitalOcean](https://www.digitalocean.com) introduced new product called [Spaces](https://blog.digitalocean.com/introducing-spaces-object-storage/) which is Object Storage very similar to Amazon's S3. This really peaked my interest, because this was something I was missing and even the thought of going over the internet for such functionality was in no interest to me. Also in fashion with their previous pricing this also is very cheap and pricing page is a no-brainer compared to AWS or GCE. [Prices are clearly and precisely defined and outlined](https://www.digitalocean.com/pricing/). You must love them for that :) |
| 8 | 17 | ||
| 9 | ### Initial requirements | 18 | ### Initial requirements |
| 10 | 19 | ||
| 11 | * [Is it possible to use them as a mounted drive with FUSE?](#fuse) (tl;dr YES) | 20 | * Is it possible to use them as a mounted drive with FUSE? (tl;dr YES) |
| 12 | * [Will the performance degrade over time and over different sizes of objects?](#performanse) (tl;dr NO&YES) | 21 | * Will the performance degrade over time and over different sizes of objects? (tl;dr NO&YES) |
| 13 | * [Can storage be mounted on multiple machines at the same time and be writable?](#multiuser) (tl;dr YES) | 22 | * Can storage be mounted on multiple machines at the same time and be writable? (tl;dr YES) |
| 14 | 23 | ||
| 15 | > Let me be clear. This scripts I use are made just for benchmarking and are not intended to be used in real-life situations. Besides that, I am looking into using this approaches but adding caching service in front of it and then dumping everything as an object to storage. This could potentially be some interesting post of itself. But in case you would need real-time data without eventual consistency please take this scripts as they are: not usable in such situations. | 24 | > Let me be clear. This scripts I use are made just for benchmarking and are not intended to be used in real-life situations. Besides that, I am looking into using this approaches but adding caching service in front of it and then dumping everything as an object to storage. This could potentially be some interesting post of itself. But in case you would need real-time data without eventual consistency please take this scripts as they are: not usable in such situations. |
| 16 | 25 | ||
| 17 | ## <a name="fuse"></a>Is it possible to use them as a mounted drive with FUSE? | 26 | ## Is it possible to use them as a mounted drive with FUSE? |
| 18 | 27 | ||
| 19 | Well, actually they can be used in such manor. Because they are similar to [AWS S3](https://aws.amazon.com/s3/) many tools are available and you can find many articles and [Stackoverflow items](https://stackoverflow.com/search?q=s3+fuse). | 28 | Well, actually they can be used in such manor. Because they are similar to [AWS S3](https://aws.amazon.com/s3/) many tools are available and you can find many articles and [Stackoverflow items](https://stackoverflow.com/search?q=s3+fuse). |
| 20 | 29 | ||
| @@ -67,7 +76,7 @@ Additional information on FUSE: | |||
| 67 | * [Github project page for s3fs](https://github.com/s3fs-fuse/s3fs-fuse) | 76 | * [Github project page for s3fs](https://github.com/s3fs-fuse/s3fs-fuse) |
| 68 | * [FUSE - Filesystem in Userspace](https://en.wikipedia.org/wiki/Filesystem_in_Userspace) | 77 | * [FUSE - Filesystem in Userspace](https://en.wikipedia.org/wiki/Filesystem_in_Userspace) |
| 69 | 78 | ||
| 70 | ## <a name="performanse"></a>Will the performance degrade over time and over different sizes of objects? | 79 | ## Will the performance degrade over time and over different sizes of objects? |
| 71 | 80 | ||
| 72 | For this task I didn't want to just read and write text files or uploading images. I actually wanted to figure out if using something like SQlite is viable in this case. | 81 | For this task I didn't want to just read and write text files or uploading images. I actually wanted to figure out if using something like SQlite is viable in this case. |
| 73 | 82 | ||
| @@ -249,7 +258,7 @@ You can download [raw result here](/files/sqlite-benchmarks.tsv). And again, the | |||
| 249 | })(); | 258 | })(); |
| 250 | </script> | 259 | </script> |
| 251 | 260 | ||
| 252 | ## <a name="multiuser"></a>Can storage be mounted on multiple machines at the same time and be writable? | 261 | ## Can storage be mounted on multiple machines at the same time and be writable? |
| 253 | 262 | ||
| 254 | Well, this one didn't take long to test. And the answer is **YES**. I mounted space on both machines and measured same performance on both machines. But because file is downloaded before write and then uploaded on complete there could potentially be problems is another process is trying to access the same file. | 263 | Well, this one didn't take long to test. And the answer is **YES**. I mounted space on both machines and measured same performance on both machines. But because file is downloaded before write and then uploaded on complete there could potentially be problems is another process is trying to access the same file. |
| 255 | 264 | ||
