aboutsummaryrefslogtreecommitdiff
path: root/src
diff options
context:
space:
mode:
authorMitja Felicijan <mitja.felicijan@gmail.com>2019-10-22 03:40:14 +0200
committerMitja Felicijan <mitja.felicijan@gmail.com>2019-10-22 03:40:14 +0200
commit28dd784a088a35739cdfdc4ce79f8ee6d50bf816 (patch)
treec198abb97177f60864530ee46f5cdcf0ae88d2bf /src
parent421677613114bb40780d3a5516b6930d386d0b09 (diff)
downloadmitjafelicijan.com-28dd784a088a35739cdfdc4ce79f8ee6d50bf816.tar.gz
Cleanup of repo and move to gostatic
Diffstat (limited to 'src')
-rw-r--r--src/blog/golang-profiling-simplified.md110
-rw-r--r--src/blog/profiling-python-web-applications-with-visual-tools.md184
-rw-r--r--src/blog/simple-iot-application.md486
-rw-r--r--src/blog/simplifying-and-reducing-clutter.md21
-rw-r--r--src/blog/what-i-ve-learned-developing-ad-server.md133
-rw-r--r--src/curriculum-vitae.md69
-rw-r--r--src/feed.atom55
-rw-r--r--src/files/dna-sequence/benchmarks.odsbin0 -> 21911 bytes
-rw-r--r--src/files/dna-sequence/chart-encoding-speed.pngbin0 -> 14201 bytes
-rw-r--r--src/files/dna-sequence/chart-file-sizes.pngbin0 -> 12391 bytes
-rw-r--r--src/files/dna-sequence/dna-basics.jpgbin0 -> 165883 bytes
-rw-r--r--src/files/dna-sequence/quote.pngbin0 -> 1068 bytes
-rw-r--r--src/files/dna-sequence/sample-binary-file.pngbin0 -> 66417 bytes
-rw-r--r--src/files/dna-sequence/sample.pngbin0 -> 65930 bytes
-rw-r--r--src/files/do-fuse/copy-benchmarks.tsv101
-rw-r--r--src/files/do-fuse/fuse-droplets.pngbin0 -> 42891 bytes
-rw-r--r--src/files/do-fuse/fuse-spaces.pngbin0 -> 32450 bytes
-rw-r--r--src/files/do-fuse/sqlite-benchmarks.tsv1001
-rw-r--r--src/files/go-profiling/golang-profiling-cpu.pdfbin0 -> 16518 bytes
-rw-r--r--src/files/go-profiling/golang-profiling-mem.pdfbin0 -> 19221 bytes
-rw-r--r--src/files/iot-application/iot-app-output.pngbin0 -> 23767 bytes
-rw-r--r--src/files/iot-application/iot-rest-example.pngbin0 -> 33912 bytes
-rw-r--r--src/files/iot-application/iot-sqlite-db.pngbin0 -> 199821 bytes
-rw-r--r--src/files/iot-application/kcachegrind.pngbin0 -> 88486 bytes
-rw-r--r--src/files/iot-application/profiling-viewer.pngbin0 -> 173672 bytes
-rw-r--r--src/files/iot-application/simple-iot-application-overview.svg2
-rw-r--r--src/files/iot-application/simple-iot-application.zipbin0 -> 6406 bytes
-rw-r--r--src/files/iot-application/snakeviz.pngbin0 -> 59601 bytes
-rw-r--r--src/index.html19
-rw-r--r--src/research/encoding-binary-data-into-dna-sequence.md345
-rw-r--r--src/research/using-digitalocean-spaces-object-storage-with-fuse.md260
-rw-r--r--src/static/avatar-512x512.pngbin0 -> 244569 bytes
-rw-r--r--src/static/avatar-64x64.pngbin0 -> 16769 bytes
-rw-r--r--src/static/style.css116
34 files changed, 2902 insertions, 0 deletions
diff --git a/src/blog/golang-profiling-simplified.md b/src/blog/golang-profiling-simplified.md
new file mode 100644
index 0000000..a49de67
--- /dev/null
+++ b/src/blog/golang-profiling-simplified.md
@@ -0,0 +1,110 @@
1title: Golang profiling simplified
2date: 2017-03-07
3tags: blog
4hide: false
5----
6
7Many posts have been written regarding profiling in Golang and I haven’t found proper tutorial regarding this. Almost all of them are missing some part of important information and it gets pretty frustrating when you have a deadline and are not finding simple distilled solution.
8
9Nevertheless, after searching and experimenting I have found a solution that works for me and probably should also for you.
10
11## Where are my pprof files?
12
13By default pprof files are generated in /tmp/ folder. You can override folder where this files are generated programmatically in your golang code as we will see below in example.
14
15## Why is my CPU profile empty?
16
17I have found out that sometimes CPU profile is empty because program was not executing long enough. Programs, that execute too quickly don’t produce pprof file in my cases. Well, file is generated but only contains 4KB of information.
18
19## Profiling
20
21As you can see from examples we are executing dummy_benchmark functions to ensure some sort of execution. Memory profiling can be done without such a “complex” function. But CPU profiling needs it.
22
23Both memory and CPU profiling examples are almost the same. Only parameters in main function when calling profile.Start are different. When we set profile.ProfilePath(“.”) we tell profiler to store pprof files in the same folder as our program.
24
25### Memory profiling
26
27```go
28package main
29
30import (
31 "fmt"
32 "time"
33 "github.com/pkg/profile"
34)
35
36func dummy_benchmark() {
37
38 fmt.Println("first set ...")
39 for i := 0; i < 918231333; i++ {
40 i *= 2
41 i /= 2
42 }
43
44 <-time.After(time.Second*3)
45
46 fmt.Println("sencond set ...")
47 for i := 0; i < 9182312232; i++ {
48 i *= 2
49 i /= 2
50 }
51}
52
53func main() {
54 defer profile.Start(profile.MemProfile, profile.ProfilePath("."), profile.NoShutdownHook).Stop()
55 dummy_benchmark()
56}
57```
58
59### CPU profiling
60
61```go
62package main
63
64import (
65 "fmt"
66 "time"
67 "github.com/pkg/profile"
68)
69
70func dummy_benchmark() {
71
72 fmt.Println("first set ...")
73 for i := 0; i < 918231333; i++ {
74 i *= 2
75 i /= 2
76 }
77
78 <-time.After(time.Second*3)
79
80 fmt.Println("sencond set ...")
81 for i := 0; i < 9182312232; i++ {
82 i *= 2
83 i /= 2
84 }
85}
86
87func main() {
88 defer profile.Start(profile.CPUProfile, profile.ProfilePath("."), profile.NoShutdownHook).Stop()
89 dummy_benchmark()
90}
91```
92
93### Generating profiling reports
94
95```bash
96# memory profiling
97go build mem.go
98./mem
99go tool pprof -pdf ./mem mem.pprof > mem.pdf
100
101# cpu profiling
102go build cpu.go
103./cpu
104go tool pprof -pdf ./cpu cpu.pprof > cpu.pdf
105```
106
107This will generate PDF document with visualized profile.
108
109- [Memory PDF profile example](/files/go-profiling/golang-profiling-mem.pdf)
110- [CPU PDF profile example](/files/go-profiling/golang-profiling-cpu.pdf)
diff --git a/src/blog/profiling-python-web-applications-with-visual-tools.md b/src/blog/profiling-python-web-applications-with-visual-tools.md
new file mode 100644
index 0000000..e99b9ff
--- /dev/null
+++ b/src/blog/profiling-python-web-applications-with-visual-tools.md
@@ -0,0 +1,184 @@
1title: Profiling Python web applications with visual tools
2date: 2017-04-21
3tags: blog
4hide: false
5----
6
7I have been profiling my software with KCachegrind for a long time now and I was missing this option when I am developing API's or other web services. I always knew that this is possible but never really took the time and dive into it.
8
9Before we begin there are some requirements. We will need to:
10
11- implement [cProfile](https://docs.python.org/2/library/profile.html#module-cProfile) into our web app,
12- convert output to [callgrind](http://valgrind.org/docs/manual/cl-manual.html) format with [pyprof2calltree](https://pypi.python.org/pypi/pyprof2calltree/),
13- visualize data with [KCachegrind](http://kcachegrind.sourceforge.net/html/Home.html) or [Profiling Viewer](http://www.profilingviewer.com/).
14
15
16If you are using MacOS you should check out [Profiling Viewer](http://www.profilingviewer.com/) or [MacCallGrind](http://www.maccallgrind.com/).
17
18![KCachegrind](/files/kcachegrind.png)
19
20We will be dividing this post into two main categories:
21
22- writing simple web-service,
23- visualize profile of this web-service.
24
25## Simple web-service
26
27Let's use virtualenv so we won't pollute our base system. If you don't have virtualenv installed on your system you can install it with pip command.
28
29```bash
30# let's install virtualenv globally
31$ sudo pip install virtualenv
32
33# let's also install pyprof2calltree globally
34$ sudo pip install pyprof2calltree
35
36# now we create project
37$ mkdir demo-project
38$ cd demo-project/
39
40# now let's create folder where we will store profiles
41$ mkdir prof
42
43# now we create empty virtualenv in venv/ folder
44$ virtualenv --no-site-packages venv
45
46# we now need to activate virtualenv
47$ source venv/bin/activate
48
49# you can check if virtualenv was correctly initialized by
50# checking where your python interpreter is located
51# if command bellow points to your created directory and not some
52# system dir like /usr/bin/python then everything is fine
53$ which python
54
55# we can check now if all is good ➜ if ok couple of
56# lines will be displayed
57$ pip freeze
58# appdirs==1.4.3
59# packaging==16.8
60# pyparsing==2.2.0
61# six==1.10.0
62
63# now we are ready to install bottlepy ➜ web micro-framework
64$ pip install bottle
65
66# you can deactivate virtualenv but you will then go
67# under system domain ➜ for now don't deactivate
68$ deactivate
69```
70
71We are now ready to write simple web service. Let's create file app.py and paste code bellow in this newly created file.
72
73```python
74# -*- coding: utf-8 -*-
75
76import bottle
77import random
78import cProfile
79
80app = bottle.Bottle()
81
82# this function is a decorator and encapsulates function
83# and performs profiling and then saves it to subfolder
84# prof/function-name.prof
85# in our example only awesome_random_number function will
86# be profiled because it has do_cprofile defined
87def do_cprofile(func):
88 def profiled_func(*args, **kwargs):
89 profile = cProfile.Profile()
90 try:
91 profile.enable()
92 result = func(*args, **kwargs)
93 profile.disable()
94 return result
95 finally:
96 profile.dump_stats("prof/" + str(func.__name__) + ".prof")
97 return profiled_func
98
99
100# we use profiling over specific function with including
101# @do_cprofile above function declaration
102@app.route("/")
103@do_cprofile
104def awesome_random_number():
105 awesome_random_number = random.randint(0, 100)
106 return "awesome random number is " + str(awesome_random_number)
107
108@app.route("/test")
109def test():
110 return "dummy test"
111
112if __name__ == '__main__':
113 bottle.run(
114 app = app,
115 host = "0.0.0.0",
116 port = 4000
117 )
118
119# run with 'python app.py'
120# open browser 'http://0.0.0.0:4000'
121```
122
123When browser hits awesome\_random\_number() function profile is created in prof/ subfolder.
124
125## Visualize profile
126
127Now let's create callgrind format from this cProfile output.
128
129```bash
130$ cd prof/
131$ pyprof2calltree -i awesome_random_number.prof
132# this creates 'awesome_random_number.prof.log' file in the same folder
133```
134
135This file can be opened with visualizing tools listed above. In this case we will be using Profilling Viewer under MacOS. You can open image in new tab. As you can see from this example there is hierarchy of execution order of your code.
136
137![Profilling Viewer](/files/profiling-viewer.png)
138
139> Make sure you convert output of the cProfile output every time you want to refresh and take a look at your possible optimizations because cProfile updates .prof file every time browser hits the function.
140
141This is just a simple example but when you are developing real-life applications this can be very illuminating, especially to see which parts of your code are bottlenecks and need to be optimized.
142
143## Update 2017-04-22
144
145Reddit user [mvt](https://www.reddit.com/user/mvt) also recommended this awesome web based profile visualizer [SnakeViz](https://jiffyclub.github.io/snakeviz/) that directly takes output from [cProfile](https://docs.python.org/2/library/profile.html#module-cProfile) module.
146
147<div class="reddit-embed" data-embed-media="www.redditmedia.com" data-embed-parent="false" data-embed-live="false" data-embed-uuid="583880c1-002e-41ed-a373-020a0ef2cff9" data-embed-created="2017-04-22T19:46:54.810Z"><a href="https://www.reddit.com/r/Python/comments/66v373/profiling_python_web_applications_with_visual/dgljhsb/">Comment</a> from discussion <a href="https://www.reddit.com/r/Python/comments/66v373/profiling_python_web_applications_with_visual/">Profiling Python web applications with visual tools</a>.</div><script async src="https://www.redditstatic.com/comment-embed.js"></script>
148
149```bash
150# let's install it globally as well
151$ sudo pip install snakeviz
152
153# now let's visualize
154$ cd prof/
155$ snakeviz awesome_random_number.prof
156# this automatically opens browser window and
157# shows visualized profile
158```
159
160![SnakeViz](/files/snakeviz.png)
161
162Reddit user [ccharles](https://www.reddit.com/user/ccharles) suggested a better way for installing pip software by targeting user level instead of using sudo.
163
164<div class="reddit-embed" data-embed-media="www.redditmedia.com" data-embed-parent="false" data-embed-live="false" data-embed-uuid="f4f0459e-684d-441e-bebe-eb49b2f0a31d" data-embed-created="2017-04-22T19:46:10.874Z"><a href="https://www.reddit.com/r/Python/comments/66v373/profiling_python_web_applications_with_visual/dglpzkx/">Comment</a> from discussion <a href="https://www.reddit.com/r/Python/comments/66v373/profiling_python_web_applications_with_visual/">Profiling Python web applications with visual tools</a>.</div><script async src="https://www.redditstatic.com/comment-embed.js"></script>
165
166```bash
167# now we need to add this path to our $PATH variable
168# we do this my adding this line at the end of your
169# ~/.bashrc file
170PATH=$PATH:$HOME/.local/bin/
171
172# in order to use this new configuration you can close
173# and reopen terminal or reload .bashrc file
174$ source ~/.bashrc
175
176# now let's test if new directory is present in $PATH
177$ echo $PATH
178
179# now we can install on user level by adding --user
180# without use of sudo
181$ pip install snakeviz --user
182```
183
184Or as suggested by [mvt](https://www.reddit.com/user/mvt) you can use [pipsi](https://github.com/mitsuhiko/pipsi).
diff --git a/src/blog/simple-iot-application.md b/src/blog/simple-iot-application.md
new file mode 100644
index 0000000..2b7d67f
--- /dev/null
+++ b/src/blog/simple-iot-application.md
@@ -0,0 +1,486 @@
1title: Simple IOT application supported by real-time monitoring and data history
2date: 2017-08-11
3tags: blog
4hide: false
5----
6
7## Initial thoughts
8
9I have been developing these kind of application for the better part of my last 5 years and people keep asking me how to approach developing such application and I will give a try explaining it here.
10
11IOT applications are really no different than any other kind of applications. We have data that needs to be collected and visualized in some form of tables or charts. The main difference here is that most of the times these data is collected by some kind of device foreign to developer that mainly operates in web domain. But fear not, it's not that different than writing some JavaScript.
12
13There are many devices able to transmit data via wireless or wired network by default but for the sake of example we will be using commonly known Arduino with wireless module already on the board → [Arduino MKR1000](https://store.arduino.cc/arduino-mkr1000).
14
15In order to make this little project as accessible to others as possible I will try to make it as inexpensive as possible. And by this I mean that I will avoid using hosted virtual servers and will be using my own laptop as a server. But you must buy Arduino MKR1000 to follow steps below. But if you would want to deploy this software I would suggest using [DigitalOcean](https://www.digitalocean.com) → smallest VPS is only per month making this one of the most affordable option out there. Please notice that this software will not run on stock web hosting that only supports LAMP (Linux, Apache, MySQL, and PHP).
16
17_But before we begin please take notice that this is strictly experimental code and not well optimized and there are much better ways in handling some aspects of the application but that requires much deeper knowledge of technology that is not needed for an example like this._
18
19**Development steps**
20
211. Simple Python API that will receive and store incoming data.
222. Prototype C++ code that will read "sensor data" and transmit it to API.
233. Data visualization with charts → extends Python web application.
24
25Step 1. and 3. will share the same web application. One route will be dedicated to API and another to serving HTML with chart.
26
27Schema below represents what we will try to achieve and how different parts correlates to each other.
28
29![Overview](/files/iot-application/simple-iot-application-overview.svg)
30
31## Simple Python API
32
33I have always been a fan of simplicity so we will be using [Bottle: Python Web Framework](https://bottlepy.org/docs/dev/). It is a single file web framework that seriously simplifies working with routes, templating and has built-in web server that satisfies our need in this case.
34
35First we need to install bottle package. This can be done by downloading ```bottle.py``` and placing it in the root of your application or by using pip software ```pip install bottle --user```.
36
37If you are using Linux or MacOS then Python is already installed. If you will try to test this on Windows please install [Python for Windows](https://www.python.org/downloads/windows/). There may be some problems with path when you will try to launch ```python webapp.py``` so please take care of this before you continue.
38
39### Basic web application
40
41Most basic bottle application is quite simple. Paste code below in ```webapp.py``` file and save.
42
43```python
44# -*- coding: utf-8 -*-
45
46import bottle
47
48# initializing bottle app
49app = bottle.Bottle()
50
51# triggered when / is accessed from browser
52# only accepts GET → no POST allowed
53@app.route("/", method=["GET"])
54def route_default():
55 return "howdy from python"
56
57# starting server on http://0.0.0.0:5000
58if __name__ == "__main__":
59 bottle.run(
60 app = app,
61 host = "0.0.0.0",
62 port = 5000,
63 debug = True,
64 reloader = True,
65 catchall = True,
66 )
67```
68
69To run this simple application you should open command prompt or terminal on your machine and go to the folder containing your file and type ```python webapp.py```. If everything goes ok then open your web browser and point it to ```http://0.0.0.0:5000```.
70
71If you would like change the port of your application (like port 80) and not use root to run your app this will present a problem. The TCP/IP port numbers below 1024 are privileged ports → this is a security feature. So in order of simplicity and security use a port number above 1024 like I have used port 5000.
72
73If this fails at any time please fix it before you continue, because nothing below will work otherwise.
74
75We use 0.0.0.0 as default host so that this app is available over your local network. If you find your local ip ```ifconfig``` and try accessing this site with your phone (if on same network/router as your machine) this should work as well (example of such ip ```http://192.168.1.15:5000```). This is a must have because Arduino will be accessing this application to send it's data.
76
77### Web application security
78
79There is a lot to be said about security and is a topic of many books. Of course all this can not be written here but to just establish some basic security → you should always use SSL with your application. Some fantastic free certificates are available by [Let's Encrypt - Free SSL/TLS Certificates](https://letsencrypt.org). With SSL certificate installed you should then make use of HTTP headers and send your "API key" via a header. If your key is send via header then this key is encrypted by SSL and send encrypted over the network. Never send your api keys by GET parameter like ```http://example.com/?api_key=somekeyvalue```. The problem that this kind of sending presents is that this key is visible in logs and by network sniffers.
80
81There is a fantastic article describing some aspects about security: [11 Web Application Security Best Practices](https://www.keycdn.com/blog/web-application-security-best-practices/). Please check it out.
82
83### Simple API for writing data-points
84
85We will now be using boilerplate code from example above and extend it to be able to write data received by API to local storage. For example use I will use SQLite3 because it plays well with Python and can store quite large amount of data. I have been using it to collect gigabytes of data in a single database without any corruption or problems → your experience may vary.
86
87To avoid learning SQLite I will be using [Dataset: databases for lazy people](https://dataset.readthedocs.io/en/latest/index.html). This package abstracts SQL and simplifies writing and reading data from database. You should install this package with pip software ```pip install dataset --user```.
88
89Because API will use POST method I will be testing if code works correctly by using [Restlet Client for Google Chrome](https://chrome.google.com/webstore/detail/restlet-client-rest-api-t/aejoelaoggembcahagimdiliamlcdmfm). This software also allows you to set headers → for basic security with API_KEY.
90
91To quickly generate passwords or API keys I usually use this nifty website [RandomKeygen](https://randomkeygen.com/).
92
93Copy and paste code below over your previous code in file ```webapp.py```.
94
95```python
96# -*- coding: utf-8 -*-
97
98import time
99import bottle
100import random
101import dataset
102
103# initializing bottle app
104app = bottle.Bottle()
105
106# connects to sqlite database
107# check_same_thread=False allows using it in multi-threaded mode
108app.config["dsn"] = dataset.connect("sqlite:///data.db?check_same_thread=False")
109
110# api key that will be used in Arduino code
111app.config["api_key"] = "JtF2aUE5SGHfVJBCG5SH"
112
113# triggered when /api is accessed from browser
114# only accepts POST → no GET allowed
115@app.route("/api", method=["POST"])
116def route_default():
117 status = 400
118 ts = int(time.time()) # current timestamp
119 value = bottle.request.body.read() # data from device
120 api_key = bottle.request.get_header("Api_Key") # api key from header
121
122 # outputs to console received data for debug reason
123 print ">>> {} :: {}".format(value, api_key)
124
125 # if api_key is correct and value is present
126 # then writes attribute to point table
127 if api_key == app.config["api_key"] and value:
128 app.config["dsn"]["point"].insert(dict(ts=ts, value=value))
129 status = 200
130
131 # we only need to return status
132 return bottle.HTTPResponse(status=status, body="")
133
134# starting server on http://0.0.0.0:5000
135if __name__ == "__main__":
136 bottle.run(
137 app = app,
138 host = "0.0.0.0",
139 port = 5000,
140 debug = True,
141 reloader = True,
142 catchall = True,
143 )
144```
145
146To run this simply go to folder containing python file and run ```python webapp.py``` from terminal. If everything goes ok you should have simple API available via POST method on /api route.
147
148After testing the service with Restlet Client you should be able to view your data in a database file ```data.db```.
149
150![REST settings example](/files/iot-application/iot-rest-example.png)
151
152You can also check the contents of new database file by using desktop client for SQLite → [DB Browser for SQLite](http://sqlitebrowser.org/).
153
154![SQLite database example](/files/iot-application/iot-sqlite-db.png)
155
156Table structure is as simple as it can be. We have ts (timestamp) and value (value from Arduino). As you can see timestamp is generated on API side. If you would happen to have atomic clock on Arduino it would be then better to generate and send timestamp with the value. This would be particularity useful if we would be collecting sensor data at a higher frequency and then sending this data in bulk to API.
157
158If you will deploy this app with uWSGI and multi-threaded, use DSN (Data Source Name) url with ```?check_same_thread=False```.
159
160Ok, now that we have some sort of a working API with some basic security so unwanted people can not post data to your database can we proceed further and try to program Arduino to send data to API.
161
162## Sending data to API with Arduino MKR1000
163
164First of all you should have MKR1000 module and microUSB cable to proceed. If you have ever done any work with Arduino you should know that you also need [Arduino IDE](https://www.arduino.cc/en/Main/Software). On provided link you should be able to download and install IDE. Once that task is completed and you have successfully run blink example you should proceed to the next step.
165
166In order to use wireless capabilities of MKR1000 you need to first install [WiFi101 library](https://www.arduino.cc/en/Reference/WiFi101) in Arduino IDE. Please check before you install, you may already have it installed.
167
168Code below is a working example that sends data to API. Before you try to test your code make sure you have run Python web application. Then change settings for wifi, api endpoint and api_key. If by some reason code bellow doesn't work for you please leave a comment and I'll try to help.
169
170Once you have opened IDE and copied this code try to compile and upload it. Then open "Serial monitor" to see if any output is presented by Arduino.
171
172```c
173#include <WiFi101.h>
174
175// wifi settings
176char ssid[] = "ssid-name";
177char pass[] = "ssid-password";
178
179// api server enpoint
180char server[] = "192.168.6.22";
181int port = 5000;
182
183// api key that must be the same as the one in Python code
184String api_key = "JtF2aUE5SGHfVJBCG5SH";
185
186// frequency data is sent in ms - every 5 seconds
187int timeout = 1000 * 5;
188
189int status = WL_IDLE_STATUS;
190
191void setup() {
192
193 // initialize serial and wait for port to open:
194 Serial.begin(9600);
195 delay(1000);
196
197 // check for the presence of the shield
198 if (WiFi.status() == WL_NO_SHIELD) {
199 Serial.println("WiFi shield not present");
200 while (true);
201 }
202
203 // attempt to connect to wifi network
204 while (status != WL_CONNECTED) {
205 Serial.print("Attempting to connect to SSID: ");
206 Serial.println(ssid);
207 status = WiFi.begin(ssid, pass);
208 // wait 10 seconds for connection
209 delay(10000);
210 }
211
212 // output wifi status to serial monitor
213 Serial.print("SSID: ");
214 Serial.println(WiFi.SSID());
215
216 IPAddress ip = WiFi.localIP();
217 Serial.print("IP Address: ");
218 Serial.println(ip);
219
220 long rssi = WiFi.RSSI();
221 Serial.print("signal strength (RSSI):");
222 Serial.print(rssi);
223 Serial.println(" dBm");
224}
225
226void loop() {
227
228 WiFiClient client;
229
230 if (client.connect(server, port)) {
231
232 // I use random number generator for this example
233 // but you can use analog or digital inputs from arduino
234 String content = String(random(1000));
235
236 client.println("POST /api HTTP/1.1");
237 client.println("Connection: close");
238 client.println("Api-Key: " + api_key);
239 client.println("Content-Length: " + String(content.length()));
240 client.println();
241 client.println(content);
242
243 delay(100);
244 client.stop();
245 Serial.println("Data sent successfully ...");
246
247 } else {
248 Serial.println("Problem sending data ...");
249 }
250
251 // waits for x seconds and continue looping
252 delay(timeout);
253
254}
255```
256
257As seen from example you can notice that Arduino is generating random integer between [ 0 .. 1000 ]. You can easily replace this with a temperature sensor or any other kind of sensor.
258
259Now that we have API under the hood and Arduino is sending demo data we can now focus on data visualization.
260
261## Data visualization
262
263Before we continue we should examine our project folder structure. Currently we only have two files in our project:
264
265_simple-iot-app/_
266
267* _webapp.py_
268* _data.db_
269
270We will now add HTML template that will contain CSS and JavaScript code inline for the simplicity reason. And for the bottle framework to be able to scan root application folder for templates we will add ```bottle.TEMPLATE_PATH.insert(0, "./")``` in ```webapp.py```. By default bottle framework uses ```views/``` subfolder to store templates. This is not the ideal situation and if you will use bottle to develop web applications you should use native behavior and store templates in it's predefined folder. But for the sake of example we will over-ride this. Be careful to fully replace your code with new code that is provided below. Avoid partially replacing code in file :) Also new code for reading data-points is provided in Python example below.
271
272First we add new route to our web application. It should be trigger when browser hits root of application ```http://0.0.0.0:5000/```. This route will do nothing more than render ```frontend.html``` template. This is done by ```return bottle.template("frontend.html")```. Check code below to further examine how exactly this is done.
273
274Now we will expand ```/api``` route and use different methods to write or read data-points. For writing data-point we will use POST method and for reading points we will use GET method. GET method will return JSON object with latest readings and historical data.
275
276There is a fantastic JavaScript library for plotting time-series charts called [MetricsGraphics.js](https://www.metricsgraphicsjs.org) that is based on [D3.js](https://d3js.org/) library for visualizing data.
277
278Data schema required by MetricsGraphics.js → to achieve this we need to transform data from database into this format:
279
280```json
281[
282 {
283 "date": "2017-08-11 01:07:20",
284 "value": 933
285 },
286 {
287 "date": "2017-08-11 01:07:30",
288 "value": 743
289 }
290]
291```
292
293Web application is now complete and we only need ```frontend.html``` that we will develop now. If you would try to start web app now and go to root app this will return error because we don't have frontend.html yet.
294
295```python
296# -*- coding: utf-8 -*-
297
298import time
299import bottle
300import json
301import datetime
302import random
303import dataset
304
305# initializing bottle app
306app = bottle.Bottle()
307
308# adds root directory as template folder
309bottle.TEMPLATE_PATH.insert(0, "./")
310
311# connects to sqlite database
312# check_same_thread=False allows using it in multi-threaded mode
313app.config["db"] = dataset.connect("sqlite:///data.db?check_same_thread=False")
314
315# api key that will be used in Arduino code
316app.config["api_key"] = "JtF2aUE5SGHfVJBCG5SH"
317
318# triggered when / is accessed from browser
319# only accepts GET → no POST allowed
320@app.route("/", method=["GET"])
321def route_default():
322 return bottle.template("frontend.html")
323
324# triggered when /api is accessed from browser
325# accepts POST and GET
326@app.route("/api", method=["GET", "POST"])
327def route_default():
328
329 # if method is POST then we write datapoint
330 if bottle.request.method == "POST":
331 status = 400
332 ts = int(time.time()) # current timestamp
333 value = bottle.request.body.read() # data from device
334 api_key = bottle.request.get_header("Api-Key") # api key from header
335
336 # outputs to console recieved data for debug reason
337 print ">>> {} :: {}".format(value, api_key)
338
339 # if api_key is correct and value is present
340 # then writes attribute to point table
341 if api_key == app.config["api_key"] and value:
342 app.config["db"]["point"].insert(dict(ts=ts, value=value))
343 status = 200
344
345 # we only need to return status
346 return bottle.HTTPResponse(status=status, body="")
347
348 # if method is GET then we read datapoint
349 else:
350 response = []
351 datapoints = app.config["db"]["point"].all()
352
353 for point in datapoints:
354 response.append({
355 "date": datetime.datetime.fromtimestamp(int(point["ts"])).strftime("%Y-%m-%d %H:%M:%S"),
356 "value": point["value"]
357 })
358
359 bottle.response.content_type = "application/json"
360 return json.dumps(response)
361
362# starting server on http://0.0.0.0:5000
363if __name__ == "__main__":
364 bottle.run(
365 app = app,
366 host = "0.0.0.0",
367 port = 5000,
368 debug = True,
369 reloader = True,
370 catchall = True,
371 )
372```
373
374And now finally we can implement ```frontend.html```. Create file with this name and copy code below. When you are done you can start web application. Steps for this part are listed below the code.
375
376```html
377<!DOCTYPE html>
378<html>
379
380 <head>
381 <meta charset="utf-8">
382 <title>Simple IOT application</title>
383 </head>
384
385 <body>
386
387 <h1>Simple IOT application</h1>
388
389 <div class="chart-placeholder">
390 <div id="chart"></div>
391 </div>
392
393 <!-- application main script -->
394 <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.2.1/jquery.min.js"></script>
395 <script src="https://cdnjs.cloudflare.com/ajax/libs/d3/4.10.0/d3.min.js"></script>
396 <script src="https://cdnjs.cloudflare.com/ajax/libs/metrics-graphics/2.11.0/metricsgraphics.min.js"></script>
397 <script>
398 function fetch_and_render() {
399 d3.json("/api", function(data) {
400 data = MG.convert.date(data, "date", "%Y-%m-%d %H:%M:%S");
401 MG.data_graphic({
402 data: data,
403 chart_type: "line",
404 full_width: true,
405 height: 270,
406 target: document.getElementById("chart"),
407 x_accessor: "date",
408 y_accessor: "value"
409 });
410 });
411 }
412 window.onload = function() {
413 // initial call for rendering
414 fetch_and_render();
415
416 // updates chart every 5 seconds
417 setInterval(function() {
418 fetch_and_render();
419 }, 5000);
420 }
421 </script>
422
423 <!-- application styles -->
424 <style>
425 body {
426 font: 13px sans-serif;
427 padding: 20px 50px;
428 }
429 .chart-placeholder {
430 border: 2px solid #ccc;
431 width: 100%;
432 user-select: none;
433 }
434 /* chart styles */
435 .mg-line1-color {
436 stroke: red;
437 stroke-width: 2;
438 }
439 .mg-main-area, .mg-main-line {
440 fill: #fff;
441 }
442 .mg-x-axis line, .mg-y-axis line {
443 stroke: #b3b2b2;
444 stroke-width: 1px;
445 }
446 </style>
447
448 </body>
449
450</html>
451```
452
453Now the folder structure should look like:
454
455_simple-iot-app/_
456
457* _webapp.py_
458* _data.db_
459* _frontend.html_
460
461Ok, lets now start application and start feeding it data.
462
4631. ```python webapp.py```
4642. connect Arduino MKR1000 to power source
4653. open browser and go to ```http://0.0.0.0:5000```
466
467If everything goes well you should be seeing new data-points rendered on chart every 5 seconds.
468
469If you navigate to ```http://0.0.0.0:5000``` you should see rendered chart as shown on picture below.
470
471![Application output](/files/iot-application/iot-app-output.png)
472
473Complete application with all the code is available for [download](/files/iot-application/simple-iot-application.zip).
474
475## Conclusion
476
477I hope this clarifies some aspects of IOT application development. Of course this is a minimal example and is far from what can be done in real life with some further dive into other technologies.
478
479If you would like to continue exploring IOT world here are some interesting resources for you to examine:
480
481* [Reading Sensors with an Arduino](https://www.allaboutcircuits.com/projects/reading-sensors-with-an-arduino/)
482* [MQTT 101 – How to Get Started with the lightweight IoT Protocol](http://www.hivemq.com/blog/how-to-get-started-with-mqtt)
483* [Stream Updates with Server-Sent Events](https://www.html5rocks.com/en/tutorials/eventsource/basics/)
484* [Internet of Things (IoT) Tutorials](http://www.tutorialspoint.com/internet_of_things/)
485
486Any comment or additional ideas are welcomed in comments below.
diff --git a/src/blog/simplifying-and-reducing-clutter.md b/src/blog/simplifying-and-reducing-clutter.md
new file mode 100644
index 0000000..b435834
--- /dev/null
+++ b/src/blog/simplifying-and-reducing-clutter.md
@@ -0,0 +1,21 @@
1title: Simplifying and reducing clutter in my life and work
2date: 2019-10-14
3tags: blog
4hide: false
5----
6
7I recently moved my main working machine back from Hachintosh to Linux. Well the experiment was interesting and I have done some great work on macOS but it was time to move back.
8
9I actually really missed Linux. The simplicity of `apt-get` or just the amount of software that exists for Linux should be a no-brainer. I spent most of my time on macOS finding solutions to make things work. Using [Brew](https://brew.sh/) was just a horrible experience and far from package managers of Linux. At least they managed to get that `sudo` debacle sorted.
10
11Not all was bad. macOS in general was a perfectly good environment. Things like Docker and tooling like this worked without any hiccups. My normal tools like coding IDE worked flawlessly and the whole look and feel is just superb. I have been using MacBook Air for couple of years so I was used to the system but never as a daily driver.
12
13One of the things I did after I installed Linux back on my machine was cleaning up my Dropbox folder. I have everything on Dropbox. Even projects folder. I write code for living so my whole life revolves around couple of megs of code (with assets). So it's not like I have huge files on my machine. I don't have movies or music or pictures on my PC. All of that stuff is in cloud. I use Google music and I have Netflix account which is more than enough for me.
14
15I also went and deleted some of the repositories on my Github account. I have deleted more code than deployed. People find this strange but for me deleting something feels so cathartic and also forces me to write better code next time around when I am faced with similar problem. That was a huge relief if I am being totally honest.
16
17Next step was to do something with my webpage. I have been using some scripts I wrote a while ago to generate static pages from markdown source posts. I kept on adding and adding stuff on top of it and it became a source of a frustration. And this is just a simple blog and I was using gulp and npm. Anyways after couple of hours of searching and testing static generators I found an interesting one [https://github.com/piranha/gostatic](https://github.com/piranha/gostatic) and I just decided to use this one. It was the only one that had a simple templating engine, not that I really need one. But others had this convoluted way of trying to solve everything and at the end just required quite bigger learning curve I was ready to go with. So I deleted couple of old posts, simplified HTML, trashed most of the CSS and went with [https://motherfuckingwebsite.com/](https://motherfuckingwebsite.com/) aesthetics. Yeah, the previous site was more visually stimulating but all I really care is the content at this point. And Times New Roman font is kind of awesome.
18
19I stopped working on most of the projects in the past couple of months because the overhead was just too insane. There comes a point when you stretch yourself too much and then you stop progressing and with that comes dissatisfaction.
20
21So that's about it. Moving forward minimal style.
diff --git a/src/blog/what-i-ve-learned-developing-ad-server.md b/src/blog/what-i-ve-learned-developing-ad-server.md
new file mode 100644
index 0000000..527f9d0
--- /dev/null
+++ b/src/blog/what-i-ve-learned-developing-ad-server.md
@@ -0,0 +1,133 @@
1title: What I've learned developing ad server
2date: 2017-04-17
3tags: blog
4hide: false
5----
6
7For the past year and half I have been developing native advertising server that contextually matches ads and displays them in different template forms on variety of websites. This project grew from serving thousands of ads per day to millions.
8
9The system is made from couple of core components:
10
11- API for serving ads,
12- Utils - cronjobs and queue management tools,
13- Dashboard UI.
14
15Initial release was using [MongoDB](https://www.mongodb.com/) for full-text search but was later replaced by [Elasticsearch](https://www.elastic.co/) for better CPU utilization and better search performance. This provided us with many amazing functionalities of [Elasticsearch](https://www.elastic.co/). You should check it out if you do any search related operations.
16
17Because the premise of the server is to provide native ad experience, they are rendered on the client side via simple templating engine. This ensures that ads can be displayed number of different ways based on the visual style of the page. And this makes JavaScript client library quite complex.
18
19So now that you know basic information about the product lets get into the lessons we learned.
20
21## Aggregate everything
22
23After beta version was released everything (impressions, clicks, etc) was written in nanosecond resolution in the database. At that time we were using [PostgreSQL](https://www.postgresql.org/) and database quickly grew way above 200GB in disk space. And that was problematic. Statistics took disturbingly long time to aggregate. Also using indexes on stats table in database was no help after we reached 500 million datapoints.
24
25> There is a marketing product information and there is real life experience. And the tend to be quite the opposite.
26
27This was the reason that now everything is aggregated on daily basis and this data is then fed to Elastic in form of daily summary. With this we achieved we can now track many more dimensions such as zone, channel and platform information. And with this information we can now adapt occurrences of ads on specific places more precisely.
28
29We have also adapted [Redis](https://redis.io/) as a full-time citizen in our stack. Because Redis also stores information on a local disk we have some sort of backup if server would accidentally suffer some failure.
30
31All the real-time statistics for ad serving and redirecting is presented as counters in Redis instance and daily extracted and pushed to Elastic.
32
33## Measure everything
34
35The thing about software is that we really don't know how well it is performing under load until such load is presented. When testing locally everything is fine but when on production things tend to fall apart.
36
37As a solution for this we are measuring everything we can. Function execution time (by encapsulating functions with timers), server performance (cpu, memory, disk, etc), Nginx and [uWSGI](https://uwsgi-docs.readthedocs.io/) performance. We sacrifice a bit of performance for the sake of this information. And we store all this information for later analysis.
38
39**Example of function execution time**
40
41```json
42{
43 "get_final_filtered_ads": {
44 "counter": 1931250,
45 "avg": 0.0066143431,
46 "elapsed": 12773.9500310003
47 },
48 "store_keywords_statistics": {
49 "counter": 1931011,
50 "avg": 0.0004605267,
51 "elapsed": 889.2821669996
52 },
53 "match_by_context": {
54 "counter": 1931011,
55 "avg": 0.0055960716,
56 "elapsed": 10806.0758889999
57 },
58 "match_by_high_performance": {
59 "counter": 262,
60 "avg": 0.0152770229,
61 "elapsed": 4.00258
62 },
63 "store_impression_stats": {
64 "counter": 1931250,
65 "avg": 0.0006189991,
66 "elapsed": 1195.4419869999
67 }
68}
69```
70
71We have also started profiling with [cProfile](https://pymotw.com/2/profile/) and then visualizing with [KCachegrind](http://kcachegrind.sourceforge.net/). This provides much more detailed look into code execution.
72
73## Cache control is your friend
74
75Because we use Javascript library for rendering ads we rely on this script extensively and when in need we need to be able to change behavior of the script quickly.
76
77In our case we can not simply replace javascript url in html code. It usually takes a day or two for the guys who maintain sites to change code or add ?ver=xxx attribute. And this makes rapid deployment and testing very difficult and time consuming. There is a limitation of how much you can test locally.
78
79We are now in the process of integrating [Google Tag Manager](https://www.google.com/analytics/tag-manager/) but couple of websites are developed on ASP.net platform that have some problems with tag manager. With a solution below we are certain that we are serving latest version of the script.
80
81And it only takes one mistake and users have the script cached and in case of caching it for 1 year you probably know where the problem is.
82
83```nginx
84# nginx ➜ /etc/nginx/sites-available/default
85location /static/ {
86 alias /path-to-static-content/;
87 autoindex off;
88 charset utf-8;
89 gzip on;
90 gzip_types text/plain application/javascript application/x-javascript text/javascript text/xml text/css;
91 location ~* \.(ico|gif|jpeg|jpg|png|woff|ttf|otf|svg|woff2|eot)$ {
92 expires 1y;
93 add_header Pragma public;
94 add_header Cache-Control "public";
95 }
96 location ~* \.(css|js|txt)$ {
97 expires 3600s;
98 add_header Pragma public;
99 add_header Cache-Control "public, must-revalidate";
100 }
101}
102```
103
104Also be careful when redirecting to url in your python code. We noticed that if we didn't precisely setup cache control and expire headers in response we didn't get the request on the server and therefore couldn't measure clicks. So when redirecting do as follows and there will be no problems.
105
106```python
107# python ➜ bottlepy web micro-framework
108response = bottle.HTTPResponse(status=302)
109response.set_header("Cache-Control", "no-store, no-cache, must-revalidate")
110response.set_header("Expires", "Thu, 01 Jan 1970 00:00:00 GMT")
111response.set_header("Location", url)
112return response
113```
114
115> Cache control in browsers is quite aggressive and you need to be precise to avoid future problems. We learned that lesson the hard way.
116
117## Learn NGINX
118
119When deciding on a web server we went with Nginx as a reverse proxy for our applications. We adapted micro-service oriented architecture early in the project to ensure when we scale we can easily add additional servers to our cluster. And Nginx was crucial to perform load balancing and static content delivery.
120
121At first our config file was quite simple and later grew larger. After patching and adding new settings I sat down and learned more about the guts of Nginx. This proved to be very useful and we were able to squeeze much more out of our setup. So I advise you to take your time and read through the [documentation](https://nginx.org/en/docs/). This saved us a lot of headache. Googling for solutions only goes so far.
122
123## Use Redis/Memcached
124
125As explained above we are using caching basically for everything. It is the corner stone of our services. At first we were very careful about the quantity of things we stored in [Redis](https://redis.io/). But we later found out that the memory footprint is very low even when storing large amount of data in it.
126
127So we gradually increased our usage to caching whole HTML outputs of dashboard. This improved our performance in order of magnitude. And by using native TTL support this goes hand in hand with our needs.
128
129The reason why we choose [Redis](https://redis.io/) over [Memcached](https://memcached.org/) was the nature of scalability of Redis out of the box. But all this can be achieved with Memcached.
130
131## Conclusion
132
133There are a lot more details that could have been written and every single topic in here deserves it's own post but you probably got the idea about the problems we faced.
diff --git a/src/curriculum-vitae.md b/src/curriculum-vitae.md
new file mode 100644
index 0000000..29c576d
--- /dev/null
+++ b/src/curriculum-vitae.md
@@ -0,0 +1,69 @@
1title: Curriculum Vitae
2date: 2018-01-16
3tags: research
4hide: false
5----
6
7**Mitja Felicijan**
8
9*[mitja.felicijan@gmail.com](mailto:mitja.felicijan@gmail.com?subject=Website+CV+Contact)*
10
11*Slovenia, EU*
12
13## Technical experience
14
15- **Key languages:** Golang, Python, C, Bash.
16- **Platforms:** GNU/Linux, macOS.
17- **Fields of study:** Zigbee, KNX, Modbus, Machine to Machine, Embedded systems, Operating systems, Distributed systems, IOT, RDBMS, Algorithms, Database engine design, SQL, NoSQL, NewSQL, Big data analytics, Machine learning, Prediction algorithms, Realtime analytics, Systems automation, Natural language processing, Bioinformatics.
18
19## Major projects
20
21- SMS marketing system (2007)
22- Yacht management software (2008)
23- Smart Home Gateway (2009)
24- Moxa UPort 1130 USB to RS485 Universal Linux driver (2009)
25- Remote management of electricity meter (2009)
26- Remote management of blood pressure monitor (2010)
27- Infomat automation system (2010)
28- GPS Tourist - GIS Software (2011)
29- Minimal GNU/Linux distribution for embedded platforms (2011)
30- Digital Jukebox system (2012)
31- NanoCloudLogger - Machine to Machine (2012)
32- Street Lightning System (2012)
33- Smart cabins with hardware sensor management (2013)
34- Contextual advertising server (2015)
35- Network accessible database engine for caching and in-memory storage (2016)
36- Tick database engine specifically designed for storing and processing large amount of sensor data with high write throughput (2016)
37- Wireless industrial lighting management system - hardware and software (2016)
38- Minimal configuration reverse proxy (2017)
39- Industrial IOT platform for deployment on on-premise (2018)
40- Custom Platform as a service based on Docker Swarm (2018)
41- Toolkit for encoding binary data into DNA sequence (2019)
42- Minimal configuration reverse proxy with load balancing and rate limiting (2019)
43- E-ink conference room occupancy display, hadrware and software solution (2019)
44
45## Employment history
46
47- Freelancer (2001 – Present)
48- Software developer at Mobinia (2005 – 2007)
49- CTO at Milk (2007 – 2009)
50- Co-Founder of UTS (2009 – 2014)
51- Senior Software Engineer at TSmedia (2015 - 2017)
52- Senior Software Engineer at Renderspace (2017 - 2019)
53- IT Consultant (2017 – Present)
54
55## Awards
56
57- Regional Award for Innovation by Chamber of Commerce and Industry of Slovenia for project Intelligent system management and regulation of Street Lighting, 2010
58- National Award for Innovation by Chamber of Commerce and Industry of Slovenia for project Intelligent system management and regulation of Street Lighting, 2010
59
60## Key responsibilities
61
62- Embedded platform development.
63- Hardware design and driver development.
64- Designing, developing and testing systems.
65- Implementation of the systems.
66- Writing and maintaining user and technical documents.
67- Development and maintenance of the project.
68- Code revision, testing and output.
69- Work on the enhancement suggested by the customers and fixes the bugs reported.
diff --git a/src/feed.atom b/src/feed.atom
new file mode 100644
index 0000000..9dc15ae
--- /dev/null
+++ b/src/feed.atom
@@ -0,0 +1,55 @@
1<?xml version="1.0" encoding="utf-8"?>
2<feed xmlns="http://www.w3.org/2005/Atom" xmlns:thr="http://purl.org/syndication/thread/1.0">
3 <id>{{ .Site.Other.Url }}</id>
4 <title>{{ .Site.Other.Title }}</title>
5 {{ with .Site.Pages.Children "blog/" }}
6 <updated>{{ .First.Date.Format "2006-01-02T15:04:05Z07:00" }}</updated>
7 {{ end }}
8 <author><name>{{ .Site.Other.Author }}</name></author>
9 <link href="{{ .Site.Other.Url }}" rel="alternate"></link>
10 <generator uri="https://github.com/piranha/gostatic">gostatic</generator>
11
12{{ with .Site.Pages.Children "blog/" }}
13{{ range .Slice 0 5 }}
14<entry>
15 <id>{{ .Url }}</id>
16 <author><name>{{ or .Other.Author .Site.Other.Author }}</name></author>
17 <title type="html">{{ html .Title }}</title>
18 <published>{{ .Date.Format "2006-01-02T15:04:05Z07:00" }}</published>
19 {{ range .Tags }}
20 <category term="{{ . }}"></category>
21 {{ end }}
22 <link href="{{ .Site.Other.Url }}/{{ .Url }}" rel="alternate"></link>
23 <content type="html">
24 {{/* .Process runs here in case only feed changed */}}
25 {{ with cut "<section>" "</section>" .Process.Content }}
26 {{ html . }}
27 {{ end }}
28 </content>
29</entry>
30{{ end }}
31{{ end }}
32
33{{ with .Site.Pages.Children "research/" }}
34{{ range .Slice 0 5 }}
35<entry>
36 <id>{{ .Url }}</id>
37 <author><name>{{ or .Other.Author .Site.Other.Author }}</name></author>
38 <title type="html">{{ html .Title }}</title>
39 <published>{{ .Date.Format "2006-01-02T15:04:05Z07:00" }}</published>
40 {{ range .Tags }}
41 <category term="{{ . }}"></category>
42 {{ end }}
43 <link href="{{ .Site.Other.Url }}/{{ .Url }}" rel="alternate"></link>
44 <content type="html">
45 {{/* .Process runs here in case only feed changed */}}
46 {{ with cut "<section>" "</section>" .Process.Content }}
47 {{ html . }}
48 {{ end }}
49 </content>
50</entry>
51{{ end }}
52{{ end }}
53
54
55</feed>
diff --git a/src/files/dna-sequence/benchmarks.ods b/src/files/dna-sequence/benchmarks.ods
new file mode 100644
index 0000000..62a8e30
--- /dev/null
+++ b/src/files/dna-sequence/benchmarks.ods
Binary files differ
diff --git a/src/files/dna-sequence/chart-encoding-speed.png b/src/files/dna-sequence/chart-encoding-speed.png
new file mode 100644
index 0000000..7fb106d
--- /dev/null
+++ b/src/files/dna-sequence/chart-encoding-speed.png
Binary files differ
diff --git a/src/files/dna-sequence/chart-file-sizes.png b/src/files/dna-sequence/chart-file-sizes.png
new file mode 100644
index 0000000..31bfa66
--- /dev/null
+++ b/src/files/dna-sequence/chart-file-sizes.png
Binary files differ
diff --git a/src/files/dna-sequence/dna-basics.jpg b/src/files/dna-sequence/dna-basics.jpg
new file mode 100644
index 0000000..c2e7f52
--- /dev/null
+++ b/src/files/dna-sequence/dna-basics.jpg
Binary files differ
diff --git a/src/files/dna-sequence/quote.png b/src/files/dna-sequence/quote.png
new file mode 100644
index 0000000..09fb01c
--- /dev/null
+++ b/src/files/dna-sequence/quote.png
Binary files differ
diff --git a/src/files/dna-sequence/sample-binary-file.png b/src/files/dna-sequence/sample-binary-file.png
new file mode 100644
index 0000000..1e4622a
--- /dev/null
+++ b/src/files/dna-sequence/sample-binary-file.png
Binary files differ
diff --git a/src/files/dna-sequence/sample.png b/src/files/dna-sequence/sample.png
new file mode 100644
index 0000000..30f12da
--- /dev/null
+++ b/src/files/dna-sequence/sample.png
Binary files differ
diff --git a/src/files/do-fuse/copy-benchmarks.tsv b/src/files/do-fuse/copy-benchmarks.tsv
new file mode 100644
index 0000000..c7a7af4
--- /dev/null
+++ b/src/files/do-fuse/copy-benchmarks.tsv
@@ -0,0 +1,101 @@
110KB 100KB 1MB 10MB
20.15 0.187 0.317 0.653
30.158 0.237 0.192 0.659
40.134 0.359 0.236 0.604
50.136 0.292 0.196 0.501
64.411 4.479 4.376 0.649
70.134 0.481 0.265 0.608
80.146 0.266 0.28 0.516
94.282 0.307 4.549 0.562
100.152 0.28 0.229 0.512
110.162 0.37 0.315 0.652
120.13 4.735 0.222 5.171
134.29 8.767 0.283 5.076
144.555 4.682 0.318 4.941
154.658 4.691 0.177 9.624
164.778 4.791 4.415 5.114
178.794 8.604 0.311 5.223
184.582 4.727 0.234 9.28
194.596 4.638 0.212 5.064
204.7 4.65 4.458 5.221
218.822 9.159 0.191 5.032
224.628 4.641 0.324 9.226
234.6 4.921 0.197 5.22
248.85 4.58 4.405 5.245
254.65 9.142 0.215 5.168
264.884 6.67 0.248 9.273
274.581 4.594 0.248 5.082
288.864 4.844 4.502 5.121
294.704 4.656 0.177 5.173
304.616 8.883 0.209 9.334
314.729 4.962 4.366 4.966
328.918 4.682 0.186 6.702
334.686 4.58 0.168 5.111
345.123 8.84 4.747 5.084
354.846 4.732 8.85 5.065
368.887 4.639 4.824 9.286
374.681 8.897 4.791 5.104
384.649 4.682 4.835 5.194
398.847 4.663 8.929 5.271
404.568 4.604 4.762 9.444
414.657 8.74 4.772 5.076
424.636 4.724 4.838 5.168
438.778 4.846 9.065 5.057
444.995 4.571 5.074 9.314
452.343 9.222 4.818 5.732
464.742 4.646 8.909 5.32
474.82 4.842 4.778 5.167
488.791 4.66 4.759 5.157
494.835 8.944 4.804 9.323
504.599 5.594 8.952 5.299
514.809 4.628 1.567 5.294
528.744 4.771 5.59 5.018
534.71 8.919 4.771 9.257
544.704 4.7 9.003 5.064
554.765 4.605 4.781 5.185
568.866 4.669 4.844 5.392
574.897 8.925 4.786 9.279
584.568 5.168 8.893 5.1
594.679 4.757 5.41 5.232
608.922 4.702 4.7 1.984
614.669 8.721 4.906 5.366
624.707 4.555 8.96 5.245
638.938 4.615 4.89 5.216
644.608 4.621 4.677 9.237
654.58 8.954 4.908 5.194
664.707 4.575 8.968 5.017
678.822 4.781 4.882 9.714
684.674 8.833 4.834 5.02
695.005 4.689 4.762 5.312
704.732 4.799 9.111 5.286
718.894 4.675 4.936 5.185
724.747 8.764 4.739 9.312
734.785 4.749 4.845 5.34
744.656 4.705 9.181 5.256
758.899 4.601 4.739 5.261
764.594 8.813 4.576 9.329
774.585 4.716 8.813 5.343
788.718 4.723 4.819 5.092
794.725 4.757 4.83 5.061
804.737 8.899 4.772 9.488
814.692 4.717 8.831 5.13
828.841 4.951 4.787 5.309
834.66 8.895 4.746 5.228
844.749 4.595 4.833 5.26
854.715 4.615 8.928 9.381
868.849 4.651 4.826 5.289
874.66 8.897 4.802 5.197
884.588 4.844 4.883 9.311
894.753 4.888 9.053 5.072
908.841 4.737 4.75 5.157
914.794 8.976 5.063 5.196
924.544 4.673 9.036 9.335
938.74 4.654 6.377 5.29
944.729 4.752 5.001 5.048
954.654 8.98 4.873 5.544
964.9 4.606 4.723 5.192
978.757 4.802 5.427 9.056
984.859 8.969 4.816 5.3
994.701 4.662 9.002 5.138
1004.943 4.813 4.894 5.15
1018.772 4.721 4.785 9.168
diff --git a/src/files/do-fuse/fuse-droplets.png b/src/files/do-fuse/fuse-droplets.png
new file mode 100644
index 0000000..d7ce243
--- /dev/null
+++ b/src/files/do-fuse/fuse-droplets.png
Binary files differ
diff --git a/src/files/do-fuse/fuse-spaces.png b/src/files/do-fuse/fuse-spaces.png
new file mode 100644
index 0000000..4dcc1c5
--- /dev/null
+++ b/src/files/do-fuse/fuse-spaces.png
Binary files differ
diff --git a/src/files/do-fuse/sqlite-benchmarks.tsv b/src/files/do-fuse/sqlite-benchmarks.tsv
new file mode 100644
index 0000000..daa2c21
--- /dev/null
+++ b/src/files/do-fuse/sqlite-benchmarks.tsv
@@ -0,0 +1,1001 @@
1DROPTABLE CREATETABLE INSERTMANY FETCHALL COMMIT
20.000732 0.000400 0.008133 0.000065 0.000166
30.000200 0.000214 0.003105 0.000043 0.000171
40.000246 0.000170 0.006594 0.000044 0.000101
50.000182 0.000166 0.003892 0.000043 0.000112
60.000248 0.000654 0.002308 0.000041 0.000090
70.000240 0.000184 0.002253 0.000053 0.000110
80.000698 0.000483 0.003737 0.000041 0.000165
90.000217 0.000179 0.002470 0.000049 0.000107
100.000243 0.000160 0.002668 0.000054 0.000340
110.000196 0.000169 0.002247 0.000040 0.000096
120.000191 0.000162 0.003522 0.000260 0.000102
130.000195 0.000188 0.002325 0.000041 0.000132
140.000194 0.000202 0.002291 0.000039 0.000091
150.000195 0.000196 0.004114 0.000042 0.000108
160.000204 0.000200 0.002971 0.000040 0.000106
170.000227 0.000159 0.002208 0.000039 0.000117
180.000207 0.000176 0.003558 0.000040 0.000124
190.000255 0.000179 0.002870 0.000040 0.000125
200.000209 0.000176 0.002248 0.000040 0.000176
210.000211 0.000174 0.002661 0.000039 0.000180
220.000208 0.000219 0.002321 0.000039 0.000151
230.000212 0.000178 0.002609 0.000040 0.000132
240.000205 0.000209 0.002666 0.000039 0.000126
250.000205 0.000176 0.002501 0.000041 0.000133
260.000243 0.000183 0.002220 0.000037 0.000117
270.000504 0.000173 0.002230 0.000121 0.000414
280.000270 0.000200 0.002325 0.000040 0.000154
290.000208 0.000176 0.002386 0.000038 0.000123
300.000229 0.000182 0.002245 0.000039 0.000127
310.000211 0.000176 0.002544 0.000039 0.000136
320.000204 0.000180 0.002133 0.000037 0.000129
330.000205 0.000178 0.002330 0.000048 0.000146
340.000210 0.000178 0.002242 0.000039 0.000109
350.000210 0.000259 0.002766 0.000039 0.000118
360.000317 0.000495 0.002237 0.000039 0.000195
370.000454 0.000246 0.002447 0.000040 0.000172
380.000936 0.000200 0.002305 0.000057 0.000173
390.000263 0.000178 0.002251 0.000038 0.000166
400.000240 0.000183 0.002169 0.000068 0.000176
410.000251 0.000189 0.002221 0.000038 0.000141
420.000268 0.000215 0.002322 0.000039 0.000226
430.000287 0.000223 0.002696 0.000045 0.000247
440.000362 0.000229 0.002551 0.000043 0.000133
450.000239 0.000200 0.002621 0.000045 0.000133
460.000634 0.000208 0.002619 0.000046 0.000138
470.000236 0.000205 0.002589 0.000046 0.000137
480.000262 0.000205 0.002607 0.000045 0.000142
490.000239 0.000198 0.002754 0.000044 0.000185
500.000238 0.000198 0.002593 0.000057 0.000160
510.000242 0.000221 0.003784 0.000122 0.000174
520.000242 0.000201 0.002625 0.000054 0.000148
530.000296 0.000225 0.002934 0.000044 0.000134
540.000239 0.000245 0.003428 0.000046 0.000158
550.000261 0.000251 0.002569 0.000046 0.000139
560.000260 0.000230 0.002603 0.000045 0.000145
570.000302 0.000212 0.002580 0.000045 0.000176
580.000794 0.000197 0.002856 0.000046 0.000141
590.000273 0.000209 0.003173 0.000045 0.000217
600.000240 0.000201 0.002844 0.000043 0.000167
610.000389 0.000175 0.004315 0.000055 0.000091
620.000275 0.000534 0.004991 0.000053 0.000092
630.000229 0.000215 0.004084 0.000045 0.000074
640.000172 0.000474 0.002611 0.000043 0.000069
650.000201 0.000174 0.002485 0.000043 0.000069
660.000173 0.000220 0.002541 0.000045 0.000068
670.000167 0.000161 0.002827 0.000043 0.000071
680.000168 0.000160 0.003512 0.000068 0.000075
690.000211 0.000167 0.002530 0.000044 0.000069
700.000193 0.000230 0.003664 0.000046 0.000074
710.000171 0.000161 0.002575 0.000076 0.000075
720.000169 0.000161 0.002595 0.000044 0.000076
730.000981 0.000174 0.002556 0.000045 0.000072
740.000168 0.000163 0.002568 0.000043 0.000072
750.000163 0.000158 0.002579 0.000043 0.000386
760.000168 0.000160 0.002579 0.000059 0.000088
770.000176 0.000163 0.002559 0.000044 0.000075
780.000167 0.000161 0.002558 0.000043 0.000075
790.000169 0.000161 0.002599 0.000043 0.000095
800.000174 0.000163 0.002633 0.000046 0.000076
810.000170 0.000165 0.002576 0.000858 0.000079
820.000169 0.000162 0.002611 0.000044 0.000075
830.000170 0.000199 0.002621 0.000043 0.000074
840.000170 0.000167 0.003611 0.000043 0.000073
850.000171 0.000159 0.002764 0.000046 0.000076
860.000171 0.000165 0.002639 0.000044 0.000073
870.000168 0.000162 0.003131 0.000046 0.000075
880.000170 0.000162 0.002858 0.000044 0.000074
890.000171 0.000164 0.002841 0.000043 0.000075
900.000167 0.000161 0.002971 0.000043 0.000074
910.000170 0.000226 0.002842 0.000044 0.000074
920.000171 0.000165 0.002822 0.000044 0.000075
930.000173 0.000160 0.002895 0.000045 0.000073
940.000167 0.000217 0.002697 0.000044 0.000076
950.000170 0.000197 0.002699 0.000044 0.000075
960.000171 0.000163 0.003230 0.000045 0.000097
970.000170 0.000164 0.003167 0.000046 0.000082
980.000172 0.000196 0.002559 0.000043 0.000075
990.000168 0.000165 0.003006 0.000045 0.000075
1000.000176 0.000160 0.002567 0.000043 0.000075
1010.000167 0.000163 0.002757 0.000045 0.000076
1020.000171 0.000162 0.002802 0.000045 0.000076
1030.000169 0.000162 0.003102 0.000043 0.000072
1040.000167 0.000162 0.002624 0.000043 0.000075
1050.000170 0.000161 0.002589 0.000043 0.000072
1060.000222 0.000253 0.002657 0.000045 0.000075
1070.000172 0.000162 0.002586 0.000044 0.000084
1080.000172 0.000165 0.002933 0.000044 0.000075
1090.000169 0.000192 0.002609 0.000044 0.000074
1100.000194 0.000162 0.003020 0.000045 0.000081
1110.000170 0.000164 0.002908 0.000045 0.000076
1120.000169 0.000163 0.002567 0.000042 0.000073
1130.000167 0.000159 0.003071 0.000042 0.000074
1140.000222 0.000163 0.003175 0.000043 0.000076
1150.000167 0.000160 0.002641 0.000046 0.000099
1160.000171 0.000168 0.002586 0.000057 0.000075
1170.000170 0.000168 0.003148 0.000046 0.000075
1180.000171 0.000159 0.002770 0.000041 0.000074
1190.000173 0.000158 0.002643 0.000055 0.000077
1200.000313 0.000174 0.002920 0.000045 0.000075
1210.000170 0.000163 0.002551 0.000044 0.000072
1220.000173 0.000161 0.002599 0.000045 0.000073
1230.000167 0.000160 0.003505 0.000046 0.000075
1240.000171 0.000161 0.002894 0.000045 0.000074
1250.000171 0.000166 0.002572 0.000042 0.000073
1260.000166 0.000160 0.004099 0.000044 0.000102
1270.000181 0.000160 0.002499 0.000046 0.000071
1280.000174 0.000175 0.002560 0.000043 0.000068
1290.000165 0.000168 0.003083 0.000044 0.000070
1300.000210 0.000163 0.002535 0.000040 0.000068
1310.000164 0.000177 0.002906 0.000044 0.000075
1320.000175 0.000227 0.002971 0.000043 0.000073
1330.000167 0.000175 0.003409 0.000046 0.000078
1340.000172 0.000166 0.002640 0.000046 0.000074
1350.000177 0.000164 0.002574 0.000046 0.000076
1360.000170 0.000163 0.002631 0.000046 0.000075
1370.000216 0.000168 0.002596 0.000046 0.000076
1380.000170 0.000163 0.002659 0.000045 0.000074
1390.000172 0.000162 0.002677 0.000046 0.000075
1400.000170 0.000159 0.002604 0.000044 0.000081
1410.000171 0.000161 0.003163 0.000046 0.000076
1420.000171 0.000162 0.002574 0.000313 0.000075
1430.000170 0.000186 0.002988 0.000046 0.000074
1440.000171 0.000162 0.002596 0.000043 0.000077
1450.000168 0.000160 0.002640 0.000055 0.000074
1460.000169 0.000161 0.002567 0.000043 0.000371
1470.000170 0.000162 0.002704 0.000057 0.000078
1480.000255 0.000185 0.002453 0.000293 0.000066
1490.000148 0.000143 0.002169 0.000037 0.000066
1500.000173 0.000141 0.002238 0.000039 0.000085
1510.000154 0.000174 0.002679 0.000041 0.000065
1520.000149 0.000144 0.002187 0.000037 0.000065
1530.000146 0.000140 0.002760 0.000039 0.000071
1540.000147 0.000151 0.002193 0.000039 0.000065
1550.000150 0.000172 0.002207 0.000039 0.000067
1560.000147 0.000141 0.002126 0.000037 0.000060
1570.000191 0.000141 0.002119 0.000036 0.000086
1580.000149 0.000144 0.002440 0.000039 0.000065
1590.000148 0.000143 0.003287 0.000041 0.000068
1600.000152 0.000149 0.002555 0.000040 0.000069
1610.000148 0.000141 0.002203 0.000038 0.000065
1620.000147 0.000139 0.002371 0.000052 0.000075
1630.000148 0.000143 0.002201 0.000037 0.000066
1640.000149 0.000140 0.002186 0.000038 0.000062
1650.000152 0.000154 0.002215 0.000038 0.000062
1660.000149 0.000144 0.002505 0.000039 0.000067
1670.000148 0.000140 0.002216 0.000038 0.000101
1680.000160 0.000144 0.002574 0.000039 0.000067
1690.000150 0.000144 0.002266 0.000040 0.000068
1700.000151 0.000142 0.003640 0.000040 0.000068
1710.000150 0.000142 0.002207 0.000038 0.000066
1720.000148 0.000140 0.002337 0.000041 0.000068
1730.000151 0.000144 0.002138 0.000038 0.000063
1740.000146 0.000178 0.002369 0.000039 0.000060
1750.000150 0.000141 0.002290 0.000039 0.000067
1760.000149 0.000143 0.002569 0.000050 0.000070
1770.000149 0.000143 0.002797 0.000040 0.000068
1780.000149 0.000143 0.002720 0.000039 0.000066
1790.000273 0.000154 0.002255 0.000039 0.000066
1800.000147 0.000141 0.002180 0.000037 0.000065
1810.000884 0.000142 0.002164 0.000036 0.000060
1820.000188 0.000143 0.002248 0.000039 0.000062
1830.000148 0.000142 0.002178 0.000038 0.000064
1840.000151 0.000140 0.002705 0.000038 0.000063
1850.000145 0.000144 0.002588 0.000039 0.000064
1860.000147 0.000142 0.002196 0.000037 0.000064
1870.000147 0.000139 0.002169 0.000035 0.000060
1880.000151 0.000894 0.002267 0.000039 0.000061
1890.000152 0.000145 0.002178 0.000038 0.000061
1900.000185 0.000142 0.002148 0.000036 0.000062
1910.000147 0.000141 0.002845 0.000040 0.000065
1920.000159 0.000178 0.002193 0.000039 0.000063
1930.000145 0.000141 0.002571 0.000039 0.000066
1940.000149 0.000141 0.003380 0.000038 0.000065
1950.000200 0.000149 0.002439 0.000039 0.000066
1960.000152 0.000140 0.002193 0.000037 0.000065
1970.000147 0.000139 0.002239 0.000037 0.000066
1980.000200 0.000143 0.002190 0.000039 0.000066
1990.000147 0.000139 0.002243 0.000038 0.000062
2000.000421 0.000144 0.002229 0.000038 0.000062
2010.000147 0.000149 0.002715 0.000038 0.000063
2020.000151 0.000176 0.002144 0.000036 0.000060
2030.000145 0.000138 0.002184 0.000038 0.000064
2040.000146 0.000207 0.002526 0.000040 0.000067
2050.000163 0.000142 0.002366 0.000038 0.000070
2060.000149 0.000143 0.002143 0.000038 0.000065
2070.000150 0.000142 0.002146 0.000035 0.000059
2080.000162 0.000147 0.002736 0.000038 0.000067
2090.000149 0.000146 0.002383 0.000040 0.000071
2100.000147 0.000139 0.002485 0.000038 0.000065
2110.000147 0.000143 0.002811 0.000039 0.000098
2120.000181 0.000142 0.002503 0.000039 0.000066
2130.000150 0.000143 0.002227 0.000039 0.000065
2140.000149 0.000143 0.002182 0.000036 0.000061
2150.000148 0.000387 0.002159 0.000036 0.000059
2160.000147 0.000173 0.002267 0.000039 0.000063
2170.000147 0.000143 0.002729 0.000039 0.000066
2180.000149 0.000142 0.002574 0.000040 0.000069
2190.000149 0.000143 0.002560 0.000040 0.000068
2200.000152 0.000141 0.002203 0.000038 0.000066
2210.000151 0.000139 0.002234 0.000038 0.000087
2220.000148 0.000140 0.002152 0.000036 0.000060
2230.000185 0.000140 0.002274 0.000039 0.000063
2240.000148 0.000144 0.002211 0.000038 0.000066
2250.000149 0.000141 0.002692 0.000039 0.000066
2260.000148 0.000145 0.002519 0.000039 0.000066
2270.000147 0.000143 0.002188 0.000038 0.000066
2280.000149 0.000171 0.002171 0.000038 0.000093
2290.000150 0.000182 0.002185 0.000038 0.000068
2300.000191 0.000154 0.002172 0.000037 0.000061
2310.000145 0.000140 0.002253 0.000043 0.000065
2320.000147 0.000139 0.002673 0.000038 0.000066
2330.000191 0.000144 0.002740 0.000038 0.000066
2340.000147 0.000142 0.002187 0.000038 0.000064
2350.000146 0.000181 0.002180 0.000038 0.000066
2360.000176 0.000142 0.002152 0.000039 0.000061
2370.000149 0.000142 0.002164 0.000037 0.000064
2380.000245 0.000150 0.002771 0.000055 0.000084
2390.000149 0.000145 0.003006 0.000040 0.000069
2400.000153 0.000144 0.002701 0.000040 0.000067
2410.000149 0.000144 0.002192 0.000038 0.000065
2420.000148 0.000143 0.002220 0.000038 0.000063
2430.000146 0.000140 0.002210 0.000038 0.000062
2440.000157 0.000144 0.002174 0.000038 0.000060
2450.000148 0.000171 0.002208 0.000039 0.000061
2460.000146 0.000141 0.002685 0.000039 0.000064
2470.000146 0.000139 0.002811 0.000038 0.000064
2480.000147 0.000140 0.002234 0.000037 0.000063
2490.000143 0.000143 0.002209 0.000040 0.000066
2500.000149 0.000144 0.002162 0.000037 0.000091
2510.000408 0.000141 0.002140 0.000036 0.000060
2520.000142 0.000149 0.002208 0.000132 0.000061
2530.000148 0.000142 0.002706 0.000040 0.000066
2540.000148 0.000142 0.002502 0.000039 0.000065
2550.000176 0.000144 0.002265 0.000039 0.000066
2560.000150 0.000142 0.002199 0.000039 0.000065
2570.000147 0.000154 0.002201 0.000040 0.000067
2580.000150 0.000142 0.002164 0.000036 0.000094
2590.000183 0.000177 0.002253 0.000039 0.000063
2600.000189 0.000143 0.002480 0.000039 0.000066
2610.000148 0.000141 0.002212 0.000037 0.000064
2620.000150 0.000137 0.002192 0.000037 0.000065
2630.000144 0.000140 0.002271 0.000039 0.000062
2640.000190 0.000171 0.002145 0.000037 0.000061
2650.000146 0.000141 0.005865 0.000099 0.000083
2660.000178 0.000165 0.002792 0.000040 0.000066
2670.000148 0.000233 0.002742 0.000039 0.000079
2680.000157 0.000151 0.002225 0.000039 0.000066
2690.000149 0.000142 0.002215 0.000039 0.000081
2700.000165 0.000141 0.002239 0.000039 0.000081
2710.000150 0.000154 0.002154 0.000036 0.000060
2720.000152 0.000151 0.002216 0.000039 0.000075
2730.000172 0.000141 0.004471 0.000060 0.000092
2740.000250 0.000210 0.002881 0.000040 0.000066
2750.000176 0.000152 0.002262 0.000038 0.000337
2760.000164 0.000154 0.002485 0.000039 0.000074
2770.000149 0.000180 0.002148 0.000039 0.000078
2780.000194 0.000145 0.002345 0.000044 0.000064
2790.000164 0.000201 0.002483 0.000040 0.000062
2800.000148 0.000140 0.002249 0.000038 0.000076
2810.000155 0.000144 0.002504 0.000039 0.000067
2820.000166 0.000150 0.002780 0.000040 0.000079
2830.000150 0.000142 0.002194 0.000038 0.000086
2840.000178 0.000153 0.002360 0.000039 0.000079
2850.000160 0.000154 0.002159 0.000036 0.000079
2860.000195 0.000445 0.002203 0.000038 0.000074
2870.000171 0.000161 0.002220 0.000038 0.000087
2880.000165 0.000151 0.002231 0.000038 0.000088
2890.000149 0.000141 0.003445 0.000040 0.000068
2900.000148 0.000143 0.002465 0.000039 0.000081
2910.000165 0.000150 0.002228 0.000038 0.000067
2920.000160 0.000142 0.003231 0.000039 0.000066
2930.000149 0.000141 0.002215 0.000038 0.000078
2940.000146 0.000152 0.002152 0.000038 0.000077
2950.000168 0.000140 0.002258 0.000040 0.000076
2960.000193 0.000142 0.002266 0.000039 0.000085
2970.000261 0.000164 0.002160 0.000037 0.000061
2980.000151 0.000419 0.002217 0.000037 0.000073
2990.000163 0.000148 0.002856 0.000038 0.000106
3000.000258 0.000204 0.002267 0.000040 0.000075
3010.000178 0.000159 0.002266 0.000038 0.000070
3020.000158 0.000149 0.002665 0.000039 0.000085
3030.000164 0.000154 0.002478 0.000039 0.000077
3040.000148 0.000140 0.002459 0.000038 0.000066
3050.000161 0.000142 0.002206 0.000038 0.000074
3060.000155 0.000151 0.002230 0.000039 0.000083
3070.000161 0.000142 0.002225 0.000037 0.000072
3080.000161 0.000187 0.002450 0.000038 0.000063
3090.000145 0.000155 0.002438 0.000039 0.000079
3100.000166 0.000138 0.002296 0.000039 0.000076
3110.000170 0.000156 0.002446 0.000038 0.000078
3120.000160 0.000159 0.002211 0.000038 0.000078
3130.000159 0.000142 0.002190 0.000036 0.000110
3140.000157 0.000150 0.002336 0.000039 0.000073
3150.000165 0.000182 0.002132 0.000038 0.000072
3160.000160 0.000140 0.002641 0.000066 0.000066
3170.000147 0.000153 0.002153 0.000039 0.000080
3180.000148 0.000156 0.002165 0.000037 0.000077
3190.000147 0.000151 0.002201 0.000038 0.000067
3200.000162 0.000143 0.002216 0.000040 0.000080
3210.000165 0.000148 0.002223 0.000055 0.000080
3220.000193 0.000143 0.002155 0.000037 0.000078
3230.000165 0.000143 0.003005 0.000040 0.000067
3240.000151 0.000145 0.002511 0.000039 0.000070
3250.000149 0.000173 0.002246 0.000039 0.000066
3260.000148 0.000143 0.002808 0.000040 0.000067
3270.000148 0.000142 0.002513 0.000038 0.000066
3280.000148 0.000143 0.002203 0.000037 0.000065
3290.000146 0.000138 0.002123 0.000038 0.000061
3300.000170 0.000149 0.002165 0.000036 0.000062
3310.000144 0.000145 0.002186 0.000037 0.000059
3320.000144 0.000139 0.002520 0.000037 0.000065
3330.000146 0.000139 0.002559 0.000038 0.000066
3340.000153 0.000142 0.002537 0.000038 0.000067
3350.000168 0.000144 0.002217 0.000048 0.000066
3360.000147 0.000141 0.002120 0.000037 0.000063
3370.000188 0.001725 0.002541 0.000040 0.000067
3380.000149 0.000143 0.002229 0.000038 0.000076
3390.000147 0.000143 0.002233 0.000037 0.000062
3400.000182 0.000142 0.002150 0.000037 0.000061
3410.000148 0.000140 0.002196 0.000037 0.000065
3420.000145 0.000140 0.002473 0.000037 0.000065
3430.000147 0.000139 0.002725 0.000040 0.000067
3440.000149 0.000142 0.002217 0.000039 0.000065
3450.000146 0.000140 0.002167 0.000037 0.000061
3460.000176 0.000144 0.002415 0.000039 0.000064
3470.000171 0.000144 0.002925 0.000040 0.000068
3480.000152 0.000167 0.002190 0.000039 0.000066
3490.000149 0.000142 0.002530 0.000039 0.000067
3500.000150 0.000142 0.003059 0.000040 0.000068
3510.000149 0.000142 0.002417 0.000038 0.000072
3520.000149 0.000143 0.002569 0.000038 0.000068
3530.000148 0.000141 0.002262 0.000040 0.000068
3540.000152 0.000144 0.002253 0.000038 0.000066
3550.000149 0.000142 0.002134 0.000037 0.000061
3560.000277 0.000427 0.002186 0.000036 0.000060
3570.000145 0.000139 0.002791 0.000039 0.000065
3580.000149 0.000144 0.002238 0.000039 0.000066
3590.000147 0.000144 0.002514 0.000039 0.000066
3600.000148 0.000143 0.002683 0.000038 0.000063
3610.000147 0.000139 0.002214 0.000037 0.000068
3620.000145 0.000139 0.002149 0.000036 0.000059
3630.000185 0.000139 0.002214 0.000037 0.000060
3640.000145 0.000140 0.003549 0.000039 0.000066
3650.000187 0.000142 0.002160 0.000037 0.000059
3660.000147 0.000158 0.002212 0.000038 0.000065
3670.000148 0.000140 0.002483 0.000039 0.000067
3680.000147 0.000142 0.003034 0.000039 0.000066
3690.000148 0.000142 0.002228 0.000039 0.000066
3700.000145 0.000151 0.002225 0.000040 0.000067
3710.000149 0.000142 0.002858 0.000048 0.000083
3720.000203 0.000185 0.004022 0.000049 0.000086
3730.000212 0.000188 0.005086 0.000056 0.000093
3740.000220 0.000203 0.004209 0.000051 0.000085
3750.000208 0.000247 0.009261 0.000098 0.000089
3760.000211 0.000262 0.002546 0.000041 0.000066
3770.000198 0.000150 0.002534 0.000039 0.000079
3780.000159 0.000143 0.002207 0.000038 0.000094
3790.000157 0.000143 0.002173 0.000038 0.000062
3800.000198 0.000505 0.002157 0.000039 0.000079
3810.000164 0.000143 0.002172 0.000038 0.000076
3820.000156 0.000148 0.002259 0.000039 0.000080
3830.000161 0.000142 0.002219 0.000039 0.000076
3840.000161 0.000143 0.002266 0.000039 0.000085
3850.000161 0.000141 0.002150 0.000036 0.000077
3860.000179 0.000140 0.002140 0.000036 0.000071
3870.000157 0.000151 0.002316 0.000040 0.000079
3880.000149 0.000143 0.002269 0.000039 0.000066
3890.000161 0.000142 0.002206 0.000040 0.000091
3900.000172 0.000143 0.002244 0.000039 0.000067
3910.000168 0.000142 0.002189 0.000039 0.000083
3920.000163 0.000188 0.002156 0.000037 0.000077
3930.000168 0.000143 0.002266 0.000039 0.000084
3940.000166 0.000147 0.002205 0.000325 0.000078
3950.000175 0.000140 0.002173 0.000037 0.000106
3960.000170 0.000153 0.002158 0.000036 0.000083
3970.000168 0.000147 0.002825 0.000039 0.000108
3980.000172 0.000151 0.002483 0.000038 0.000085
3990.000160 0.000143 0.002163 0.000038 0.000066
4000.000161 0.000154 0.002493 0.000039 0.000084
4010.000167 0.000153 0.002564 0.000040 0.000082
4020.000159 0.000151 0.002185 0.000046 0.000088
4030.000157 0.000156 0.002175 0.000039 0.000076
4040.000150 0.000144 0.002151 0.000038 0.000063
4050.000160 0.000140 0.002429 0.000038 0.000064
4060.000160 0.000154 0.002184 0.000048 0.000077
4070.000168 0.000142 0.002686 0.000040 0.000119
4080.000164 0.000152 0.002279 0.000039 0.000075
4090.000161 0.000143 0.002192 0.000068 0.000067
4100.000161 0.000154 0.002190 0.000040 0.000092
4110.000246 0.000146 0.003064 0.000038 0.000072
4120.000163 0.000158 0.002171 0.000037 0.000073
4130.000216 0.000144 0.002209 0.000039 0.000115
4140.000159 0.000141 0.003338 0.000039 0.000079
4150.000277 0.000158 0.002464 0.000039 0.000082
4160.000168 0.000150 0.002227 0.000037 0.000079
4170.000168 0.000146 0.002775 0.000038 0.000077
4180.000146 0.000147 0.002694 0.000042 0.000084
4190.000160 0.000145 0.002807 0.000039 0.000066
4200.000162 0.000177 0.002187 0.000063 0.000066
4210.000147 0.000141 0.002220 0.000038 0.000085
4220.000160 0.000142 0.002216 0.000037 0.000077
4230.000166 0.000159 0.002224 0.000039 0.000108
4240.000147 0.000141 0.002746 0.000039 0.000078
4250.000159 0.000141 0.002194 0.000037 0.000063
4260.000164 0.000143 0.002164 0.000039 0.000067
4270.000169 0.000152 0.002278 0.000074 0.000088
4280.000157 0.000157 0.002155 0.000068 0.000076
4290.000159 0.000140 0.002170 0.000035 0.000078
4300.000156 0.000141 0.002299 0.000040 0.000066
4310.000192 0.000160 0.002241 0.000039 0.000082
4320.000149 0.000143 0.002288 0.000039 0.000079
4330.000161 0.000142 0.002185 0.000049 0.000077
4340.000147 0.000149 0.002284 0.000039 0.000063
4350.000456 0.000144 0.002203 0.000046 0.000064
4360.000187 0.000144 0.002147 0.000037 0.000061
4370.000147 0.000140 0.002238 0.000040 0.000067
4380.000147 0.000140 0.003077 0.000041 0.000068
4390.000151 0.000142 0.002226 0.000038 0.000065
4400.000146 0.000142 0.002188 0.000039 0.000065
4410.000145 0.000141 0.002156 0.000036 0.000061
4420.000143 0.000172 0.002379 0.000037 0.000060
4430.000152 0.000231 0.002172 0.000038 0.000065
4440.000153 0.000142 0.002181 0.000039 0.000065
4450.000148 0.000142 0.002567 0.000039 0.000067
4460.000150 0.000142 0.002177 0.000038 0.000072
4470.000147 0.000146 0.002328 0.000038 0.000063
4480.000146 0.000150 0.002211 0.000038 0.000063
4490.000149 0.000143 0.002222 0.000040 0.000072
4500.000150 0.000144 0.002455 0.000039 0.000065
4510.000147 0.000144 0.002206 0.000039 0.000066
4520.000145 0.000141 0.002153 0.000055 0.000070
4530.000443 0.000144 0.002139 0.000036 0.000069
4540.000147 0.000182 0.002188 0.000037 0.000061
4550.000146 0.000138 0.002248 0.000038 0.000067
4560.000147 0.000142 0.002817 0.000039 0.000067
4570.000148 0.000144 0.002230 0.000038 0.000066
4580.000148 0.000142 0.002239 0.000039 0.000067
4590.000149 0.000142 0.002197 0.000038 0.000063
4600.000181 0.000674 0.002170 0.000038 0.000061
4610.000146 0.000195 0.002204 0.000037 0.000061
4620.000146 0.000141 0.002260 0.000039 0.000067
4630.000150 0.000142 0.002193 0.000045 0.000065
4640.000147 0.000140 0.002229 0.000036 0.000066
4650.000146 0.000137 0.002197 0.000037 0.000062
4660.000152 0.000159 0.002187 0.000036 0.000060
4670.000145 0.000139 0.002224 0.000037 0.000064
4680.000149 0.000144 0.002175 0.000038 0.000066
4690.000150 0.000143 0.002187 0.000038 0.000066
4700.000148 0.000141 0.002152 0.000036 0.000061
4710.000185 0.000141 0.002176 0.000036 0.000064
4720.000169 0.000145 0.002483 0.000038 0.000067
4730.000149 0.000141 0.002225 0.000036 0.000064
4740.000244 0.000149 0.002538 0.000038 0.000065
4750.000156 0.000143 0.002317 0.000039 0.000297
4760.000228 0.000172 0.002222 0.000039 0.000300
4770.000149 0.000145 0.002173 0.000040 0.000066
4780.000154 0.000145 0.002155 0.000038 0.000093
4790.000161 0.000145 0.002178 0.000039 0.000063
4800.000147 0.000170 0.002299 0.000039 0.000066
4810.000149 0.000142 0.003494 0.000040 0.000066
4820.000149 0.000178 0.002237 0.000038 0.000062
4830.000148 0.000143 0.002150 0.000037 0.000064
4840.000146 0.000139 0.002315 0.000038 0.000065
4850.000147 0.000141 0.002269 0.000039 0.000067
4860.000173 0.000145 0.002191 0.000037 0.000065
4870.000166 0.000144 0.002247 0.000038 0.000061
4880.000146 0.000140 0.002551 0.000038 0.000065
4890.000148 0.000175 0.002202 0.000037 0.000064
4900.000145 0.000141 0.002217 0.000038 0.000063
4910.000146 0.000138 0.002164 0.000132 0.000547
4920.000148 0.000144 0.008140 0.000160 0.000893
4930.000311 0.000221 0.004526 0.000058 0.000109
4940.000238 0.000225 0.003475 0.000044 0.000094
4950.000178 0.000177 0.002537 0.000041 0.000087
4960.000172 0.000161 0.002194 0.000048 0.000084
4970.000172 0.000163 0.002177 0.000040 0.000084
4980.001177 0.000156 0.002351 0.000041 0.000325
4990.000167 0.000163 0.002273 0.000040 0.000088
5000.000170 0.000151 0.002245 0.000040 0.000077
5010.000172 0.000896 0.002181 0.000038 0.000080
5020.000202 0.000164 0.002449 0.000038 0.000076
5030.000162 0.000161 0.002188 0.000037 0.000078
5040.000165 0.000154 0.002440 0.000074 0.000091
5050.000167 0.000149 0.002185 0.000039 0.000081
5060.000176 0.000154 0.002427 0.000040 0.000093
5070.000168 0.000154 0.002304 0.000038 0.000105
5080.000672 0.000160 0.002260 0.000038 0.000088
5090.000686 0.000159 0.002207 0.000038 0.000084
5100.000163 0.000154 0.002186 0.000037 0.000077
5110.000173 0.000153 0.002399 0.000038 0.000082
5120.000166 0.000157 0.002709 0.000039 0.000077
5130.000155 0.000149 0.002143 0.000038 0.000097
5140.000166 0.000154 0.003454 0.000051 0.000106
5150.000166 0.000160 0.002539 0.000039 0.000128
5160.000169 0.000149 0.002307 0.000039 0.000085
5170.000170 0.000158 0.002225 0.000040 0.000088
5180.000170 0.000180 0.002165 0.000036 0.000103
5190.000203 0.000160 0.002345 0.000039 0.000075
5200.000173 0.000191 0.002160 0.000038 0.000074
5210.000165 0.000156 0.002243 0.000039 0.000085
5220.000172 0.000154 0.002260 0.000040 0.000090
5230.000163 0.000164 0.002258 0.000040 0.000085
5240.000168 0.000143 0.002755 0.000039 0.000086
5250.000178 0.000155 0.002202 0.000039 0.000075
5260.000164 0.000153 0.002267 0.000038 0.000081
5270.000161 0.000154 0.002158 0.000036 0.000090
5280.000169 0.000158 0.002454 0.000037 0.000061
5290.000162 0.000154 0.002543 0.000038 0.000091
5300.000170 0.000154 0.002168 0.000037 0.000085
5310.000166 0.000151 0.002852 0.000038 0.000087
5320.000167 0.000165 0.002484 0.000039 0.000089
5330.000374 0.000197 0.002217 0.000038 0.000082
5340.000156 0.000150 0.002213 0.000038 0.000112
5350.000683 0.000155 0.002131 0.000038 0.000077
5360.000162 0.000164 0.002199 0.000038 0.000076
5370.000176 0.000154 0.002345 0.000038 0.000089
5380.000175 0.000150 0.002928 0.000039 0.000082
5390.000161 0.000140 0.002528 0.000039 0.000066
5400.000159 0.000151 0.002256 0.000039 0.000075
5410.000155 0.000156 0.002233 0.000040 0.000066
5420.000171 0.000156 0.002149 0.000066 0.000084
5430.000182 0.000154 0.002233 0.000037 0.000117
5440.000166 0.000160 0.002460 0.000037 0.000088
5450.000159 0.000165 0.002891 0.000043 0.000075
5460.000169 0.000143 0.002383 0.000038 0.000084
5470.000162 0.000149 0.002313 0.000039 0.000078
5480.000166 0.000161 0.003837 0.000041 0.000092
5490.000166 0.000144 0.002389 0.000038 0.000078
5500.000185 0.000153 0.002548 0.000040 0.000090
5510.000166 0.000152 0.002943 0.000037 0.000063
5520.000147 0.000140 0.002284 0.000038 0.000066
5530.000145 0.000141 0.002555 0.000038 0.000071
5540.000189 0.000143 0.002235 0.000038 0.000359
5550.000149 0.000140 0.002779 0.000053 0.000089
5560.000211 0.000206 0.002744 0.000040 0.000067
5570.000150 0.000144 0.002471 0.000039 0.000065
5580.000151 0.000140 0.002563 0.000040 0.000064
5590.000148 0.000138 0.002305 0.000039 0.000066
5600.000148 0.000141 0.002162 0.000036 0.000060
5610.000182 0.000145 0.002403 0.000042 0.000063
5620.000152 0.000141 0.002311 0.000039 0.000065
5630.000148 0.000180 0.002192 0.000038 0.000065
5640.000149 0.000141 0.002516 0.000039 0.000066
5650.000147 0.000142 0.002193 0.000040 0.000064
5660.000146 0.000138 0.002194 0.000036 0.000060
5670.000197 0.000142 0.002291 0.000038 0.000063
5680.000148 0.000142 0.002440 0.000039 0.000066
5690.000148 0.000143 0.002228 0.000039 0.000066
5700.000149 0.000140 0.002216 0.000038 0.000067
5710.000148 0.000145 0.002196 0.000038 0.000066
5720.000148 0.000141 0.002157 0.000036 0.000061
5730.000144 0.000175 0.002491 0.000039 0.000063
5740.000147 0.000141 0.002290 0.000039 0.000066
5750.000149 0.000143 0.002508 0.000039 0.000067
5760.000149 0.000142 0.002536 0.000039 0.000067
5770.000150 0.000141 0.003132 0.000046 0.000070
5780.000153 0.000145 0.002202 0.000039 0.000067
5790.000149 0.000143 0.002102 0.000037 0.000067
5800.000989 0.000142 0.002188 0.000063 0.000068
5810.000151 0.000142 0.002229 0.000038 0.000068
5820.001481 0.000141 0.002238 0.000039 0.000070
5830.000148 0.000142 0.002204 0.000037 0.000093
5840.000160 0.000141 0.002138 0.000038 0.000062
5850.000145 0.000141 0.002708 0.000039 0.000065
5860.000147 0.000142 0.002218 0.000039 0.000067
5870.000148 0.000140 0.002759 0.000038 0.000066
5880.000148 0.000139 0.003156 0.000037 0.000067
5890.000185 0.000141 0.002259 0.000040 0.000066
5900.000148 0.000142 0.002226 0.000047 0.000068
5910.000148 0.000142 0.002305 0.000040 0.000090
5920.001000 0.000155 0.002217 0.000064 0.000068
5930.000154 0.000144 0.002554 0.000038 0.000065
5940.000148 0.000141 0.002151 0.000038 0.000066
5950.000146 0.000181 0.003031 0.000039 0.000062
5960.000146 0.000180 0.002254 0.000039 0.000061
5970.000147 0.000143 0.002188 0.000039 0.000065
5980.000147 0.000140 0.002259 0.000039 0.000063
5990.000146 0.000141 0.002238 0.000038 0.000076
6000.000148 0.000141 0.002163 0.000038 0.000061
6010.000153 0.000143 0.002195 0.000043 0.000072
6020.000149 0.000177 0.003291 0.000039 0.000063
6030.000258 0.000153 0.002150 0.000039 0.000066
6040.000157 0.000144 0.002155 0.000037 0.000060
6050.000160 0.001194 0.002269 0.000040 0.000100
6060.000164 0.000151 0.002162 0.000038 0.000078
6070.000163 0.000424 0.002178 0.000036 0.000069
6080.001333 0.000389 0.002249 0.000039 0.000066
6090.000175 0.000142 0.002208 0.000037 0.000102
6100.000443 0.000156 0.002249 0.000040 0.000062
6110.000244 0.001562 0.003049 0.000041 0.000083
6120.000208 0.000183 0.002483 0.000040 0.000068
6130.000164 0.000156 0.002220 0.000040 0.000078
6140.000169 0.000142 0.002694 0.000040 0.000083
6150.000162 0.000152 0.002453 0.000038 0.000077
6160.000157 0.000189 0.002306 0.000040 0.000077
6170.000162 0.000151 0.002200 0.000039 0.000325
6180.000150 0.000142 0.002251 0.000039 0.000066
6190.000172 0.000157 0.002184 0.000039 0.000073
6200.000160 0.000150 0.002678 0.000038 0.000326
6210.000165 0.000151 0.002292 0.000038 0.000094
6220.000162 0.000156 0.002203 0.000037 0.000083
6230.000170 0.000141 0.002175 0.000037 0.000074
6240.000149 0.000166 0.002235 0.000039 0.000071
6250.000161 0.000143 0.002423 0.000036 0.000180
6260.000164 0.000152 0.003095 0.000039 0.000076
6270.000172 0.000153 0.002466 0.000039 0.000115
6280.000151 0.000153 0.002274 0.000039 0.000066
6290.000150 0.000142 0.003179 0.000040 0.000080
6300.000172 0.000159 0.002421 0.000039 0.000083
6310.000159 0.000142 0.002165 0.000037 0.000068
6320.000155 0.000150 0.002233 0.000041 0.000123
6330.000153 0.000158 0.002253 0.000039 0.000571
6340.000203 0.000145 0.002269 0.000041 0.000077
6350.000164 0.000158 0.002176 0.000038 0.000086
6360.000197 0.000144 0.002220 0.000041 0.000080
6370.000174 0.000403 0.002224 0.000039 0.000063
6380.000218 0.000144 0.002150 0.000036 0.000069
6390.000149 0.000141 0.002479 0.000040 0.000079
6400.000163 0.000145 0.002664 0.000039 0.000082
6410.000150 0.000152 0.002446 0.000040 0.000069
6420.000203 0.000154 0.002205 0.000043 0.000077
6430.000160 0.000143 0.002210 0.000039 0.000087
6440.000194 0.000145 0.002167 0.000038 0.000069
6450.000151 0.000154 0.002137 0.000036 0.000079
6460.000162 0.000140 0.002697 0.000037 0.000085
6470.000162 0.000143 0.002233 0.000039 0.000076
6480.000148 0.000144 0.002210 0.000039 0.000065
6490.000151 0.000152 0.003015 0.000041 0.000084
6500.000158 0.000156 0.002730 0.000039 0.000079
6510.000312 0.000165 0.002207 0.000038 0.000076
6520.000167 0.000139 0.002297 0.000040 0.000065
6530.000172 0.000154 0.002205 0.000037 0.000080
6540.000146 0.000149 0.002286 0.000039 0.000076
6550.000164 0.000151 0.002214 0.000038 0.000073
6560.000162 0.000169 0.003110 0.000038 0.000067
6570.000293 0.000144 0.002182 0.000038 0.000060
6580.000157 0.000153 0.003778 0.000049 0.000095
6590.001735 0.000210 0.004360 0.000050 0.000083
6600.000297 0.000198 0.002532 0.000039 0.000072
6610.000185 0.000163 0.002173 0.000039 0.000070
6620.000183 0.000142 0.002122 0.000038 0.000062
6630.000147 0.000145 0.002443 0.000039 0.000066
6640.000149 0.000144 0.002473 0.000040 0.000066
6650.000147 0.000139 0.002949 0.000038 0.000063
6660.000147 0.000139 0.002737 0.000039 0.000066
6670.000199 0.000142 0.002927 0.000038 0.000066
6680.000149 0.000141 0.002188 0.000038 0.000065
6690.000147 0.000144 0.002203 0.000038 0.000066
6700.000149 0.000141 0.002154 0.000037 0.000062
6710.000144 0.000137 0.003526 0.000037 0.000066
6720.000151 0.000153 0.002150 0.000036 0.000060
6730.000145 0.000138 0.002202 0.000037 0.000065
6740.000272 0.000187 0.002477 0.000038 0.000306
6750.000148 0.000141 0.002421 0.000038 0.000067
6760.000147 0.000141 0.002252 0.000039 0.000065
6770.000150 0.000140 0.002144 0.000037 0.000061
6780.000191 0.000144 0.002229 0.000038 0.000060
6790.000145 0.000145 0.002202 0.000038 0.000061
6800.000146 0.000142 0.002418 0.000038 0.000065
6810.000189 0.000171 0.002568 0.000040 0.000066
6820.000150 0.000141 0.002300 0.000039 0.000067
6830.000151 0.000141 0.002199 0.000038 0.000347
6840.000147 0.000140 0.002165 0.000035 0.000061
6850.000151 0.000646 0.002310 0.000040 0.000062
6860.000161 0.000410 0.002195 0.000038 0.000061
6870.000147 0.000141 0.002466 0.000039 0.000066
6880.000147 0.000141 0.003026 0.000038 0.000066
6890.000148 0.000142 0.002223 0.000038 0.000065
6900.000147 0.000142 0.002196 0.000038 0.000067
6910.000147 0.000141 0.002155 0.000044 0.000064
6920.000146 0.000140 0.002354 0.000039 0.000067
6930.000149 0.000143 0.002186 0.000037 0.000062
6940.000150 0.000144 0.002498 0.000040 0.000063
6950.000178 0.000212 0.002453 0.000039 0.000062
6960.000149 0.000177 0.002463 0.000038 0.000063
6970.000147 0.000142 0.002507 0.000038 0.000067
6980.000149 0.000142 0.002717 0.000038 0.000066
6990.000148 0.000141 0.002452 0.000037 0.000065
7000.000147 0.000140 0.002266 0.000039 0.000066
7010.000149 0.000141 0.002183 0.000037 0.000066
7020.000153 0.000142 0.002203 0.000039 0.000067
7030.000152 0.000419 0.002245 0.000040 0.000062
7040.000149 0.000181 0.002181 0.000038 0.000063
7050.000147 0.000142 0.002224 0.000039 0.000066
7060.000147 0.000142 0.002204 0.000038 0.000066
7070.000146 0.000141 0.002250 0.000038 0.000065
7080.000148 0.000141 0.002142 0.000038 0.000063
7090.000156 0.000139 0.002176 0.000036 0.000060
7100.000243 0.000148 0.002768 0.000039 0.000069
7110.000146 0.000204 0.002194 0.000037 0.000065
7120.000147 0.000143 0.003071 0.000039 0.000066
7130.000148 0.000144 0.003489 0.000042 0.000073
7140.000151 0.000151 0.002173 0.000039 0.000064
7150.000146 0.000140 0.003509 0.000038 0.000067
7160.000148 0.000142 0.002191 0.000038 0.000064
7170.000146 0.000139 0.002441 0.000039 0.000117
7180.000174 0.000141 0.002133 0.000038 0.000065
7190.000151 0.000142 0.002257 0.000039 0.000073
7200.000163 0.000147 0.002187 0.000038 0.000061
7210.000146 0.000222 0.002193 0.000038 0.000062
7220.000145 0.000143 0.002434 0.000037 0.000064
7230.000145 0.000139 0.002933 0.000041 0.000066
7240.000146 0.000140 0.002680 0.000037 0.000065
7250.000143 0.000139 0.002217 0.001029 0.000065
7260.000145 0.000139 0.002361 0.000039 0.000067
7270.000150 0.000143 0.002186 0.000068 0.000066
7280.000148 0.000142 0.002149 0.000037 0.000061
7290.000147 0.000181 0.002183 0.000037 0.000061
7300.000146 0.000455 0.002305 0.000038 0.000074
7310.000148 0.000143 0.002223 0.000038 0.000066
7320.000148 0.000141 0.002547 0.000038 0.000066
7330.000148 0.000143 0.002180 0.000038 0.000336
7340.000146 0.000141 0.002102 0.000037 0.000063
7350.000150 0.000145 0.002170 0.000037 0.000067
7360.000152 0.000138 0.002982 0.000038 0.000067
7370.000149 0.000143 0.002419 0.000037 0.000064
7380.000145 0.000195 0.002228 0.000040 0.000067
7390.000148 0.000143 0.002193 0.000038 0.000064
7400.000155 0.000141 0.002166 0.000067 0.000066
7410.000454 0.000176 0.002193 0.000038 0.000063
7420.000186 0.000142 0.002165 0.000035 0.000066
7430.000144 0.000138 0.002542 0.000038 0.000066
7440.000148 0.000143 0.002733 0.000039 0.000066
7450.000147 0.000141 0.002227 0.000038 0.000067
7460.000145 0.000142 0.002764 0.000037 0.000064
7470.000144 0.000138 0.002207 0.000037 0.000065
7480.000147 0.000185 0.002262 0.000038 0.000062
7490.000154 0.000160 0.002163 0.000038 0.000063
7500.000150 0.000145 0.002719 0.000038 0.000065
7510.000145 0.000139 0.002226 0.000037 0.000074
7520.000148 0.000140 0.002517 0.000038 0.000067
7530.000148 0.000142 0.003734 0.000039 0.000067
7540.000147 0.000143 0.002508 0.000039 0.000067
7550.000146 0.000143 0.002288 0.000038 0.000067
7560.000149 0.000143 0.002899 0.000039 0.000067
7570.000150 0.000145 0.002232 0.000037 0.000065
7580.000148 0.000142 0.002169 0.000039 0.000067
7590.000161 0.000141 0.002196 0.000036 0.000060
7600.000145 0.000137 0.002467 0.000040 0.000064
7610.000147 0.000141 0.002168 0.000037 0.000063
7620.000147 0.000139 0.002165 0.000037 0.000064
7630.000146 0.000138 0.002167 0.000036 0.000060
7640.000150 0.000141 0.002326 0.000039 0.000063
7650.000149 0.000179 0.002197 0.000039 0.000063
7660.000148 0.000142 0.002538 0.000039 0.000067
7670.000148 0.000148 0.002555 0.000039 0.000067
7680.000150 0.000144 0.002180 0.000038 0.000066
7690.000245 0.000152 0.002203 0.000038 0.000065
7700.000146 0.000142 0.002118 0.000036 0.000091
7710.000648 0.000141 0.002173 0.000035 0.000058
7720.000142 0.000149 0.002137 0.000037 0.000059
7730.000144 0.000138 0.002191 0.000037 0.000063
7740.000143 0.000137 0.002795 0.000039 0.000065
7750.000147 0.000256 0.002250 0.000038 0.000064
7760.000148 0.000142 0.002231 0.000040 0.000075
7770.000149 0.000143 0.002174 0.000038 0.000061
7780.000182 0.000708 0.002255 0.000038 0.000061
7790.000181 0.000170 0.002222 0.000038 0.000060
7800.000148 0.000141 0.002177 0.000038 0.000065
7810.000147 0.000141 0.002478 0.000039 0.000065
7820.000148 0.000141 0.002191 0.000039 0.000065
7830.000146 0.000139 0.002161 0.000067 0.000063
7840.000157 0.000138 0.002174 0.000036 0.000059
7850.000143 0.000165 0.002396 0.000040 0.000067
7860.000148 0.000141 0.002302 0.000044 0.000067
7870.000148 0.000142 0.002226 0.000043 0.000065
7880.000149 0.000142 0.002198 0.000038 0.000087
7890.000147 0.000143 0.002221 0.000039 0.000066
7900.000146 0.000142 0.002376 0.000065 0.000063
7910.000152 0.000154 0.002201 0.000038 0.000062
7920.000150 0.000142 0.002705 0.000039 0.000067
7930.000149 0.000142 0.002267 0.000039 0.000067
7940.000194 0.000149 0.002347 0.000039 0.000066
7950.000155 0.000141 0.002594 0.000038 0.000066
7960.000148 0.000141 0.002189 0.000038 0.000064
7970.000202 0.000142 0.002155 0.000039 0.000062
7980.000182 0.000146 0.002204 0.000037 0.000061
7990.000146 0.000139 0.002466 0.000037 0.000065
8000.000146 0.000140 0.002463 0.000036 0.000065
8010.000146 0.000139 0.002209 0.000037 0.000063
8020.000145 0.000138 0.002146 0.000036 0.000060
8030.000181 0.000142 0.003356 0.000038 0.000068
8040.000161 0.000142 0.002169 0.000038 0.000062
8050.000146 0.000175 0.002538 0.000039 0.000061
8060.000148 0.000141 0.002482 0.000039 0.000067
8070.000148 0.000144 0.002450 0.000040 0.000066
8080.000149 0.000143 0.002466 0.000043 0.000068
8090.000148 0.000144 0.003551 0.000038 0.000068
8100.000149 0.000142 0.002482 0.000039 0.000066
8110.000149 0.000142 0.002220 0.000039 0.000066
8120.000151 0.000140 0.002199 0.000038 0.000064
8130.000148 0.000184 0.002185 0.000038 0.000066
8140.000145 0.000140 0.002158 0.000036 0.000092
8150.000158 0.000140 0.002262 0.000038 0.000062
8160.000148 0.000143 0.002674 0.000039 0.000066
8170.000148 0.000140 0.002421 0.000039 0.000066
8180.000149 0.000149 0.002433 0.000038 0.000065
8190.000146 0.000172 0.002187 0.000038 0.000065
8200.000146 0.000140 0.002311 0.000039 0.000323
8210.000149 0.000142 0.002180 0.000038 0.000091
8220.000420 0.000143 0.002483 0.000038 0.000063
8230.000685 0.000145 0.002136 0.000035 0.000064
8240.000146 0.000145 0.002433 0.000038 0.000062
8250.000146 0.000139 0.002496 0.000039 0.000066
8260.000149 0.000139 0.003626 0.000041 0.000068
8270.000153 0.000147 0.002272 0.000042 0.000067
8280.000248 0.000155 0.002208 0.000038 0.000063
8290.000146 0.000138 0.002524 0.000038 0.000068
8300.000147 0.000140 0.002176 0.000210 0.000065
8310.000147 0.000140 0.002166 0.000036 0.000060
8320.000144 0.000146 0.002169 0.000036 0.000057
8330.000144 0.000138 0.002207 0.000037 0.000063
8340.000145 0.000138 0.002183 0.000037 0.000062
8350.000145 0.000137 0.002167 0.000036 0.000059
8360.000148 0.000453 0.002310 0.000038 0.000061
8370.000183 0.000855 0.002326 0.000037 0.000061
8380.000146 0.000175 0.002672 0.000036 0.000060
8390.000143 0.000140 0.002238 0.000039 0.000065
8400.000146 0.000139 0.002473 0.000037 0.000064
8410.000146 0.000139 0.002196 0.000039 0.000065
8420.000145 0.000139 0.002141 0.000036 0.000061
8430.000174 0.000397 0.002175 0.000036 0.000059
8440.000143 0.000139 0.002647 0.000037 0.000065
8450.000147 0.000138 0.002196 0.000037 0.000064
8460.000146 0.000138 0.002199 0.000037 0.000063
8470.000146 0.000138 0.002167 0.000036 0.000066
8480.000169 0.000141 0.002156 0.000036 0.000060
8490.000143 0.000139 0.002180 0.000037 0.000065
8500.000144 0.000136 0.002756 0.000039 0.000066
8510.000150 0.000141 0.002919 0.000039 0.000066
8520.000147 0.000140 0.002184 0.000036 0.000065
8530.000145 0.000138 0.002168 0.000036 0.000091
8540.000156 0.000139 0.002169 0.000036 0.000059
8550.000143 0.000139 0.002741 0.000038 0.000065
8560.000147 0.000140 0.002429 0.000037 0.000063
8570.000145 0.000139 0.002226 0.000037 0.000064
8580.000145 0.000139 0.003381 0.000040 0.000066
8590.000153 0.000141 0.002262 0.000038 0.000064
8600.000145 0.000140 0.002137 0.000036 0.000062
8610.000154 0.000650 0.002217 0.000038 0.000063
8620.000184 0.000143 0.002209 0.000038 0.000062
8630.000153 0.000142 0.002907 0.000039 0.000066
8640.000147 0.000142 0.002158 0.000038 0.000064
8650.000146 0.000140 0.002953 0.000039 0.000068
8660.000148 0.000143 0.002208 0.000039 0.000065
8670.000149 0.000139 0.002187 0.000036 0.000065
8680.000144 0.000139 0.002157 0.000036 0.000061
8690.000154 0.000926 0.002139 0.000036 0.000059
8700.000183 0.000140 0.002526 0.000038 0.000062
8710.000148 0.000142 0.002207 0.000038 0.000066
8720.000147 0.000139 0.002790 0.000039 0.000069
8730.000149 0.000144 0.002251 0.000038 0.000066
8740.000151 0.000140 0.002220 0.000039 0.000066
8750.000148 0.000142 0.002523 0.000038 0.000064
8760.000151 0.000138 0.002151 0.000037 0.000065
8770.000147 0.000140 0.002251 0.000037 0.000062
8780.000149 0.000139 0.002607 0.000037 0.000065
8790.000147 0.000141 0.003380 0.000037 0.000066
8800.000147 0.000139 0.002285 0.000069 0.000066
8810.000149 0.000142 0.002566 0.000038 0.000067
8820.000147 0.000142 0.002523 0.000038 0.000067
8830.000152 0.000143 0.002215 0.000038 0.000067
8840.000150 0.000144 0.002243 0.000038 0.000075
8850.000149 0.000141 0.002148 0.000036 0.000063
8860.000182 0.000144 0.002167 0.000036 0.000062
8870.000278 0.000155 0.002631 0.000036 0.000061
8880.000149 0.000139 0.003175 0.000040 0.000066
8890.000156 0.000140 0.002660 0.000038 0.000065
8900.000148 0.000141 0.006171 0.000067 0.000069
8910.000164 0.000142 0.002713 0.000038 0.000064
8920.000161 0.000150 0.002270 0.000038 0.000081
8930.000160 0.000283 0.002276 0.000038 0.000083
8940.000168 0.000150 0.002207 0.000037 0.000072
8950.000151 0.000669 0.002160 0.000038 0.000062
8960.000196 0.000156 0.002363 0.000036 0.000061
8970.000162 0.000141 0.002160 0.000037 0.000077
8980.000147 0.000141 0.002676 0.000038 0.000096
8990.000162 0.000143 0.002263 0.000037 0.000065
9000.000162 0.000141 0.002206 0.000036 0.000080
9010.000146 0.000139 0.002149 0.000036 0.000060
9020.000169 0.000884 0.002163 0.000036 0.000076
9030.000187 0.000140 0.002222 0.000036 0.000061
9040.000145 0.000140 0.002192 0.000037 0.000084
9050.000145 0.000138 0.002619 0.000039 0.000116
9060.000158 0.000149 0.002213 0.000038 0.000089
9070.000145 0.000183 0.002154 0.000038 0.000089
9080.000162 0.000142 0.002142 0.000037 0.000061
9090.000146 0.000178 0.002401 0.000038 0.000062
9100.000145 0.000150 0.002741 0.000037 0.000081
9110.000147 0.000139 0.002360 0.000040 0.000067
9120.000151 0.000153 0.002459 0.000039 0.000075
9130.000148 0.000155 0.002459 0.000037 0.000091
9140.000153 0.000152 0.002174 0.000036 0.000064
9150.000424 0.000149 0.002116 0.000036 0.000068
9160.000166 0.000168 0.002625 0.000038 0.000076
9170.000146 0.000141 0.002957 0.000038 0.000067
9180.000160 0.000142 0.002501 0.000039 0.000079
9190.000147 0.000143 0.002219 0.000038 0.000066
9200.000160 0.000143 0.002771 0.000040 0.000079
9210.000148 0.000150 0.002426 0.000037 0.000082
9220.000146 0.000138 0.002134 0.000036 0.000103
9230.000659 0.000143 0.002197 0.000036 0.000073
9240.000179 0.000153 0.002301 0.000038 0.000074
9250.000147 0.000142 0.002258 0.000038 0.000066
9260.000146 0.000141 0.002210 0.000038 0.000066
9270.000161 0.000141 0.002235 0.000038 0.000084
9280.000145 0.000138 0.002131 0.000036 0.000060
9290.000151 0.000211 0.002265 0.000038 0.000062
9300.000147 0.000142 0.002254 0.000038 0.000067
9310.000148 0.000143 0.002217 0.000038 0.000079
9320.000160 0.000155 0.002229 0.000038 0.000066
9330.000145 0.000142 0.002129 0.000038 0.000065
9340.000165 0.000140 0.002140 0.000036 0.000076
9350.000162 0.000142 0.002452 0.000039 0.000079
9360.000148 0.000143 0.002253 0.000059 0.000068
9370.000164 0.000142 0.003378 0.000039 0.000096
9380.000150 0.000194 0.002192 0.000039 0.000067
9390.000161 0.000152 0.002202 0.000037 0.000077
9400.000160 0.000141 0.002258 0.000039 0.000067
9410.000167 0.000143 0.002706 0.000039 0.000067
9420.000149 0.000155 0.002280 0.000037 0.000100
9430.000174 0.000144 0.002134 0.000037 0.000090
9440.001167 0.000154 0.002224 0.000038 0.000067
9450.000162 0.000155 0.002181 0.000035 0.000065
9460.000773 0.000153 0.002145 0.000036 0.000060
9470.000149 0.000161 0.002160 0.000036 0.000071
9480.000208 0.000144 0.002164 0.000035 0.000060
9490.000143 0.000138 0.002156 0.000036 0.000064
9500.000143 0.000138 0.002225 0.000055 0.000066
9510.000147 0.000141 0.002734 0.000038 0.000065
9520.000145 0.000147 0.002173 0.000037 0.000064
9530.000146 0.000139 0.002112 0.000037 0.000060
9540.000144 0.000137 0.002708 0.000038 0.000064
9550.000144 0.000139 0.002421 0.000037 0.000064
9560.000145 0.000140 0.002449 0.000037 0.000063
9570.000143 0.000138 0.002278 0.000038 0.000064
9580.000145 0.000140 0.002427 0.000040 0.000130
9590.000151 0.000142 0.002155 0.000036 0.000064
9600.000181 0.000139 0.002435 0.000036 0.000060
9610.000145 0.000138 0.003527 0.000038 0.000065
9620.000146 0.000178 0.002178 0.000036 0.000060
9630.000145 0.000138 0.002139 0.000037 0.000065
9640.000145 0.000137 0.003006 0.000037 0.000064
9650.000146 0.000139 0.002204 0.000037 0.000065
9660.000145 0.000139 0.002211 0.000038 0.000062
9670.000182 0.000140 0.002221 0.000036 0.000061
9680.000145 0.000139 0.003169 0.000038 0.000068
9690.000149 0.000174 0.002414 0.000038 0.000066
9700.000147 0.000142 0.002234 0.000038 0.000066
9710.000149 0.000143 0.002678 0.000038 0.000065
9720.000148 0.000141 0.002886 0.000038 0.000066
9730.000145 0.000140 0.002250 0.000038 0.000065
9740.000148 0.000139 0.002181 0.000035 0.000065
9750.000718 0.000142 0.002141 0.000035 0.000059
9760.000189 0.000140 0.002383 0.000036 0.000059
9770.000145 0.000139 0.002206 0.000039 0.000065
9780.000154 0.000186 0.002457 0.000038 0.000066
9790.000190 0.000141 0.002224 0.000038 0.000066
9800.000149 0.000141 0.002151 0.000037 0.000066
9810.000215 0.000143 0.002151 0.000035 0.000061
9820.000144 0.000138 0.002822 0.000039 0.000065
9830.000147 0.000139 0.002275 0.000038 0.000065
9840.000148 0.000141 0.002211 0.000036 0.000064
9850.000146 0.000138 0.002201 0.000037 0.000066
9860.000148 0.000140 0.002273 0.000038 0.000068
9870.000150 0.000144 0.002188 0.000037 0.000063
9880.000152 0.000151 0.002190 0.000037 0.000062
9890.000146 0.000142 0.003145 0.000039 0.000067
9900.000151 0.000139 0.002218 0.000037 0.000065
9910.000145 0.000138 0.002264 0.000037 0.000066
9920.000148 0.000142 0.003011 0.000039 0.000067
9930.000149 0.000141 0.002196 0.000038 0.000065
9940.000146 0.000141 0.002188 0.000036 0.000060
9950.000149 0.000140 0.002190 0.000035 0.000060
9960.000144 0.000137 0.002641 0.000038 0.000064
9970.000146 0.000138 0.002182 0.000043 0.000065
9980.000146 0.000141 0.002216 0.000036 0.000064
9990.000147 0.000139 0.002294 0.000039 0.000068
10000.000657 0.000145 0.002143 0.000037 0.000062
10010.000154 0.000415 0.002237 0.000040 0.000084
diff --git a/src/files/go-profiling/golang-profiling-cpu.pdf b/src/files/go-profiling/golang-profiling-cpu.pdf
new file mode 100644
index 0000000..15241cb
--- /dev/null
+++ b/src/files/go-profiling/golang-profiling-cpu.pdf
Binary files differ
diff --git a/src/files/go-profiling/golang-profiling-mem.pdf b/src/files/go-profiling/golang-profiling-mem.pdf
new file mode 100644
index 0000000..822e445
--- /dev/null
+++ b/src/files/go-profiling/golang-profiling-mem.pdf
Binary files differ
diff --git a/src/files/iot-application/iot-app-output.png b/src/files/iot-application/iot-app-output.png
new file mode 100644
index 0000000..1c80589
--- /dev/null
+++ b/src/files/iot-application/iot-app-output.png
Binary files differ
diff --git a/src/files/iot-application/iot-rest-example.png b/src/files/iot-application/iot-rest-example.png
new file mode 100644
index 0000000..3ed86aa
--- /dev/null
+++ b/src/files/iot-application/iot-rest-example.png
Binary files differ
diff --git a/src/files/iot-application/iot-sqlite-db.png b/src/files/iot-application/iot-sqlite-db.png
new file mode 100644
index 0000000..82e1e29
--- /dev/null
+++ b/src/files/iot-application/iot-sqlite-db.png
Binary files differ
diff --git a/src/files/iot-application/kcachegrind.png b/src/files/iot-application/kcachegrind.png
new file mode 100644
index 0000000..0dc48ab
--- /dev/null
+++ b/src/files/iot-application/kcachegrind.png
Binary files differ
diff --git a/src/files/iot-application/profiling-viewer.png b/src/files/iot-application/profiling-viewer.png
new file mode 100644
index 0000000..a450513
--- /dev/null
+++ b/src/files/iot-application/profiling-viewer.png
Binary files differ
diff --git a/src/files/iot-application/simple-iot-application-overview.svg b/src/files/iot-application/simple-iot-application-overview.svg
new file mode 100644
index 0000000..817666d
--- /dev/null
+++ b/src/files/iot-application/simple-iot-application-overview.svg
@@ -0,0 +1,2 @@
1<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
2<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="623px" height="482px" version="1.1" content="&lt;mxfile userAgent=&quot;Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36&quot; version=&quot;7.1.3&quot; editor=&quot;www.draw.io&quot; type=&quot;google&quot;&gt;&lt;diagram id=&quot;d1a10cf0-d877-4170-d86f-ec45460d7050&quot; name=&quot;Page-1&quot;&gt;7Vnbbts4EP0aPzYQdbP9mKTNFtjtokgWSPpIi7RNrCRqKfq2X79DiZREUU5sV267RREgoI6o4XDmkHNIT4L7bP+bwMX6Eyc0nfge2U+C9xPfn8Yz+K+AQw1Ecw2sBCM1hFrgif1LNehpdMMILa2OkvNUssIGE57nNJEWhoXgO7vbkqf2qAVeUQd4SnDqos+MyHWNziKvxT9StlqbkZGn32TYdNZAucaE7zpQ8GES3AvOZd3K9vc0VbEzcam/ezjytnFM0Fye8oFff7DF6UbP7T2WeIFLqv2TBzNpcLVQzeSQspxQMQnudmsm6VOBE4XvINWArWWWwhOC5oJvoCP5Y2GAUgr+dxMymOzdkufySY+hnd9SIen+6HxQEyVgF+UZleIAXfQHwVzPSDMrjHWgd22e4qiG1p0UzXQ3rJmxaiy3wYOGjt9wLAMnlhM/TqUKBDRWqmEANW0ruvE/G25evCsrwt9CBxQV+/alsfJMF8rVokhZgiXjuTELHtaW7dEAdjwgbNuHjnpp1p/xI3RdamyIgcFP9QmwAbdO8XRw2Dft9wguarrCg/c2t10qj8Fesy0ces8d9vpmL+nSt9lgvoa/4QB/eyGiOblV+yc85TynVRywkAZLUlyWLLEjBXMXhxcV1RsUzgzwpQLm8VQBeyZfdNhVu34XVV3JA1MOV+/cmFemPlPBYLJqR6q61V5T4uzhvTTAzPhGJNTaCWE6Kyo7C9pNVicZ0UAuDCZoCotzazsxlB89wmfOquXWcCGyuBD1t6jaef1Vd4vvG5r2DAU9Q/WUHUOQVHzodCtUh/IVhyObvJFnlR5o1BZbLjYxPYme0Vn0bKloMVSzdpies2japSe68VCXnsihZ2X6F0H/JwQN5+iqBI1RMItmOF7OEY0hPO9cxj4LKCoDhY2A6Cpqb08qVVBi5FA9uucpFy3Nl0DNHoRTtsrV+gCqVRpOFSyQEemtfpExQtQwgwVwhBrn9xVa6Na4cIC1/ggVzsnQ1MnQI8XktQSVP3+GIhT9OBmaORm6FWTDcj6QpE+/P4IQ8s6We+gSubdSBUZv6GOfYvzYLqVo4BSD/IEUxNdIwdzdxqoDyEIdpIGhIwc24Zmq21cJbPhDBdZcZ4ytuI/IkJeOrDYSp6u+b/yoq3Ac+X2ZbAlc2TK8yL+Xjukvtst1DOoZupbQ7p0SzeXF1XQMMkvvq6T3qFSd9ql6PXLOf5HzDHJG35yc/iWbKMHluilRr/LRlm0TP7ir/s6g3PfiTjwbTsXZ3EHxG4aOcOeC9CK8REmYYN+fYoo9OnCG6t1OLnFi305+pOmWKsHs3lI+8o3UN6vKyWM3lKNLeVHn9jwlb8uortzsK6QHnLFU5eYvllFYm96fdAf/H3mG82uIqGBq0SGYuyJqeqXzgcOP+Hr8wAX7dhRJ6fLnYUgQeG8yZD4OQ+Cx/bWs3nDanxyDD/8B&lt;/diagram&gt;&lt;/mxfile&gt;"><defs/><g transform="translate(0.5,0.5)"><path d="M 283 367 C 283 345.67 348 345.67 348 367 L 348 415 C 348 436.33 283 436.33 283 415 Z" fill="#ffffff" stroke="#000000" stroke-width="2" stroke-miterlimit="10" pointer-events="none"/><path d="M 283 367 C 283 383 348 383 348 367" fill="none" stroke="#000000" stroke-width="2" stroke-miterlimit="10" pointer-events="none"/><g transform="translate(287.5,395.5)"><switch><foreignObject style="overflow:visible;" pointer-events="all" width="54" height="12" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; vertical-align: top; width: 54px; white-space: nowrap; word-wrap: normal; font-weight: bold; text-align: center;"><div xmlns="http://www.w3.org/1999/xhtml" style="display:inline-block;text-align:inherit;text-decoration:inherit;">Database</div></div></foreignObject><text x="27" y="12" fill="#000000" text-anchor="middle" font-size="12px" font-family="Helvetica" font-weight="bold">Database</text></switch></g><rect x="211" y="211" width="200" height="100" fill="#ffffff" stroke="#000000" stroke-width="2" pointer-events="none"/><g transform="translate(252.5,234.5)"><switch><foreignObject style="overflow:visible;" pointer-events="all" width="115" height="51" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; vertical-align: top; width: 115px; white-space: nowrap; word-wrap: normal; text-align: center;"><div xmlns="http://www.w3.org/1999/xhtml" style="display:inline-block;text-align:inherit;text-decoration:inherit;"><b><font style="font-size: 15px">Web application</font></b><div><b><font size="4"><br /></font></b></div><div><b><br /></b></div></div></div></foreignObject><text x="58" y="32" fill="#000000" text-anchor="middle" font-size="12px" font-family="Helvetica">[Not supported by viewer]</text></switch></g><path d="M 274.76 391 L 251 391 Q 241 391 241 381 L 241 308" fill="none" stroke="#000000" stroke-width="2" stroke-miterlimit="10" pointer-events="none"/><path d="M 280.76 391 L 272.76 395 L 274.76 391 L 272.76 387 Z" fill="#000000" stroke="#000000" stroke-width="2" stroke-miterlimit="10" pointer-events="none"/><path d="M 348 391 L 372 391 Q 382 391 382 381 L 382 321.24" fill="none" stroke="#000000" stroke-width="2" stroke-miterlimit="10" pointer-events="none"/><path d="M 382 315.24 L 386 323.24 L 382 321.24 L 378 323.24 Z" fill="#000000" stroke="#000000" stroke-width="2" stroke-miterlimit="10" pointer-events="none"/><g transform="translate(177.5,327.5)"><switch><foreignObject style="overflow:visible;" pointer-events="all" width="50" height="26" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; vertical-align: top; width: 50px; white-space: nowrap; word-wrap: normal; text-align: center;"><div xmlns="http://www.w3.org/1999/xhtml" style="display:inline-block;text-align:inherit;text-decoration:inherit;">Write<div>datapoint</div></div></div></foreignObject><text x="25" y="19" fill="#000000" text-anchor="middle" font-size="12px" font-family="Helvetica">[Not supported by viewer]</text></switch></g><g transform="translate(397.5,327.5)"><switch><foreignObject style="overflow:visible;" pointer-events="all" width="56" height="26" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; vertical-align: top; width: 56px; white-space: nowrap; word-wrap: normal; text-align: center;"><div xmlns="http://www.w3.org/1999/xhtml" style="display:inline-block;text-align:inherit;text-decoration:inherit;">Read<div>datapoints</div></div></div></foreignObject><text x="28" y="19" fill="#000000" text-anchor="middle" font-size="12px" font-family="Helvetica">[Not supported by viewer]</text></switch></g><rect x="151" y="51" width="120" height="60" rx="9" ry="9" fill="#ffffff" stroke="#000000" stroke-width="2" pointer-events="none"/><g transform="translate(182.5,67.5)"><switch><foreignObject style="overflow:visible;" pointer-events="all" width="55" height="26" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; vertical-align: top; width: 55px; white-space: nowrap; word-wrap: normal; font-weight: bold; text-align: center;"><div xmlns="http://www.w3.org/1999/xhtml" style="display:inline-block;text-align:inherit;text-decoration:inherit;">Arduino<div>MKR1000</div></div></div></foreignObject><text x="28" y="19" fill="#000000" text-anchor="middle" font-size="12px" font-family="Helvetica" font-weight="bold">[Not supported by viewer]</text></switch></g><rect x="351" y="51" width="120" height="60" rx="9" ry="9" fill="#ffffff" stroke="#000000" stroke-width="2" pointer-events="none"/><g transform="translate(372.5,74.5)"><switch><foreignObject style="overflow:visible;" pointer-events="all" width="75" height="12" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; vertical-align: top; width: 77px; white-space: nowrap; word-wrap: normal; font-weight: bold; text-align: center;"><div xmlns="http://www.w3.org/1999/xhtml" style="display:inline-block;text-align:inherit;text-decoration:inherit;">Web browser</div></div></foreignObject><text x="38" y="12" fill="#000000" text-anchor="middle" font-size="12px" font-family="Helvetica" font-weight="bold">Web browser</text></switch></g><path d="M 254.57 205.86 L 218.81 177.25 Q 211 171 211 161 L 211 111" fill="none" stroke="#000000" stroke-width="2" stroke-miterlimit="10" pointer-events="none"/><path d="M 259.25 209.6 L 250.51 207.73 L 254.57 205.86 L 255.51 201.48 Z" fill="#000000" stroke="#000000" stroke-width="2" stroke-miterlimit="10" pointer-events="none"/><path d="M 367.43 205.86 L 403.19 177.25 Q 411 171 411 161 L 411 119.24" fill="none" stroke="#000000" stroke-width="2" stroke-miterlimit="10" pointer-events="none"/><path d="M 362.75 209.6 L 366.49 201.48 L 367.43 205.86 L 371.49 207.73 Z" fill="#000000" stroke="#000000" stroke-width="2" stroke-miterlimit="10" pointer-events="none"/><path d="M 411 113.24 L 415 121.24 L 411 119.24 L 407 121.24 Z" fill="#000000" stroke="#000000" stroke-width="2" stroke-miterlimit="10" pointer-events="none"/><path d="M 571 171 L 51 171" fill="none" stroke="#b3b3b3" stroke-width="2" stroke-miterlimit="10" stroke-dasharray="6 6" pointer-events="none"/><g transform="translate(350.5,284.5)"><switch><foreignObject style="overflow:visible;" pointer-events="all" width="45" height="12" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; font-size: 12px; font-family: &quot;Times New Roman&quot;; color: rgb(0, 0, 0); line-height: 1.2; vertical-align: top; width: 46px; white-space: nowrap; word-wrap: normal; font-weight: bold; text-align: right;"><div xmlns="http://www.w3.org/1999/xhtml" style="display:inline-block;text-align:inherit;text-decoration:inherit;"><font face="Helvetica">Route: /</font></div></div></foreignObject><text x="23" y="12" fill="#000000" text-anchor="middle" font-size="12px" font-family="Times New Roman" font-weight="bold">[Not supported by viewer]</text></switch></g><g transform="translate(222.5,284.5)"><switch><foreignObject style="overflow:visible;" pointer-events="all" width="62" height="12" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; font-size: 12px; font-family: &quot;Times New Roman&quot;; color: rgb(0, 0, 0); line-height: 1.2; vertical-align: top; width: 63px; white-space: nowrap; word-wrap: normal; font-weight: bold;"><div xmlns="http://www.w3.org/1999/xhtml" style="display:inline-block;text-align:inherit;text-decoration:inherit;"><font face="Helvetica">Route: /api</font></div></div></foreignObject><text x="31" y="12" fill="#000000" text-anchor="middle" font-size="12px" font-family="Times New Roman" font-weight="bold">[Not supported by viewer]</text></switch></g></g></svg> \ No newline at end of file
diff --git a/src/files/iot-application/simple-iot-application.zip b/src/files/iot-application/simple-iot-application.zip
new file mode 100644
index 0000000..46d3205
--- /dev/null
+++ b/src/files/iot-application/simple-iot-application.zip
Binary files differ
diff --git a/src/files/iot-application/snakeviz.png b/src/files/iot-application/snakeviz.png
new file mode 100644
index 0000000..5bab395
--- /dev/null
+++ b/src/files/iot-application/snakeviz.png
Binary files differ
diff --git a/src/index.html b/src/index.html
new file mode 100644
index 0000000..383737b
--- /dev/null
+++ b/src/index.html
@@ -0,0 +1,19 @@
1<h2>Research</h2>
2<ul class="article-list">
3{{ range .Site.Pages.Children "research/" }}
4 <li>
5 {{ template "date" .Date }}
6 <a href="{{ $.Rel .Url }}" title="{{ .Title }}">{{ .Title }}</a>
7 </li>
8{{ end }}
9</ul>
10
11<h2>Blog posts</h2>
12<ul class="article-list">
13{{ range .Site.Pages.Children "blog/" }}
14 <li>
15 {{ template "date" .Date }}
16 <a href="{{ $.Rel .Url }}" title="{{ .Title }}">{{ .Title }}</a>
17 </li>
18{{ end }}
19</ul>
diff --git a/src/research/encoding-binary-data-into-dna-sequence.md b/src/research/encoding-binary-data-into-dna-sequence.md
new file mode 100644
index 0000000..3bfeaab
--- /dev/null
+++ b/src/research/encoding-binary-data-into-dna-sequence.md
@@ -0,0 +1,345 @@
1title: Encoding binary data into DNA sequence
2date: 2019-01-03
3tags: research
4hide: false
5----
6
7## Initial thoughts
8
9Imagine a world where you could go outside and take a leaf from a tree and put it through your personal DNA sequencer and get data like music, videos or computer programs from it. Well, this is all possible now. It was not done on a large scale because it is quite expensive to create DNA strands but it's possible.
10
11Encoding data into DNA sequence is relatively simple process once you understand the relationship between binary data and nucleotides and scientists have been making large leaps in this field in order to provide viable long-term storage solution for our data that would potentially survive our specie if case of global disaster. We could imprint all the world's knowledge into plants and ensure the survival of our knowledge.
12
13More optimistic usage for this technology would be easier storage of ever growing data we produce every day. Once machines for sequencing DNA become fast enough and cheaper this could mean the next evolution of storing data and abandoning classical hard and solid state drives in data warehouses.
14
15As we currently stand this is still not viable but it is quite an amazing and cool technology.
16
17My interests in this field are purely in encoding processes and experimental testing mainly because I don't have the access to this expensive machines. My initial goal was to create a toolkit that can be used by everybody to encode their data into a proper DNA sequence.
18
19## Glossary
20
21**deoxyribose**
22: A five-carbon sugar molecule with a hydrogen atom rather than a hydroxyl group in the 2′ position; the sugar component of DNA nucleotides.
23
24**double helix**
25: The molecular shape of DNA in which two strands of nucleotides wind around each other in a spiral shape.
26
27**nitrogenous base**
28: A nitrogen-containing molecule that acts as a base; often referring to one of the purine or pyrimidine components of nucleic acids.
29
30**phosphate group**
31: A molecular group consisting of a central phosphorus atom bound to four oxygen atoms.
32
33**RGB**
34: The RGB color model is an additive color model in which red, green and blue light are added together in various ways to reproduce a broad array of colors.
35
36**GCC**
37: The GNU Compiler Collection is a compiler system produced by the GNU Project supporting various programming languages.
38
39## Data encoding
40
41**TL;DR:** Encoding involves the use of a code to change original data into a form that can be used by an external process [^1].
42
43Encoding is the process of converting data into a format required for a number of information processing needs, including:
44
45- Program compiling and execution
46- Data transmission, storage and compression/decompression
47- Application data processing, such as file conversion
48
49Encoding can have two meanings[^1]:
50
51- In computer technology, encoding is the process of applying a specific code, such as letters, symbols and numbers, to data for conversion into an equivalent cipher.
52- In electronics, encoding refers to analog to digital conversion.
53
54## Quick history of DNA
55
56- **1869** - Friedrich Miescher identifies "nuclein".
57- **1900s** - The Eugenics Movement.
58- **1900** – Mendel's theories are rediscovered by researchers.
59- **1944** - Oswald Avery identifies DNA as the 'transforming principle'.
60- **1952** - Rosalind Franklin photographs crystallized DNA fibres.
61- **1953** - James Watson and Francis Crick discover the double helix structure of DNA.
62- **1965** - Marshall Nirenberg is the first person to sequence the bases in each codon.
63- **1983** - Huntington's disease is the first mapped genetic disease.
64- **1990** - The Human Genome Project begins.
65- **1995** - Haemophilus Influenzae is the first bacterium genome sequenced.
66- **1996** - Dolly the sheep is cloned.
67- **1999** - First human chromosome is decoded.
68- **2000** – Genetic code of the fruit fly is decoded.
69- **2002** – Mouse is the first mammal to have its genome decoded.
70- **2003** – The Human Genome Project is completed.
71- **2013** – DNA Worldwide and Eurofins Forensic discover identical twins have differences in their genetic makeup [^2].
72
73## What is DNA?
74
75Deoxyribonucleic acid, a self-replicating material which is **present in nearly all living organisms** as the main constituent of chromosomes. It is the **carrier of genetic information**.
76
77> The nitrogen in our DNA, the calcium in our teeth, the iron in our blood, the carbon in our apple pies were made in the interiors of collapsing stars. We are made of starstuff.
78>
79> **-- Carl Sagan, Cosmos**
80
81The nucleotide in DNA consists of a sugar (deoxyribose), one of four bases (cytosine (C), thymine (T), adenine (A), guanine (G)), and a phosphate. Cytosine and thymine are pyrimidine bases, while adenine and guanine are purine bases. The sugar and the base together are called a nucleoside.
82
83![DNA](/files/dna-sequence/dna-basics.jpg#center)
84
85*DNA (a) forms a double stranded helix, and (b) adenine pairs with thymine and cytosine pairs with guanine. (credit a: modification of work by Jerome Walker, Dennis Myts) [^3]*
86
87## Encode binary data into DNA sequence
88
89As an input file you can use any file you want:
90- ASCII files,
91- Compiled programs,
92- Multimedia files (MP3, MP4, MVK, etc),
93- Images,
94- Database files,
95- etc.
96
97Note: If you would copy all the bytes from RAM to file or pipe data to file you could encode also this data as long as you provide file pointer to the encoder.
98
99### Basic Encoding
100
101As already mentioned, the Basic Encoding is based on a simple mapping. Since DNA is composed of 4 nucleotides (Adenine, Cytosine, Guanine, Thymine; usually referred using the first letter). Using this technique we can encode
102
103$$ log_2(4) = log_2(2^2) = 2 bits $$
104
105using a single nucleotide. In this way, we are able to use the 4 bases that compose the DNA strand to encode each byte of data.
106
107| Two bits | Nucleotides |
108| -------- | ---------------- |
109| 00 | **A** (Adenine) |
110| 10 | **G** (Guanine) |
111| 01 | **C** (Cytosine) |
112| 11 | **T** (Thymine) |
113
114With this in mind we can simply encode any data by using two-bit to Nucleotides conversion
115
116```pascal
117{ Algorithm 1: Naive byte array to DNA encode }
118procedure EncodeToDNASequence(f) string
119begin
120 enc string
121 while not eof(f) do
122 c byte := buffer[0] { Read 1 byte from buffer }
123 bin integer := sprintf('08b', c) { Convert to string binary }
124 for e in range[0, 2, 4, 6] do
125 if e[0] == 48 and e[1] == 48 then { 0x00 - A (Adenine) }
126 enc += 'A'
127 else if e[0] == 48 and e[1] == 49 then { 0x01 - G (Guanine) }
128 enc += 'G'
129 else if e[0] == 49 and e[1] == 48 then { 0x10 - C (Cytosine) }
130 enc += 'C'
131 else if e[0] == 49 and e[1] == 49 then { 0x11 - T (Thymine) }
132 enc += 'T'
133 return enc { Return DNA sequence }
134end
135```
136
137Another encoding would be **Goldman encoding**. Using this encoding helps with Nonsense mutation (amino acids replaced by a stop codon) that occurs and is the most problematic during translation because it leads to truncated amino acid sequences, which in turn results in truncated proteins. [^4]
138
139[Where to store big data? In DNA: Nick Goldman at TEDxPrague](https://www.youtube.com/watch?v=a4PiGWNsIEU)
140
141### FASTA file format
142
143In bioinformatics, FASTA format is a text-based format for representing either nucleotide sequences or peptide sequences, in which nucleotides or amino acids are represented using single-letter codes. The format also allows for sequence names and comments to precede the sequences. The format originates from the FASTA software package, but has now become a standard in the field of bioinformatics. [^5]
144
145The first line in a FASTA file started either with a ">" (greater-than) symbol or, less frequently, a ";" (semicolon) was taken as a comment. Subsequent lines starting with a semicolon would be ignored by software. Since the only comment used was the first, it quickly became used to hold a summary description of the sequence, often starting with a unique library accession number, and with time it has become commonplace to always use ">" for the first line and to not use ";" comments (which would otherwise be ignored).
146
147```text
148;LCBO - Prolactin precursor - Bovine
149; a sample sequence in FASTA format
150MDSKGSSQKGSRLLLLLVVSNLLLCQGVVSTPVCPNGPGNCQVSLRDLFDRAVMVSHYIHDLSS
151EMFNEFDKRYAQGKGFITMALNSCHTSSLPTPEDKEQAQQTHHEVLMSLILGLLRSWNDPLYHL
152VTEVRGMKGAPDAILSRAIEIEEENKRLLEGMEMIFGQVIPGAKETEPYPVWSGLPSLQTKDED
153ARYSAFYNLLHCLRRDSSKIDTYLKLLNCRIIYNNNC*
154
155>MCHU - Calmodulin - Human, rabbit, bovine, rat, and chicken
156ADQLTEEQIAEFKEAFSLFDKDGDGTITTKELGTVMRSLGQNPTEAELQDMINEVDADGNGTID
157FPEFLTMMARKMKDTDSEEEIREAFRVFDKDGNGYISAAELRHVMTNLGEKLTDEEVDEMIREA
158DIDGDGQVNYEEFVQMMTAK*
159
160>gi|5524211|gb|AAD44166.1| cytochrome b [Elephas maximus maximus]
161LCLYTHIGRNIYYGSYLYSETWNTGIMLLLITMATAFMGYVLPWGQMSFWGATVITNLFSAIPYIGTNLV
162EWIWGGFSVDKATLNRFFAFHFILPFTMVALAGVHLTFLHETGSNNPLGLTSDSDKIPFHPYYTIKDFLG
163LLILILLLLLLALLSPDMLGDPDNHMPADPLNTPLHIKPEWYFLFAYAILRSVPNKLGGVLALFLSIVIL
164GLMPFLHTSKHRSMMLRPLSQALFWTLTMDLLTLTWIGSQPVEYPYTIIGQMASILYFSIILAFLPIAGX
165IENY
166```
167
168FASTA format was extended by [FASTQ](https://en.wikipedia.org/wiki/FASTQ_format) format from the [Sanger Centre](https://www.sanger.ac.uk/) in Cambridge.
169
170### PNG encoded DNA sequence
171
172| Nucleotides | RGB | Color name |
173| ------------- | ----------- | ---------- |
174| A -> Adenine | (0,0,255) | Blue |
175| G -> Guanine | (0,100,0) | Green |
176| C -> Cytosine | (255,0,0) | Red |
177| T -> Thymine | (255,255,0) | Yellow |
178
179With this in mind we can create a simple algorithm to create PNG representation of a DNA sequence.
180
181```pascal
182{ Algorithm 2: Naive DNA to PNG encode from FASTA file }
183procedure EncodeDNASequenceToPNG(f)
184begin
185 i image
186 while not eof(f) do
187 c char := buffer[0] { Read 1 char from buffer }
188 case c of
189 'A': color := RGB(0, 0, 255) { Blue }
190 'G': color := RGB(0, 100, 0) { Green }
191 'C': color := RGB(255, 0, 0) { Red }
192 'T': color := RGB(255, 255, 0) { Yellow }
193 drawRect(i, [x, y], color)
194 save(i) { Save PNG image }
195end
196```
197
198## Encoding text file in practice
199
200In this example we will take a simple text file as our input stream for encoding. This file will have a quote from Niels Bohr and saved as txt file.
201
202> How wonderful that we have met with a paradox. Now we have some hope of making progress.
203> ― Niels Bohr
204
205First we encode text file into FASTA file.
206
207```bash
208./dnae-encode -i quote.txt -o quote.fa
2092019/01/10 00:38:29 Gathering input file stats
2102019/01/10 00:38:29 Starting encoding ...
211 106 B / 106 B [==================================] 100.00% 0s
2122019/01/10 00:38:29 Saving to FASTA file ...
2132019/01/10 00:38:29 Output FASTA file length is 438 B
2142019/01/10 00:38:29 Process took 987.263µs
2152019/01/10 00:38:29 Done ...
216```
217
218Output of `quote.fa` file contains the encoded DNA sequence in ASCII format.
219
220```text
221>SEQ1
222GACAGCTTGTGTACAAGTGTGCTTGCTCGCGAGCGGGTACGCGCGTGGGCTAACAAGTGA
223GCCAGCAGGTGAACAAGTGTGCGGACAAGCCAGCAGGTGCGCGGACAAGCTGGCGGGTGA
224ACAAGTGTGCCGGTGAGCCAACAAGCAGACAAGTAAGCAGGTACGCAGGCGAGCTTGTCA
225ACTCACAAGATCGCTTGTGTACAAGTGTGCGGACAAGCCAGCAGGTGCGCGGACAAGTAT
226GCTTGCTGGCGGACAAGCCAGCTTGTAAGCGGACAAGCTTGCGCACAAGCTGGCAGGCCT
227GCCGGCTCGCGTACAAATTCACAAGTAAGTACGCTTGCGTGTACGCGGGTATGTATACTC
228AACCTCACCAAACGGGACAAGATCGCCGGCGGGCTAGTATACAAGAACGCTTGCCAGTAC
229AACC
230```
231
232Then we encode FASTA file from previous operation to encode this data into PNG.
233
234```bash
235./dnae-png -i quote.fa -o quote.png
2362019/01/10 00:40:09 Gathering input file stats ...
2372019/01/10 00:40:09 Deconstructing FASTA file ...
2382019/01/10 00:40:09 Compositing image file ...
239 424 / 424 [==================================] 100.00% 0s
2402019/01/10 00:40:09 Saving output file ...
2412019/01/10 00:40:09 Output image file length is 1.1 kB
2422019/01/10 00:40:09 Process took 19.036117ms
2432019/01/10 00:40:09 Done ...
244```
245
246After encoding into PNG format this file looks like this.
247
248![Encoded Quote in PNG format](/files/dna-sequence/quote.png)
249
250The larger the input stream is the larger the PNG file would be.
251
252Compiled basic Hello World C program with [GCC](https://www.gnu.org/software/gcc/) would [look like](/files/dna-sequence/sample.png).
253
254```c
255// gcc -O3 -o sample sample.c
256#include <stdio.h>
257
258main() {
259 printf("Hello, world!\n");
260 return 0;
261}
262```
263
264## Toolkit for encoding data
265
266I have created a toolkit with two main programs:
267- dnae-encode (encodes file into FASTA file)
268- dnae-png (encodes FASTA file into PNG)
269
270Toolkit with full source code is available on [github.com/mitjafelicijan/dna-encoding](https://github.com/mitjafelicijan/dna-encoding).
271
272### dnae-encode
273
274```bash
275> ./dnae-encode --help
276usage: dnae-encode --input=INPUT [<flags>]
277
278A command-line application that encodes file into DNA sequence.
279
280Flags:
281 --help Show context-sensitive help (also try --help-long and --help-man).
282 -i, --input=INPUT Input file (ASCII or binary) which will be encoded into DNA sequence.
283 -o, --output="out.fa" Output file which stores DNA sequence in FASTA format.
284 -s, --sequence=SEQ1 The description line (defline) or header/identifier line, gives a name and/or a unique identifier for the sequence.
285 -c, --columns=60 Row characters length (no more than 120 characters). Devices preallocate fixed line sizes in software.
286 --version Show application version.
287```
288
289### dnae-png
290
291```bash
292> ./dnae-png --help
293usage: dnae-png --input=INPUT [<flags>]
294
295A command-line application that encodes FASTA file into PNG image.
296
297Flags:
298 --help Show context-sensitive help (also try --help-long and --help-man).
299 -i, --input=INPUT Input FASTA file which will be encoded into PNG image.
300 -o, --output="out.png" Output file in PNG format that represents DNA sequence in graphical way.
301 -s, --size=10 Size of pairings of DNA bases on image in pixels (lower resolution lower file size).
302 --version Show application version.
303```
304
305## Benchmarks
306
307First we generate some binary sample data with dd.
308
309```bash
310dd if=<(openssl enc -aes-256-ctr -pass pass:"$(dd if=/dev/urandom bs=128 count=1 2>/dev/null | base64)" -nosalt < /dev/zero) of=1KB.bin bs=1KB count=1 iflag=fullblock
311```
312
313Our freshly generated 1KB file looks something like this (its full of garbage data as intended).
314
315![Sample binary file 1KB](/files/dna-sequence/sample-binary-file.png)
316
317We create following binary files:
318- 1KB.bin
319- 10KB.bin
320- 100KB.bin
321- 1MB.bin
322- 10MB.bin
323- 100MB.bin
324
325After this we create FASTA files for all the binary files by encoding them into DNA sequence.
326
327```bash
328./dnae-encode -i 100MB.bin -o 100MB.fa
329```
330
331Then we GZIP all the FASTA files to see how much the can be compressed.
332
333```bash
334gzip -9 < 10MB.fa > 10MB.fa.gz
335```
336
337[Download ODS file with benchmarks](/files/dna-sequence/benchmarks.ods).
338
339## References
340
341[^1]: https://www.techopedia.com/definition/948/encoding
342[^2]: https://www.dna-worldwide.com/resource/160/history-dna-timeline
343[^3]: https://opentextbc.ca/biology/chapter/9-1-the-structure-of-dna/
344[^4]: https://arxiv.org/abs/1801.04774
345[^5]: https://en.wikipedia.org/wiki/FASTA_format
diff --git a/src/research/using-digitalocean-spaces-object-storage-with-fuse.md b/src/research/using-digitalocean-spaces-object-storage-with-fuse.md
new file mode 100644
index 0000000..099fbef
--- /dev/null
+++ b/src/research/using-digitalocean-spaces-object-storage-with-fuse.md
@@ -0,0 +1,260 @@
1title: Using DigitalOcean Spaces Object Storage with FUSE
2date: 2018-01-16
3tags: research
4hide: false
5----
6
7Couple of months ago [DigitalOcean](https://www.digitalocean.com) introduced new product called [Spaces](https://blog.digitalocean.com/introducing-spaces-object-storage/) which is Object Storage very similar to Amazon's S3. This really peaked my interest, because this was something I was missing and even the thought of going over the internet for such functionality was in no interest to me. Also in fashion with their previous pricing this also is very cheap and pricing page is a no-brainer compared to AWS or GCE. [Prices are clearly and precisely defined and outlined](https://www.digitalocean.com/pricing/). You must love them for that :)
8
9### Initial requirements
10
11* Is it possible to use them as a mounted drive with FUSE? (tl;dr YES)
12* Will the performance degrade over time and over different sizes of objects? (tl;dr NO&YES)
13* Can storage be mounted on multiple machines at the same time and be writable? (tl;dr YES)
14
15> Let me be clear. This scripts I use are made just for benchmarking and are not intended to be used in real-life situations. Besides that, I am looking into using this approaches but adding caching service in front of it and then dumping everything as an object to storage. This could potentially be some interesting post of itself. But in case you would need real-time data without eventual consistency please take this scripts as they are: not usable in such situations.
16
17## Is it possible to use them as a mounted drive with FUSE?
18
19Well, actually they can be used in such manor. Because they are similar to [AWS S3](https://aws.amazon.com/s3/) many tools are available and you can find many articles and [Stackoverflow items](https://stackoverflow.com/search?q=s3+fuse).
20
21To make this work you will need DigitalOcean account. If you don't have one you will not be able to test this code. But if you have an account then you go and [create new Droplet](https://cloud.digitalocean.com/droplets/new?size=s-1vcpu-1gb&region=ams3&distro=debian&distroImage=debian-9-x64&options=private_networking,install_agent). If you click on this link you will already have preselected Debian 9 with smallest VM option.
22
23* Please be sure to add you SSH key, because we will login to this machine remotely.
24* If you change your region please remember which one you choose because we will need this information when we try to mount space to our machine.
25
26Instuctions on how to use SSH keys and how to setup them are available in article [How To Use SSH Keys with DigitalOcean Droplets](https://www.digitalocean.com/community/tutorials/how-to-use-ssh-keys-with-digitalocean-droplets).
27
28![DigitalOcean Droplets](/files/do-fuse/fuse-droplets.png)
29
30After we created Droplet it's time to create new Space. This is done by clicking on a button [Create](https://cloud.digitalocean.com/spaces/new) (right top corner) and selecting Spaces. Choose pronounceable ```Unique name``` because we will use it in examples below. You can either choose Private or Public, it doesn't matter in our case. And you can always change that in the future.
31
32When you have created new Space we should [generate Access key](https://cloud.digitalocean.com/settings/api/tokens). This link will guide to the page when you can generate this key. After you create new one, please save provided Key and Secret because Secret will not be shown again.
33
34![DigitalOcean Spaces](/files/do-fuse/fuse-spaces.png)
35
36Now that we have new Space and Access key we should SSH into our machine.
37
38```bash
39# replace IP with the ip of your newly created droplet
40ssh root@IP
41
42# this will install utilities for mounting storage objects as FUSE
43apt install s3fs
44
45# we now need to provide credentials (access key we created earlier)
46# replace KEY and SECRET with your own credentials but leave the colon between them
47# we also need to set proper permissions
48echo "KEY:SECRET" > .passwd-s3fs
49chmod 600 .passwd-s3fs
50
51# now we mount space to our machine
52# replace UNIQUE-NAME with the name you choose earlier
53# if you choose different region for your space be careful about -ourl option (ams3)
54s3fs UNIQUE-NAME /mnt/ -ourl=https://ams3.digitaloceanspaces.com -ouse_cache=/tmp
55
56# now we try to create a file
57# once you mount it may take a couple of seconds to retrieve data
58echo "Hello cruel world" > /mnt/hello.txt
59```
60
61After all this you can return to your browser and go to [DigitalOcean Spaces](https://cloud.digitalocean.com/spaces) and click on your created space. If file hello.txt is present you have successfully mounted space to your machine and wrote data to it.
62
63I choose the same region for my Droplet and my Space but you don't have to. You can have different regions. What this actually does to performance I don't know.
64
65Additional information on FUSE:
66
67* [Github project page for s3fs](https://github.com/s3fs-fuse/s3fs-fuse)
68* [FUSE - Filesystem in Userspace](https://en.wikipedia.org/wiki/Filesystem_in_Userspace)
69
70## Will the performance degrade over time and over different sizes of objects?
71
72For this task I didn't want to just read and write text files or uploading images. I actually wanted to figure out if using something like SQlite is viable in this case.
73
74### Measurement experiment 1: File copy
75
76```bash
77# first we create some dummy files at different sizes
78dd if=/dev/zero of=10KB.dat bs=1024 count=10 #10KB
79dd if=/dev/zero of=100KB.dat bs=1024 count=100 #100KB
80dd if=/dev/zero of=1MB.dat bs=1024 count=1024 #1MB
81dd if=/dev/zero of=10MB.dat bs=1024 count=10240 #10MB
82
83# now we set time command to only return real
84TIMEFORMAT=%R
85
86# now lets test it
87(time cp 10KB.dat /mnt/) |& tee -a 10KB.results.txt
88
89# and now we automate
90# this will perform the same operation 100 times
91# this will output results into separated files based on objecty size
92n=0; while (( n++ < 100 )); do (time cp 10KB.dat /mnt/10KB.$n.dat) |& tee -a 10KB.results.txt; done
93n=0; while (( n++ < 100 )); do (time cp 100KB.dat /mnt/100KB.$n.dat) |& tee -a 100KB.results.txt; done
94n=0; while (( n++ < 100 )); do (time cp 1MB.dat /mnt/1MB.$n.dat) |& tee -a 1MB.results.txt; done
95n=0; while (( n++ < 100 )); do (time cp 10MB.dat /mnt/10MB.$n.dat) |& tee -a 10MB.results.txt; done
96```
97
98Files of size 100MB were not successfully transferred and ended up displaying error (cp: failed to close '/mnt/100MB.1.dat': Operation not permitted).
99
100As I suspected, object size is not really that important. Sadly I don't have the time to test performance over periods of time. But if some of you would do it please send me your data. I would be interested in seeing results.
101
102**Here are plotted results**
103
104You can download [raw result here](/files/do-fuse/copy-benchmarks.tsv). Measurements are in seconds.
105
106<script src="//cdn.plot.ly/plotly-latest.min.js"></script>
107<div id="copy-benchmarks"></div>
108<script>
109(function(){
110 var request = new XMLHttpRequest();
111 request.open("GET", "/files/do-fuse/copy-benchmarks.tsv", true);
112 request.onload = function() {
113 if (request.status >= 200 && request.status < 400) {
114 var payload = request.responseText.trim();
115 var tsv = payload.split("\n");
116 for (var i=0; i<tsv.length; i++) { tsv[i] = tsv[i].split("\t"); }
117 var traces = [];
118 var headers = tsv[0];
119 tsv.shift();
120 Array.prototype.forEach.call(headers, function(el, idx) {
121 var x = [];
122 var y = [];
123 for (var j=0; j<tsv.length; j++) {
124 x.push(j);
125 y.push(parseFloat(tsv[j][idx].replace(",", ".")));
126 }
127 traces.push({ x: x, y: y, type: "scatter", name: el, line: { width: 1, shape: "spline" } });
128 });
129 var copy = Plotly.newPlot("copy-benchmarks", traces, { legend: {"orientation": "h"}, height: 400, margin: { l: 40, r: 0, b: 20, t: 30, pad: 0 }, yaxis: { title: "execution time in seconds", titlefont: { size: 12 } }, xaxis: { title: "fn(i)", titlefont: { size: 12 } } });
130 } else { }
131 };
132 request.onerror = function() { };
133 request.send(null);
134})();
135</script>
136
137As far as these tests show, performance is quite stable and can be predicted which is fantastic. But this is a small test and spans only over couple of hours. So you should not completely trust them.
138
139### Measurement experiment 2: SQLite performanse
140
141I was unable to use database file directly from mounted drive so this is a no-go as I suspected. So I executed code below on a local disk just to get some benchmarks. I inserted 1000 records with DROPTABLE, CREATETABLE, INSERTMANY, FETCHALL, COMMIT for 1000 times to generate statistics. As you can see performance of SQLite is quite amazing. You could then potentially just copy file to mounted drive and be done with it.
142
143```python
144import time
145import sqlite3
146import sys
147
148if len(sys.argv) < 3:
149 print("usage: python sqlite-benchmark.py DB_PATH NUM_RECORDS REPEAT")
150 exit()
151
152def data_iter(x):
153 for i in range(x):
154 yield "m" + str(i), "f" + str(i*i)
155
156header_line = "%s\t%s\t%s\t%s\t%s\n" % ("DROPTABLE", "CREATETABLE", "INSERTMANY", "FETCHALL", "COMMIT")
157with open("sqlite-benchmarks.tsv", "w") as fp:
158 fp.write(header_line)
159
160start_time = time.time()
161conn = sqlite3.connect(sys.argv[1])
162c = conn.cursor()
163end_time = time.time()
164result_time = CONNECT = end_time - start_time
165print("CONNECT: %g seconds" % (result_time))
166
167start_time = time.time()
168c.execute("PRAGMA journal_mode=WAL")
169c.execute("PRAGMA temp_store=MEMORY")
170c.execute("PRAGMA synchronous=OFF")
171result_time = PRAGMA = end_time - start_time
172print("PRAGMA: %g seconds" % (result_time))
173
174for i in range(int(sys.argv[3])):
175 print("#%i" % (i))
176
177 start_time = time.time()
178 c.execute("drop table if exists test")
179 end_time = time.time()
180 result_time = DROPTABLE = end_time - start_time
181 print("DROPTABLE: %g seconds" % (result_time))
182
183 start_time = time.time()
184 c.execute("create table if not exists test(a,b)")
185 end_time = time.time()
186 result_time = CREATETABLE = end_time - start_time
187 print("CREATETABLE: %g seconds" % (result_time))
188
189 start_time = time.time()
190 c.executemany("INSERT INTO test VALUES (?, ?)", data_iter(int(sys.argv[2])))
191 end_time = time.time()
192 result_time = INSERTMANY = end_time - start_time
193 print("INSERTMANY: %g seconds" % (result_time))
194
195 start_time = time.time()
196 c.execute("select count(*) from test")
197 res = c.fetchall()
198 end_time = time.time()
199 result_time = FETCHALL = end_time - start_time
200 print("FETCHALL: %g seconds" % (result_time))
201
202 start_time = time.time()
203 conn.commit()
204 end_time = time.time()
205 result_time = COMMIT = end_time - start_time
206 print("COMMIT: %g seconds" % (result_time))
207
208 print
209 log_line = "%f\t%f\t%f\t%f\t%f\n" % (DROPTABLE, CREATETABLE, INSERTMANY, FETCHALL, COMMIT)
210 with open("sqlite-benchmarks.tsv", "a") as fp:
211 fp.write(log_line)
212
213start_time = time.time()
214conn.close()
215end_time = time.time()
216result_time = CLOSE = end_time - start_time
217print("CLOSE: %g seconds" % (result_time))
218```
219
220You can download [raw result here](/files/do-fuse/sqlite-benchmarks.tsv). And again, these results are done on a local block storage and do not represent capabilities of object storage. With my current approach and state of the test code these can not be done. I would need to make Python code much more robust and check locking etc.
221
222<div id="sqlite-benchmarks"></div>
223<script>
224(function(){
225 var request = new XMLHttpRequest();
226 request.open("GET", "/files/do-fuse/sqlite-benchmarks.tsv", true);
227 request.onload = function() {
228 if (request.status >= 200 && request.status < 400) {
229 var payload = request.responseText.trim();
230 var tsv = payload.split("\n");
231 for (var i=0; i<tsv.length; i++) { tsv[i] = tsv[i].split("\t"); }
232 var traces = [];
233 var headers = tsv[0];
234 tsv.shift();
235 Array.prototype.forEach.call(headers, function(el, idx) {
236 var x = [];
237 var y = [];
238 for (var j=0; j<tsv.length; j++) {
239 x.push(j);
240 y.push(parseFloat(tsv[j][idx].replace(",", ".")));
241 }
242 traces.push({ x: x, y: y, type: "scatter", name: el, line: { width: 1, shape: "spline" } });
243 });
244 var sqlite = Plotly.newPlot("sqlite-benchmarks", traces, { legend: {"orientation": "h"}, height: 400, margin: { l: 50, r: 0, b: 20, t: 30, pad: 0 }, yaxis: { title: "execution time in seconds", titlefont: { size: 12 } } });
245 } else { }
246 };
247 request.onerror = function() { };
248 request.send(null);
249})();
250</script>
251
252## Can storage be mounted on multiple machines at the same time and be writable?
253
254Well, this one didn't take long to test. And the answer is **YES**. I mounted space on both machines and measured same performance on both machines. But because file is downloaded before write and then uploaded on complete there could potentially be problems is another process is trying to access the same file.
255
256## Observations and conslusion
257
258Using Spaces in this way makes it easier to access and manage files. But besides that you would need to write additional code to make this one play nice with you applications.
259
260Nevertheless, this was extremely simple to setup and use and this is just another excellent product in DigitalOcean product line. I found this exercise very valuable and am thinking about implementing some sort of mechanism for SQLite, so data can be stored on Spaces and accessed by many VM's. For a project where data doesn't need to be accessible in real-time and can have couple of minutes old data this would be very interesting. If any of you find this proposal interesting please write in a comment box below or shoot me an email and I will keep you posted.
diff --git a/src/static/avatar-512x512.png b/src/static/avatar-512x512.png
new file mode 100644
index 0000000..fc34eee
--- /dev/null
+++ b/src/static/avatar-512x512.png
Binary files differ
diff --git a/src/static/avatar-64x64.png b/src/static/avatar-64x64.png
new file mode 100644
index 0000000..1406f46
--- /dev/null
+++ b/src/static/avatar-64x64.png
Binary files differ
diff --git a/src/static/style.css b/src/static/style.css
new file mode 100644
index 0000000..efb65c0
--- /dev/null
+++ b/src/static/style.css
@@ -0,0 +1,116 @@
1body {
2 line-height: 150%;
3 margin-bottom: 100px;
4}
5
6main {
7 max-width: 680px;
8 padding: 20px 30px;
9}
10
11nav {
12 margin-bottom: 50px;
13}
14
15nav a {
16 display: inline-block;
17 margin-right: 20px;
18}
19
20h1 {
21 font-size: 280%;
22 line-height: initial;
23}
24
25h2 {
26 font-size: 200%;
27 line-height: initial;
28}
29
30h3 {
31 font-size: 160%;
32 line-height: initial;
33}
34
35img {
36 max-width: 100%;
37 margin: 30px auto;
38 display: block;
39}
40
41ul.article-list li {
42 margin-bottom: 10px;
43}
44
45ul.article-list time {
46 display: inline-block;
47 min-width: 120px;
48}
49
50article .info {
51 font-style: oblique;
52}
53
54pre {
55 overflow: auto;
56}
57
58table {
59 width: 100%;
60}
61
62table, th, td {
63 border: 1px solid black;
64 text-align: left;
65}
66
67th, td {
68 padding: 5px 10px;
69}
70
71::selection {
72 background: #ff0;
73 color: #000;
74}
75
76::-moz-selection {
77 background: #ff0;
78 color: #000;
79}
80
81@media only screen and (max-width:480px) {
82 nav {
83 text-align: center;
84 margin-bottom: initial;
85 }
86
87 nav a {
88 margin-bottom: 10px;
89 }
90
91 ul.article-list {
92 padding-left: 20px;
93
94 }
95
96 ul.article-list li {
97 margin-bottom: 20px;
98
99 }
100
101 ul.article-list a {
102 display: block;
103 }
104
105 h1 {
106 font-size: 220%;
107 }
108
109 h2 {
110 font-size: 180%;
111 }
112
113 h3 {
114 font-size: 160%;
115 }
116}