diff options
| author | Mitja Felicijan <mitja.felicijan@gmail.com> | 2023-10-31 10:17:43 +0100 |
|---|---|---|
| committer | Mitja Felicijan <mitja.felicijan@gmail.com> | 2023-10-31 10:17:43 +0100 |
| commit | ae24d9a8869c497537839f330384cbadb2cf687c (patch) | |
| tree | 79184f1d1db6f0b8e2f7aad3a262b3a2bccdde58 /public/profiling-python-web-applications-with-visual-tools.html | |
| parent | 409dfae3983b55ffa469a556683fab4ec70b1e99 (diff) | |
| download | mitjafelicijan.com-ae24d9a8869c497537839f330384cbadb2cf687c.tar.gz | |
Updated theme
Diffstat (limited to 'public/profiling-python-web-applications-with-visual-tools.html')
| -rwxr-xr-x | public/profiling-python-web-applications-with-visual-tools.html | 16 |
1 files changed, 8 insertions, 8 deletions
diff --git a/public/profiling-python-web-applications-with-visual-tools.html b/public/profiling-python-web-applications-with-visual-tools.html index 77ba60a..c5126c7 100755 --- a/public/profiling-python-web-applications-with-visual-tools.html +++ b/public/profiling-python-web-applications-with-visual-tools.html | |||
| @@ -10,7 +10,7 @@ | |||
| 10 | missing this option when I am developing API's or other web services. I always | 10 | missing this option when I am developing API's or other web services. I always |
| 11 | knew that this is possible but never really took the time and dive into it.<p>Before we begin there are some requirements. We will need to:<ul><li>implement <a href=https://docs.python.org/2/library/profile.html#module-cProfile>cProfile</a> into our web app,<li>convert output to <a href=http://valgrind.org/docs/manual/cl-manual.html>callgrind</a> format with <a href=https://pypi.python.org/pypi/pyprof2calltree/>pyprof2calltree</a>,<li>visualize data with <a href=http://kcachegrind.sourceforge.net/html/Home.html>KCachegrind</a> or <a href=http://www.profilingviewer.com/>Profiling Viewer</a>.</ul><p>If you are using MacOS you should check out <a href=http://www.profilingviewer.com/>Profiling | 11 | knew that this is possible but never really took the time and dive into it.<p>Before we begin there are some requirements. We will need to:<ul><li>implement <a href=https://docs.python.org/2/library/profile.html#module-cProfile>cProfile</a> into our web app,<li>convert output to <a href=http://valgrind.org/docs/manual/cl-manual.html>callgrind</a> format with <a href=https://pypi.python.org/pypi/pyprof2calltree/>pyprof2calltree</a>,<li>visualize data with <a href=http://kcachegrind.sourceforge.net/html/Home.html>KCachegrind</a> or <a href=http://www.profilingviewer.com/>Profiling Viewer</a>.</ul><p>If you are using MacOS you should check out <a href=http://www.profilingviewer.com/>Profiling |
| 12 | Viewer</a> or | 12 | Viewer</a> or |
| 13 | <a href=http://www.maccallgrind.com/>MacCallGrind</a>.<figure><img src=/posts/python-profiling/kcachegrind.png alt=KCachegrind></figure><p>We will be dividing this post into two main categories:<ul><li>writing simple web-service,<li>visualize profile of this web-service.</ul><h2 id=simple-web-service>Simple web-service</h2><p>Let's use virtualenv so we won't pollute our base system. If you don't have | 13 | <a href=http://www.maccallgrind.com/>MacCallGrind</a>.<figure><img loading="lazy" src=/posts/python-profiling/kcachegrind.png alt=KCachegrind></figure><p>We will be dividing this post into two main categories:<ul><li>writing simple web-service,<li>visualize profile of this web-service.</ul><h2 id=simple-web-service>Simple web-service</h2><p>Let's use virtualenv so we won't pollute our base system. If you don't have |
| 14 | virtualenv installed on your system you can install it with pip command.<pre tabindex=0 style=background-color:#fff><code><span style=display:flex><span><span style=color:green># let's install virtualenv globally</span> | 14 | virtualenv installed on your system you can install it with pip command.<pre tabindex=0 style=background-color:#fff><code><span style=display:flex><span><span style=color:green># let's install virtualenv globally</span> |
| 15 | </span></span><span style=display:flex><span>$ sudo pip install virtualenv | 15 | </span></span><span style=display:flex><span>$ sudo pip install virtualenv |
| 16 | </span></span><span style=display:flex><span> | 16 | </span></span><span style=display:flex><span> |
| @@ -105,7 +105,7 @@ subfolder.<h2 id=visualize-profile>Visualize profile</h2><p>Now let's create cal | |||
| 105 | </span></span></code></pre><p>This file can be opened with visualizing tools listed above. In this case we | 105 | </span></span></code></pre><p>This file can be opened with visualizing tools listed above. In this case we |
| 106 | will be using Profilling Viewer under MacOS. You can open image in new tab. As | 106 | will be using Profilling Viewer under MacOS. You can open image in new tab. As |
| 107 | you can see from this example there is hierarchy of execution order of your | 107 | you can see from this example there is hierarchy of execution order of your |
| 108 | code.<figure><img src=/posts/python-profiling/profiling-viewer.png alt="Profilling Viewer"></figure><blockquote><p>Make sure you convert output of the cProfile output every time you want to | 108 | code.<figure><img loading="lazy" src=/posts/python-profiling/profiling-viewer.png alt="Profilling Viewer"></figure><blockquote><p>Make sure you convert output of the cProfile output every time you want to |
| 109 | refresh and take a look at your possible optimizations because cProfile updates | 109 | refresh and take a look at your possible optimizations because cProfile updates |
| 110 | .prof file every time browser hits the function.</blockquote><p>This is just a simple example but when you are developing real-life applications | 110 | .prof file every time browser hits the function.</blockquote><p>This is just a simple example but when you are developing real-life applications |
| 111 | this can be very illuminating, especially to see which parts of your code are | 111 | this can be very illuminating, especially to see which parts of your code are |
| @@ -121,7 +121,7 @@ module.<div class=reddit-embed data-embed-media=www.redditmedia.com data-embed-p | |||
| 121 | </span></span><span style=display:flex><span>$ snakeviz awesome_random_number.prof | 121 | </span></span><span style=display:flex><span>$ snakeviz awesome_random_number.prof |
| 122 | </span></span><span style=display:flex><span><span style=color:green># this automatically opens browser window and</span> | 122 | </span></span><span style=display:flex><span><span style=color:green># this automatically opens browser window and</span> |
| 123 | </span></span><span style=display:flex><span><span style=color:green># shows visualized profile</span> | 123 | </span></span><span style=display:flex><span><span style=color:green># shows visualized profile</span> |
| 124 | </span></span></code></pre><figure><img src=/posts/python-profiling/snakeviz.png alt=SnakeViz></figure><p>Reddit user <a href=https://www.reddit.com/user/ccharles>ccharles</a> suggested a better | 124 | </span></span></code></pre><figure><img loading="lazy" src=/posts/python-profiling/snakeviz.png alt=SnakeViz></figure><p>Reddit user <a href=https://www.reddit.com/user/ccharles>ccharles</a> suggested a better |
| 125 | way for installing pip software by targeting user level instead of using sudo.<div class=reddit-embed data-embed-media=www.redditmedia.com data-embed-parent=false data-embed-live=false data-embed-uuid=f4f0459e-684d-441e-bebe-eb49b2f0a31d data-embed-created=2017-04-22T19:46:10.874Z><a href=https://www.reddit.com/r/Python/comments/66v373/profiling_python_web_applications_with_visual/dglpzkx/>Comment</a> from discussion <a href=https://www.reddit.com/r/Python/comments/66v373/profiling_python_web_applications_with_visual/>Profiling Python web applications with visual tools</a>.</div><script async src=https://www.redditstatic.com/comment-embed.js></script><pre tabindex=0 style=background-color:#fff><code><span style=display:flex><span><span style=color:green># now we need to add this path to our $PATH variable</span> | 125 | way for installing pip software by targeting user level instead of using sudo.<div class=reddit-embed data-embed-media=www.redditmedia.com data-embed-parent=false data-embed-live=false data-embed-uuid=f4f0459e-684d-441e-bebe-eb49b2f0a31d data-embed-created=2017-04-22T19:46:10.874Z><a href=https://www.reddit.com/r/Python/comments/66v373/profiling_python_web_applications_with_visual/dglpzkx/>Comment</a> from discussion <a href=https://www.reddit.com/r/Python/comments/66v373/profiling_python_web_applications_with_visual/>Profiling Python web applications with visual tools</a>.</div><script async src=https://www.redditstatic.com/comment-embed.js></script><pre tabindex=0 style=background-color:#fff><code><span style=display:flex><span><span style=color:green># now we need to add this path to our $PATH variable</span> |
| 126 | </span></span><span style=display:flex><span><span style=color:green># we do this my adding this line at the end of your</span> | 126 | </span></span><span style=display:flex><span><span style=color:green># we do this my adding this line at the end of your</span> |
| 127 | </span></span><span style=display:flex><span><span style=color:green># ~/.bashrc file</span> | 127 | </span></span><span style=display:flex><span><span style=color:green># ~/.bashrc file</span> |
| @@ -138,11 +138,11 @@ way for installing pip software by targeting user level instead of using sudo.<d | |||
| 138 | </span></span><span style=display:flex><span><span style=color:green># without use of sudo</span> | 138 | </span></span><span style=display:flex><span><span style=color:green># without use of sudo</span> |
| 139 | </span></span><span style=display:flex><span>$ pip install snakeviz --user | 139 | </span></span><span style=display:flex><span>$ pip install snakeviz --user |
| 140 | </span></span></code></pre><p>Or as suggested by <a href=https://www.reddit.com/user/mvt>mvt</a> you can | 140 | </span></span></code></pre><p>Or as suggested by <a href=https://www.reddit.com/user/mvt>mvt</a> you can |
| 141 | use <a href=https://github.com/mitsuhiko/pipsi>pipsi</a>.</div></article></main><section><hr><h2>Posts from blogs I follow around the net</h2><ul><li><a href=https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSWhyNotDirectoryToFilesystem target=_blank rel=noopener>One reason that ZFS can't turn a directory into a filesystem</a> — <a href=https://utcc.utoronto.ca/~cks/space/blog/>Chris's Wiki :: blog</a><div>One of the wishes that I and other people frequently have for ZFS | 141 | use <a href=https://github.com/mitsuhiko/pipsi>pipsi</a>.</div></article></main><section><hr><h2>Posts from blogs I follow around the net</h2><ul><li><a href=https://utcc.utoronto.ca/~cks/space/blog/linux/NFSv4ServerLockClients target=_blank rel=noopener>Finding which NFSv4 client owns a lock on a Linux NFS(v4) server</a> — <a href=https://utcc.utoronto.ca/~cks/space/blog/>Chris's Wiki :: blog</a><div>A while back I wrote an entry about finding which NFS client owns |
| 142 | is the ability to take an existing directory (and everything | 142 | a lock on a Linux NFS server, which turned |
| 143 | underneath it) in a ZFS filesystem and turn it into a sub-filesystem | 143 | out to be specific to NFS v3 (which I really should have seen coming, |
| 144 | of its own. One reason for wanting this is that a number of things | 144 | since it involved NLM and lockd). Finding the NFS v4 client that |
| 145 | are set and controlled on a per-filesyst…<li><a href=http://www.landley.net/notes-2023.html#28-10-2023 target=_blank rel=noopener>October 28, 2023</a> — <a href=http://www.landley.net/notes-2023.html>Rob Landley's Blog Thing for 2023</a><div>Oh good grief, two of my least favorite licensing people, Larry Rosen | 145 | owns a lock is, depending on your perspective, either simpl…<li><a href=http://www.landley.net/notes-2023.html#28-10-2023 target=_blank rel=noopener>October 28, 2023</a> — <a href=http://www.landley.net/notes-2023.html>Rob Landley's Blog Thing for 2023</a><div>Oh good grief, two of my least favorite licensing people, Larry Rosen |
| 146 | and Bradley Kuhn, are interacting on the OSI's license-discuss | 146 | and Bradley Kuhn, are interacting on the OSI's license-discuss |
| 147 | list where the're doing | 147 | list where the're doing |
| 148 | bad computer history and insisting that a guy Larry Rosen | 148 | bad computer history and insisting that a guy Larry Rosen |
