This commit is contained in:
Alfred Melch 2019-08-15 20:46:02 +02:00
parent 019d281952
commit c67793cacd
4 changed files with 20 additions and 7 deletions

View File

@ -19,7 +19,9 @@ Each section in this chapter describes a set of benchmarks run on the same syste
At first it will be observed how the algorithms perform under different browsers. The chart to use for this is the "Simplify.js vs Simplify.wasm" chart. For that a Windows system was chosen as it allows to run benchmarks under three of the four browsers in question. The dataset is the Simplify.js example which will be simplified with and without the high quality mode.
The device is a \textsf{HP Pavilion x360 - 14-ba101ng}\footnote{\url{https://support.hp.com/us-en/product/hp-pavilion-14-ba100-x360-convertible-pc/16851098/model/18280360/document/c05691748}} convertible. It contains an Intel® Core™ i5-8250U Processor with 4 cores, 6MB cache. The operating system is Windows 10 and the browsers are on their newest versions with Chrome 75, Firefox 68 and Edge 44.18362.1.0.
The device is a \textsf{HP Pavilion x360 - 14-ba101ng}\footnote{\url{https://support.hp.com/us-en/product/hp-pavilion-14-ba100-x360-convertible-pc/16851098/model/18280360/document/c05691748}} convertible. It contains an Intel® Core™ i5-8250U Processor with 4 cores, 6MB cache. The operating system is Windows 10 and the browsers are on their newest versions with Chrome 75, Firefox 68 and Edge 44.18362.1.0.
Table \ref{tbl:dimensions-1} summarizes the setting. For each problem dimension the chosen characteristics are highlighted in green color. The number of benchmark diagrams in a chapter is determined by the multitude of characteristics selected. In the case here there are three browsers tested each with two quality options resulting in six diagrams to be produced.
\input{./results-benchmark/win_ffox_simplify_vs_false.tex}
@ -43,10 +45,11 @@ Without high quality mode the Simplify.wasm gets overtaken by the Simplify.js al
Interestingly in the Edge browser the two JavaScript algorithms perform more alike when high quality disabled. As can be seen in figure \ref{fig:win_edge_simplify_vs_false} The turning point where WebAssembly is not the fastest is at around 0.45 to 0.6. When turning high quality on the graph in figure \ref{fig:win_edge_simplify_vs_true} resembles the chart from Chrome only with more consistent results for the original implementation.
\FloatBarrier
\subsection{Case 2 - Windows - wasm runtime analysis}
\label{ch:case2}
\begin{table}[htb]
\begin{table}[!htb]
\centering
\includegraphics[width=.75\linewidth]{./images/dimensions-2.png}
\label{tbl:dimensions-2}
@ -54,7 +57,7 @@ Interestingly in the Edge browser the two JavaScript algorithms perform more ali
\end{table}
For this case the same device as in the former case is used. To compare the results of the two cases the same dataset is used. Under the Edge browser the Simplify.wasm runtime analysis was measured.
For this case the same device as in the former case is used. To compare the results of the two cases the same dataset is used. Under the Edge browser the Simplify.wasm runtime analysis was measured. Table \ref{tbl:dimensions-2} summarizes this.
\input{./results-benchmark/win_edge_simplify_stack_false.tex}
\input{./results-benchmark/win_edge_simplify_stack_true.tex}
@ -65,10 +68,11 @@ Inspecting figures \ref{fig:win_edge_simplify_stack_false} and \ref{fig:win_edge
In the case of high quality disabled the results show a very steep curve of the execution time. Quickly the time span for preparing the memory dominates in the process. In the second graph it can be seen that the fraction is significantly lower due to the execution time being consistently higher.
\FloatBarrier
\subsection{Case 3 - MacBook Pro - wasm vs js}
\label{ch:case3}
\begin{table}[htb]
\begin{table}[!htb]
\centering
\includegraphics[width=.75\linewidth]{./images/dimensions-3.png}
\label{tbl:dimensions-3}
@ -89,11 +93,11 @@ The results of the Safari browser with high quality disabled (figure \ref{fig:ma
When turning on high quality mode the JavaScript implementations still perform alike. However Simplify.wasm is clearly faster as seen in figure \ref{fig:mac_safa_bavaria_vs_true}. Simplify.wasm performs here about twice as fast as the algorithms implemented in JavaScript. Those however have a steeper decrease as the tolerance numbers go up.
\FloatBarrier
\subsection{Case 4 - Ubuntu - turf.js analysis}
\label{ch:case4}
\begin{table}[htb]
\begin{table}[!htb]
\centering
\includegraphics[width=.75\linewidth]{./images/dimensions-4.png}
\label{tbl:dimensions-4}
@ -114,17 +118,25 @@ Figure \ref{fig:ubu_ffox_bavaria_vs_true} shows how the JavaScript versions perf
The next two figures show the case when high quality is disabled. In figure \ref{fig:ubu_ffox_bavaria_vs_false} two algorithms seem to converge. And when looking at figure \ref{fig:ubu_ffox_bavaria_jsstack_false} one can see that the data preparation gets more costly as the tolerance rises. From a tolerance of 0.0014 on the alternative Simplify.js implementation is faster than the Turf.js method.
\FloatBarrier
\subsection{Case 5 - iPad - mobile testing}
\label{ch:case5}
\begin{table}[htb]
\begin{table}[!htb]
\centering
\includegraphics[width=.75\linewidth]{./images/dimensions-5.png}
\label{tbl:dimensions-5}
\caption{Problem dimensions of Case 5}
\end{table}
At last the results from a mobile device will be shown. The device is an iPad Air with iOS version 12.4. The Simplify.js example is being generalized using Safari and the Firefox browser. Again both quality settings are used for the benchmarks.
\input{./results-benchmark/ipad_safa_simplify_vs_false.tex}
\input{./results-benchmark/ipad_safa_simplify_vs_true.tex}
When the high quality parameter is left in its default state the WebAssembly solution is fastest on low tolerance numbers (figure \ref{fig:ipad_safa_simplify_vs_false}). As seen before the JavaScript versions are getting faster when the tolerance increases. The original Simplify.js version surpasses the WebAssembly performance while the alternative tangents it. As it was the case on the desktop system the algorithms perform similarly when high quality is set to \textsf{true}. Figure \ref{fig:ipad_safa_simplify_vs_true} shows that Simplify.wasm is also here the faster method.
\input{./results-benchmark/ipad_ffox_simplify_vs_false.tex}
\input{./results-benchmark/ipad_ffox_simplify_vs_true.tex}
Interestingly the results in figure \ref{fig:ipad_ffox_simplify_vs_false} and \ref{fig:ipad_ffox_simplify_vs_true} show the exact same results as the Safari results. In chapter \ref{ch:discussion} this will be further examined.

Binary file not shown.

Before

Width:  |  Height:  |  Size: 19 KiB

After

Width:  |  Height:  |  Size: 19 KiB

Binary file not shown.

View File

@ -16,6 +16,7 @@
\usepackage{url} % for filepaths and urls
\usepackage{hyperref} % for hyperlinks
\usepackage{float}
\usepackage[section]{placeins} % for using \FloatBarrier in results
% configure headers
\usepackage{fancyhdr} % for headers