102 lines
4.9 KiB
TeX
102 lines
4.9 KiB
TeX
\section[Results]{Benchmark results}
|
|
|
|
In this chapter the results are presented. There were a multitude of tests to make. Multiple devices were used to run several benchmarks on different browsers and under various parameters. To organize which benchmarks had to run, first all the problem dimensions were clarified. Devices will be categorized into desktop and mobile devices. The browsers to test will come from the four major browser vendors which were involved in WebAssembly development. Those are Firefox from Mozilla, Chrome from Google, Edge from Microsoft and Safari from Apple. For either of the two data sets a fixed range of tolerances is set to maintain consistency across the diagrams. The values are explained in chapter \ref{ch:benchmark-data}. The other parameter "high quality" can be either switched on or off. The three chart types are explained in chapter \ref{ch:benchmark-cases}.
|
|
|
|
Each section in this chapter describes a set of benchmarks run on the same system. A table in the beginning will indicate the problem dimensions chosen to inspect. After a description of the system and a short summary of the case the results will be presented in the form of graphs. Those are the graphs produced from the application described in chapter \ref{ch:benchmark-app}
|
|
|
|
|
|
\subsection{Case 1 - Windows - wasm vs js}
|
|
\marginpar{hp pavilion}
|
|
\marginpar{6 charts}
|
|
\marginpar{questions: 1, 3, 5}
|
|
|
|
\begin{table}[htb]
|
|
\centering
|
|
\includegraphics[width=.75\linewidth]{./images/dimensions-1.png}
|
|
\label{tbl:dimensions-1}
|
|
\caption{Problem dimensions of Case 1}
|
|
\end{table}
|
|
|
|
At first it will be observed how the algorithms perform under different browsers. The chart to use for this is the "Simplify.js vs Simplify.wasm" chart. For that a Windows system was chosen as it allows to run benchmarks under three of the four browsers in question. The dataset is the Simplify.js example which will be simplified with and without the high quality mode.
|
|
|
|
The device is a \textsf{HP Pavilion x360 - 14-ba101ng}\footnote{\path{https://support.hp.com/us-en/product/hp-pavilion-14-ba100-x360-convertible-pc/16851098/model/18280360/document/c05691748}} convertible. It contains an Intel® Core™ i5-8250U Processor with 4 cores, 6MB cache. The operating system is Windows 10 and the browsers are on their newest versions with Chrome 76, Firefox 68 and Edge 44.18362.1.0.
|
|
|
|
\input{./results-benchmark/win_chro_simplify_vs_false.tex}
|
|
\input{./results-benchmark/win_ffox_simplify_vs_false.tex}
|
|
\input{./results-benchmark/win_edge_simplify_vs_false.tex}
|
|
\input{./results-benchmark/win_chro_simplify_vs_true.tex}
|
|
\input{./results-benchmark/win_ffox_simplify_vs_true.tex}
|
|
\input{./results-benchmark/win_edge_simplify_vs_true.tex}
|
|
\clearpage
|
|
|
|
\subsection{Case 2 - Windows - wasm runtime analysis}
|
|
\marginpar{hp pavilion}
|
|
\marginpar{2 charts}
|
|
\marginpar{questions: 2, 3, 5}
|
|
\todo{maybe add ffox}
|
|
|
|
\begin{table}[htb]
|
|
\centering
|
|
\includegraphics[width=.75\linewidth]{./images/dimensions-2.png}
|
|
\label{tbl:dimensions-2}
|
|
\caption{Problem dimensions of Case 2}
|
|
\end{table}
|
|
|
|
|
|
For this case the same device as in the former case is used. To compare the results of the two cases the same dataset is used. Under the Edge browser the Simplify.wasm runtime analysis was measured.
|
|
|
|
\input{./results-benchmark/win_edge_simplify_stack_false.tex}
|
|
\input{./results-benchmark/win_edge_simplify_stack_true.tex}
|
|
\clearpage
|
|
|
|
\subsection{Case 3 - MacBook Pro - wasm vs js}
|
|
\marginpar{MacBook Pro 15}
|
|
\marginpar{4 charts}
|
|
\marginpar{Chrome and FF comparable to results above}
|
|
|
|
A 2018 MacBook Pro 15" will be used to test the safari browser. For comparison the benchmarks will also be held under Firefox on MacOS. This time the bavarian boundary will be simplified with both preprocessing enabled and disabled.
|
|
|
|
\begin{table}[htb]
|
|
\centering
|
|
\includegraphics[width=.75\linewidth]{./images/dimensions-3.png}
|
|
\label{tbl:dimensions-3}
|
|
\caption{Problem dimensions of Case 3}
|
|
\end{table}
|
|
|
|
|
|
\input{./results-benchmark/mac_safa_bavaria_vs_false.tex}
|
|
\input{./results-benchmark/mac_ffox_bavaria_vs_false.tex}
|
|
\input{./results-benchmark/mac_safa_bavaria_vs_true.tex}
|
|
\input{./results-benchmark/mac_ffox_bavaria_vs_true.tex}
|
|
\clearpage
|
|
|
|
\subsection{Case 4 - Ubuntu - turf.js analysis}
|
|
\marginpar{Lenovo Miix 510}
|
|
\marginpar{4 charts}
|
|
\marginpar{Firefox because orig simplify is faster}
|
|
|
|
\begin{table}[htb]
|
|
\centering
|
|
\includegraphics[width=.75\linewidth]{./images/dimensions-4.png}
|
|
\label{tbl:dimensions-4}
|
|
\caption{Problem dimensions of Case 4}
|
|
\end{table}
|
|
|
|
\input{./results-benchmark/ubu_ffox_bavaria_vs_false.tex}
|
|
\input{./results-benchmark/ubu_ffox_bavaria_jsstack_false.tex}
|
|
\input{./results-benchmark/ubu_ffox_bavaria_vs_true.tex}
|
|
\input{./results-benchmark/ubu_ffox_bavaria_jsstack_true.tex}
|
|
\clearpage
|
|
|
|
\subsection{Case 5 - iPhone - mobile testing}
|
|
|
|
\marginpar{iPhone??}
|
|
\marginpar{6 charts}
|
|
|
|
\begin{table}[htb]
|
|
\centering
|
|
\includegraphics[width=.75\linewidth]{./images/dimensions-5.png}
|
|
\label{tbl:dimensions-5}
|
|
\caption{Problem dimensions of Case 5}
|
|
\end{table}
|