add results for bpv

This commit is contained in:
Arthur Grisel-Davy 2023-08-01 16:26:16 -04:00
parent fee0323ee1
commit 66397ac8d5
3 changed files with 52 additions and 15 deletions

View file

@ -176,24 +176,42 @@ The distance of each new trace to the reference average is computed and compared
If the distance is above the pre-computed threshold, the new trace is considered anomalous. If the distance is above the pre-computed threshold, the new trace is considered anomalous.
\subsection{Results} \subsection{Results}
We evaluated the \gls{bpv} on three occasions. We evaluated the \gls{bpv} on two occasions.
First, we assembled a panel of relevant devices, including switches, \gls{wap} and \gls{pc}. First, we assembled a panel of relevant devices, including switches, \gls{wap} and \gls{pc}.
The evaluations revealed that the \gls{bpv} performed better on simpler devices like switches and \gls{wap} compared to general-purpose computers. The evaluations revealed that the \gls{bpv} performed better on simpler devices like switches and \gls{wap} compared to general-purpose computers.
This is mainly due to the reduced variability and noise in the traces captured from simpler devices that produce a more robust model. This is mainly due to the reduced variability and noise in the traces captured from simpler devices that produce a more robust model.
This first study leads to the publication of a work-in-progress paper in the EMSOFT 2022 conference \cite{grisel2022work} that describes the design and capabilities of the \gls{bpv} in its first version. This first study leads to the publication of a work-in-progress paper in the EMSOFT 2022 conference \cite{grisel2022work} that describes the design and capabilities of the \gls{bpv} in its first version.
Then, we performed a case study with an industry partner on \gls{rtu}. %Then, we performed a case study with an industry partner on \gls{rtu}.
The \gls{rtu} was composed of one low-complexity embedded system and one main general-purpose computer. %The \gls{rtu} was composed of one low-complexity embedded system and one main general-purpose computer.
The computer's activity overtook most of the other information in the trace and made it more difficult to detect subtle variations. %The computer's activity overtook most of the other information in the trace and made it more difficult to detect subtle variations.
However, the \gls{bpv} could still detect intrusions in the computer from the global trace. %However, the \gls{bpv} could still detect intrusions in the computer from the global trace.
For example, a user modifying some settings through the \gls{bios} or booting into a different \gls{os} was detected. %For example, a user modifying some settings through the \gls{bios} or booting into a different \gls{os} was detected.
This case study revealed that some systems could have multiple valid modes of the boot sequence. %This case study revealed that some systems could have multiple valid modes of the boot sequence.
This discovery enabled us to rethink the model of the \gls{bpv} to allow such variations. %This discovery enabled us to rethink the model of the \gls{bpv} to allow such variations.
We performed the final evaluation on a drone. We performed the second evaluation on a drone.
A drone is a prime machine for the \gls{bpv} as its low complexity allows for consistent boot traces. A drone is a prime machine for the \gls{bpv} as its low complexity allows for consistent boot traces.
We successfully detected different firmware versions by leveraging the information from the two previous experiments. We successfully detected different firmware versions by leveraging the information from the two previous experiments.
Along the evaluations, the \gls{bpv} capabilities have been modified to adapt to specific cases and enable anomalous training samples, multi-model evaluations, and autonomous learning. Along the evaluations, the \gls{bpv} capabilities have been modified to adapt to specific cases and enable anomalous training samples, multi-model evaluations, and autonomous learning.
\agd{add results} \begin{table}[ht]
\centering
\begin{tabular}{lccc}
\toprule
\textbf{Test Case} & \textbf{Experiment} & \textbf{F1 Score} \tabularnewline
\toprule
\multirow{4}*{Network Devices} & TP-Link switch & 0.87\tabularnewline
& HP switch & 0.98 \tabularnewline
& Asus Router & 1.00\tabularnewline
& Linksys Router & 0.92\tabularnewline
\midrule
\multirow{4}*{Drone} & Original & 1.00\tabularnewline
& Compiled & 1.00\tabularnewline
& Low Battery & 1.00\tabularnewline
& Bootloader Bug & 1.00\tabularnewline
\bottomrule
\end{tabular}
\label{tab:fw-results}
\end{table}
\newpage \newpage
\section{State Detection and Segmentation} \section{State Detection and Segmentation}

View file

@ -98,7 +98,6 @@
& AC SVM & & \numprint[\%]{99.5} \tabularnewline & AC SVM & & \numprint[\%]{99.5} \tabularnewline
\bottomrule \bottomrule
\end{tabular} \end{tabular}
\label{tab:fw-results}
\end{table} \end{table}
\footnote{Published in \textit{Side-channel Based Runtime Intrusion Detection for Network Equipment} at MLCS (Workshop of ECML-PKDD 2023)} \footnote{Published in \textit{Side-channel Based Runtime Intrusion Detection for Network Equipment} at MLCS (Workshop of ECML-PKDD 2023)}
@ -110,9 +109,6 @@
\includegraphics[height=0.9\textheight]{images/xpsu_illustration.pdf} \includegraphics[height=0.9\textheight]{images/xpsu_illustration.pdf}
\end{figure} \end{figure}
\end{frame} \end{frame}
\begin{frame}{ Preliminary Work - xPSU}
\end{frame}
\begin{frame}{Preliminary Work - SDS \& BPV} \begin{frame}{Preliminary Work - SDS \& BPV}
\begin{figure} \begin{figure}
@ -124,8 +120,29 @@
\begin{frame}{Preliminary Work - SDS \& BPV} \begin{frame}{Preliminary Work - SDS \& BPV}
\begin{figure} \begin{figure}
\centering \centering
\includegraphics[width=0.8\textwidth]{images/training_bpv.pdf} \includegraphics[width=0.9\textwidth]{images/training_bpv.pdf}
\end{figure} \end{figure}
\end{frame}
\begin{frame}{Preliminary Work - SDS \& BPV}
\begin{table}[ht]
\centering
\begin{tabular}{lccc}
\toprule
\textbf{Test Case} & \textbf{Experiment} & \textbf{F1 Score} \tabularnewline
\toprule
\multirow{4}*{Network Devices} & TP-Link switch & 0.87\tabularnewline
& HP switch & 0.98 \tabularnewline
& Asus Router & 1.00\tabularnewline
& Linksys Router & 0.92\tabularnewline
\midrule
\multirow{4}*{Drone} & Original & 1.00\tabularnewline
& Compiled & 1.00\tabularnewline
& Low Battery & 1.00\tabularnewline
& Bootloader Bug & 1.00\tabularnewline
\bottomrule
\end{tabular}
\end{table}
\footnote{Published in \textit{Work-in-Progress: Boot Sequence Integrity Verification with Power Analysis} at EMSOFT 2022\\ \footnote{Published in \textit{Work-in-Progress: Boot Sequence Integrity Verification with Power Analysis} at EMSOFT 2022\\
and in \textit{Independent Boot Process Verification using Side-Channel Power Analysis} at QRS 2023} and in \textit{Independent Boot Process Verification using Side-Channel Power Analysis} at QRS 2023}
\end{frame} \end{frame}

View file

@ -133,6 +133,8 @@
}{} % end of ifthenelse (no else) }{} % end of ifthenelse (no else)
\usepackage{cite} \usepackage{cite}
\usepackage{multirow}
\usepackage{booktabs}
\usepackage[acronyms]{glossaries} % Exception to the rule of hyperref being the last add-on package \usepackage[acronyms]{glossaries} % Exception to the rule of hyperref being the last add-on package
% If glossaries-extra is not in your LaTeX distribution, get it from CTAN (http://ctan.org/pkg/glossaries-extra), % If glossaries-extra is not in your LaTeX distribution, get it from CTAN (http://ctan.org/pkg/glossaries-extra),
% although it's supposed to be in both the TeX Live and MikTeX distributions. There are also documentation and % although it's supposed to be in both the TeX Live and MikTeX distributions. There are also documentation and