220 lines
No EOL
14 KiB
TeX
220 lines
No EOL
14 KiB
TeX
\section{Case Studies}
|
|
|
|
\begin{figure}
|
|
\centering
|
|
\includegraphics[width=0.45\textwidth]{images/diagram}
|
|
\caption{Overview of the data pipeline for the case studies.}
|
|
\label{fig:cs_diagram}
|
|
\end{figure}
|
|
|
|
Both case studies follow the same data pipeline for applying the trust framework to power consumption data.
|
|
Figure~\ref{fig:cs_diagram} provides an overview of the processing steps both in and around the trust framework.
|
|
|
|
The system under test --- a personal computer in both studies --- is instrumented to collect power consumption data either from the complete machine or the \ac{cpu} only.
|
|
For each input time series, an activity detector provides an array of labels corresponding to the detected state of the machine for each timestamp.
|
|
The activity detector, although different in each study, always provides output labels that propagate the uncertainty in the input trace.
|
|
For example, if the input time series contains patterns unrecognizable for the detector or missing data, then the output contains \textit{unknown} labels for the corresponding segments.
|
|
Where a classic system may not be able to process \textit{unknown} labels, the trust framework is designed with uncertain information in mind and thus allows the detector to provide an honest output.
|
|
|
|
The labels provided by the activity detector represent the state of the system but the trust framework expect ternary labels representing STL satisfaction.
|
|
The pipeline considers a combination of two \ac{stl} formulae to evaluate the STL satisfaction from state labels and return a ternary value.
|
|
The first formula determines if the output is uncertain or not.
|
|
If the output is not uncertain, then the second formula determines the satisfaction.
|
|
The output of the checker is an array of ternary values $\{1,0,-1\}$ representing the satisfaction of the trace against a predefined policy.
|
|
These study proposes examples of policies to verify, but it is important to note that there is no restrictions on the design of the policy.
|
|
Any condition that is verifiable with the available state labels from the activity detector can provide valid information for the trust framework.
|
|
This lack of constraint makes this approach applicable to a wide variety of security problems.
|
|
|
|
|
|
\subsection{Case Study 1: Controlled Data}
|
|
The first case study considers a dataset collected in the lab.
|
|
The monitored machine is a NUC mini-PC from Intel.
|
|
A Hall effect sensor \cite{palitronica} measures the power consumption at the cable level.
|
|
The capture system measures the current --- proportional to the power --- at 10kHz.
|
|
Prior to the activity detection, a preprocessing step downsamples the trace to 20Hz with an average filter.
|
|
|
|
The machine performs a scenario simulating the activity of an office computer during a working day.
|
|
For ease of experiment, the scenario is compressed into four hours.
|
|
The scenario starts with an \textit{off} period representing the time between midnight and the start of the day.
|
|
The computer then boots up and remains \textit{on} for the rest of the day.
|
|
When evening starts, the computer goes to sleep until the maintenance phase, just before midnight.
|
|
During the maintenance phase, the computer reboots one time to mimic update installations.
|
|
Attacks and malfunctions are randomly triggered using three mechanisms.
|
|
First, the computer can remain \textit{on} either during the night or the evening.
|
|
Then, crypto-mining software can turn on, generating a high load on the system.
|
|
Finally, the scenario can randomly skip rebooting the machine.
|
|
Each abnormal activity symbolizes a different family of attacks.
|
|
|
|
The policy for this first case study is a restriction on the high load that the system can experience.
|
|
A high load of any kind (CPU, GPU or I/O, for example) for a significant amount of time is a suspicious activity.
|
|
These loads can indicate malware like crypto-miners or ransomware or result from a software malfunction that puts the machine into a frozen state.
|
|
The policy verification considers chunks of time to produce a single ternary value depending on the state labels in the chunk.
|
|
Because the goal is to leverage uncertainty, the verification uses a combination of two \ac{stl} formulae to provide a ternary output.
|
|
|
|
\begin{align}
|
|
S_1 &= \lozenge_{[i,i+n]}[(state \neq unknown) \wedge (state \neq high)] \label{stl1}\\
|
|
S_2 &= \neg S_1 \wedge \lozenge_{[i,i+n]}[(state = unknown)] \label{stl2}
|
|
\end{align}
|
|
|
|
Because the proposed \ac{stl} formulae are simple, this study does not employ a full-featured \ac{stl} checker.
|
|
Instead, a simple purpose-built code (see pseudo-code \ref{alg:code1}) verifies the \ac{stl} formulae.
|
|
The checker first divides the power trace in non-overlapping chunks.
|
|
A satisfaction value --- $1=$satisfaction, $0=$uncertain, or $-1=$violation --- corresponds to the data in exactly one chunk.
|
|
To compute the satisfaction value, the checker searches for sequences of a predefined length that contains \textit{high} or \textit{unknown} values (Equation~\ref{stl1}).
|
|
If no such sequence is found, the chunk of trace is compliant.
|
|
If a sequence is found, the chunk is either uncertain or in violation whether it contains respectively \textit{unknown} labels or not (Equation~\ref{stl2}).
|
|
The checker returns the value after finding the first sequence.
|
|
|
|
|
|
\begin{algorithm}
|
|
\caption{Pseudo-code for detecting high load violations.}
|
|
\label{alg:code1}
|
|
\begin{algorithmic}[1]
|
|
\Require A label array $l$, chunk length $L_c$, and sequence length $L_s$.
|
|
\Procedure{checkChunk}{chunk}
|
|
\For{$j \in range(i,i+L_c)$}
|
|
\If{$j=2$ or $j=-1$}
|
|
\State $k=1$
|
|
\While{$j+k < i+L_c$ and $l[i+k] \in \{2,-1\}$}
|
|
\State $k \gets k+1$
|
|
\If{$k=L_c$}
|
|
\If{$-1 \in l[i:i+k]$}
|
|
\State \Return $0$
|
|
\Else
|
|
\State \Return $-1$
|
|
\EndIf
|
|
\EndIf
|
|
\EndWhile
|
|
\EndIf
|
|
\EndFor
|
|
\State \Return $1$
|
|
\EndProcedure
|
|
|
|
\State $start,step,end \gets 0,L_c,length(l)$
|
|
\State $compliances \gets emptyList()$
|
|
\For{$i \in range(start,end,step)$}
|
|
\State $chunk \gets l[i:i+L_c]$
|
|
\State $compliances.append(checkChunk(chunk))$
|
|
\EndFor
|
|
\State \Return $compliances$
|
|
\end{algorithmic}
|
|
\end{algorithm}
|
|
|
|
The output of verifying the STL formulae is an array of ternary labels, where each value represents the satisfaction of a chunk of time series.
|
|
The interval window is set to \( \mathcal{I} = 60 \hspace{0.5em} minutes\) in our experiment to calculate the trust snapshot opinion (\( \omega_{Y}^{S} \)).
|
|
The trust index opinion (\( \omega_{Y}^{I} \)) is then updated at the end of each interval using Eq.~\eqref{eq:trust_idx}.
|
|
|
|
|
|
\begin{figure}
|
|
\centering
|
|
\includegraphics[width=0.45\textwidth]{images/combined_plot_dsd_raw.pdf}
|
|
\caption{Case Study 1: Trust observations for a lab machine.}
|
|
\label{fig:trust_CSI}
|
|
\end{figure}
|
|
|
|
|
|
Figure~\ref{fig:trust_CSI} illustrates the information the trust management framework provides.
|
|
Subplot A is the input data from the \ac{stl}-checker, delineated into daily increments, with three discreet values: satisfied (+1), uncertain (0), and violation (-1), reflecting the system's state over the eight-day period.
|
|
|
|
Subplot B plots the trust index (\( T_{Y}^{I} \)), which represents the comprehensive trust metric of the system as it evolves throughout the experiment.
|
|
The trust index initially experiences a notable decline because uncertain or violated observations influence subjective opinions, attributable to the \ac{sl} 's reliance on a relatively sparse dataset for opinion formulation.
|
|
However, as the volume of observations increases, the sensitivity to violations and uncertainties reduces because the trust snapshot opinion is fused with the trust index opinion.
|
|
This plot illustrates the adaptive nature of our proposed trust management framework, as it effectively incorporates new evidence to update the trust index opinion continuously, ensuring an accurate representation of the system's reliability.
|
|
|
|
Subplots C, D and E present the values of the trust snapshot opinion (\( \omega_{Y}^{S} \)) projected probabilities for --- satisfied (+1), uncertain (0), and violation (-1) --- respectively, over the defined interval window \( \mathcal{I} \).
|
|
These instantaneous assessments within the window help identify short-term trends that may not be as noticeable when examining long-term aggregated data.
|
|
The sum of probabilities across these three states is equal to one at all times.
|
|
|
|
Our framework offers a more refined understanding of trust dynamics by allowing for a clear distinction between satisfied, violation and uncertain states.
|
|
The diminished sensitivity in the trust index opinion suggests a potential vulnerability.
|
|
Namely, consistent adversarial intrusions could remain undetected if solely assessed through the trust index opinion.
|
|
However, we can identify trends as significant shifts in the projected probabilities by examining the granular components of the trust snapshot opinions.
|
|
If an attacker targets a system in periodic intervals, observing only the trust index will not reveal any noticeable patterns.
|
|
However, analyzing the breakdown plot of trust snapshot opinions can aid in detecting such trends.
|
|
|
|
|
|
\begin{figure}
|
|
\centering
|
|
\includegraphics[width=0.45\textwidth]{images/trust_index_opinion_component_dsd_raw.pdf}
|
|
\caption{Case Study 1: Trust index opinion components.}
|
|
\label{fig:trust_index_opinion_component}
|
|
\end{figure}
|
|
|
|
Figure~\ref{fig:trust_index_opinion_component} illustrates the components of the trust index opinion across several days, showcasing the nuanced dynamics of trust assessment within the framework of our case study.
|
|
This visual representation helps understanding how trust in our \ac{cpss} evolves over time. The graph breaks down the trust index opinion into segmented categories of satisfaction, uncertainty, and violation.
|
|
These fluctuations are indicative of the system's responses to varying conditions and how they contribute to each component.
|
|
|
|
|
|
|
|
\subsection{Case Study 2: Production Data}
|
|
The second case study considers power consumption data from production systems.
|
|
The systems are classroom computers available to students for lab courses.
|
|
The current measurement uses the same capture system as for the first case study.
|
|
The traces contain measurements of the power consumption of the CPU only.
|
|
|
|
The machines in this classroom should never turn off.
|
|
For remote maintenance reasons, it is expected that the students log off from the machines after using them but should not turn them off completely.
|
|
For this result, the policy that this case study considers is the constant \textit{on} state from the machine.
|
|
|
|
To verify the state of the machine, the activity detection algorithm compares the current values to a threshold.
|
|
The threshold represents the separation between the \textit{off} and \textit{on} states.
|
|
In the same way as the first case study, the checker algorithm divides the input trace into multiple chunks that --- for this analysis --- represent 30 seconds of consumption.
|
|
|
|
The uncertainties in this case study take the form of missing data.
|
|
The captured system being deployed in a remote location outside of the controlled environment of a lab is subjected to network issues.
|
|
Instead of preprocessing the trace to fill empty segments using some heuristic or discarding the chunks containing missing measurements, the policy checker can leverage them by passing along \textit{uncertain} values to the trust framework.
|
|
|
|
To determine the output value for each chunk, the checker verifies the presence of \textit{off} values or missing data in each chunk.
|
|
If the chunk contains values below the threshold, then the output is $-1=$violation.
|
|
If there are no \textit{off} values but the chunk contains missing data, the checker cannot guarantee that the machine was not off at some point.
|
|
In this case, the output is $0=$uncertain.
|
|
Finally, if the chunk contains no missing values and only values above the threshold, then the output is $1=$satisfy.
|
|
Algorithm~\ref{alg:code2} presents the pseudo-code of the policy checker.
|
|
|
|
\begin{algorithm}
|
|
\caption{Pseudo-code for detecting off violations.}
|
|
\label{alg:code2}
|
|
\begin{algorithmic}[1]
|
|
\Require A power trace $t_s$, chunk length $L_c$, and a threshold value $T$.
|
|
\Procedure{checkChunk}{chunk}
|
|
\State $compliance \gets 1$
|
|
\For{$j \in range(i,i+L_c)$}
|
|
\If{$chunk[j] < T$}
|
|
\State \Return 0
|
|
\ElsIf{$chunk[j] = NaN$}
|
|
\State $compliance = 0$
|
|
\EndIf
|
|
\EndFor
|
|
\State \Return $compliance$
|
|
\EndProcedure
|
|
|
|
\State $start,step,end \gets 0,L_c,length(t_s)$
|
|
\State $compliances \gets emptyList()$
|
|
\For{$i \in range(start,end,step)$}
|
|
\State $chunk \gets t_s[i:i+L_c]$
|
|
\State $compliances.append(checkChunk(chunk))$
|
|
\EndFor
|
|
\State \Return $compliances$
|
|
\end{algorithmic}
|
|
\end{algorithm}
|
|
|
|
|
|
\begin{figure}
|
|
\centering
|
|
\includegraphics[width=0.45\textwidth]{images/combined_plot_ARTHUR-103.pdf}
|
|
\caption{Case Study 2: Trust observations for a production machine.}
|
|
\label{fig:trust_CSII}
|
|
\end{figure}
|
|
|
|
|
|
Figure~\ref{fig:trust_CSII}, illustrates the application of our trust management framework to production data from classroom computers, showcasing the framework's performance in a real-world scenario.
|
|
Subplot A illustrates the input signal to the framework over multiple days --- marked by vertical dashed lines ---, containing discrete evaluations of system states: satisfied (+1), uncertain (0), and violated (-1).
|
|
|
|
Subplot B reveals the evolution of the trust index (\( T_{Y}^{I} \)) across the observed time frame.
|
|
Initial observations show periods marked by uncertain (0) and violated (-1) states, resulting in a significant decline in trust.
|
|
The gradual increase in the trust index suggests an enhancement in system dependability as operations advance and more positive observations are noted.
|
|
|
|
Subplots C, D, and E delineate the projected probabilities for satisfaction, uncertainty, and violation states derived from the trust snapshot opinion (\( \omega_{Y}^{S} \)) over the same period.
|
|
These emphasize the nuanced understanding of trust dynamics, allowing for identifying transient anomalies and operational trends that may not be immediately evident with the trust index alone.
|
|
|
|
Subplots D (Uncertain) and E (Violation) exhibit intermittent peaks, which highlight moments when the system's operational state was either not well understood or deviated from expected behavior. |