#let act(body) = [ #text(fill: rgb(230, 76, 0))[Action: #body] ] #let dact(body) = [ #strike(stroke: 2pt+black)[#text(fill: rgb(230, 76, 0))[Action: #body]] ] #align(center)[*Trust Management with Subjective Logic for Safety Critical Systems in Uncertain Environments*] Review \#374A ------------------------------------------------------------------------ Overall merit *1.* Reject Paper summary The paper under review proposes a new system for trust management of cyber physical system by using subjective logic and signal temporal logic for trust assessment in uncertain environments. The theoretical contribution of the paper is a development of a new cumulative fusion formalism for subjective logic. This operator fuses multiple opinions about the same proposition into a single, combined opinion, taking into account the uncertainty inherent in each opinion. The authors prove that this operator respects the goal relationship of the belief function. Based on this new fusion formalism, they build a trust management framework that aggregates data, generates a trust opinion over each time interval end combines them with the ne fusion operator. The trust management framework is then applied to two case studies. *Strengths* The new fusion operator is rigorously formalized and its correctness is proven. *Weaknesses* - the use of STL for labeling the data is not explained, the used formulas are very simple #act[We could remove references to STL. The DSD does not truly use STL and the little it does is very simple and does not _require_ STL formalism. I agree it is cool to show that STL can be used but this is not the core of the proposed approach and it seems to confuse readers.] - there is a large body of work on SLT based monitoring of CPS from the formal methods community; it is unclear how this paper compares and why these existing monitoring methods are not employed #act[Again, maybe remove STL.] - the overall motivation did not become clear and I do not see why I need to use subjective logic on top of signal temporal logic #act[Remove STL and reinforce the interest of subjective logic in the intro or presentation of the proposed approach] - it is not obvious to me that combining opinions on trusts over different time-windows is beneficial in CPS that are subject to dynamic changes over time #act[Provide a deeper analysis of the temporal relationship and the interest in not taking only the latest observation of trust.] Review \#374B ------------------------------------------------------------------------ Overall merit *2.* Weak reject Paper summary This paper presents a novel trust management framework for Cyber-Physical Systems (CPSs). Utilizing observations from target systems, such as power consumption, the framework employs an activity detector and an STL checker to classify these observations into "satisfaction (+1)," "uncertainty (0)," or "violation (-1)" based on predefined STL properties. For any fixed interval window, the framework leverages subjective logic (SL) to compute a trust snapshot opinion, which evaluates the system's trustworthiness within the current time interval, as well as a trust index opinion, which provides an assessment of the system's overall trustworthiness. Additionally, the framework optionally include trust calibration actions, which can be manually designed to bolster trust in the system if it falls below a predefined threshold. Meta review: The reviewers thought the problem considered in this paper is important and interesting, but it should be improved at least from the following aspects: 1, the motivation is unclear, particular, why subjective logic is necessary for dealing with trust, some real-world examples are needed. 2, there is a huge bulk of work on monitoring of CPS, but unfortunately, no comparison with them. *Strengths* 1. Unlike existing works, the proposed framework accommodates the possibility of "unknown" evidence, enabling it to handle uncertain information effectively. 2. By leveraging subjective logic, the proposed framework offers a quantitative assessment of the system's trustworthiness. *Weaknesses* 1. The main technical contribution — extending beyond traditional binary trust assessments to accommodate uncertainties — appears limited. Incorporating 'unknown' into the domain of subjective logic does not present significant challenges. #act[Well *you* try to do it and *you* make a paper then!] 2. It is difficult to assess or reason why the obtained trust snapshot opinion and trust index opinion accurately capture the "trustworthiness" of the system. Detailed comments Based on the observed data series, the proposed framework calculates values intended to capture the system's "trustworthiness." It is crucial to demonstrate, either theoretically or experimentally, why the proposed metrics effectively represent "trustworthiness," considering that numerous alternative metrics could be proposed. #act[Difficult to provide as there are no ground truth. Need to think about that.] Minor: On page 4, right column, the notation "W_2^F \oplus W_3^G" should be corrected to "W_2^F \oplus W_2^G". #act[fix that] Review \#374C ------------------------------------------------------------------------ Overall merit *4.* Accept Paper summary The paper proposes a framework for utilizing Subjective Logic (SL) for trust management. Core is the adoption of SL in two case studies (power consumption of a NUC mini-PC from Intel, production data) for providing a trust score even with uncertain or incomplete data. The framework is intended for observations in cyber physical systems (CPS). *Strengths* - accountable derivation of adoption of SL for the case studies - comprehensive description of both trust management framework and case studies *Weaknesses* - just a short discussion part - a more comprehensive description of system reaction in the case of violations and some evaluated evidence would support understanding the effectiveness of the proposed framework for the use in CPS #act[Not clear. Maybe talking about calibration actions (_reaction in the case of violation_)] Detailed comments the derivation of formulas given in Joesang "Subjective Logic" could be shortened to more concise descriptions in favour of more detailed description of interaction of the framework with the CPS in case of violation Review \#374D ------------------------------------------------------------------------ Overall merit *1.* Reject Paper summary This paper proposes a model for quantifying trust in a system that uses subjective logic. The model includes a trust index and a trust snapshot for purposes of trust measurement, with trust calibration to allow the user to adjust the trust. The design is evaluated with two case studies. *Strengths* Trust is an important concern in CPS, and it's interesting to see it applies subjective logic for trust management purpose. *Weaknesses* - Poor motivation #act[No. But we could still put more emphasis on the motivations.] - Lack connection between the concepts and real-world scenarios #act[Very no! Read the paper!] - Unclear usefulness #act[WHAT!? READ THE PAPER!] - Lack comparisons to existing systems #act[ok we could try to provide a baseline for comparison.] Detailed comments Managing trust in CPS is an important research problem, and the use of subjective logics to this setting is interesting. Unfortunately, I feel that the paper is rather immature and has a number of problems. - The paper does not provide a convincing motivation to the proposed framework. In the introduction, the description and importance of trust management systems are so vague that it makes it impossible to relate to real systems. A concrete example or use case should be discussed to show where these systems are used and how bad undertrust and overtrust would be. - The concepts are not very well described. Where is the definition of “integrity” (used in the problem statement)? Is it the same thing as “trustworthiness”? How are “off”, “boot”, and “high load” in the example related to integrity or trustworthiness? Real-world examples should be provided to show what is a system with integrity and what is a system without integrity. #act[I still diasgree but for the few that read diagonaly we could maybe review the paragraphs explaining the real-world experiment and make it clearer.] - The authors fail to address why the properties they prove in Sec. IV-C are useful for real systems. What will be the consequences if they don’t hold? - There are existing intrusion detection systems using different techniques. How is the proposed method superior to existing solutions? The case study does not have any metrics to show how good/accurate a method is. #act[First, it does not need to be superior to be interesting. We are not selling vacume cleaner we are doing research. Second, we can look into providing a baseline with other methods.] In short, while I find trust an interesting topic, the paper falls short in several important aspects, making its benefits and contributions questionable.