Evaluation
All research track submissions in compliance with the submission policy and within the scope of the conference will be evaluated through a single-blind, two-stage process involving a program committee and a program board. Eligible submissions will be evaluated by at least three members of the program committee, with the program board and program committee working together to make the final acceptance decisions. Authors will have the opportunity to see their reviews from the program committee and provide a short clarification before the final decisions are made.
Research track submissions will be evaluated based on the following criteria:
Soundness: Claimed contributions should be supported through the rigorous application of appropriate research methods. The claims should be scoped to what can be supported, and limitations should be discussed.
Significance: Contributions will be evaluated for their novelty, originality, and importance with respect to the existing body of knowledge. Submissions will be expected to explicitly argue for the relevance and usefulness of the research and discuss the novelty of the claimed contributions through a comparison with pertinent related work.
Verifiability: The evaluation of submissions will take into account to what extent sufficient information is available to support the full or partial independent verification or replication of the claimed contributions. The committee will consider independent verifiability an important factor that can significantly strengthen the value of a submission. Authors are strongly encouraged to explicitly discuss the verifiability of their contributions, including any links to relevant research artifacts, in a sub-section or paragraph titled "Verifiability".
Presentation Quality: Submissions will be expected to meet high standards of presentation, including adequate use of the English language, absence of major ambiguity, clearly readable figures and tables, and respect of the formatting instructions provided below.
Evaluation
All papers will be evaluated in terms of the following criteria.
Value: the problem is worth exploring;
Impact: the potential for disruption of current practice;
Originality: of the paper's insight;
Synergy: the paper appropriately connects a set of things that were previously treated separately;
Validity: soundness of the rationale;
Scholarship: appropriate consideration of relevant literature;
Quality: overall paper quality;
Surprise: startling and unexpected findings.