ISMAR is a high-quality conference with a competitive submission process. In 2011, ISMAR accepted only 18% of the papers submitted. ISMAR has a rigorous reviewing process that is similar to the processes used by ACM SIGGRAPH and ACM UIST. Every submitted paper and poster is subjected to this process.
As a reviewer, your work is of vital importance to the conference and we rely on your expertise and good work to be able to generate an outstanding program. The key points in undertaking reviews are:
First, log into the Precision Conference System (PCS) and complete your profile describing your expertise in the different topic areas for submissions. List all topics that you feel qualified for. Be really honest, that will help the program committee members to judge adequately if you are the best person to do a specific review.
In your profile you can also add comments about your previous experience as a reviewer: indicate for which ISMAR conference you reviewed before, any other related conferences or journals. If it’s your first review, the program committee will be able to see it and take it into account as well.
The main criterion for accepting a paper is the novelty of the contribution. Describe, what is stimulating and novel about the presented work. A good submission should also be technically correct and adhere to the standards of academic writing. Minor errors and flaws can be corrected and should not be reason to reject a paper.
Submissions are rated in the review form on a numeric scale from 1 to 5 starting at 1 (Definite reject:) to 5 (Definite accept). Be as precise in your rating of a submission as possible. Moreover, your written evaluation must support the numeric rating. The written evaluation should always indicate why the specific rating was given; if a submission should be accepted, be sure to state the reasons for it; if a submission should be rejected, again list the arguments against it. For example, your review may be discounted, if you rate a paper highly, but only describe its flaws.
ISMAR invites submissions of papers between 4 and 10 pages. The acceptance for publication will be partly based on the contribution per page of the paper. For example, a paper in eight pages will be expected to have more contribution than one with four pages. Please, evaluate the paper based on the presented length and form. Do not discount a short paper, because the authors could have shown more applications or evaluated against another concept or system. Similarly, if possible, do not recommend to shorten a long submission to a shorter version. If the paper cannot be accepted in it’s submitted form, then describe the reasons for this.
Posters, as in previous years, are considered to be the first presentations of novel ideas without necessity for substantial implementation or testing, or novel applications without significant new technical advancement so far. The main criteria for judging poster submissions are novelty and usefulness to the ISMAR community.
ISMAR posters do not present earlier work for ISMAR submissions. Therefore, if a paper builds on related work of a poster presented at ISMAR in former years, this is valid, but should be cited accordingly in the paper.
Reviewers should be careful to not immediately dismiss a submission that appears to have a flaw (e.g., in a user study) but is otherwise innovative or interesting. It is possible that such a paper would be accepted if the issues were addressed in some way during the shepherding process, e.g., by removing the associated prose, or adding prose (e.g., in the conclusions of the paper) that acknowledges the issue.
After all reviews are in, authors have access to the submission system, can read their reviews and write a short rebuttal message (about 500 words). This rebuttal should be used to comment or correct any factual errors in the reviews or clarify questions the reviewers had. It must not include new results, data, or other material. Any such information will be disregarded.
As a reviewer, please keep in mind that you can ask specific questions and may get a clarification. It also implies that authors will see your review before the final decision.
Following the review and the rebuttal period, the program committee member assigned to the submission will open a discussion about the evaluation to arrive at a common opinion and decision on the submission. This step is important to make the final decision more comprehensible to the authors. Please, follow the invitation and participate in the discussion. You will be able to read the other reviews, post messages and revise and update your own review. The program committee member will usually also let you know about their recommendation.
In writing your review, please be polite to authors. Even if you rate a paper poorly, you can critique it in a positive voice. As part of polite reviewing practice, you should always state what is good about a paper first, followed by your criticisms. If possible, you should offer suggestions for improvement along with your criticism.
Please, finish your review on time. After all reviews are completed, the discussion period starts, and finally, the assigned program committee member will write a meta-review capturing the outcome of the reviewing process, before the program committee meeting. This process needs some time, and relies on all reviews being completed by the deadline.