2017 ACL 2017:TPMS
Jump to navigation
Jump to search
Sent to Track Chairs.
Subject: Toronto Paper Matching System Information
Dear [reviewerFirstName] [reviewerLastName], This email briefs you on how the TPMS is currently integrated. It is a bit clunky but we hope you will try to make use of it as it is a new development that we hope ACL can adopt for the long term. We believe it is best for one AC to take charge of this development just for the short-term assignment that is happening today -- it is not a good idea to have multiple ACs working on this for the area. Your meta AC will delegate this responsibility. 1) TPMS relies on 'queries' that we have to design and put to use. We think it is most useful to have a single query that represents all papers and all reviewers in a track (one for long and one for short, since they are on different portals). You may design other queries but this is what we have tried to run for you to pilot it. To build a query, you go to Beta and Legacy Tools > Manage the Toronto Paper Matching System > Manage Queries. There you'll see the query that I have created earlier and you can retrieve its results. However, the results from there are from earlier today, and we anticipate a change in the number of registered accounts. So let's create a query! Click Create Query. To do as above, go to the reviewers and papers panels and select all for each one (You may have to change the pager to display more than 10 results). Then go to the description panel and write a name. The previously run query's name is similar to "All (<Area> - Long)". 2) When constructed, the queries are deposited on the TPMS server and run. It takes time for the system to process it (it doesn't seem very deterministic). You must retrieve the results later -- sometimes it can take 20 minutes or longer. If you want to re-run the query later, you have to construct a new query. 3) When it finishes, the system (as of right now) sometimes incorrectly states that there are 0 reviewers in the results. This is a bug that we have discovered, and we are not sure whether it will get fixed in time. But there's a simple workaround -- construct a new query with the same parameters and try again. 4) When you get somewhat useful results (we believe around 30-40% of reviewers currently have a workable TPMS account), you can choose to have these scores show up in the assignment interface. Do this by checking the "canonical" checkbox. This will cause the scores to show up in the sub menus under "Make Assignments". We've done the preliminary queries for you for the long papers, but please try to do it for the short papers, to get some familiarity for it -- we have not run any queries for you on that side. We're pretty sure that there will be questions on how to use TPMS, so please feel free to let us know. We'll try to answer these as timely as we can. Thanks again for your cooperation! -- ACL 2017 Long - https://www.softconf.com/acl2017/papers ACL 2017 Short - https://www.softconf.com/acl2017/shortpapers Shared Spreadsheet - <shortlink/>