From TeraGrid Wiki
The Large Resource Allocations Committee (LRAC) and Medium Resource Allocations Committee (MRAC) -- jointly referred to as the "xRAC" -- is the group which reviews proposals for use of NSF resources made available to the national research community. As of April 2006, all resources allocated by the xRAC are TeraGrid resources, so while the xRAC committee is not a TeraGrid committee, it is a critical component of our operational framework. The 35-40 members of the xRAC are computational experts from many institutions, and the committee and process are designed to provide independent merit-review of proposals for the TeraGrid. Proposals for allocations are submitted and managed through the POPS system, and there is documentation about the operational practices of that group on the POPS site.
xRAC committee meetings are held quarterly in March, June, September, and December for decisions on Medium requests (up to 500,000 SUs as of March 2007), with the March and September meetings including review of Large requests as well. As of September 2007, proposals can include requests for storage resources, with disk requests up to 25 TB and tape requests up to 100 TB considered by the MRAC, and larger requests by the LRAC. A request that combines compute and storage requests is categorized for review according to the largest of its compute, disk or tape components.
For the official policy document for the TeraGrid allocations process, go to TeraGrid Allocations Policies.
TeraGrid Allocation Proposal Guidelines Teleconferences
- The first teleconference, supported by readytalk, was held on Thursday, June 26, 2:00-3:300 CDT
- 16 non-TG Attendees and 5 TG Attendees
- TG Allocation Proposal Guidelines Teleconference documents
Step 1. Making Resources Available for Allocation
To include a resource in the xRAC/POPS process, the following timeline applies (using the March xRAC meeting as an example):
Step 1a: Get resource into POPS. We need a name and a few "attributes", which in POPS means a short set of Q&A, for which I can provide examples. At this point, we don't even need to know how much they plan to make available, just that the resource will be available. Must be done by prior to the proposal submission period, which is generally just after every xRAC meeting. So for a resource to be allocable in March, it needs to be added to POPS in December.
Step 1b: Resourse owner/sponsors need to promote the resource's availability so that potential requestor's know to ask for time. There is nothing formal about this step, and they can start at any time. A news submission for posting to TeraGrid News is one easy way suggested at a minimum.
Step 1c: A few weeks prior to the xRAC meeting, the Resource Provider will need to specify a number of SUs that they are willing to allocate at the meeting.
Step 1d: The Resource Provider needs to include their system in TGCDB so allocations can be made. This presumes the RP has AMIE up and running, and they are plugged into the accounting system. I have some prelim documentation on this step. It involves preparing a short text file. Needs to be done prior to the allocation award start date ideally by early March for April 1 allocations.
Step 2. Proposal Submission
This is done through the POPS system.
Step 2a: The proposal submitter logs into POPS. They can use their POPS login/password (creating one if they do not have one or any other TeraGrid authentication). Submitters can also authenticate with their TG Portal login, or NCSA or SDSC Kerberos credentials.
Anyone is allowed a POPS login/password. The web page to obtain the ID/PW is open, and the process is automatic, with no checking or verification of applicants. POPS logins and passwords remain valid indefinitely as long as you remember the username/password. If you forget your POPS login, you can e-mail the POPS allocation group via the link on the POPS web page. One of the allocation group must then verify their identity, reset their password, and FAX their new password back to them. This is a potential issue, since the nominal hours of the allocation group are eight hours per day, five days per week. Typically used only once per year (we can get data on this: frequency of use and frequency of reset). Requests for POPS password resets peak just before the MRAC/LRAC submission deadlines.
Step 2b: The proposer completes the required information for submitting an allocation proposal. See CI-Partnership Allocations for more information about types of requests. New proposers may contact the POPS allocation group during this process to ask questions (this is relatively infrequent).
Step 2c: The system stores the submitted information in the POPS database. Some of the fields are drop-downs, but much of the data entered is free text that is not subject to any editing.
Step 2d: Proposer gets notified by e-mail when s/he finalizes his/her proposal.
- Initiator: Proposer (PI)
- Involved Parties: PI, allocations staff, POPs development team
- Result: A proposal (with all supporting information) ready for review
- Result holder: Allocations staff
Step 3. Proposal Review
Step 3a. POPS administrators can view POPS "List of proposals" for a meeting to see all submitted proposals for a particular meeting. Some of the information in proposals, such as adherence to NSF Cyberinfrastructure Resource Allocation Policies, must be checked by the meeting administrator(s) prior to distribution to the reviewers. This is a significant effort and requires familiarity with the policies.
Step 3b. The xRAC meeting administrator sends a list of conflicts of interest (CoI) to the POPS development team to be entered into the system so that conflicts will show up in the reports and will ensure reviewers with conflicts won't be able to see proposals they have a conflict with. Additional conflicts can be added later in the process.
Step 3c. The xRAC meeting administrator assigns reviewers (via POPS Admin page) for each proposal. The meeting administrator then requests the development team to open the POPS link for reviewers and notifies the reviewers that the link is available so that they can begin reviewing the proposals assigned to them.
Step 3d. xRAC members are generally given 3-4 weeks to download, read, and submit their reviews via the POPS Review site. On rare occasions, a reviewer can e-mail the meeting administrator asking them to contact a proposer with a question. The proposer's response is forwarded to the reviewer (to maintain anonymity of reviewers). The allocations group watches the reviews to make sure that they are entered into POPS as required and to remind reviewers of deadlines.
Step 3e. During the review period, the xRAC meeting administrator prepares additional meeting materials: a spreadsheet of prior usage (for PIs with prior allocations), and the spreadsheet used by the allocations staff to track recommendations against resource availability. Also during this period, the meeting administrator solicits the available SUs (or TBs) from RPs for the resources each site is making available. The available units are entered into POPS for meeting reports and into the allocation spreadsheet. Prior usage data is currently taken from the TGCDB and plans include linking to TGCDB directly from POPS-Review to permit reviewers to access this data directly.
Step 3f. After the review submission deadline, the meeting administrator assembles the final meeting materials, including meeting agendas (summary of proposals and requests, generated by POPS-Admin), and meeting books (proposal info including abstracts plus full reviews, also generated by POPS-Admin).
Step 4. The xRAC Meeting
Step 4a. The xRAC meeting event begins the night before at a 'caucus' session. During the caucus, the reviewers on a given proposal confer and agree upon a consensus recommendation. This is a self-organizing process and proceeds informally as reviewers form and reform small groups.
Step 4b. The xRAC meeting proper starts in the morning. The meeting generally begins with RP announcements and resource updates. Occasionally, other topics are discussed as time permits to solicit input from the xRAC members. The meeting chair (appointed for each meeting) guides the xRAC through all of the submitted proposals. For each proposal, one reviewer presents the proposal and the consensus recommendation. Other xRAC members can comment as appropriate. As each recommendation is made, the allocations staff enters it into a spreadsheet. Reviewers are instructed to award SUs or TBs based on each proposal's merits, without consideration of final resource availability.
Step 4c. After all recommendations are made, allocations officers confer to ensure recommendations were consistently recorded. Then, to assess whether recommended awards exceeded the availability on any resource. For those resources where awards exceed availability, every effort is made to move awards to comparable resources with available time. Only as a last resort, when all resources are fully allocated, are awards reduced. xRAC reviewers are consulted as needed to provide input to the appropriateness of award moves.
Step 4d. The xRAC is presented with a summary of any awards moved or reduced, if any, and the meeting is adjourned.
Step 5. Processing Allocations Awards and Follow-Up
Step 5a. Following the xRAC meeting, the process to make awards is begun to add the allocation awards into the TeraGrid accounting system. The meeting administrator enters award amounts into POPS via POPS-Admin.
Step 5b. In addition, the meeting administrator notifies PIs of the results of the requests, including anonymized reviewer comments. Currently, this process is manual and involves sending postal or electronic mail with award letters and reviewer comments (review reports are provided via POPS-Admin). Under development are mechanisms within POPS to allow PIs to both check status of submitted proposals during the review process, and following entry of awards made and any meeting- or proposal-specific comments by the meeting administrator, to permit PIs to be notified by email that award results and reviewer comments are available via POPS.
Step 5c. If they are dissatisfied with their award results and reviewer comments, PIs have the option of submitting "justification" proposals via POPS within four weeks of being notified of their awards. The meeting administrator assigns these submissions to reviewers and requests reviewers to evaluate them. After reviewer comments have been received, awards (if any) are made and processed. Policies state that PIs are entitled to a decision within five weeks of submission; if reviewer comments have not been received, the allocations officers agree upon an award.
Step 5d. In addition to participation in the meetings, xRAC members are also involved in the review of Supplemental proposals. These proposals can be submitted at any time, and policy states that the PI is entitled to a response within five weeks. In practice, however, consideration of large Supplemental requests are often deferred to the next available xRAC meeting for full discussion.
Selection of xRAC Members
Generally, xRAC members serve three-year terms, although those terms can be extended up to one-year by invitation or cut short by the reviewer for various reasons.
Candidates to fill xRAC slots are solicited from current and departing xRAC members, the TeraGrid RPs, the TeraGrid Science Advisory Board, NSF, and other sources. Based on discussion among the allocations officers and others, as well as the current balance of disciplinary expertise, candidates are contacted and invited to participate. Contacts and recruiting are coordinated, but the responsibilities are shared among RPs. The xRAC meeting administrator is tasked with monitoring the current xRAC roster and term expirations to ensure that recruiting efforts are appropriately targeted.