Science Gateways: Internal
From TeraGrid Wiki
[edit]
Page info
- Last updated 4/1/2011; created 12/17/2008
- Contact Nancy Wilkins-Diehr
- Parent page: Science Gateways
- Content: Internal pages for management of TG SGW project; Current information; links to historical content migrated from Repo/WG pages.
[edit]
Attribute-based Authentication
This is a major TeraGrid effort for 2009. By September, 2009 all jobs run using community accounts will need to include attributes that uniquely identify the end users on the gateway. This will allow TeraGrid to programmatically count end gateway users and fulfill an NSF requirement. Plenty of help is available for making this change, contact Jim Basney. For more information for both gateways using Globus and those using gsi-ssh for job submission, see https://www.teragrid.org/web/science-gateways/security#saml.
Current status of gateway implementation at http://www.teragridforum.org/mediawiki/index.php?title=Science_Gateway_Credential_with_Attributes_Status.
[edit]
Gram 5 porting
External transition guide: https://www.teragrid.org/web/user-support/gram
Internal porting experiences: http://www.teragridforum.org/mediawiki/index.php?title=Gateways_Gram5_Porting
[edit]
Software listing
Manual entries: https://www.teragrid.org/web/science-gateways/register_software
Web services entries: http://www.renci.org/~jdr0887/gawsr-howto/
[edit]
TeraGrid Science Gateway Powerpoint Templates
[edit]
Teleconference & Mailing List
[edit]
Teleconference Info
866-740-1260 Toll-free U.S. & Canada
303-248-0285 Alternate
8229457 Passcode
- Fridays
- Biweekly
- 9:00 am, Pacific
Contact Nancy Wilkins-Diehr for scheduling details: wilkinsn at sdsc dot edu
[edit]
Mailing List
Gateway list email address: gateways at teragrid dot org
Subscribe, unsubscribe, list subscribers, browse archives : http://teragrid.org/mailman/listinfo/gateways
Search the Archives [coming back soon]: Visit the Science Gateways section of the TeraGrid Web site. Each page has a Search Archives option at the top of the left navigation bar.
[edit]
TeraGrid Science Gateway Staffing
This table displays the TeraGrid staff members who are receiving GIG funding in the Extension year to work on Science Gateway related work. See the Gateway List on the public TeraGrid Web site for a full list of current TeraGrid Science Gateways.
| Institution | Name | Funding | Projects (tasks/gateway) |
|---|---|---|---|
| IU | Suresh Marru | 100% | targeted support (NOAA) |
| Raminder Singh | 100% | targeted support (PolarGrid, Ultrascan), CI Tutor tutorial for OGCE (10%) | |
| Mike Lowe | 50% | Quarry, VM support for OGCE, SimpleGrid instances on Quarry | |
| LSU | Michael Thomas (student) | targeted support (Cactus) | |
| Erik Schnetter | 53% FTE total, both positions | targeted support (Cactus) | |
| NCAR | Don Middleton | 37% | targeted support (GridAMP, others) |
| Matthew Woitaszek | 37% | targeted support (ESG-CCSM) | |
| Cecelia DeLuca | 32% | targeted support (ESG-CCSM) | |
| NCSA | Yan Liu | 100% | help desk support for gateway questions targeted support (new communities using GIS, CIPRES, NBCR), SimpleGrid, CI Tutor tutorial for SimpleGrid (10%), Quarry instance with SimpleGrid (10%) |
| Shaowen Wang | 27% | targeted support (GISolve, SimpleGrid new communities using GIS) | |
| Eric Shook | 25% | help desk support for gateway questions, example-based documentation and multimedia materials (16%) | |
| TBN (Wenwu Tang replacement) | 27% | help desk support for gateway questions | |
| NICS | Victor Hazlewood | 30% | working toward standardized treatment of community accounts via security-wg |
| ORNL | John Cobb | 59% | targeted support, remote vis (27%) |
| Vickie Lynch | 26% | remote vis, targeted support (5%) | |
| TBN | 27% | remote vis | |
| TBN | 27% | remote vis | |
| Purdue | Lan Zhao | 50% | targeted support (ESG-CCSM, C4E4) |
| Preston Smith | 25% | consulting support for condor | |
| Christopher Thompson | 50% | CCSM and classroom support | |
| Graduate student (at 50%) | 25% | CCSM and classroom support | |
| SDSC | Nancy Wilkins-Diehr | 94% | area director |
| Diana Diehl | 27% | Documentation | |
| Terri Schwartz | 53% | CIPRES and other targeted support | |
| TACC | Rion Dooley | 50% | gateway accounting tools |
| U Chicago | Stuart Martin | 50% | gateway coordination |
| | | | |
| Thomas Uram | 100% | targeted support (SIDGrid, OLSG) |
[edit]
Current Activities
Extension year; Aug. 1, 2010 - Jul. 31, 2011
The activity list below was excerpted from the Extension year Integrated Project Plan (IPP). Caveat: work is described with quarterly milestones and a July, 2011 end date. In actuality the TeraGrid follow-on program, XD, will begin April 1, 2011. This may necessitate compressed timelines with a re-focus on activities that will ensure a smooth transition to the new program. Stay tuned.
| Task | Site | Begin | End | Staff member | |
|---|---|---|---|---|---|
| Helpdesk, new communities | |||||
| Design SimpleGrid Cloud framework. Key success factor: Package available for rapid prototyping and evaluation of the TeraGrid science gateway approach. | NCSA | Aug-10 | Sep-10 | Yan (80%), Shaowen (27%), Eric (11%), TBN (27%) | |
| Develop "Front to TeraGrid Science Gateways" (F-SGW) service. Key success factor: F-SGW provides gateway software listings and access to gateway demos through the TGUP. | NCSA | Oct-10 | Dec-10 | Yan (80%), Shaowen (27%), Eric (11%), TBN (27%) | |
| Develop GISolve-X. Key success factor: New GIS communities able to use GISolve infrastructure. | NCSA | Jan-11 | Mar-11 | Yan (80%), Shaowen (27%), Eric (11%), TBN (27%) | |
| UCGIS CI workshop | NCSA, SDSC | Feb-10 | Feb-10 | Shaowen (27%), Nancy (89%) | |
| Support users, new communities using Condor. | Purdue | Aug-10 | Jul-11 | Preston (25%) | |
| Accounting | |||||
| Services to allow gateways to query and display user and job specific information such as gateway profile information, community allocation, user usage and user jobs. The project will be successful if we have released a production quality service that can be used by science gateways and the TeraGrid User Portal. | TACC | Aug-10 | Sep-10 | Rion (50%) | |
| Expand gateway information in TGUP by adding the ability for users to view their gateway jobs from the user portal. The project will be successful if we see increased usage and adoption of the job services gateway released in quarter 1 and the production release of an expanded gateway area in the TeraGrid user portal. | TACC | Oct-10 | Dec-10 | Rion (50%) | |
| Expand Gateway Service to include additional features and capabilities based on gateway and user requirements, this can include queries such as getting all gateway jobs during a specific time period, etc. | TACC | Jan-11 | Mar-11 | Rion (50%) | |
| Expand community account management and gateway tracking for PIs of community accounts. previous quarter. This project will be successful if we increase usage of gateway services, increase access to gateway area in the user portal, and complete the production release of additional gateway PI capabilities. | TACC | Apr-11 | July-11 | Rion (50%) | |
| Community Accounts | |||||
| Completion of standardized community account approach across RPs | NICS | Aug-10 | July-11 | Victor (30%) | |
| Virtual Machines | |||||
| Gateway image hosting on the IU gateway hosting service. We will offer OGCE, Simple Grid images available on request from early users. Key success factor: Availability of VMs with OGCE and SimpleGrid installed. | IU, NCSA | Oct-10 | Dec-10 | Mike Lowe (60%), Yan (10%) | |
| Web service control over gateway images on gateway hosting service. Key success factor: Ability to control gateway images on hosting service via Web services. | IU | Jan-11 | Mar-11 | Mike (60%) | |
| Implement initial "custom build" virtual machines for gateway hosting. Key success factor: Prototype custom-built VMs available. | IU | Jan-11 | Mar-11 | Mike (60%) | |
| Custom build virtual machines on gateway hosting service made. Key success factor: Production custom-built VMs available. | IU | Apr-11 | July-11 | Mike (60%) | |
| Web service and Web dashboard for controlling gateway VMs made available. Key success factor: Dashboard for VM control made available. | IU | Apr-11 | July-11 | Mike (60%) | |
| Education and Training | |||||
| Gateways tutorial for CI Tutor organized with Sandy Kappes, NCSA. Tutorial can cover OGCE topics, advanced workflow support or Open Social gadget-based gateways. Key success factor: Availability of OGCE CI Tutor tutorial. | IU | Aug-10 | July-11 | Raminder (10%) | |
| Develop CI Tutor tutorial for SimpleGrid. Key success factor: Availability of SimpleGrid CI Tutor tutorial. | NCSA | Aug-10 | July-11 | Yan (10%) | |
| Gateway Documentation | |||||
| Transition documentation from wordy text descriptions to many more examples and demonstrations. Incorporation of topics developed by others. General availability for keeping gateway documentation current. TG to XD transition documentation. | SDSC | Aug-10 | July-11 | Diana (27%) | |
| Provide example-based documentation and multimedia materials (e.g., short videos) based on help desk questions. Develop content for how to quickly build and customize a gateway prototype from SimpleGrid on a TeraGrid science gateway host cluster (i.e., Quarry@IU); | NCSA | Aug-10 | July-11 | Eric (16%) | |
| Gateway Targeted Support | |||||
| SimFactory extended to allow Cactus to be compiled and deployed on all TeraGrid machines. Logging and database capabilities added to store persistant information. Software improved for release. Documentation provided for the use of SimFactory. SimFactory version 1 released to community as a set of command line tools. Metric: Release of software and documentation. Demonstration that SimFactory can compile a real physics code (e.g. the open community McLachlan code) on all TeraGrid resources. | LSU | Aug-10 | Sep-10 | Michael | |
| TeraGrid tools for profiling and debugging integrated into SimFactory work process and tested with a suite of community codes. SimFactory version 2 released. Metric: Five different research groups are using SimFactory | LSU | Oct-10 | Dec-10 | Michael | |
| Support for on-demand debugging integrated into SimFactory. SimFactory version 3 released. Metric: SimFactory able to support on-demand computing on more than 500 cores of at least one TeraGrid resource | LSU | Jan-11 | Mar-11 | Michael | |
| Simple GUI interface provided for SimFactory. SimFactory version 4 released. Metric: Ten different research groups from at least two different application areas are using SimFactory on TeraGrid resources. | LSU | Apr-11 | July-11 | Michael | |
| Flexible gateway support for incoming requests. Past work includes development of workflow tools, scheduling strategies, software environment and visualization tools, building several small Matlab programs to act as filters, called through Web interfaces, on data sets, incorporation of community accounts, attribute-based authentication, transition to GRAM5 and improved error handling and database hosting. | IU | Aug-10 | July-11 | Suresh, Raminder | |
| CCSM-ESG. Test and debug the prototype systems developed in PY5. | Purdue | Aug-10 | Sep-10 | Lan (50%), Christopher Thompson (50%), grad student (50%) | |
| CCSM-ESG. Integration of the prototype systems so that an end-to-end CCSM workflow is supported that includes publication of the data and metadata from a CCSM run back to the ESGC gateway. | Purdue | Aug-10 | Dec-10 | Lan (50%), Christopher (50%), student (50%) | |
| CCSM-ESG. Improve performance and scalability by running simulations in multiple TeraGrid resources with scheduling algorithms based on resource availability and network bandwidth. | Purdue | Aug-10 | Mar-11 | Lan (50%), Christopher (50%), student (50%) | |
| CCSM-ESG. Use the portal system in classrooms. | Purdue | Jan-11 | July-11 | Lan (50%), Christopher (50%), student (50%) | |
| CCSM-ESG. Develop tutorials, education modules. | Purdue | Apr-11 | July-11 | Lan (50%), Christopher (50%), student (50%) | |
| CCSM-ESG. Replicate one or more CCSM workflows used by internal (NCAR) CCSM model developers on the TeraGrid, using the ESGC Gateway. | NCAR | Aug-10 | Sep-10 | Cecelia (32%), Don (37%) | |
| CCSM-ESG. Demonstrate a workflow that includes publication of the data and metadata from a CCSM run back to the ESGC Gateway. | NCAR | Oct-10 | Dec-10 | Cecelia (32%), Don (37%) | |
| CCSM-ESG. Replicate one or more CCSM workflows used by the CCSM external community on the TeraGrid, using the ESGC Gateway. | NCAR | Jan-11 | Mar-11 | Cecelia (32%), Don (37%) | |
| CCSM-ESG. Demonstrate federation of Purdue’s data holdings so that they appear as part of the ESG collections. Utilize CCSM workflows managed by the ESGC Gateway in a scientific or educational activity, such as a summer school, university course, workshop or tutorial. | NCAR | Apr-11 | July-11 | Cecelia (32%), Don (37%) | |
| Flexible gateway support for an incoming earth science-related gateway. Develop requirements specification through in-situ deployment of software on TeraGrid resources with domain researchers. | NCAR | Aug-10 | Sep-10 | Matthew (37%) | |
| Prototype job and data management solution. | NCAR | Oct-10 | Dec-10 | Matthew (37%) | |
| Develop and deploy revised gateway. | NCAR | Jan-11 | Mar-11 | Matthew (37%) | |
| Finalize development and solution testing | NCAR | Apr-11 | July-11 | Matthew (37%) | |
| Spallation Neutron Source support. Maintenance for the RP aggregation switch and a portion of the site lambda connectivity. Coordination with REDDnet group and the provision of storage and data transport both inside and outside of the TeraGrid network as well as coordination between TeraGrid and DataNet partners. | ORNL | Aug-10 | July-11 | John (37%), Vickie (5%) | |
| CIPRES support. Ability to handle individual user accounts, attribute-based authentication, expansion to other TG platforms and beyond as necessary. | SDSC | Aug-10 | July-11 | Terri (50%) | |
| OpenSocial frameworks. Release a stable File Transfer Gadget. Through the gadget, users can browse their TeraGrid data on each TeraGrid site, and reliably transfer files between sites. | UC | Aug-10 | Sep-10 | Tom (100%) | |
| Release a Job Management Gadget. Through the gadget, users can submit jobs to TeraGrid and check job status. Release a social network management service allowing users to set up their own social relationship and import their existing social data from commercial social sites like MySpace. | UC | Oct-10 | Dec-10 | Tom (100%) | |
| Release a collaborative data management service. Users can make their data files sharable for their friends or a group of collaborators, and tag these files collaboratively. | UC | Jan-11 | Mar-11 | Tom (100%) | |
| Release a collaborative workflow management service. Users can use this service to run and monitor the progress of their workflows with their collaborators. | UC | Apr-11 | July-11 | Tom (100%) | |
| GRAM gadgets. Develop a way to easily monitor all GRAM jobs for a user. Design doc - design the new interface for getting a list of a user's jobs from a gram service. | UC | Aug-10 | Sep-10 | TBN (21%) | |
| New code enhancement - Implement design in GRAM5 service and C client API | UC | Oct-10 | Dec-10 | TBN (21%) | |
| New code enhancement - Implement design in Java CoG's jGlobus GRAM client API | UC | Jan-11 | Mar-11 | TBN (21%) | |
| New code enhancement - Work with Google Gadget developers to design and implement a generic GRAM job monitoring gadget | UC | Apr-11 | July-11 | TBN (21%) | |
| Remote Visualization for Gateways | |||||
| Develop web interface for visualization algorithms for standard file formats. | ORNL | Aug-10 | July-11 | John (27%), Vickie (21%), TBN1 (27%), TBN2 (27%) | |
[edit]
Important Science Gateways Management Links
[edit]
RP Community account information (password protected)
[edit]
TeraGrid Science Gateways registration (password protected)
[edit]
Science Gateways Past Management (includes internal info migrated from Repo)
[edit]
Science Gateways Policies
[edit]
TeraGrid Science Gateways site
[edit]
SG in the TeraGrid Wiki
[edit]
Science Gateway Use Cases
[edit]
LifeRay editing
[edit]
Form Finder Links to all the forms used by PIs to get a TG SGW up and running.
[edit]
Larger Gateway Events
[edit]
Gateway Security Summit, January, 2008
[edit]
