Updates related on Grid since last meeting in December 2008: Service, resource, and recent achievements to utilize distributed computing infrastructures
This document summarizes updates related to grid computing since the last meeting in December 2008. It discusses resource upgrades at KEK, including an additional 2.5 million SPECint2000 by December 2009 and another 2.5 million by March 2010. Plans are outlined to deploy a storage service in early 2010 to expose tape storage to the grid. Operational statistics show over 1,000 ILC jobs run, accounting for around 10% of CPU time and over 1 petabyte of storage used. ILC is reported as the heaviest computing resource consumer at KEK in the past 6 months. Several pending items are also noted including FTS deployment and reconstruction of the KEK-1 grid site.
Similar to Updates related on Grid since last meeting in December 2008: Service, resource, and recent achievements to utilize distributed computing infrastructures
Data Plane Evolution: Towards Openness and FlexibilityAPNIC
Similar to Updates related on Grid since last meeting in December 2008: Service, resource, and recent achievements to utilize distributed computing infrastructures (20)
Unraveling Multimodality with Large Language Models.pdf
Updates related on Grid since last meeting in December 2008: Service, resource, and recent achievements to utilize distributed computing infrastructures
1. 4th ILC Detector Workshop 2-4 December 2009 @ KEK, Tsukuba, Japan Day2: Software (15min) Updates related on Grid since last meeting in December 2008 http://kds.kek.jp/conferenceTimeTable.py?confId=3363 Service, resource, and recent achievements to utilize distributed computing infrastructures Go Iwai, KEK/CRC go.iwai@kek.jp Agenda of last meeting can be found at http://kds.kek.jp/conferenceTimeTable.py?confId=2260
2. Mt. Tsukuba (N) Central Tsukuba (S) KEK-2 KEK-1 Quoted from last meeting in December 2008 upgrade 2009 upgrade 2009 upgrade 2009 Belle Computing System (BCS) Storage Belle Computing System (BCS) CPU Super Computer System Central Information System (CIS) ILC Detector Workshop -- Updates related on Grid since last meeting in December 2008
3. Major HEP projects: Belle, J-PARC, ATLAS ongoing projects ILC, Belle2 future projects Also covering Material science, bio-chemistry and so on using synchrotron light and neutron source Radiotherapy as technology transfer We have a role to support university groups in these fields. including Grid deployment/operation. Tohoku Univ. Nagoya Univ. Kobe Univ. KEK Hiroshima IT Univ. of Tsukuba Introduction Dec 3, 2009 ILC Detector Workshop -- Updates related on Grid since last meeting in December 2008 3 Hiroshima Univ. (Alice) U of Tokyo (Atlas)
4. KEK’s Contribution in EGEE-III TSA1.1: Grid Management Interoperability and collaboration TSA1.2: Grid operations and support 1st line support for operations problems Middleware deployment and support Coordination of middleware deployment and support for problems. Regional certification of middleware releases if needed (outside of PPS and SA3 involvement). This is anticipated to be very rare and will require specific justification. Dec 3, 2009 ILC Detector Workshop -- Updates related on Grid since last meeting in December 2008 4
5. SINET Both for domestic & international institutes Logical Network ConnectionFocused ONLY on Grid Dec 3, 2009 ILC Detector Workshop -- Updates related on Grid since last meeting in December 2008 5 Univ. Grid Univ. Grid Facility Site Univ. Grid Univ. Grid Service Univ. Grid Univ. Grid VO-Central Service Generic Service at Grid Level Upgraded VOMS VOMRS Upgraded Retired CA KEK-1 gLite KEK-2 gLite VOMS Computing Facility for Belle Exp. LFC SRB SRB-DSI CD-1 NAREGI MCAT HSM CD-2 NAREGI DMZ Intranet GRID-LAN CC-GRID VO-Specific Service
6. Recent Changes in VOMS VOMS.kek.jp has been shutdown on 2nd Oct 2009, end of validity of its host certificate. Migrated to voms.cc.kek.jp Validity of proxy cert is extended 7days, i.e. 168hours MyProxy does not work properly without special settings User enrollment https://vomrs.cc.kek.jp:8443/vo/voname/vomrs Should be kept up to date on CIC portal https://cic.gridops.org/index.php?section=vo&vo=voname Dec 3, 2009 ILC Detector Workshop -- Updates related on Grid since last meeting in December 2008 6 People in ILC are not affected by this change! VOMS VOMRS voms.kek.jp VOMS voms.cc.kek.jp DMZ GRID-LAN
7. Recent Change in KEK-1 CE/RB based on gLite-3.0/SL3X has been decommissioned in Oct 2009. Still not yet deployed news Dec 3, 2009 ILC Detector Workshop -- Updates related on Grid since last meeting in December 2008 7 KEK-1 gLite KEK-2 gLite No CE can be found at GStat http://gstat.gridops.org/gstat/JP-KEK-CRC-01/
8. Resource Scale (KEK-2) Storage service (DPM) ~10TB with HPSS as backend storage ~30,000 JPY/TB Except for HSM through SRB-DSI Not published information by Glue Computing service System A: As a part of Central Information System (CIS) System B: Recycle-use of old blades in Belle Computing System (BCS) Dec 3, 2009 ILC Detector Workshop -- Updates related on Grid since last meeting in December 2008 8 VOMS VOMRS Computing Facility for Belle Exp. SRB KEK-2 gLite LFC MCAT HSM System B System A Reuse of old resources Upgraded in March Computational specification of system A & B as of Oct 2009
9. Dec 3, 2009 ILC Detector Workshop -- Updates related on Grid since last meeting in December 2008 9 Recent WorksIntegration of BCS and CIS Computing Facility for Belle Exp. SRB MCAT HSM MT Ops Room @ South Bldg. of CRC 10G x 2 1st Machine Room @ North Bldg. of CRC (1) Done: BW expansion is Oct 2009 (2) In deploying SRM-SRB 1Gx12 Done Pend. Pend. CE CE Done CE CE SRB-DSI (GridFTP) (3) Done half, pending others SRB HSM KEK-2 gLite Existing computing resource 192 nodes ~ 1,536 cores ~ 4.6 MSI2K
10. Snapshot Dec 3, 2009 ILC Detector Workshop -- Updates related on Grid since last meeting in December 2008 10 http://gstat.gridops.org/gstat/JP-KEK-CRC-02/ 4 CEs will be available step-by-step System B (Old blades in BCS) are once again re-used for NAREGI First half of 4 CEs are now available Second half is depend on works in Belle/Belle2 In Dec 2009 In Mar 2010? System B: down in Dec 2009 System A
11. 252 MSI2K as of Oct 2009 Resource Scale – LCG Infrastructure Dec 3, 2009 ILC Detector Workshop -- Updates related on Grid since last meeting in December 2008 11 http://gridmap.cern.ch KEK 1MSI2K
12. 252 MSI2K as of Oct 2009 Dec 2009 (Now) Dec 3, 2009 ILC Detector Workshop -- Updates related on Grid since last meeting in December 2008 12 KEK 2.8MSI2K
13. 252 MSI2K as of Oct 2009 Mar 2010 Dec 3, 2009 ILC Detector Workshop -- Updates related on Grid since last meeting in December 2008 13 KEK 5.0MSI2K Half of Asia-Pacific & regional largest center
14. Summary Table of Resource Scale Dec 3, 2009 ILC Detector Workshop -- Updates related on Grid since last meeting in December 2008 14 Sorry for confusion! A bit revised: “System C is now available and D will be in operation in Next March” is incorrect. It should be corrected that “Half of both System C and System D is now available and another half will be in operation in Next March”. Up Down Legend
15. Upcoming Storage Service To expose HSM to Grid SRB-DSI to bridge SRB inside BCS Just for GridFTP access, not for SRM and LFC To enable access with logical file name, e.g. lcg-xxx lfn://logical-file-name SRM-SRB for Site URL, e.g. srm://srm.kek.jp/ SRB-DSI for Transfer URL, e.g. gridftp://disk.kek.jp/ A bit behind of schedule Dec 3, 2009 ILC Detector Workshop -- Updates related on Grid since last meeting in December 2008 15
16. Ops stats @ KEK in recent 6 months More than 1K jobs have been processed forILC ~10% of CPU time ILC is the heaviest consumer! Dec 3, 2009 ILC Detector Workshop -- Updates related on Grid since last meeting in December 2008 16 1,439 of 34,462
17. Dec 3, 2009 ILC Detector Workshop -- Updates related on Grid since last meeting in December 2008 17 1,207 of 1,285 GB Read 2,837 of 3,830 GB Write
18. Summary of Updates on Grid Resource upgrade +2.5MSI2K in Dec 2009 +2.5MSI2K in Mar 2010 (tentative) Storage service In operation early next year hopefully Ops Stats Storage: Read: 1,204 GB in 6 month Write: 2,837 GB in 6 month CPU: 1K jobs (~10% of CPU time) are submitted (Many) pending works FTS Common requirement KEK-1 re-construction Dec 3, 2009 ILC Detector Workshop -- Updates related on Grid since last meeting in December 2008 18 ILC is the heaviest consumer in previous 6 months at KEK!
19. Many Thanks for Your Attention! Go Iwai, KEK/CRC go.iwai@kek.jp