Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

IDC Technology Spotlight: Big Memory Computing Emerges to Better Enable Data-Instensive IT

848 Aufrufe

Veröffentlicht am

In this report, Eric Burgener, Research Vice President at IDC provides an overview of key Big Memory market dynamics. He proposes that as real-time analytics workloads become more prevalent, the gap in the memory/storage hierarchy is highlighting a significant market opportunity that is addressed by Big Memory Computing.

Eric talks about a confluence of market and technology enablers, definitions of big memory technologies, the benefits of Big Memory, and key early use cases for Big Memory Computing. Read this report if you want a 3600 view of Big Memory in 10 minutes.

Veröffentlicht in: Technologie
  • Als Erste(r) kommentieren

IDC Technology Spotlight: Big Memory Computing Emerges to Better Enable Data-Instensive IT

  1. 1. IDC TECHNOLOGY SPOTLIGHT Sponsoredby:MemVerge Big Memory Computing Emerges to Better Enable Data-Intensive IT June 2020 Written by: Eric Burgener, Research Vice President, Infrastructure Systems, Platforms, and Technologies Group Introduction Digitaltransformation (DX)—theevolutionamongenterprisesofallsizes towardmuchmoredata-drivenbusinessmodels—isunderwayinmost industries.Moredatathaneverbeforeisbeingcaptured,stored, protected,andanalyzed,and ITorganizationsare morefocusedthanever ondrivingdirectbusinessvaluefromthedatatheycollecton customers, productsandservices,markets,andinternalprocessesandworkflows. Next-generationapplications(NGAs)arebeingdeployedtohelpdrivethis value,andmanyofthememploya"bigdataanalytics"orientationthat leveragesdata-intensivetechnologiessuchasartificialintelligence (AI), machinelearning (ML),anddeeplearningtogleaninsightsthatresultin betterbusinessdecisions.Real-timeworkloadsareappearinginmany industries—examplesincludefraud analyticsinfinancialservices, customerprofilinginsocialmediaand streaming, InternetofThings(IoT) applications,andsecurityandcyberthreatdetectionacrossmanydifferent industries—andthepercentageofreal-timedataisontherise. Thesetypesofreal-timebigdataanalyticsworkloadsareputtingperformancedemandsonITinfrastructurethatarevery difficulttomeet costeffectivelywithlegacyarchitectures.Butperformance isnottheonlychallenge.Asenterprises evolvetheirbusinessmodels,manyoftheseNGAsformthefoundationforcompetitivedifferentiation,andtheyare increasinglybeingviewedasmissioncritical.Indeed,IDCpredictsthatby2021,60–70%ofthe Global2000willhaveat leastone real-timebigdataanalyticsworkloadthatisalso considered missioncritical.AsenterprisesmodernizetheirIT infrastructuretomeetthese evolvingperformance and availabilityrequirements,the largeperformancegapinthe memory/storagehierarchybetween volatile,byte-addressable,andexpensivemain memoryandblock-addressable solid statedisks(SSDs)(which arebothpersistent and lessexpensiveona$/GBbasis)isbecomingincreasingly problematic.Althoughsome applicationshavebeenwrittento run inmainmemory(e.g.,OracleDatabaseIn-Memory, SAPHANA)toimprovetheperformanceandefficiencyof resourceutilization,today'smainmemorycapacityisstillquite limited. Inaddition,thecatalogofofferingsthathavebeenrewritteninthismannerisextremelylimitedbecause itisa major software engineeringprojecttoretoolanapplicationforin-memoryoperation. As real-time analytics workloads become more prevalent, the gap in the memory/storage hierarchy is highlighting a significant market opportunity that is addressed by a new market category called Big Memory Computing. KEY TAKEAWAYS » Many new workloads being deployed by enterprises undergoing digital transformation demand memory-class performance. » By combining Persistent Memory technology with software-defined memory virtualization, Big Memory Computing meets new workflow needs. » Because Big Memory Computing runs on industry-standard server hardware and does not require any application modifications, enterprises should be able to easily evaluate how the technology can apply to them. AT A GLANCE
  2. 2. Page 2#US46573120 IDC TECHNOLOGY SPOTLIGHT Big Memory Computing Emerges to Better Enable Data-Intensive IT Asreal-time,mission-criticalbigdataanalyticsworkloadsbecomemoreprevalent,this"gap"inthememory/storage hierarchyishighlightingasignificantmarketopportunity. Whatisneeded isanabilitytosignificantlyextendthe capacity ofmainmemorytoaccommodatemuch largerdatasetswitha solution exhibitingnear-DRAMperformanceatlower costthatisbothpersistentand appropriateformission-criticalworkloads.Tomaximizenear-termadoption,thissolution shoulddeliverthesecapabilitieswithoutrequiringexistingapplicationstobe rewritten. A Confluence of Market and Technology Enablers Thepreviouslydefinedsolutionshouldcome asnosurprisetoITpractitioners,butuntilrecently,themarket imperative andthetechnologytosolvetheproblemdidnotexist.Atthebeginningofthepastdecade,severalsmallstart-ups attemptedtoaddressthisissue,buta strongmarketrequirementandthekeytechnologiesneededto solveitwerenot there.Thesituation isverydifferenttoday.WiththerecentexplosioninAI/ML-drivenreal-timebigdataanalytics,the marketneedforaviablenear-term solution isclear,directlyrelatedtobusinesssuccessformanyenterprises,andwill becomewidespreadoverthenext12–24months. Unlikeadecadeago,severalnewtechnologieshavearisenthatmakeitpracticaltoaddressthisneedtoday.Thefirstis the concentratedcomputepower,nowwidespread,that'sneededtoleveragetechnologiessuchasAI/MLforbigdata analytics.MulticoreCPUsandgeneral-purpose GPUsareavailableinvolumefrommultiplevendors,affordablydelivering themassive computepower neededforreal-timeAI/ML-drivenworkloads. Thesecondisanewclassofstorage,referredto asPersistentMemory(PM),that ispersistent,byteaddressable, availableinmuchlargercapacitiesthantraditionalDRAMDIMMs,costslessona$/GBbasisthanDRAM,anddelivers accesslatenciesintheseveralhundrednanosecondrange.PMproductsareavailabletodayfromasinglevendorbutare expectedtobeavailableinvolumefrommultiplevendorsby2022.IDCexpectsthatfrom2019to2023,thePM market willgrowata248%compoundannualgrowthrate (CAGR)tocrest$2.6billioninannualrevenue. Thethirdtechnologyisasoftware-definedmemoryvirtualizationlayerthat createsalogicalpoolofpersistentstorage composedofbothDRAMandPMthat canbe sharedacrossserversthrougha switched remotedirectmemoryaccess (RDMA)network.Thisisnot anNVMeoverFabricsnetwork(which supportsblock-addressabletraffic);thisisan RDMA networkthat supportsbyte-addressabletraffic. This"memorylake"willprovide significantlyhighercapacitiesthanDRAM technologyaloneatablendedcost/GBthatisalreadylowerthanthatofDRAMandwillcontinuetodropasmultiple vendorsbeginto shipproductsin volume. TomeetNGArequirements,thismemoryvirtualizationlayershouldincludeseveralkeyfeaturesbesidesjust memory pooling. First,itshouldprovideintelligentdataplacementwithinthememorylaketo optimizeaccesslatenciesand generalperformance.Second,itshouldincludeenterprise-classdataservicesthatenable mission-criticaloperation. Ataminimum,thisshouldincludedataprotectionfeaturessuchasRAIDand/orerasure codingandfastrecovery featuressuchassnapshotsand replication.Giventhattradingandfinancialmarketanalyticsare anearlyusecase, encryptionwouldalsobean attractive earlyfeature.Third,itshouldbeusablewithexistingworkloadswithoutrequiring anyapplicationrewrites.
  3. 3. Page 3#US46573120 IDC TECHNOLOGY SPOTLIGHT Big Memory Computing Emerges to Better Enable Data-Intensive IT Definitions IDCbelievesthatthepreviouslymentioned marketopportunityandsolutiondefinitionwillbecomethebasisfor anew market category.An aptnameforthismarketthat isbothdescriptive and shortisBigMemoryComputing. The solution outlinedpreviouslyrequirestechnologiesfromatleastseveralvendors,anditisimportanttonotethatmultiplevendors arealreadyworkingonboththehardware componentsandthesoftwarecomponentsforthisplatform. Figure 1 providesasimple,nonvendor-specificdefinitionofBigMemoryComputing. FIGURE 1: The DefinitionofBig Memory Computing Source: IDC, 2020 Benefits Thebusinessbenefitsof BigMemoryComputingareclear.First,itputsmoredatacloserto computeresourcesand makesthedataaccessibleatDRAM-like speeds,providingsignificantlymore consistentandscalableperformancethan is possiblewithtoday'smemory/storagehierarchy. Bigdataanalyticsworkloadsthatrequirehighdegreesofconcurrency whileworkingonverylargedatasetswillbe abletodrivebetterresultsfasterwiththisnewparadigm. Infact,manyof thesenewworkloadswouldnotevenbefeasiblewithoutthecombinationofacceleratedcomputeand BigMemory Computing.Asdata setsgrowandreal-timeapplicationsbecomemorepervasive,BigMemoryComputingwillbecomea requiredpartofthe ITinfrastructureformoreenterprises. Second,the software-definedmemoryvirtualizationlayerunlocksthetruepotentialofthenewPMhardware.Muchlike the server virtualization (hypervisor)layerenabledtheparallelismtotakefulladvantageofmulticoreCPUarchitectures, thememoryvirtualizationtierwilldothesameforthenewPMmedia.Thiswilldrive improvedefficienciesintermsof increased ITinfrastructureandmixedenterpriseworkloaddensity,streamliningdatacentersbyloweringenergyand floorspaceconsumptioncosts.Withsupportforenterprise-classdataservicessuchasRAID,snapshots,andreplicationfor memory-residentdatabuiltintothismemoryvirtualizationtier,ITmanagershavethetoolstoprovidethehighavailability andfastrecoveryneededformission-criticalworkloads.Inaddition,theabilitytocreateamemorylakethatsignificantly
  4. 4. Page 4#US46573120 IDC TECHNOLOGY SPOTLIGHT Big Memory Computing Emerges to Better Enable Data-Intensive IT transcendsthememorycapacityofanysingle serverandenablessharingacrossmanyserversaddressestoday'sDRAM capacitylimitations.Ultimately,BigMemoryComputingshouldsupportmemorylakeshundredsofterabytesin size, deliveringthehighconcurrencynecessaryevenasthescaleof"bigdata"growsovertime. Third,theintroductionofamemorylakewillstreamlineinfrastructure,allowingsmallerconfigurationstomeetanygiven performancegoal.FormanyofthetargetNGAworkloads,theintroductionofBigMemoryComputingwilltranscendthe needforolder-stylepersistent"performancestorage,"allowingthattiertoberemovedentirely.Themuchloweraverage latenciesdrivenbytheBigMemorytier(relativetotheolderperformancestorage)willdrivehigherCPUutilization,allowing customerstogetmoreoutofcomputeresources.ThisimprovedefficiencycantranslatetofewerCPUcores(andultimately servers)neededtomeetagivenperformancelevel,whichinturncandrivelowerapplicationsoftwarelicensingcosts. TheabilitytodelivertheadvantagesofBigMemoryComputingfortoday'sworkloadswithoutrequiringanyapplication modificationswillbecriticaltothesuccessofthisnewcategory.Givingenterprisestheabilitytoaccesstheperformance advantagesofthisnewapproachquicklyandeasilywillbe keytorapidadoption.Thevisionof BigMemoryComputingis thatcustomerscanuseindustry-standard servers,runningwidelyavailableLinuxoperatingsystems,addPMhardware and memoryvirtualization software,andthen immediatelyenjoytheperformancebenefitsofin-memorycomputing. Despitetheclearperformanceadvantagesofin-memorycomputing,wehavealreadyseenthatwhen applicationsmust bespecificallymodifiedtotake advantageofitandmain memorycapacityremainslimited,adoptionproceedsvery slowly. Sotheabilitytoquicklyandeasilyadopt in-memorycomputingforexistingworkloadswithoutmodificationwith largememorylakeswillbecriticaltorapidmarketpenetration. Considering MemVerge Foundedin2017andheadquarteredinMilpitas,California,MemVergeisastoragesoftwarecompanythatprovidesthe memoryvirtualizationlayerforBigMemoryComputing.Thevendor'sBigMemoryComputingvisionhasgarneredthesupport ofmanykeyindustryplayersthathavebecomeinvestors,includingIntel,Cisco,NetApp,andSKhynix.MemVerge'sMemory Machinesoftwareinstallsonindustry-standardIntelArchitectureserversrunningLinux,allofwhichareinterconnectedusing anInfiniBandRDMAnetwork.DRAMandPMcapacityfromallattachedserversarepooledintoamemorylakeaccessibleby allservers.TheMemoryMachineincludesenterprise-classdataservicessuchasZeroIOSnapshot,RDMA-basedreplication, andmemorytiering,givingadministratorsthetoolsneededtorunmission-criticalworkloadsinMemoryMachine environments.Anyoftoday'sworkloadscanberuninthisenvironmentwithoutrequiringanymodifications,enabling applicationstoquicklyandeasilybenefitfromtheperformanceandefficiencyofin-memoryoperation. MemVerge'simplementation,whileprovidingbackward compatibilitywithexistingapplications,offersseveral APIs. WithTransparentMemoryService,theMemoryMachineusescombinedDRAMandPMcapacityasvolatilememory, extendingmainmemorycapacitywhilemakingitoverallmoreaffordable(duetotheblended$/GBcostofDRAMand PM). Inthat case,theoperatingsystem just seestheadditionalPMcapacityasmore mainmemory. In addition, MemVergedeliversanin-memorysnapshotfeature,utilizingthepersistenceoftheunderlyingmemory,ontopofthe TransparentMemoryService.Suchsnapshotsare instantandnondisruptivetoapplications,enablingapplicationrollback, crashrecovery,and cloning.MemVergeMemoryMachinealsoincludesaSoftwareDevelopmentKit (SDK)with APIs,and if ITorganizationswanttomodifyorbuildapplicationsusingtheSDK,theycan getmoregranularcontrolofthe memory virtualizationlayerforimprovedefficiencyofoperation,havemorecomplete accesstodataservices,andpossiblyenjoy moreperformance(dependingonaworkload'sI/Oprofileandworkflows).
  5. 5. Page 5#US46573120 IDC TECHNOLOGY SPOTLIGHT Big Memory Computing Emerges to Better Enable Data-Intensive IT MemVergeistargetingseveralusecaseswithitsBigMemoryComputingplatform(seeFigure2).Thefirstusecaseisinreal- timedataanalyticsandordermanagementwherelowerlatenciestranslatedirectlytoimprovedrevenuestreams. Tradingfloorapplicationsandotherworkloadsthathavesingle-threadedsequencerscanbenefitsignificantlyfromthe muchlowerlatenciesofBigMemoryComputing(whichcanbeinthe200–400nanosecondrange).Thesecondusecaseis forin-memorydatabasecloning.MemVerge'spatentedZeroIOSnapshotenablesscalableclonecreationwithout noticeablelatencyimpactswhenthesamedatasetneedstobeusedfordifferentworkflows(suchasdev/test,analytics,or reporting),whilethepersistentbutsharedmemorypoolsupportsalmostinstantaneousrecoveryintheeventofserveror applicationcrashes.ThethirdusecaseisinAI/ML-assistedworkloadssuchasfrauddetectionorinferenceengineswhere theperformancecandropdramaticallywhentherelevantdatasetislargerthanthemainmemorycapacity.Theabilityto recoverrapidlyusingMemVerge'ssnapshottechnologyisasignificantadvantageinthisusecaseaswell. FIGURE 2: Key Early UseCases forBig Memory Computing Source: IDC, 2020 MemVergeisbenefitingfrom someverylargeandwell-knownnamesinenterprise computinginitsgo-to-market strategy.KeyinvestorsincludeCisco,Intel,NetApp,andSKhynixaswellasseveralventurecapitalists.Cisco isa major servervendorlookingtodifferentiatefromitscompetitorsbybeingfirsttomarketwithconverged infrastructure offeringsthatsupportBigMemoryComputing. IntelisafirstmoverinthePMhardware arena,NetAppalreadyoffersa PMsolutionforitsONTAP-basedenterprise storagearrays,andSKhynixisamarketleaderinsolidstatememory technologies.MemVerge'scurrent go-to-marketapproachisbasedaround a100%indirectchannelstrategy,butthe companywillbenefitsignificantlyfromthemarketpowerofthese vendorsastheyestablish marketbeachheadsthatwill introduce BigMemoryComputingtotheenterpriseinfrastructure. IntelwasthefirstvendortointroducegenerallyavailablePMproducts(whichthevendorbrandsunderthe IntelOptane Data CenterPersistentMemoryname).Optaneproductsuseanewmediatypethatthevendorjointlydevelopedwith Microncalled3DXPoint,andultimatelyMicronisexpectedtoshipPMproductsinvolumeaswell. Intelusesthe3D XPointmediainbothbyte-addressable (PM)andblock-addressable (storage-classmemoryorSCM)storagedevices, whichthevendorisalreadysellingtoday. IntelOptaneData CenterPersistentMemoryproductsplugdirectlyinto standardmemoryslotsonIntelArchitecture systemsandareaccessibleusingthebyte-addressableDDR4interface.
  6. 6. Page 6#US46573120 IDC TECHNOLOGY SPOTLIGHT Big Memory Computing Emerges to Better Enable Data-Intensive IT Today, BigMemoryComputingspecificallyusesIntel'sPMproducts (whichhavebuilt-inencryptionfordata atrest),but the vendor'sOptaneSSDs(whilesignificantlylessperformantthan Intel'sPMproducts)arethefastestblock-addressable storagedevicesavailableonthemarkettoday. IntelOptaneData CenterPersistentMemorydeliversroughlyanorderof magnitudelowerlatenciesthan Optane SSDs.GivenIntel'smarketleadershippositioninPMproductstoday,itmakes perfectsensethatthecompanywouldbeoneoftheearlyinvestorsinMemVerge. Challenges Despitetheclearmarketneedforandappealof BigMemoryComputing,thisisanewmarketthatwillrequire evangelicalworktogenerate awareness.ThefactthatMemVerge'spartnerecosystemincludesmajorplayers suchas Cisco,Intel,NetApp,andSKhynixwillhelp amplifyMemVerge'smessageevenasthe vendorprovidesthecredibilityto encourageITorganizationstoseriouslyconsideritasanear-term solution.MemVerge'sdecisiontosupporta compatibilitymodewasa verysmartonebecause itmakesitrelativelyeasyforprospectivebuyerstotryitout and validatethebenefits.ThePMhardwarenecessaryforthissolution isalsorelativelynew (thefirstIntelOptaneData CenterPersistentMemoryproductsshippedincalendar2Q19)andstillpricedabithigherthanitwillbeoncethe technologyisshippinginvolumefrom multiplevendors. Still,evenatthisearlystage,there areclearusecaseswhere BigMemoryComputingmakescompetitive andeconomicsense. Conclusion WithMemVerge'srecentMemoryMachineannouncement,wehaveenteredthe eraof BigMemoryComputing.Burgeoningrequirementsinreal-timebigdataanalyticsdrivea strongmarketneed,whilea confluenceofotherhardware and softwaretechnologies thatonlyrecentlybecameavailablemakethisaviableplatformformission-critical computingtoday.BigMemoryComputingpromisestounlockthetrueperformance potentialofthenewPersistentMemorytechnology,enablingthe creationof cost-effectivemainstream computingplatformsfor manyoftheNGAsthatfuel competitivedifferentiationfordigitallytransformedenterprises.IDChasdefinedthe BigMemoryComputingcategoryinthispaperandexpectsthatwithintwo yearsthere willbea vibrantmultivendor communityofferingthesetypesofplatforms. Today,MemVergeistheplayerthat isalreadyofferingthesesolutions.Cofounder CharlesFan(whowasthegeneralmanagerofVMware'svSANbusiness)haspulledtogetheracompellingecosystemof bellwetherpartnerstofuelMemVerge'sgo-to-marketefforts,andthatecosystemisexpectedtodrivemarquee end userstoseriouslyevaluateBigMemoryComputingastheplatformofchoiceformission-criticalreal-timeanalytics workloads.IDCbelievesthat BigMemoryComputingaddressesanimportant marketneed,thatthenecessary technologyenablersareinplace,andthatenterpriseswithreal-timebigdataanalyticsandotherin-memorytype workloadsshouldtakeaseriouslookatMemVerge'sMemoryMachineplatform.Thenecessaryhardwareproducts and memoryvirtualizationfeaturesaretheretoday,andenterpriseslookingformemory-classperformancefor mission-criticalworkloadsshouldtakenote. Big Memory Computing promises to unlock the true performance potential of the new Persistent Memory technology.
  7. 7. Page 7#US46573120 IDC TECHNOLOGY SPOTLIGHT Big Memory Computing Emerges to Better Enable Data-Intensive IT About the Analyst EricBurgener,Research Vice President, Infrastructure Systems, Platforms, and Technologies EricBurgenerisResearchVicePresidentwithinIDC'sEnterpriseInfrastructurepractice.Mr.Burgener'score researchcoverageincludesstoragesystems,softwareandsolutions,quarterlytrackers,andend-user researchaswellasadvisoryservicesandconsultingprograms.Basedonhisbackgroundcoveringenterprise storage,Mr.Burgener'sresearchincludesaparticularemphasisonsolidstatetechnologiesinenterprise storagesystemsaswellassoftware-definedinfrastructure.HewasawardedtheAlexanderMotsenigos MemorialAwardforOutstandingInnovationinMarketResearchin2017byIDC,wasrecognizedasoneof theArchitectAnalystPower100in2019byindependentresearcherARInsights,andisanactiveparticipant intheITBuyer'sResearchProgramatIDC. The content in this paper was adapted from existing IDC research published on www.idc.com. IDC Research,Inc. 5 Speen Street Framingham, MA 01701, USA T 508.872.8200 F 508.935.4015 Twitter @IDC idc-insights-community.com www.idc.com This publication was produced by IDC Custom Solutions. The opinion, analysis, and research results presented herein are drawn from more detailed research and analysis independently conducted and published by IDC, unless specific vendor sponsorship is noted. IDC Custom Solutions makes IDC content available in a wide range of formats for distribution by various companies. A license to distribute IDC content does not imply endorsement of or opinion about the licensee. External Publication of IDC Information and Data — Any IDC information that is to be used in advertising, press releases, or promotional materials requires prior written approval from the appropriate IDC Vice President or Country Manager. A draft of the proposed document should accompany any such request. IDC reserves the right to deny approval of external usage for any reason. Copyright 2020 IDC. Reproduction without written permission is completely forbidden.

×