NVDIMM block drivers with NFIT pictures: http://suselabtw.blogspot.com/2016/03/nvdimm-block-drivers-with-nfit.html

1. NVDIMM block drivers with
NFIT
March, 2016, SUSE Labs Taipei technology sharing day,
Taipei
Joey Lee
SUSE Labs Taipei

2. 2
Agenda
• NVDIMM
• NFIT
• Conceptual model
‒ nvdimm.txt
‒ Namespace/Region
‒ PMEM
‒ BLK
‒ NFIT
‒ BTT
• Q&A

3. NVDIMM

4. 4
NVDIMM
• NVDIMM: Non-Volatile DIMM. Non-volatile memory
in a DIMM form factor. [1]
http://www.enterprisetech.com/2014/02/20/micron-pushes-memory-roadm
ap-several-routes/

5. 5
NVDIMM Architecture

6. 6
NVDIMM Architecture - Block

7. 7
SPA/DPA
• SPA: System Physical Address. A physical address
on the host operating system. [1]
• DPA: DIMM Physical Address. An address within the
memory in an NVDIMM. [1]

8. 8
SPA/DPA
NVDIMM Namespace Specification, Intel

9. 9
SPA/DPA Interleave
• DPA: DIMM Physical Address, is a DIMM-relative
offset. With one DIMM in the system there would be
a 1:1 system-physical-address:DPA association.
Once more DIMMs are added an memory controller
interleave must be decoded to determine the DPA
associated with a given system-physical-address.
BLK capacity always has a 1:1 relationship with a
single-dimm's DPA range. [2]

10. 10
SPA/DPA Interleave
NVDIMM Namespace Specification, Intel

11. 11
PMEM
• Persistent Memory (or PMEM): Byte‐addressable
memory which retains its contents across power
loss.A DIMM primarily containing non‐volatile
memory. [3]
• PMEM: A system physical address range where
writes are persistent. A block device composed of
PMEM is capable of DAX. A PMEM address range
may span/interleave several DIMMs. [2]
• DAX: File system extensions to bypass the page
cache and block layer to mmap persistent memory,
from a PMEM block device, directly into a process
address space. [2]

12. 12
Block Window
• BW(Block Window): A set of registers consisting of a
command register, a status register, and an aperture
allowing the NVDIMM driver to read and write blocks
of data to any persistent area on an NVDIMM. [1]
• BLK: A set of one or more programmable memory
mapped apertures provided by a DIMM to access its
media. This indirection precludes the performance
benefit of interleaving, but enables DIMM-bounded
failure modes. [2]

13. 13
Block Window

14. 14
BTT
• Block Translation Table. A software data structure,
defined in the NVDIMM Namespace Specification,
which prevents torn blocks when a write is
interrupted by a system crash, hang, or power
failure. [1]
• BTT: Existing software may have an expectation that
the power-fail-atomicity of writes is at least one
sector, 512 bytes. The BTT is an indirection table
with atomic update semantics to front a PMEM/BLK
block device driver and present arbitrary atomic
sector sizes. [2]

15. 15
Namespace
• NVDIMM Namespace: Similar to an NVMe
Namespace or a Logical Unit (LUN) on a SCSI disk,
this is a software mechanism for managing ranges
of persistence on NVDIMMs. [1]
• NVDIMM Namespace Label: Labels, stored at a
known location on NVDIMMs, that define the
DIMM’s contribution to NVDIMM Namespaces. This
is a software mechanism; the DIMM itself just sees
the labels as part of the overall data stored on the
DIMM. [1]

16. 16
PMEM Namespace
NVDIMM Namespace Specification, Intel

17. 17
BLK Namespace
NVDIMM Namespace Specification, Intel

18. 18
Namespace
NVDIMM Namespace Specification, Intel

19. 19
I/O on Multi-Range BLK Namspaces
NVDIMM Namespace Specification, Intel

20. 20
Example Software Organization for
Block I/O
NVDIMM Namespace Specification, Intel

21. 21
Example Software Organization for
Namespace Management
NVDIMM Namespace Specification, Intel

22. NFIT

23. 23
NFIT
• NFIT: The NVDIMM Firmware Interface Table, which
defines the ACPI-like information created by the
BIOS to inform the OS about NVDIMMs in the
system. [1]
• Device Specific Method (DSM): The NVDIMM root
device and the NVDIMM devices can have device
specific methods (_DSM) to provide additional
functions specific to a particular NVDIMM
implementation. [4]

24. 24
NFIT Overview
ACPI 6.0 spec

25. 25
NFIT
NVDIMM Block Window Driver Writer’s Guide, Intel

26. 26
NFIT Information
NVDIMM Block Window Driver Writer’s Guide, Intel

27. 27
NFIT SPA
NVDIMM Block Window Driver Writer’s Guide, Intel

28. 28
NFIT Flush Hint
NVDIMM Block Window Driver Writer’s Guide, Intel

29. 29
NFIT Block Window
NVDIMM Block Window Driver Writer’s Guide, Intel

30. 30
NFIT (fixed)
NVDIMM Block Window Driver Writer’s Guide, Intel

31. Conceptual model

32. 32
Conceptual model of nvdimm.txt

33. 33
Conceptual model of
driver/nvdimm/namsepace, region

34. 34
Conceptual model of
driver/nvdimm/pmem

35. 35
Conceptual model of
driver/nvdimm/blk

36. 36
Conceptual model of
driver/nvdimm/btt

37. 37
Conceptual model of driver/acpi/nfit

38. Q&A

39. 39
Reference
• [1] NVDIMM Block Window Driver Writer’s Guide,
Intel
• [2] Documentation/nvdimm/nvdimm.txt, Linux Kernel
• [3] NVDIMM Namespace Specification, Intel
• [4] NVDIMM DSM Interface Example, Intel

40. Thank you.
40
Feedback to
jlee@suse.com

42. Corporate Headquarters
Maxfeldstrasse 5
90409 Nuremberg
Germany
+49 911 740 53 0 (Worldwide)
www.suse.com
Join us on:
www.opensuse.org
42

43. Unpublished Work of SUSE. All Rights Reserved.
This work is an unpublished work and contains confidential, proprietary and trade secret information of SUSE.
Access to this work is restricted to SUSE employees who have a need to know to perform tasks within the scope of
their assignments. No part of this work may be practiced, performed, copied, distributed, revised, modified, translated,
abridged, condensed, expanded, collected, or adapted without the prior written consent of SUSE.
Any use or exploitation of this work without authorization could subject the perpetrator to criminal and civil liability.
General Disclaimer
This document is not to be construed as a promise by any participating company to develop, deliver, or market a
product. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making
purchasing decisions. SUSE makes no representations or warranties with respect to the contents of this document,
and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. The
development, release, and timing of features or functionality described for SUSE products remains at the sole
discretion of SUSE. Further, SUSE reserves the right to revise this document and to make changes to its content, at
any time, without obligation to notify any person or entity of such revisions or changes. All SUSE marks referenced in
this presentation are trademarks or registered trademarks of Novell, Inc. in the United States and other countries. All
third-party trademarks are the property of their respective owners.