Go的内存管理
go runtime的基本内存模型
go runtime借鉴了C++的内存模型和TCMalloc内存分片机制。
tcmalloc (Thread-Caching Malloc) 是 Google 开发的内存分配器.
先了解一些go runtime堆内存抽象的名词
- page：虚拟内存大小，将进程虚拟内存空间划分为多份同等大小的Page
- mspan：内存管理单元，多个page组成一个mSpan，包含起始Page编号Start，连续多少个Page数量Length。mSpan的数据结构为双向链表。
//go:notinheap
type mspan struct {
next *mspan // next span in list, or nil if none
prev *mspan // previous span in list, or nil if none
list *mSpanList // For debugging. TODO: Remove.
startAddr uintptr // address of first byte of span aka s.base()
npages uintptr // number of pages in span
manualFreeList gclinkptr // list of free objects in mSpanManual spans
...其中Page和mSpan的关系
- Size class：同一类别大小内存集合（集不同Span）称为Size Class，Size Class集合类似一个刻度列表，例如8字节，16字节，32字节。在申请内存时，会找到与申请内存向上最匹配的那个Size Class内存块返回给内存申请调用方。Go runtime根据刻度大小共分为67个SizeClass。https://github.com/golang/go/blob/master/src/runtime/sizeclasses.go
- ObjectSize：go routine一次向go进程内存申请的对象Object大小。Object是go runtime对内存更细化的管理单元，一个mSpan将被拆分为多个object，每一个mSpan会有一个属性Number of Object，表示持有Object的数量。例如mSpan大小为8KB，ObjectSize为8B，Number of Object为1024；ObjectSize与Size Class也有对应关系，例如1-8B的Object的Size Class为1,8-16B的Object size为2.
这里着重比对下Object和Page。Object是go内存管理内部对象存储内存的基本单元；Page则是go内存管理与操作系统内存交互时衡量容量的基本单元（即向操作系统申请内存时的内存管理单元）。 - Span Class：go runtime针对span特性定义的相关属性，一个Size Class对应的span中可能包含需要GC扫描的对象（这部分对象包含指针），和不需要GC扫描的对象（这部分对象不包含指针）。根据这两部分对象的特性，则可以表达出，一个Size Class对应的两个Span Class（一个需要GC扫描的span对象列表，一个无需GC扫描的span对象列表）https://github.com/golang/go/blob/fba83cdfc6c4818af5b773afa39e457d16a6db7a/src/runtime/mheap.go#L583
// A spanClass represents the size class and noscan-ness of a span.
//
// Each size class has a noscan spanClass and a scan spanClass. The
// noscan spanClass contains only noscan objects, which do not contain
// pointers and thus do not need to be scanned by the garbage
// collector.
type spanClass uint8
const (
numSpanClasses = _NumSizeClasses << 1
tinySpanClass = spanClass(tinySizeClass<<1 | 1)
)
func makeSpanClass(sizeclass uint8, noscan bool) spanClass {
return spanClass(sizeclass<<1) | spanClass(bool2int(noscan))
}
go runtime的三级存储体系（MCache MCentral Mheap）
go runtime 通过MCache、MCentral，Mheap这三个对象完成对进程虚拟内存的抽象和管理。MCache对应TCMalloc中的Thread Cache，因为是线程/协程独享，所以对它使用时无需加锁。MCentral为go runtime全局缓存，当MCache空间不足时，会想MCentral申请新的对象；最后是MHeap，该对象是对进程堆空间的抽象，上游是MCentral，下游是操作系统，当MCentral空间不足时，会向MHeap申请内存，MHeap空间不足时，会直接向操作系统申请内存扩容，同时，go runtime针对一些大对象的内存分配也会直接分配在这块区域。
MCache将直接与go runtime的调度模型角色“P”绑定，P向MCache申请内存的基本单元便是Object，MCache与MCentral交换内存的基本单元是Span，MCentral与MHeap交换内存的基本单元为Page。下面针对这三者一一展开描述。
MCache
根据如下代码，可得出P在初始化时，会初始化自己专属的MCache空间。MCache 与P 绑定，因为在实际进程运行中，某个时刻一个P只会跟一个M（线程）绑定，因此该部分内存使用时无需加锁，使用MCache分配内存这无疑加快了内存的分配速度。它们之间的关系（如图3）所示
func (pp *p) init(id int32) {
pp.id = id
pp.status = _Pgcstop
pp.sudogcache = pp.sudogbuf[:0]
pp.deferpool = pp.deferpoolbuf[:0]
pp.wbBuf.reset()
if pp.mcache == nil {
if id == 0 {
if mcache0 == nil {
throw("missing mcache?")
}
// Use the bootstrap mcache0. Only one P will get
// mcache0: the one with ID 0.
pp.mcache = mcache0
} else {
pp.mcache = allocmcache()
}
}
...func allocmcache() *mcache {
var c *mcache
systemstack(func() {
lock(&mheap_.lock)
c = (*mcache)(mheap_.cachealloc.alloc())
c.flushGen = mheap_.sweepgen
unlock(&mheap_.lock)
})
for i := range c.alloc {
c.alloc[i] = &emptymspan
}
c.nextSample = nextSample()
return c
}
接下来看下MCache的数据结构定义，“alloc”属性定义了MCache上可以分配的对象，MCache上每一个Span Class都对应了一个mspan对象，根据Span Class的跨度不同，mspan的长度则不同。拿Span Class为4（见图2）的mspan举例，内存大小为8KB，对外提供的每一个Object大小为16B，所以共存在512个Obejct。当goroutine向MCache申请内存时，当与申请内存匹配的Span Class的MSpan没有可提供的Object时候，就会从MCentral申请新的存储空间，当然，这一步需要加锁处理。
type mcache struct {
// The following members are accessed on every malloc,
// so they are grouped here for better caching.
nextSample uintptr // trigger heap sample after allocating this many bytes
scanAlloc uintptr // bytes of scannable heap allocated
// Allocator cache for tiny objects w/o pointers.
// See "Tiny allocator" comment in malloc.go.
// tiny points to the beginning of the current tiny block, or
// nil if there is no current tiny block.
//
// tiny is a heap pointer. Since mcache is in non-GC'd memory,
// we handle it by clearing it in releaseAll during mark
// termination.
//
// tinyAllocs is the number of tiny allocations performed
// by the P that owns this mcache.
tiny uintptr
tinyoffset uintptr
tinyAllocs uintptr
// The rest is not accessed on every malloc.
alloc [numSpanClasses]*mspan // spans to allocate from, indexed by spanClass
stackcache [_NumStackOrders]stackfreelist
// flushGen indicates the sweepgen during which this mcache
// was last flushed. If flushGen != mheap_.sweepgen, the spans
// in this mcache are stale and need to the flushed so they
// can be swept. This is done in acquirep.
flushGen uint32
}
type spanClass uint8const ( numSpanClasses = _NumSizeClasses << 1 tinySpanClass = spanClass(tinySizeClass<<1 | 1))根据（图2）可以看到，Span Class为0和1刻度的内存，MCache并没有分配任何内存（Object Size）也为0，因为go runtime针对内存为0的数据做了特殊处理。针对go routine申请内存大小为0的对象时，go runtime 直接返回一个固定内存地址（例如struct{}），这么做的好处显而易见，可以节省部分内存空间。go runtime代码如下:
func mallocgc(size uintptr, typ *_type, needzero bool) unsafe.Pointer {
...
// Short-circuit zero-sized allocation requests.
if size == 0 {
return unsafe.Pointer(&zerobase)
}
...
}
MCentral
MCentral代码如下所示，可以看到，MCentral核心的属性为“partial”和“full”，它们分别为spanSet，表示msapn的集合。
- Partial Span Set 表示的是还有可用空间的Span集合，这个集合中的所有Span元素都至少还有一个可用的Object空间；且当MCache向MCentral申请空间时，从该集合返回mSpan，在使用完退还时，也会将退换的Span加入Partial mSpan Set List中。
- Full Span Set，表示的是没有可用空间的Span链表，该集合上的Span都不确定是否还有空间的Object空间。且当MCache向MCentral申请空间时，从该集合返回Span，在使用完退还时，也会将退换的Span加入Full Span Set中。
当MCache的某个Size Class对应的mSpan，被goroutine以申请object形式一次次取走，下次再申请时MCache不能返回对应object，此时MCache会向MCentral申请一个对应的mSpan。MCentral设计为两个mspan Set集合的原因是为了更高效的管理和区分内存块的状态，以及优化GC的分配，回收效率。其中partial 集合表示mSpan只有部分被使用，当它所有对象都被分配完后，该mSpan会被移动至full集合。这样，MCentral可以优先从partial中分配内存，避免造成内存碎片。partial和full都有两个spanSet集合，分别代表GC“已经清扫” 和“尚未清扫”的mspan，在每一个GC周期中，一方面可以高效的管理mspan状态，确保未被使用的内存可以及时回收，而正在使用或者已被分配完的内存不会被过度清扫，另一方面垃圾回收器和内存分配器并行执行的情况下，通过将已清扫和为清扫的mspan分开管理，可以避免mspan在分配的同时也被清扫，减少竞态条件的发生。
type mcentral struct {
spanclass spanClass
// partial and full contain two mspan sets: one of swept in-use
// spans, and one of unswept in-use spans. These two trade
// roles on each GC cycle. The unswept set is drained either by
// allocation or by the background sweeper in every GC cycle,
// so only two roles are necessary.
//
// sweepgen is increased by 2 on each GC cycle, so the swept
// spans are in partial[sweepgen/2%2] and the unswept spans are in
// partial[1-sweepgen/2%2]. Sweeping pops spans from the
// unswept set and pushes spans that are still in-use on the
// swept set. Likewise, allocating an in-use span pushes it
// on the swept set.
//
// Some parts of the sweeper can sweep arbitrary spans, and hence
// can't remove them from the unswept set, but will add the span
// to the appropriate swept list. As a result, the parts of the
// sweeper and mcentral that do consume from the unswept list may
// encounter swept spans, and these should be ignored.
partial [2]spanSet // list of spans with a free object
full [2]spanSet // list of spans with no free objects
}
// A spanSet is a set of *mspans.
//
// spanSet is safe for concurrent push and pop operations.
type spanSet struct {
// A spanSet is a two-level data structure consisting of a
// growable spine that points to fixed-sized blocks. The spine
// can be accessed without locks, but adding a block or
// growing it requires taking the spine lock.
//
// Because each mspan covers at least 8K of heap and takes at
// most 8 bytes in the spanSet, the growth of the spine is
// quite limited.
//
// The spine and all blocks are allocated off-heap, which
// allows this to be used in the memory manager and avoids the
// need for write barriers on all of these. spanSetBlocks are
// managed in a pool, though never freed back to the operating
// system. We never release spine memory because there could be
// concurrent lock-free access and we're likely to reuse it
// anyway. (In principle, we could do this during STW.)
spineLock mutex
spine unsafe.Pointer // *[N]*spanSetBlock, accessed atomically
spineLen uintptr // Spine array length, accessed atomically
spineCap uintptr // Spine array cap, accessed under lock
MHeap
MHeap是对操作系统进程堆对象的抽象，上游是MCentral，当MCentral中的Span对象不足时，会找MHeap申请（此过程也会加锁保证线程安全）；下游是操作系统，当MHeap内存不够时，会向操作系统申请虚拟内存空间。
MHeap管理的内存基本单元为HeapArena，HeapArena是对多个"Page"的抽象，在Linux 64位的操作系统上，每一个HeapArena占用64MB的内存，每一个HeapArena包含多个Page,每个Page由不同的mSpan来管理。MHeap中，根据“arens”属性对HeapArena进行定义和管理。从下图mheap定义中，还有几个关键属性需要关注
central属性：根据该属性可以发现，MCentral为MHeap的一部分。
arenaHints属性：用于指示为Arena分配更多地址时，在堆的什么地址分配。
https://github.com/golang/go/blob/bdcd6d1b653dd7a5b3eb9a053623f85433ff9e6b/src/runtime/mheap.go#L148
type mheap struct {
...
// bitmap stores the pointer/scalar bitmap for the words in // this arena. See mbitmap.go for a description. Use the // heapBits type to access this. bitmap [heapArenaBitmapBytes]byte
// arenas is the heap arena map. It points to the metadata for // the heap for every arena frame of the entire usable virtual ... arenas [1 << arenaL1Bits]*[1 << arenaL2Bits]*heapArena ...
// arenaHints is a list of addresses at which to attempt to // add more heap arenas. This is initially populated with a // set of general hint addresses, and grown with the bounds of // actual heap arena ranges. arenaHints *arenaHint
...
// central free lists for small size classes.
// the padding makes sure that the mcentrals are
// spaced CacheLinePadSize bytes apart, so that each mcentral.lock
// gets its own cache line.
// central is indexed by spanClass.
central [numSpanClasses]struct {
mcentral mcentral
pad [cpu.CacheLinePadSize - unsafe.Sizeof(mcentral{})%cpu.CacheLinePadSize]byte
}
...
}接下来是HeapArena，它是由多个Page组成的的抽象管理单元，HeapArea的bitmap属性，表示这个HeapArena的内存使用情况，即当前标记对应的地址中是否存在对象以及是否被GC标记过。
// A heapArena stores metadata for a heap arena. heapArenas are stored
// outside of the Go heap and accessed via the mheap_.arenas index.
//
//go:notinheap
type heapArena struct {
// bitmap stores the pointer/scalar bitmap for the words in
// this arena. See mbitmap.go for a description. Use the
// heapBits type to access this.
bitmap [heapArenaBitmapBytes]byte
// spans maps from virtual address page ID within this arena to *mspan.
// For allocated spans, their pages map to the span itself.
// For free spans, only the lowest and highest pages map to the span itself.
// Internal pages map to an arbitrary span.
// For pages that have never been allocated, spans entries are nil.
//
// Modifications are protected by mheap.lock. Reads can be
// performed without locking, but ONLY from indexes that are
// known to contain in-use or stack spans. This means there
// must not be a safe-point between establishing that an
// address is live and looking it up in the spans array.
spans [pagesPerArena]*mspan
...
}
