PostgreSQL中GetSnapshotData的处理过程是什么
这篇文章主要讲解了“PostgreSQL中GetSnapshotData的处理过程是什么”,文中的讲解内容简单清晰,易于学习与理解,下面请大家跟着小编的思路慢慢深入,一起来研究和学习“PostgreSQL中GetSnapshotData的处理过程是什么”吧!
一、数据结构全局/静态变量
/**CurrentlyregisteredSnapshots.Orderedinaheapbyxmin,sothatwecan*quicklyfindtheonewithlowestxmin,toadvanceourMyPgXact->xmin.*当前已注册的快照.*按照xmin堆排序,这样我们可以快速找到xmin最小的一个,从而可以设置MyPgXact->xmin。*/staticintxmin_cmp(constpairingheap_node*a,constpairingheap_node*b,void*arg);staticpairingheapRegisteredSnapshots={&xmin_cmp,NULL,NULL};/*firstGetTransactionSnapshotcallinatransaction?*/boolFirstSnapshotSet=false;/**Remembertheserializabletransactionsnapshot,ifany.Wecannottrust*FirstSnapshotSetincombinationwithIsolationUsesXactSnapshot(),because*GUCmayberesetbeforeus,changingthevalueofIsolationUsesXactSnapshot.*如存在则记下serializable事务快照.*我们不能信任与IsolationUsesXactSnapshot()结合使用的FirstSnapshotSet,*因为GUC可能会在我们之前重置,改变IsolationUsesXactSnapshot的值。*/staticSnapshotFirstXactSnapshot=NULL;/**CurrentSnapshotpointstotheonlysnapshottakenintransaction-snapshot*mode,andtothelatestonetakeninaread-committedtransaction.*SecondarySnapshotisasnapshotthat'salwaysup-to-dateasofthecurrent*instant,evenintransaction-snapshotmode.Itshouldonlybeusedfor*special-purposecode(say,RIchecking.)CatalogSnapshotpointstoan*MVCCsnapshotintendedtobeusedforcatalogscans;wemustinvalidateit*wheneverasystemcatalogchangeoccurs.*CurrentSnapshot指向在transaction-snapshot模式下获取的唯一快照/在read-committed事务中获取的最新快照。*SecondarySnapshot是即使在transaction-snapshot模式下,也总是最新的快照。它应该只用于特殊用途码(例如,RI检查)。*CatalogSnapshot指向打算用于catalog扫描的MVCC快照;*无论何时发生systemcatalog更改,我们都必须马上使其失效。**TheseSnapshotDatastructsarestatictosimplifymemoryallocation*(seethehackinGetSnapshotDatatoavoidrepeatedmalloc/free).*这些SnapshotData结构体是静态的便于简化内存分配.*(可以回过头来看GetSnapshotData函数如何避免重复的malloc/free)*/staticSnapshotDataCurrentSnapshotData={HeapTupleSatisfiesMVCC};staticSnapshotDataSecondarySnapshotData={HeapTupleSatisfiesMVCC};SnapshotDataCatalogSnapshotData={HeapTupleSatisfiesMVCC};/*Pointerstovalidsnapshots*///指向有效的快照staticSnapshotCurrentSnapshot=NULL;staticSnapshotSecondarySnapshot=NULL;staticSnapshotCatalogSnapshot=NULL;staticSnapshotHistoricSnapshot=NULL;/**TheseareupdatedbyGetSnapshotData.Weinitializethemthisway*fortheconvenienceofTransactionIdIsInProgress:eveninbootstrap*mode,wedon'twantittosaythatBootstrapTransactionIdisinprogress.*这些变量通过函数GetSnapshotData更新.*为了便于TransactionIdIsInProgress,以这种方式初始化它们:*即使在引导模式下,我们也不希望表示BootstrapTransactionId正在进行中。**RecentGlobalXminandRecentGlobalDataXminareinitializedto*InvalidTransactionId,toensurethatnoonetriestouseastale*value.Readersshouldensurethatithasbeensettosomethingelse*beforeusingit.*RecentGlobalXmin和RecentGlobalDataXmin初始化为InvalidTransactionId,*以确保没有人尝试使用过时的值。*在使用它之前,读取进程应确保它已经被设置为其他值。*/TransactionIdTransactionXmin=FirstNormalTransactionId;TransactionIdRecentXmin=FirstNormalTransactionId;TransactionIdRecentGlobalXmin=InvalidTransactionId;TransactionIdRecentGlobalDataXmin=InvalidTransactionId;/*(table,ctid)=>(cmin,cmax)mappingduringtimetravel*/staticHTAB*tuplecid_data=NULL;
MyPgXact
当前的事务信息.
/**FlagsforPGXACT->vacuumFlags*PGXACT->vacuumFlags标记**Note:Ifyoumodifytheseflags,youneedtomodifyPROCARRAY_XXXflags*insrc/include/storage/procarray.h.*注意:如果修改了这些标记,需要更新src/include/storage/procarray.h中的PROCARRAY_XXX标记**PROC_RESERVEDmaylaterbeassignedforuseinvacuumFlags,butitsvalueis*usedforPROCARRAY_SLOTS_XMINinprocarray.h,soGetOldestXminwon'tbeable*tomatchandignoreprocesseswiththisflagset.*PROC_RESERVED可能在接下来分配给vacuumFlags使用,*但是它在procarray.h中用于标识PROCARRAY_SLOTS_XMIN,*因此GetOldestXmin不能匹配和忽略使用此标记的进程.*///是否autovacuumworker?#definePROC_IS_AUTOVACUUM0x01/*isitanautovacworker?*///正在运行lazyvacuum#definePROC_IN_VACUUM0x02/*currentlyrunninglazyvacuum*///正在运行analyze#definePROC_IN_ANALYZE0x04/*currentlyrunninganalyze*///只能通过autovacuum设置#definePROC_VACUUM_FOR_WRAPAROUND0x08/*setbyautovaconly*///在事务外部正在执行逻辑解码#definePROC_IN_LOGICAL_DECODING0x10/*currentlydoinglogical*decodingoutsidexact*///保留用于procarray#definePROC_RESERVED0x20/*reservedforprocarray*//*flagsresetatEOXact*///在EOXact时用于重置标记的MASK#definePROC_VACUUM_STATE_MASK\(PROC_IN_VACUUM|PROC_IN_ANALYZE|PROC_VACUUM_FOR_WRAPAROUND)/**PriortoPostgreSQL9.2,thefieldsbelowwerestoredaspartofthe*PGPROC.However,benchmarkingrevealedthatpackingtheseparticular*membersintoaseparatearrayastightlyaspossiblespedupGetSnapshotData*considerablyonsystemswithmanyCPUcores,byreducingthenumberof*cachelinesneedingtobefetched.Thus,thinkverycarefullybeforeadding*anythingelsehere.*/typedefstructPGXACT{//当前的顶层事务ID(非子事务)//出于优化的目的,只读事务并不会分配事务号(xid=0)TransactionIdxid;/*idoftop-leveltransactioncurrentlybeing*executedbythisproc,ifrunningandXID*isassigned;elseInvalidTransactionId*///在启动事务时,当前正在执行的最小事务号XID,但不包括LAZYVACUUM//vacuum不能清除删除事务号xid>=xmin的元组TransactionIdxmin;/*minimalrunningXIDasitwaswhenwewere*startingourxact,excludingLAZYVACUUM:*vacuummustnotremovetuplesdeletedby*xid>=xmin!*///vacuum相关的标记uint8vacuumFlags;/*vacuum-relatedflags,seeabove*/booloverflowed;booldelayChkpt;/*trueifthisprocdelayscheckpointstart;*previouslycalledInCommit*/uint8nxids;}PGXACT;externPGDLLIMPORTstructPGXACT*MyPgXact;
Snapshot
SnapshotData结构体指针,SnapshotData结构体可表达的信息囊括了所有可能的快照.
有以下几种不同类型的快照:
1.常规的MVCC快照
2.在恢复期间的MVCC快照(处于Hot-Standby模式)
3.在逻辑解码过程中使用的历史MVCC快照
4.作为参数传递给HeapTupleSatisfiesDirty()函数的快照
5.作为参数传递给HeapTupleSatisfiesNonVacuumable()函数的快照
6.用于在没有成员访问情况下SatisfiesAny、Toast和Self的快照
//SnapshotData结构体指针typedefstructSnapshotData*Snapshot;//无效的快照#defineInvalidSnapshot((Snapshot)NULL)/**WeuseSnapshotDatastructurestorepresentboth"regular"(MVCC)*snapshotsand"special"snapshotsthathavenon-MVCCsemantics.*Thespecificsemanticsofasnapshotareencodedbythe"satisfies"*function.*我们使用SnapshotData结构体表示"regular"(MVCC)snapshots和具有非MVCC语义的"special"snapshots。*///测试函数typedefbool(*SnapshotSatisfiesFunc)(HeapTuplehtup,Snapshotsnapshot,Bufferbuffer);//常见的有://HeapTupleSatisfiesMVCC:判断元组对某一快照版本是否有效//HeapTupleSatisfiesUpdate:判断元组是否可更新(同时更新同一个元组)//HeapTupleSatisfiesDirty:判断当前元组是否存在脏数据//HeapTupleSatisfiesSelf:判断tuple对自身信息是否有效//HeapTupleSatisfiesToast:判断是否TOAST表//HeapTupleSatisfiesVacuum:判断元组是否能被VACUUM删除//HeapTupleSatisfiesAny:所有元组都可见//HeapTupleSatisfiesHistoricMVCC:用于CATALOG表/**Structrepresentingallkindofpossiblesnapshots.*该结构体可表达的信息囊括了所有可能的快照.**Thereareseveraldifferentkindsofsnapshots:**NormalMVCCsnapshots**MVCCsnapshotstakenduringrecovery(inHot-Standbymode)**HistoricMVCCsnapshotsusedduringlogicaldecoding**snapshotspassedtoHeapTupleSatisfiesDirty()**snapshotspassedtoHeapTupleSatisfiesNonVacuumable()**snapshotsusedforSatisfiesAny,Toast,Selfwherenomembersare*accessed.*有以下几种不同类型的快照:**常规的MVCC快照**在恢复期间的MVCC快照(处于Hot-Standby模式)**在逻辑解码过程中使用的历史MVCC快照**作为参数传递给HeapTupleSatisfiesDirty()函数的快照**作为参数传递给HeapTupleSatisfiesNonVacuumable()函数的快照**用于在没有成员访问情况下SatisfiesAny、Toast和Self的快照**TODO:It'sprobablyagoodideatosplitthisstructusingaNodeTag*similartohowparserandexecutornodesarehandled,withonetypefor*eachdifferentkindofsnapshottoavoidoverloadingthemeaningof*individualfields.*TODO:使用类似于parser/executornodes的处理,使用NodeTag来拆分结构体会是一个好的做法,*使用OO(面向对象继承)的方法.*/typedefstructSnapshotData{//测试tuple是否可见的函数SnapshotSatisfiesFuncsatisfies;/*tupletestfunction*//**TheremainingfieldsareusedonlyforMVCCsnapshots,andarenormally*justzeroesinspecialsnapshots.(Butxminandxmaxareused*speciallybyHeapTupleSatisfiesDirty,andxminisusedspeciallyby*HeapTupleSatisfiesNonVacuumable.)*余下的字段仅用于MVCC快照,在特殊快照中通常为0。*(xmin和xmax可用于HeapTupleSatisfiesDirty,xmin可用于HeapTupleSatisfiesNonVacuumable)**AnMVCCsnapshotcanneverseetheeffectsofXIDs>=xmax.Itcansee*theeffectsofallolderXIDsexceptthoselistedinthesnapshot.xmin*isstoredasanoptimizationtoavoidneedingtosearchtheXIDarrays*formosttuples.*XIDs>=xmax的事务,对该快照是不可见的(没有任何影响).*对该快照可见的是小于xmax,但不在snapshot列表中的XIDs.*记录xmin是出于优化的目的,避免为大多数tuples搜索XID数组.*///XID∈[2,min)是可见的TransactionIdxmin;/*allXID<xminarevisibletome*///XID∈[xmax,∞)是不可见的TransactionIdxmax;/*allXID>=xmaxareinvisibletome*//**FornormalMVCCsnapshotthiscontainstheallxactIDsthatarein*progress,unlessthesnapshotwastakenduringrecoveryinwhichcase*it'sempty.ForhistoricMVCCsnapshots,themeaningisinverted,i.e.*itcontains*committed*transactionsbetweenxminandxmax.*对于普通的MVCC快照,xip存储了所有正在进行中的XIDs,除非在恢复期间产生的快照(这时候数组为空)*对于历史MVCC快照,意义相反,即它包含xmin和xmax之间的*已提交*事务。**note:allidsinxip[]satisfyxmin<=xip[i]<xmax*注意:所有在xip数组中的XIDs满足xmin<=xip[i]<xmax*/TransactionId*xip;//xip数组中的元素个数uint32xcnt;/*#ofxactidsinxip[]*//**Fornon-historicMVCCsnapshots,thiscontainssubxactIDsthatarein*progress(andothertransactionsthatareinprogressiftakenduring*recovery).Forhistoricsnapshotitcontains*all*xidsassignedtothe*replayedtransaction,includingthetoplevelxid.*对于非历史MVCC快照,下面这些域含有活动的subxactIDs.*(以及在恢复过程中状态为进行中的事务).*对于历史MVCC快照,这些域字段含有*所有*用于回放事务的快照,包括顶层事务XIDs.**note:allidsinsubxip[]are>=xmin,butwedon'tbotherfiltering*outanythatare>=xmax*注意:sbuxip数组中的元素均≥xmin,但我们不需要过滤掉任何>=xmax的项*/TransactionId*subxip;//subxip数组元素个数int32subxcnt;/*#ofxactidsinsubxip[]*///是否溢出?boolsuboverflowed;/*hasthesubxiparrayoverflowed?*///在Recovery期间的快照?booltakenDuringRecovery;/*recovery-shapedsnapshot?*///如为静态快照,则该值为Fboolcopied;/*falseifit'sastaticsnapshot*///在自身的事务中,CID<curcid是可见的CommandIdcurcid;/*inmyxact,CID<curcidarevisible*//**AnextrareturnvalueforHeapTupleSatisfiesDirty,notusedinMVCC*snapshots.*HeapTupleSatisfiesDirty返回的值,在MVCC快照中无用*/uint32speculativeToken;/**Book-keepinginformation,usedbythesnapshotmanager*用于快照管理器的Book-keeping信息*///在ActiveSnapshot栈中的引用计数uint32active_count;/*refcountonActiveSnapshotstack*///在RegisteredSnapshots中的引用计数uint32regd_count;/*refcountonRegisteredSnapshots*///RegisteredSnapshots堆中的链接pairingheap_nodeph_node;/*linkintheRegisteredSnapshotsheap*///快照"拍摄"时间戳TimestampTzwhenTaken;/*timestampwhensnapshotwastaken*///拍照时WALstream中的位置XLogRecPtrlsn;/*positionintheWALstreamwhentaken*/}SnapshotData;
ShmemVariableCache
VariableCache是共享内存中的一种数据结构,用于跟踪OID和XID分配状态。
ShmemVariableCache是VariableCache结构体指针.
/**VariableCacheisadatastructureinsharedmemorythatisusedtotrack*OIDandXIDassignmentstate.Forlargelyhistoricalreasons,thereis*justonestructwithdifferentfieldsthatareprotectedbydifferent*LWLocks.*VariableCache是共享内存中的一种数据结构,用于跟踪OID和XID分配状态。*由于历史原因,这个结构体有不同的字段,由不同的LWLocks保护。**Note:xidWrapLimitandoldestXidDBarenot"active"values,butare*usedjusttogenerateusefulmessageswhenxidWarnLimitorxidStopLimit*areexceeded.*注意:xidWrapLimit和oldestXidDB是不"活跃"的值,在xidWarnLimit或xidStopLimit*超出限制时用于产生有用的信息.*/typedefstructVariableCacheData{/**ThesefieldsareprotectedbyOidGenLock.*这些域字段通过OidGenLock字段保护*///下一个待分配的OIDOidnextOid;/*nextOIDtoassign*///在必须执行XLOGwork前可用OIDsuint32oidCount;/*OIDsavailablebeforemustdoXLOGwork*//**ThesefieldsareprotectedbyXidGenLock.*这些字段通过XidGenLock锁保护.*///下一个待分配的事务IDTransactionIdnextXid;/*nextXIDtoassign*///集群范围内最小datfrozenxidTransactionIdoldestXid;/*cluster-wideminimumdatfrozenxid*///在该XID开始强制执行autovacuumTransactionIdxidVacLimit;/*startforcingautovacuumshere*///在该XID开始提出警告TransactionIdxidWarnLimit;/*startcomplaininghere*///在该XID开外,拒绝生成下一个XIDTransactionIdxidStopLimit;/*refusetoadvancenextXidbeyondhere*///"世界末日"XID,需回卷TransactionIdxidWrapLimit;/*wheretheworldends*///持有最小datfrozenxid的DBOidoldestXidDB;/*databasewithminimumdatfrozenxid*//**ThesefieldsareprotectedbyCommitTsLock*这些字段通过CommitTsLock锁保护*/TransactionIdoldestCommitTsXid;TransactionIdnewestCommitTsXid;/**ThesefieldsareprotectedbyProcArrayLock.*这些字段通过ProcArrayLock锁保护*/TransactionIdlatestCompletedXid;/*newestXIDthathascommittedor*aborted*//**ThesefieldsareprotectedbyCLogTruncationLock*这些字段通过CLogTruncationLock锁保护*///clog中最古老的XIDTransactionIdoldestClogXid;/*oldestit'ssafetolookupinclog*/}VariableCacheData;//结构体指针typedefVariableCacheData*VariableCache;/*pointerto"variablecache"insharedmemory(setupbyshmem.c)*///共享内存中的指针(通过shmem.c设置)VariableCacheShmemVariableCache=NULL;二、源码解读
GetSnapshotData函数返回快照信息.
重点是构造xmin : xmax : xip_list,其实现逻辑简单总结如下:
1.获取xmax = ShmemVariableCache->latestCompletedXid + 1;
2.遍历全局procArray数组,构建快照信息
2.1 获取进程相应的事务信息pgxact
2.2 获取进程事务ID(pgxact->xid),取最小的xid作为xmin(不包括0)
2.3 把xid放入快照->xip数组中(不包括本进程所在的事务id)
/**GetSnapshotData--returnsinformationaboutrunningtransactions.*GetSnapshotData--返回关于正在运行中的事务的相关信息**Thereturnedsnapshotincludesxmin(loweststill-runningxactID),*xmax(highestcompletedxactID+1),andalistofrunningxactIDs*intherangexmin<=xid<xmax.Itisusedasfollows:*AllxactIDs<xminareconsideredfinished.*AllxactIDs>=xmaxareconsideredstillrunning.*ForanxactIDxmin<=xid<xmax,consultlisttoseewhether*itisconsideredrunningornot.*Thisensuresthatthesetoftransactionsseenas"running"bythe*currentxactwillnotchangeafterittakesthesnapshot.*返回的snapshot包括xmin(最小的正在运行的事务ID),xmax(已完结事务ID+1),*以及在xmin<=xid<xmax之间正在运行的事务IDs.*意义如下:*事务IDs<xmin是已确定完成的事务.*事务IDs>=xmax是正在运行的事务.*对于XID∈[xmin,xmax)的事务,需查阅列表确认是否正在运行中**Allrunningtop-levelXIDsareincludedinthesnapshot,exceptforlazy*VACUUMprocesses.WealsotrytoincluderunningsubtransactionXIDs,*butsincePGPROChasonlyalimitedcacheareaforsubxactXIDs,full*informationmaynotbeavailable.Ifwefindanyoverflowedsubxidarrays,*wehavetomarkthesnapshot'ssubxiddataasoverflowed,andextrawork**may*needtobedonetodeterminewhat'srunning(seeXidInMVCCSnapshot()*intqual.c).*所有正在运行的顶层XIDs包含在快照中,除了lazyVACUUM进程.*我们尝试包含所有正在运行的子事务XIDs,但由于PGPROC只有有限的缓存,包含所有的子事务信息暂未实现.*如果我们搜索溢出的子事务数组,我们必须标记快照的subxid数据为溢出,*而且需要执行额外的工作以确定哪些在运行(查看tqual.c中的XidInMVCCSnapshot()函数)**Wealsoupdatethefollowingbackend-globalvariables:*TransactionXmin:theoldestxminofanysnapshotinuseinthe*currenttransaction(thisisthesameasMyPgXact->xmin).*RecentXmin:thexmincomputedforthemostrecentsnapshot.XIDs*olderthanthisareknownnotrunninganymore.*RecentGlobalXmin:theglobalxmin(oldestTransactionXminacrossall*runningtransactions,exceptthoserunningLAZYVACUUM).Thisis*thesamecomputationdoneby*GetOldestXmin(NULL,PROCARRAY_FLAGS_VACUUM).*RecentGlobalDataXmin:theglobalxminfornon-catalogtables*>=RecentGlobalXmin*我们同时更新了以下后台全局变量:*TransactionXmin:当前事务中在所有仍在使用的快照中最旧的xmin(与MyPgXact->xmin一致).*RecentXmin:最近快照的xmin.小于xmin的事务已知已完结.*RecentGlobalXmin:全局的xmin(除了正在运行的LAZYVACUUM,跨越所有正在运行事务的最旧的TransactionXmin),*这是使用同样的规则,通过GetOldestXmin(NULL,PROCARRAY_FLAGS_VACUUM)处理.*RecentGlobalDataXmin:非catalog数据表的全局xmin,该值>=RecentGlobalXmin.**Note:thisfunctionshouldprobablynotbecalledwithanargumentthat's*notstaticallyallocated(seexipallocationbelow).*注意:不应该使用非静态分配的参数调用这个函数(参见下面的xip分配)。*/SnapshotGetSnapshotData(Snapshotsnapshot){ProcArrayStruct*arrayP=procArray;//进程数组TransactionIdxmin;//xminTransactionIdxmax;//xmaxTransactionIdglobalxmin;//全局xminintindex;intcount=0;intsubcount=0;boolsuboverflowed=false;TransactionIdreplication_slot_xmin=InvalidTransactionId;TransactionIdreplication_slot_catalog_xmin=InvalidTransactionId;Assert(snapshot!=NULL);/**AllocatingspaceformaxProcsxidsisusuallyoverkill;numProcswould*besufficient.Butitseemsbettertodothemallocwhilenotholding*thelock,sowecan'tlookatnumProcs.Likewise,weallocatemuch*moresubxipstoragethanisprobablyneeded.*为maxProcsxids分配空间通常是多余的;numProcs就足够了。*但是在不持有锁的情况下执行malloc似乎更好,因此我们不能查看numProcs。*同样地,我们分配的子xip存储可能比实际需要的多得多。**Thisdoesopenapossibilityforavoidingrepeatedmalloc/free:since*maxProcsdoesnotchangeatruntime,wecansimplyreusetheprevious*xiparraysifany.(Thisreliesonthefactthatallcallerspass*staticSnapshotDatastructs.)*这确实为避免重复的malloc/free创造了一种可能性:因为maxProcs在运行时不会改变,*如果有的话,我们可以简单地重用前面的xip数组。*(这依赖于所有调用者都传递静态快照数据结构这一事实。)*/if(snapshot->xip==NULL){/**Firstcallforthissnapshot.Snapshotissamesizewhetherornot*weareinrecovery,seelatercomments.*首次调用.快照的大小不管是在常规还是在恢复状态都是一样的,看稍后的注释.*/snapshot->xip=(TransactionId*)malloc(GetMaxSnapshotXidCount()*sizeof(TransactionId));if(snapshot->xip==NULL)ereport(ERROR,(errcode(ERRCODE_OUT_OF_MEMORY),errmsg("outofmemory")));Assert(snapshot->subxip==NULL);snapshot->subxip=(TransactionId*)malloc(GetMaxSnapshotSubxidCount()*sizeof(TransactionId));if(snapshot->subxip==NULL)ereport(ERROR,(errcode(ERRCODE_OUT_OF_MEMORY),errmsg("outofmemory")));}/**ItissufficienttogetsharedlockonProcArrayLock,evenifweare*goingtosetMyPgXact->xmin.*即使我们要设置MyPgXact->xmin,也需要获取锁,在ProcArrayLock上获得共享锁就足够了.**/LWLockAcquire(ProcArrayLock,LW_SHARED);/*xmaxisalwayslatestCompletedXid+1*///xmax=latestCompletedXid+1//已完结事务号+1xmax=ShmemVariableCache->latestCompletedXid;Assert(TransactionIdIsNormal(xmax));TransactionIdAdvance(xmax);//+1/*initializexmincalculationwithxmax*///初始化xmin为xmaxglobalxmin=xmin=xmax;//是否处于恢复过程中?snapshot->takenDuringRecovery=RecoveryInProgress();if(!snapshot->takenDuringRecovery){//不是,正常运行中int*pgprocnos=arrayP->pgprocnos;//进程数intnumProcs;/**SpinoverprocArraycheckingxid,xmin,andsubxids.Thegoalis*togatherallactivexids,findthelowestxmin,andtrytorecord*subxids.*SpinOverprocArray,检查xid/xmin和subxids.*目标是搜集所有活动的xids,找到最小的xmin,并尝试记录subxids.*/numProcs=arrayP->numProcs;for(index=0;index<numProcs;index++)//遍历procArray数组{intpgprocno=pgprocnos[index];//allPgXact[]索引PGXACT*pgxact=&allPgXact[pgprocno];//获取PGXACTTransactionIdxid;//事务id/**Skipoverbackendsdoinglogicaldecodingwhichmanagesxmin*separately(checkbelow)andonesrunningLAZYVACUUM.*跳过正在执行逻辑解码(单独管理xmin)和执行LAZYVACUUM的进程.**/if(pgxact->vacuumFlags&(PROC_IN_LOGICAL_DECODING|PROC_IN_VACUUM))continue;/*Updateglobalxmintobethesmallestvalidxmin*///更新globalxmin为最小有效的xminxid=UINT32_ACCESS_ONCE(pgxact->xmin);//获取进程事务的xminif(TransactionIdIsNormal(xid)&&NormalTransactionIdPrecedes(xid,globalxmin))globalxmin=xid;/*Fetchxidjustonce-seeGetNewTransactionId*///只提取一次xid--查看函数GetNewTransactionIdxid=UINT32_ACCESS_ONCE(pgxact->xid);/**IfthetransactionhasnoXIDassigned,wecanskipit;it*won'thavesub-XIDseither.IftheXIDis>=xmax,wecanalso*skipit;suchtransactionswillbetreatedasrunninganyway*(andanysub-XIDswillalsobe>=xmax).*如果事务未分配XID事务号,跳过此事务.该事务也不会含有子事务.*如果XID>=xmax,我们也可以跳过,这些事务可被处理为正在运行的思维.*(这些事务的子事务XID也同样会>=xmax)*/if(!TransactionIdIsNormal(xid)||!NormalTransactionIdPrecedes(xid,xmax))continue;/**Wedon'tincludeourownXIDs(ifany)inthesnapshot,butwe*mustincludetheminxmin.*在快照中,不会包含自己的XIDs,但必须体现在xmin中*/if(NormalTransactionIdPrecedes(xid,xmin))//xid小于xmin,设置为xidxmin=xid;if(pgxact==MyPgXact)continue;//跳过本事务/*AddXIDtosnapshot.*///添加XID到快照中snapshot->xip[count++]=xid;/**SavesubtransactionXIDsifpossible(ifwe'vealready*overflowed,there'snopoint).NotethatthesubxactXIDsmust*belaterthantheirparent,sononeedtocheckthemagainst*xmin.Wecouldfilteragainstxmax,butitseemsbetternotto*dothatmuchworkwhileholdingtheProcArrayLock.*如可能,保存子事务XIDs(如果已经溢出,那就没法了).*注意子事务XIDs必须在他们的父事务之后发生,因此无需检查xmin.*我们可以利用xmax进行过滤,但是在持有锁ProcArrayLock时最好不要做那么多的工作。**Theotherbackendcanaddmoresubxidsconcurrently,butcannot*removeany.Henceit'simportanttofetchnxidsjustonce.*Shouldbesafetousememcpy,though.(Weneedn'tworryabout*missinganyxidsaddedconcurrently,becausetheymustpostdate*xmax.)*其他后台进程可能并发增加子事务ID,但不能清除.*因此,只取一次nxids很重要.不过,使用memcpy是安全的.*(不需要担心遗漏并发增加xids,因为他们在xmax之后)**Again,ourownXIDsarenotincludedinthesnapshot.*再次,我们自己的XIDs不需要包含在快照中*/if(!suboverflowed){if(pgxact->overflowed)suboverflowed=true;else{intnxids=pgxact->nxids;if(nxids>0){PGPROC*proc=&allProcs[pgprocno];pg_read_barrier();/*pairswithGetNewTransactionId*/memcpy(snapshot->subxip+subcount,(void*)proc->subxids.xids,nxids*sizeof(TransactionId));subcount+=nxids;}}}}}else{/**We'reinhotstandby,sogetXIDsfromKnownAssignedXids.*处于hotstandby中,通过KnownAssignedXids获取XIDs.**Westoreallxidsdirectlyintosubxip[].Here'swhy:*直接存储所有的xids到subxip[]中,这是因为:**Inrecoverywedon'tknowwhichxidsaretop-levelandwhichare*subxacts,adesignchoicethatgreatlysimplifiesxidprocessing.*在恢复过程中,我们不需要知道哪些xids是顶层事务,哪些是子事务,*这可以极大的简化xid处理过程.**Itseemslikewewouldwanttotrytoputxidsintoxip[]only,but*thatisfairlysmall.Wewouldeitherneedtomakethatbiggeror*toincreasetherateatwhichweWAL-logxidassignment;neitheris*anappealingchoice.*似乎我们只想把xid放到xip[]中,但xip数组是相当小的。*我们要么需要扩展,要么提高WAL-logxid分派的速度;*但这两个选择都不吸引人。**Wecouldtrytostorexidsintoxip[]firstandthenintosubxip[]*iftherearetoomanyxids.Thatonlyworksifthesnapshotdoesn't*overflowbecausewedonotsearchsubxip[]inthatcase.Asimpler*wayistojuststoreallxidsinthesubxactarraybecausethisis*byfarthebiggerarray.Wejustleavethexiparrayempty.*如果xid太多的话,我们尝试先将xid存储到xip[]中,然后再在subxip[]中存储。*这只在快照没有溢出的情况下有效,因为在这种情况下我们不搜索subxip[]。*一种更简单的方法是将所有xid存储在subxact数组中,因为这个数组要大得多。*让xip数组为空。**EitherwayweneedtochangethewayXidInMVCCSnapshot()works*dependinguponwhenthesnapshotwastaken,orchangenormal*snapshotprocessingsoitmatches.*无论哪种方式,我们都需要根据快照的拍摄时间更改XidInMVCCSnapshot()的工作方式,*或者更改正常的快照处理,使其匹配。**Note:Itispossibleforrecoverytoendbeforewefinishtaking*thesnapshot,andfornewlyassignedtransactionidstobeaddedto*theProcArray.xmaxcannotchangewhileweholdProcArrayLock,so*thosenewlyaddedtransactionidswouldbefilteredaway,sowe*neednotbeconcernedaboutthem.*注意:在我们完成快照之前,恢复可能会结束,*并且新分配的事务id可能会添加到ProcArray中。*当我们持有锁ProcArrayLock时,xmax无法更改,*因此那些新添加的事务id将被过滤掉,因此无需担心。*/subcount=KnownAssignedXidsGetAndSetXmin(snapshot->subxip,&xmin,xmax);if(TransactionIdPrecedesOrEquals(xmin,procArray->lastOverflowedXid))suboverflowed=true;}/**FetchintolocalvariablewhileProcArrayLockisheld-the*LWLockReleasebelowisabarrier,ensuringthishappensinsidethe*lock.*持有ProcArrayLock锁时,提前到本地变量中,*下面的LWLockRelease是一个屏障,确保这发生在锁内部。*/replication_slot_xmin=procArray->replication_slot_xmin;replication_slot_catalog_xmin=procArray->replication_slot_catalog_xmin;if(!TransactionIdIsValid(MyPgXact->xmin))MyPgXact->xmin=TransactionXmin=xmin;LWLockRelease(ProcArrayLock);/**Updateglobalxmintoincludeactualprocessxids.Thisisaslightly*differentwayofcomputingitthanGetOldestXminuses,butshouldgive*thesameresult.*更新globalxmin已包含实际的进程xids.*这是一种与GetOldestXmin使用的计算方法略有不同的方法,但是应该会得到相同的结果。*/if(TransactionIdPrecedes(xmin,globalxmin))globalxmin=xmin;/*Updateglobalvariablestoo*///更新全局变量RecentGlobalXmin=globalxmin-vacuum_defer_cleanup_age;if(!TransactionIdIsNormal(RecentGlobalXmin))RecentGlobalXmin=FirstNormalTransactionId;/*Checkwhetherthere'sareplicationslotrequiringanolderxmin.*///检查是否存在正在请求更旧xmin的复制slotif(TransactionIdIsValid(replication_slot_xmin)&&NormalTransactionIdPrecedes(replication_slot_xmin,RecentGlobalXmin))RecentGlobalXmin=replication_slot_xmin;/*Non-catalogtablescanbevacuumedifolderthanthisxid*///比该xid小的非catalog表可被vacuum进程清除RecentGlobalDataXmin=RecentGlobalXmin;/**Checkwhetherthere'sareplicationslotrequiringanoldercatalog*xmin.*检查是否存在正确请求更旧catalogxmin的复制slot*/if(TransactionIdIsNormal(replication_slot_catalog_xmin)&&NormalTransactionIdPrecedes(replication_slot_catalog_xmin,RecentGlobalXmin))RecentGlobalXmin=replication_slot_catalog_xmin;RecentXmin=xmin;snapshot->xmin=xmin;snapshot->xmax=xmax;snapshot->xcnt=count;snapshot->subxcnt=subcount;snapshot->suboverflowed=suboverflowed;//当前命令idsnapshot->curcid=GetCurrentCommandId(false);/**Thisisanewsnapshot,sosetbothrefcountsarezero,andmarkitas*notcopiedinpersistentmemory.*这是一个新的快照,因此设置refcounts为0,并标记其未在持久化内存中拷贝.*/snapshot->active_count=0;snapshot->regd_count=0;snapshot->copied=false;if(old_snapshot_threshold<0){/**Ifnotusing"snapshottooold"feature,fillrelatedfieldswith*dummyvaluesthatdon'trequireanylocking.*如启用"snapshottooold"特性,使用虚拟值填充相关的字段,这里不需要锁.*/snapshot->lsn=InvalidXLogRecPtr;snapshot->whenTaken=0;}else{/**CapturethecurrenttimeandWALstreamlocationincasethis*snapshotbecomesoldenoughtoneedtofallbackonthespecial*"oldsnapshot"logic.*捕获当前时间和WAL流位置,以防快照变得足够旧时需要使用特殊的“oldsnapshot”逻辑。*/snapshot->lsn=GetXLogInsertRecPtr();snapshot->whenTaken=GetSnapshotCurrentTimestamp();MaintainOldSnapshotTimeMapping(snapshot->whenTaken,xmin);}//返回快照returnsnapshot;}三、跟踪分析
执行简单查询,可触发获取快照逻辑.
16:35:08(xdb@[local]:5432)testdb=#begin;BEGIN16:35:13(xdb@[local]:5432)testdb=#*select1;
启动gdb,设置断点
(gdb)bGetSnapshotDataBreakpoint1at0x89aef3:fileprocarray.c,line1519.(gdb)cContinuing.Breakpoint1,GetSnapshotData(snapshot=0xf9be60<CurrentSnapshotData>)atprocarray.c:15191519ProcArrayStruct*arrayP=procArray;(gdb)
输入参数snapshot,实质是全局变量CurrentSnapshotData
(gdb)p*snapshot$1={satisfies=0xa9310d<HeapTupleSatisfiesMVCC>,xmin=2354,xmax=2358,xip=0x24c7e40,xcnt=1,subxip=0x251dfa0,subxcnt=0,suboverflowed=false,takenDuringRecovery=false,copied=false,curcid=0,speculativeToken=0,active_count=0,regd_count=0,ph_node={first_child=0x0,next_sibling=0x0,prev_or_parent=0x0},whenTaken=0,lsn=0}
查看共享内存(ShmemVariableCache)中的信息.
nextXID = 2358,下一个待分配的事务ID = 2358.
(gdb)p*ShmemVariableCache$2={nextOid=42605,oidCount=8183,nextXid=2358,oldestXid=561,xidVacLimit=200000561,xidWarnLimit=2136484208,xidStopLimit=2146484208,xidWrapLimit=2147484208,oldestXidDB=16400,oldestCommitTsXid=0,newestCommitTsXid=0,latestCompletedXid=2357,oldestClogXid=561}(gdb)
获取全局进程数组procArray,赋值->arrayP.
初始化相关变量.
(gdb)n1524intcount=0;(gdb)n1525intsubcount=0;(gdb)1526boolsuboverflowed=false;(gdb)1527volatileTransactionIdreplication_slot_xmin=InvalidTransactionId;(gdb)1528volatileTransactionIdreplication_slot_catalog_xmin=InvalidTransactionId;(gdb)1530Assert(snapshot!=NULL);(gdb)1543if(snapshot->xip==NULL)(gdb)
查看进程数组信息和allPgXact[]数组编号(arrayP->pgprocnos数组).
allPgXact定义:static PGXACT *allPgXact;
(gdb)p*arrayP$3={numProcs=5,maxProcs=112,maxKnownAssignedXids=7280,numKnownAssignedXids=0,tailKnownAssignedXids=0,headKnownAssignedXids=0,known_assigned_xids_lck=0'\000',lastOverflowedXid=0,replication_slot_xmin=0,replication_slot_catalog_xmin=0,pgprocnos=0x7f8765d9a3a8}(gdb)parrayP->pgprocnos[0]$4=97(gdb)parrayP->pgprocnos[1]$5=98(gdb)parrayP->pgprocnos[2]$6=99(gdb)parrayP->pgprocnos[3]$7=103(gdb)parrayP->pgprocnos[4]$9=111
加锁,获取/修改相关信息
(gdb)1568LWLockAcquire(ProcArrayLock,LW_SHARED);
计算xmax
(gdb)n1571xmax=ShmemVariableCache->latestCompletedXid;(gdb)1572Assert(TransactionIdIsNormal(xmax));(gdb)pxmax$10=2357(gdb)n1573TransactionIdAdvance(xmax);(gdb)1576globalxmin=xmin=xmax;(gdb)1578snapshot->takenDuringRecovery=RecoveryInProgress();(gdb)pxmax$11=2358
判断是否处于恢复状态,当前不是恢复状态,进入相应的处理逻辑
(gdb)n1580if(!snapshot->takenDuringRecovery)(gdb)psnapshot->takenDuringRecovery$13=false(gdb)n1582int*pgprocnos=arrayP->pgprocnos;(gdb)
获取进程数和PGXACT索引数组,准备遍历
(gdb)n1590numProcs=arrayP->numProcs;(gdb)1591for(index=0;index<numProcs;index++)(gdb)(gdb)p*pgprocnos$14=97(gdb)pnumProcs$15=5(gdb)
获取pgxact信息
(gdb)n1593intpgprocno=pgprocnos[index];(gdb)1594volatilePGXACT*pgxact=&allPgXact[pgprocno];(gdb)1601if(pgxact->vacuumFlags&PROC_IN_LOGICAL_DECODING)(gdb)1605if(pgxact->vacuumFlags&PROC_IN_VACUUM)(gdb)1609xid=pgxact->xmin;/*fetchjustonce*/(gdb)p*pgxact$16={xid=0,xmin=0,vacuumFlags=0'\000',overflowed=false,delayChkpt=false,nxids=0'\000'}(gdb)
不是正常的xid,下一个pgxact
(gdb)n1610if(TransactionIdIsNormal(xid)&&(gdb)1615xid=pgxact->xid;(gdb)1623if(!TransactionIdIsNormal(xid)(gdb)pxid$17=0(gdb)n1625continue;(gdb)
下一个xid = 2355,正常的事务ID
(gdb)1591for(index=0;index<numProcs;index++)(gdb)1593intpgprocno=pgprocnos[index];(gdb)1594volatilePGXACT*pgxact=&allPgXact[pgprocno];(gdb)1601if(pgxact->vacuumFlags&PROC_IN_LOGICAL_DECODING)(gdb)p*pgxact$18={xid=2355,xmin=0,vacuumFlags=0'\000',overflowed=false,delayChkpt=false,nxids=0'\000'}(gdb)
进行处理
(gdb)n1605if(pgxact->vacuumFlags&PROC_IN_VACUUM)(gdb)1609xid=pgxact->xmin;/*fetchjustonce*/(gdb)1610if(TransactionIdIsNormal(xid)&&(gdb)1615xid=pgxact->xid;(gdb)1623if(!TransactionIdIsNormal(xid)(gdb)1624||!NormalTransactionIdPrecedes(xid,xmax))(gdb)1631if(NormalTransactionIdPrecedes(xid,xmin))(gdb)pxid$19=2355(gdb)pxmin$20=2358(gdb)n1632xmin=xid;(gdb)1633if(pgxact==MyPgXact)(gdb)
这是同一个xact,处理下一个xact
(gdb)1633if(pgxact==MyPgXact)(gdb)ppgxact$21=(volatilePGXACT*)0x7f8765d9a218(gdb)pMyPgXact$22=(structPGXACT*)0x7f8765d9a218(gdb)n1634continue;(gdb)
下一个是2354
...(gdb)p*pgxact$23={xid=2354,xmin=0,vacuumFlags=0'\000',overflowed=false,delayChkpt=false,nxids=0'\000'}(gdb)
xmin调整为2354
1631if(NormalTransactionIdPrecedes(xid,xmin))(gdb)1632xmin=xid;(gdb)1633if(pgxact==MyPgXact)(gdb)pxmin$24=2354(gdb)
写入到xip_list中
1637snapshot->xip[count++]=xid;(gdb)1654if(!suboverflowed)(gdb)(gdb)pcount$25=1
继续循环,完成5个pgxact的遍历
1591for(index=0;index<numProcs;index++)(gdb)1715replication_slot_xmin=procArray->replication_slot_xmin;(gdb)
无复制信息
(gdb)1715replication_slot_xmin=procArray->replication_slot_xmin;(gdb)pprocArray->replication_slot_xmin$28=0(gdb)n1716replication_slot_catalog_xmin=procArray->replication_slot_catalog_xmin;(gdb)1718if(!TransactionIdIsValid(MyPgXact->xmin))
调整本进程的事务信息
(gdb)n1719MyPgXact->xmin=TransactionXmin=xmin;(gdb)pMyPgXact->xmin$29=0(gdb)n
释放锁
1721LWLockRelease(ProcArrayLock);(gdb)1728if(TransactionIdPrecedes(xmin,globalxmin))(gdb)
调整全局xmin
(gdb)pxmin$30=2354(gdb)pglobalxmin$31=2358(gdb)n1729globalxmin=xmin;(gdb)
更新其他信息
(gdb)1732RecentGlobalXmin=globalxmin-vacuum_defer_cleanup_age;(gdb)pRecentGlobalXmin$32=2354(gdb)pvacuum_defer_cleanup_age$33=0(gdb)n1733if(!TransactionIdIsNormal(RecentGlobalXmin))(gdb)1737if(TransactionIdIsValid(replication_slot_xmin)&&(gdb)1742RecentGlobalDataXmin=RecentGlobalXmin;(gdb)pRecentGlobalXmin$34=2354(gdb)n1748if(TransactionIdIsNormal(replication_slot_catalog_xmin)&&(gdb)
填充snapshot域字段信息
(gdb)1752RecentXmin=xmin;(gdb)1754snapshot->xmin=xmin;(gdb)1755snapshot->xmax=xmax;(gdb)1756snapshot->xcnt=count;(gdb)1757snapshot->subxcnt=subcount;(gdb)1758snapshot->suboverflowed=suboverflowed;(gdb)1760snapshot->curcid=GetCurrentCommandId(false);(gdb)1766snapshot->active_count=0;(gdb)1767snapshot->regd_count=0;(gdb)1768snapshot->copied=false;(gdb)1770if(old_snapshot_threshold<0)(gdb)1776snapshot->lsn=InvalidXLogRecPtr;(gdb)1777snapshot->whenTaken=0;(gdb)1791returnsnapshot;(gdb)
返回snapshot
(gdb)psnapshot$35=(Snapshot)0xf9be60<CurrentSnapshotData>(gdb)p*snapshot$36={satisfies=0xa9310d<HeapTupleSatisfiesMVCC>,xmin=2354,xmax=2358,xip=0x24c7e40,xcnt=1,subxip=0x251dfa0,subxcnt=0,suboverflowed=false,takenDuringRecovery=false,copied=false,curcid=0,speculativeToken=0,active_count=0,regd_count=0,ph_node={first_child=0x0,next_sibling=0x0,prev_or_parent=0x0},whenTaken=0,lsn=0}(gdb)
注意:snapshot->satisfies函数在初始化该全局变量已设置为HeapTupleSatisfiesMVCC.
感谢各位的阅读,以上就是“PostgreSQL中GetSnapshotData的处理过程是什么”的内容了,经过本文的学习后,相信大家对PostgreSQL中GetSnapshotData的处理过程是什么这一问题有了更深刻的体会,具体使用情况还需要大家实践验证。这里是亿速云,小编将为大家推送更多相关知识点的文章,欢迎关注!
声明:本站所有文章资源内容,如无特殊说明或标注,均为采集网络资源。如若本站内容侵犯了原著者的合法权益,可联系本站删除。