PostgreSQL中PageAddItemExtended函数的逻辑是什么
本篇内容主要讲解“PostgreSQL中PageAddItemExtended函数的逻辑是什么”,感兴趣的朋友不妨来看看。本文介绍的方法操作简单快捷,实用性强。下面就让小编来带大家学习“PostgreSQL中PageAddItemExtended函数的逻辑是什么”吧!
一、测试数据表testdb=# drop table if exists t_insert;
NOTICE: table "t_insert" does not exist, skipping
DROP TABLE
testdb=# create table t_insert(id int,c1 char(10),c2 char(10),c3 char(10));
CREATE TABLE
插入数据主要的实现在bufpage.c中,主要的函数是PageAddItemExtended。该函数的英文注释已经说得非常清楚,不过为了更好理解,这里添加了中文注释。
变量、宏定义和结构体等说明
1、Page指向char的指针:typedefchar*Pointer;typedefPointerPage;2、Item指向char的指针:typedefPointerItem;3、Sizetypedefsize_tSize;4、OffsetNumberunsignedshort(16bits):typedefuint16OffsetNumber;5、PageHeaderPageHeaderData结构体指针typedefstructPageHeaderData{/*XXXLSNismemberof*any*block,notonlypage-organizedones*/PageXLogRecPtrpd_lsn;/*LSN:nextbyteafterlastbyteofxlog*recordforlastchangetothispage*/uint16pd_checksum;/*checksum*/uint16pd_flags;/*flagbits,seebelow*/LocationIndexpd_lower;/*offsettostartoffreespace*/LocationIndexpd_upper;/*offsettoendoffreespace*/LocationIndexpd_special;/*offsettostartofspecialspace*/uint16pd_pagesize_version;TransactionIdpd_prune_xid;/*oldestprunableXID,orzeroifnone*/ItemIdDatapd_linp[FLEXIBLE_ARRAY_MEMBER];/*linepointerarray*/}PageHeaderData;typedefPageHeaderData*PageHeader;#definePD_HAS_FREE_LINES0x0001/*arethereanyunusedlinepointers?*/#definePD_PAGE_FULL0x0002/*notenoughfreespacefornewtuple?*/#definePD_ALL_VISIBLE0x0004/*alltuplesonpagearevisibleto*everyone*/#definePD_VALID_FLAG_BITS0x0007/*ORofallvalidpd_flagsbits*/6、SizeOfPageHeaderData长整型,定义页头大小:#defineoffsetof(type,field)((long)&((type*)0)->field)#defineSizeOfPageHeaderData(offsetof(PageHeaderData,pd_linp))7、BLCKSZ#defineBLCKSZ81928、PageGetMaxOffsetNumber如果空闲空间的lower小于等于页头大小,值为0,否则值为lower减去页头大小(24Bytes)后除以ItemId大小(4Bytes)#definePageGetMaxOffsetNumber(page)\(((PageHeader)(page))->pd_lower<=SizeOfPageHeaderData?0:\((((PageHeader)(page))->pd_lower-SizeOfPageHeaderData)\/sizeof(ItemIdData)))9、InvalidOffsetNumber是否无效的偏移值#defineInvalidOffsetNumber((OffsetNumber)0)10、OffsetNumberNext输入值+1#defineOffsetNumberNext(offsetNumber)\((OffsetNumber)(1+(offsetNumber)))11、ItemId结构体ItemIdData指针typedefstructItemIdData{unsignedlp_off:15,/*offsettotuple(fromstartofpage)*/lp_flags:2,/*stateofitempointer,seebelow*/lp_len:15;/*bytelengthoftuple*/}ItemIdData;typedefItemIdData*ItemId;#defineLP_UNUSED0/*unused(shouldalwayshavelp_len=0)*/#defineLP_NORMAL1/*used(shouldalwayshavelp_len>0)*/#defineLP_REDIRECT2/*HOTredirect(shouldhavelp_len=0)*/#defineLP_DEAD3/*dead,mayormaynothavestorage*/12、PageGetItemId获取相应的ItemIdData的指针#definePageGetItemId(page,offsetNumber)\((ItemId)(&((PageHeader)(page))->pd_linp[(offsetNumber)-1]))13、PAI_OVERWRITE位标记,实际值为1#definePAI_OVERWRITE(1<<0)14、PAI_IS_HEAP位标记,实际值为2#definePAI_IS_HEAP(1<<1)15、ItemIdIsUsedItemId是否已被占用#defineItemIdIsUsed(itemId)\((itemId)->lp_flags!=LP_UNUSED)16、ItemIdHasStorage#defineItemIdHasStorage(itemId)\((itemId)->lp_len!=017、PageHasFreeLinePointers/PageClearHasFreeLinePointers判断Page从Header到Lower之间是否有空位#definePageHasFreeLinePointers(page)\(((PageHeader)(page))->pd_flags&PD_HAS_FREE_LINES)清除空位标记(标记错误时清除)#definePageClearHasFreeLinePointers(page)\(((PageHeader)(page))->pd_flags&=~PD_HAS_FREE_LINES)18、MaxHeapTuplesPerPage每个Page可以容纳的最大Tuple数,计算公式是:(Block大小-页头大小)/(对齐后行头部大小+行指针大小)#defineMaxHeapTuplesPerPage\((int)((BLCKSZ-SizeOfPageHeaderData)/\(MAXALIGN(SizeofHeapTupleHeader)+sizeof(ItemIdData))))
PageAddItemExtended函数解读
/*PageAddItemExtended函数:输入:page-指向页的指针item-指向数据的指针size-数据大小offsetNumber-指定数据存储的偏移量flags-标记位(是否覆盖/是否Heap数据)输出:OffsetNumber-数据存储实际的偏移量*/OffsetNumberPageAddItemExtended(Pagepage,Itemitem,Sizesize,OffsetNumberoffsetNumber,intflags){PageHeaderphdr=(PageHeader)page;//页头指针SizealignedSize;//对齐大小intlower;//Freespace低位intupper;//Freespace高位ItemIditemId;//行指针OffsetNumberlimit;//行偏移,Freespace中第1个可用的位置偏移boolneedshuffle=false;//是否需要移动原有数据/**Bewaryaboutcorruptedpagepointers*/if(phdr->pd_lower<SizeOfPageHeaderData||phdr->pd_lower>phdr->pd_upper||phdr->pd_upper>phdr->pd_special||phdr->pd_special>BLCKSZ)ereport(PANIC,(errcode(ERRCODE_DATA_CORRUPTED),errmsg("corruptedpagepointers:lower=%u,upper=%u,special=%u",phdr->pd_lower,phdr->pd_upper,phdr->pd_special)));/**SelectoffsetNumbertoplacethenewitemat*///获取存储数据的偏移(位于Lower和Upper之间)limit=OffsetNumberNext(PageGetMaxOffsetNumber(page));/*wasoffsetNumberpassedin?*/if(OffsetNumberIsValid(offsetNumber)){//如果指定了数据存储的偏移量(传入的偏移量参数有效)/*yes,checkit*/if((flags&PAI_OVERWRITE)!=0)//不覆盖原有数据{if(offsetNumber<limit){//获取指定偏移的ItemIditemId=PageGetItemId(phdr,offsetNumber);//指定的数据偏移已使用或者已分配存储空间,报错if(ItemIdIsUsed(itemId)||ItemIdHasStorage(itemId)){elog(WARNING,"willnotoverwriteausedItemId");returnInvalidOffsetNumber;}}}else//覆盖原有数据{//指定的行偏移不在空闲空间中,需要移动原数据为新数据腾位置if(offsetNumber<limit)needshuffle=true;/*needtomoveexistinglinp's*/}}else//没有指定数据存储行偏移{/*offsetNumberwasnotpassedin,sofindafreeslot*//*ifnofreeslot,we'llputitatlimit(1stopenslot)*/if(PageHasFreeLinePointers(phdr))//页头标记提示存在已回收的空间{/**Lookfor"recyclable"(unused)ItemId.Wecheckfornostorage*aswell,justtobeparanoid---unuseditemsshouldneverhave*storage.*///循环找出第1个可用的空闲行偏移for(offsetNumber=1;offsetNumber<limit;offsetNumber++){itemId=PageGetItemId(phdr,offsetNumber);if(!ItemIdIsUsed(itemId)&&!ItemIdHasStorage(itemId))break;}//没有找到,说明页头标记有误,需清除标记,以免误导if(offsetNumber>=limit){/*thehintiswrong,soresetit*/PageClearHasFreeLinePointers(phdr);}}else//没有已回收的空间,行指针/数据存储到FreeSpace中{/*don'tbothersearchingifhintsaysthere'snofreeslot*/offsetNumber=limit;}}/*Rejectplacingitemsbeyondthefirstunusedlinepointer*/if(offsetNumber>limit){//如果指定的偏移大于空闲空间可用的第1个位置,报错elog(WARNING,"specifieditemoffsetistoolarge");returnInvalidOffsetNumber;}/*Rejectplacingitemsbeyondheapboundary,ifheap*/if((flags&PAI_IS_HEAP)!=0&&offsetNumber>MaxHeapTuplesPerPage){//Heap数据,但偏移大于一页可存储的最大Tuple数,报错elog(WARNING,"can'tputmorethanMaxHeapTuplesPerPageitemsinaheappage");returnInvalidOffsetNumber;}/**Computenewlowerandupperpointersforpage,seeifit'llfit.**Note:doarithmeticassignedints,toavoidmistakesif,say,*alignedSize>pd_upper.*/if(offsetNumber==limit||needshuffle)//如果数据存储在Freespace中,修改lower值lower=phdr->pd_lower+sizeof(ItemIdData);elselower=phdr->pd_lower;//否则,找到了已回收的空闲位置,使用原有的loweralignedSize=MAXALIGN(size);//大小对齐upper=(int)phdr->pd_upper-(int)alignedSize;//申请存储空间if(lower>upper)//校验returnInvalidOffsetNumber;/**OKtoinserttheitem.First,shuffletheexistingpointersifneeded.*///获取行指针itemId=PageGetItemId(phdr,offsetNumber);//if(needshuffle)//如果需要腾位置,把原有的行指针往后挪一"格"memmove(itemId+1,itemId,(limit-offsetNumber)*sizeof(ItemIdData));/*settheitempointer*///设置新数据行指针ItemIdSetNormal(itemId,upper,size);/**Itemsnormallycontainnouninitializedbytes.Corebufpageconsumers*conform,butthisisnotanecessarycodingrule;anewindexAMcould*opttodepartfromit.However,datatypeinputfunctionsandother*C-languagefunctionsthatsynthesizedatumsshouldinitializeall*bytes;datumIsEqual()reliesonthis.Testinghere,alongwiththe*similarcheckinprinttup(),helpstocatchsuchmistakes.**Valuesofthe"name"typeretrievedviaindex-onlyscansmaycontain*uninitializedbytes;seecommentinbtrescan().Valgrindwillreport*thisasanerror,butitissafetoignore.*/VALGRIND_CHECK_MEM_IS_DEFINED(item,size);/*copytheitem'sdataontothepage*///把数据放在数据区memcpy((char*)page+upper,item,size);/*adjustpageheader*///更新页头的lower&upperphdr->pd_lower=(LocationIndex)lower;phdr->pd_upper=(LocationIndex)upper;//如成功,返回实际的行偏移returnoffsetNumber;}三、跟踪分析
下面使用gdb对PageAddItemExtended函数跟踪分析。
测试场景:首先插入8行数据,然后删除第2行,接着插入1行数据:
testdb=#--插入8行数据testdb=#insertintot_insertvalues(1,'11','12','13');insertintot_insertvalues(6,'11','12','13');insertintot_insertvalues(7,'11','12','13');insertintot_insertvalues(8,'11','12','13');checkpoint;INSERT01testdb=#insertintot_insertvalues(2,'11','12','13');INSERT01testdb=#insertintot_insertvalues(3,'11','12','13');INSERT01testdb=#insertintot_insertvalues(4,'11','12','13');INSERT01testdb=#insertintot_insertvalues(5,'11','12','13');INSERT01testdb=#insertintot_insertvalues(6,'11','12','13');INSERT01testdb=#insertintot_insertvalues(7,'11','12','13');INSERT01testdb=#insertintot_insertvalues(8,'11','12','13');INSERT01testdb=#testdb=#checkpoint;CHECKPOINTtestdb=#--删除第2行testdb=#deletefromt_insertwhereid=2;DELETE1testdb=#testdb=#checkpoint;CHECKPOINTtestdb=#testdb=#--获取pidtestdb=#selectpg_backend_pid();pg_backend_pid----------------1572(1row)
使用gdb跟踪最后插入1行数据的过程:
#启动gdb,绑定进程[root@localhostdemo]#gdb-p1572GNUgdb(GDB)RedHatEnterpriseLinux7.6.1-100.el7Copyright(C)2013FreeSoftwareFoundation,Inc.LicenseGPLv3+:GNUGPLversion3orlater<http://gnu.org/licenses/gpl.html>Thisisfreesoftware:youarefreetochangeandredistributeit.ThereisNOWARRANTY,totheextentpermittedbylaw.Type"showcopying"and"showwarranty"fordetails.ThisGDBwasconfiguredas"x86_64-redhat-linux-gnu".Forbugreportinginstructions,pleasesee:<http://www.gnu.org/software/gdb/bugs/>.Attachingtoprocess1572...(gdb)#设置断点(gdb)bPageAddItemExtendedBreakpoint1at0x845119:filebufpage.c,line196.(gdb)#切换到psql,执行插入语句testdb=#insertintot_insertvalues(9,'11','12','13');(挂起)#切换回gdb(gdb)cContinuing.Breakpoint1,PageAddItemExtended(page=0x7feaaefac300"\001",item=0x29859f8"2\234\030",size=61,offsetNumber=0,flags=2)atbufpage.c:196196PageHeaderphdr=(PageHeader)page;#断点在函数的第一行代码上#bt打印调用栈(gdb)bt#0PageAddItemExtended(page=0x7feaaefac300"\001",item=0x29859f8"2\234\030",size=61,offsetNumber=0,flags=2)atbufpage.c:196#10x00000000004cf4f9inRelationPutHeapTuple(relation=0x7feac6e2ccb8,buffer=141,tuple=0x29859e0,token=false)athio.c:53#20x00000000004c34ecinheap_insert(relation=0x7feac6e2ccb8,tup=0x29859e0,cid=0,options=0,bistate=0x0)atheapam.c:2487#30x00000000006c076binExecInsert(mtstate=0x2984c10,slot=0x2985250,planSlot=0x2985250,estate=0x29848c0,canSetTag=true)atnodeModifyTable.c:529#40x00000000006c29f3inExecModifyTable(pstate=0x2984c10)atnodeModifyTable.c:2126#50x000000000069a7d8inExecProcNodeFirst(node=0x2984c10)atexecProcnode.c:445#60x0000000000690994inExecProcNode(node=0x2984c10)at../../../src/include/executor/executor.h:237#70x0000000000692e5einExecutePlan(estate=0x29848c0,planstate=0x2984c10,use_parallel_mode=false,operation=CMD_INSERT,sendTuples=false,numberTuples=0,direction=ForwardScanDirection,dest=0x2990dc8,execute_once=true)atexecMain.c:1726#80x0000000000690e58instandard_ExecutorRun(queryDesc=0x2981020,direction=ForwardScanDirection,count=0,execute_once=true)atexecMain.c:363#90x0000000000690cefinExecutorRun(queryDesc=0x2981020,direction=ForwardScanDirection,count=0,execute_once=true)atexecMain.c:306#100x0000000000851d84inProcessQuery(plan=0x2990c68,sourceText=0x28c5ef0"insertintot_insertvalues(9,'11','12','13');",params=0x0,queryEnv=0x0,dest=0x2990dc8,completionTag=0x7ffdbc052d10"")atpquery.c:161#110x00000000008534f4inPortalRunMulti(portal=0x292b490,isTopLevel=true,setHoldSnapshot=false,dest=0x2990dc8,altdest=0x2990dc8,completionTag=0x7ffdbc052d10"")atpquery.c:1286#120x0000000000852b32inPortalRun(portal=0x292b490,count=9223372036854775807,isTopLevel=true,run_once=true,dest=0x2990dc8,altdest=0x2990dc8,completionTag=0x7ffdbc052d10"")atpquery.c:799#130x000000000084cebcinexec_simple_query(query_string=0x28c5ef0"insertintot_insertvalues(9,'11','12','13');")atpostgres.c:1122#140x0000000000850f3cinPostgresMain(argc=1,argv=0x28efaa8,dbname=0x28ef990"testdb",username=0x28ef978"xdb")atpostgres.c:4153#150x00000000007c0168inBackendRun(port=0x28e7970)atpostmaster.c:4361#160x00000000007bf8fcinBackendStartup(port=0x28e7970)atpostmaster.c:4033#170x00000000007bc139inServerLoop()atpostmaster.c:1706#180x00000000007bb9f9inPostmasterMain(argc=1,argv=0x28c0b60)atpostmaster.c:1379#190x00000000006f19e8inmain(argc=1,argv=0x28c0b60)atmain.c:228#使用next命令,单步调试(gdb)next207if(phdr->pd_lower<SizeOfPageHeaderData||(gdb)next208phdr->pd_lower>phdr->pd_upper||(gdb)next207if(phdr->pd_lower<SizeOfPageHeaderData||(gdb)next209phdr->pd_upper>phdr->pd_special||(gdb)next208phdr->pd_lower>phdr->pd_upper||(gdb)next210phdr->pd_special>BLCKSZ)(gdb)next209phdr->pd_upper>phdr->pd_special||(gdb)next219limit=OffsetNumberNext(PageGetMaxOffsetNumber(page));(gdb)next222if(OffsetNumberIsValid(offsetNumber))(gdb)poffsetNumber$1=0#offsetNumber为0,没有指定插入的行偏移(gdb)next247if(PageHasFreeLinePointers(phdr))#查看页头信息(gdb)p*phdr$2={pd_lsn={xlogid=1,xrecoff=3677462648},pd_checksum=0,pd_flags=0,pd_lower=56,pd_upper=7680,pd_special=8192,pd_pagesize_version=8196,pd_prune_xid=1612849,pd_linp=0x7feaaefac318}(gdb)#查看行指针信息(gdb)pphdr->pd_linp[0]$3={lp_off=8128,lp_flags=1,lp_len=61}(gdb)pphdr->pd_linp[1]$4={lp_off=8064,lp_flags=1,lp_len=61}...(gdb)pphdr->pd_linp[8]$11={lp_off=0,lp_flags=0,lp_len=0}#行指针偏移的lp_flags均为1(LP_NORMAL),表示正在使用。...298alignedSize=MAXALIGN(size);(gdb)psize$17=61(gdb)palignedSize$18=5045956(gdb)next300upper=(int)phdr->pd_upper-(int)alignedSize;(gdb)palignedSize$19=64#原为61,对齐为64(gdb)next302if(lower>upper)(gdb)plower$20=60(gdb)pupper$21=7616(gdb)next308itemId=PageGetItemId(phdr,offsetNumber);(gdb)310if(needshuffle)(gdb)next315ItemIdSetNormal(itemId,upper,size);(gdb)next332memcpy((char*)page+upper,item,size);(gdb)p*itemId$23={lp_off=7616,lp_flags=1,lp_len=61}...(gdb)next338returnoffsetNumber;(gdb)#数据插入在行偏移为9(行指针下标=8)的位置上。(gdb)poffsetNumber$24=9(gdb)pphdr->pd_linp[8]$25={lp_off=7616,lp_flags=1,lp_len=61}...#说明:之所以新插入的数据没有放在2号偏移上面,是因为被删除的第2行没有被回收。#查看Page中的数据testdb=#select*fromheap_page_items(get_raw_page('t_insert',0));lp|lp_off|lp_flags|lp_len|t_xmin|t_xmax|t_field3|t_ctid|t_infomask2|t_infomask|t_hoff|t_bits|t_oid|t_data----+--------+----------+--------+---------+---------+----------+--------+-------------+------------+--------+--------+-------+------------------------------------------------------------------------------1|8128|1|61|1612841|0|0|(0,1)|4|2306|24|||\x010000001731312020202020202020173132202020202020202017313320202020202020202|8064|1|61|1612842|1612849|0|(0,2)|8196|258|24|||\x020000001731312020202020202020173132202020202020202017313320202020202020203|8000|1|61|1612843|0|0|(0,3)|4|2306|24|||\x030000001731312020202020202020173132202020202020202017313320202020202020204|7936|1|61|1612844|0|0|(0,4)|4|2306|24|||\x040000001731312020202020202020173132202020202020202017313320202020202020205|7872|1|61|1612845|0|0|(0,5)|4|2306|24|||\x050000001731312020202020202020173132202020202020202017313320202020202020206|7808|1|61|1612846|0|0|(0,6)|4|2306|24|||\x060000001731312020202020202020173132202020202020202017313320202020202020207|7744|1|61|1612847|0|0|(0,7)|4|2306|24|||\x070000001731312020202020202020173132202020202020202017313320202020202020208|7680|1|61|1612848|0|0|(0,8)|4|2306|24|||\x080000001731312020202020202020173132202020202020202017313320202020202020209|7616|1|61|1612850|0|0|(0,9)|4|2050|24|||\x09000000173131202020202020202017313220202020202020201731332020202020202020(9rows)
到此,相信大家对“PostgreSQL中PageAddItemExtended函数的逻辑是什么”有了更深的了解,不妨来实际操作一番吧!这里是亿速云网站,更多相关内容可以进入相关频道进行查询,关注我们,继续学习!
声明:本站所有文章资源内容,如无特殊说明或标注,均为采集网络资源。如若本站内容侵犯了原著者的合法权益,可联系本站删除。