受限波尔兹曼机（Restricted Boltzmann Machines）介绍
RBM在深度学习（deep learning）算法中有着非常重要的应用，本文介绍了RBM的基本概念，并介绍了几种有代表性的算法。作者西安交大张春霞，姬楠楠，王冠伟。山国武技亡文在线应用的热潮。理论方面,RBM的CD快速学习算法促进了研究者们对随机近似理论、基于能量的模型、未归一化的统计模型的研究⑧。应用方面,RBM目前已被成功地应用于不同的机器学习问题⑨-14,如分炎、回归、降维、高维时闾序列建模、图像特征提取、协同过滤等等。2006年, Hinton等人[15提出了一种深度信念网终( Deep Belief Nets,DBN),并给出了该模型的一个髙效学习算法。这个算法成为了其后至今深度学习算法的主要框架。在该算法中,个DBN模型被视为由若干个RBM堆叠在起,训练时可通过由低到高逐层训练这些RBM来实现:(1)底部RBM以原始输入数据训练;(2)将底部RBM抽取的特征作为顶部RBM的输入训练;(3)过程(1)和(2)可以重复来训练所需要的尽可能多的层数。由于RBM可以通过CD快速训练,这一框架绕过了直接从整体上训练DBN的高复杂度,从而将其化简为对多个RBM的训练冋题。 Hinton建议,经过这种方式训练后,叮以再通过传统的全局学习算法(如反向传播算法)对网络进行微调,从而使模型收敛到局部最优点。这种学习算法,本质上等同于先通过逐层RBM训练将模型的参数初始化为较优的值,再通过少量的传统学习算法进一步训练。这样一来,不仅解决了模型训练速度慢的问题,大量试验结果也表明,这种方式能够产生非常好的参数初始值,从而大大提升了模型的建模能力。自此,机器学习领域又产生了一个新的研究方向-深度学习( Deep learning)[1618],明确提出了面向人工智能的机器学习算法的设计目标。当前,以RBM为基本构成模块的DBN模型被认为是最有效的深度学习算法之一。鉴于RBM在深度学习领域中占据的核心位置以及其本身的良好性质,为了给RBM的初学者提供入门指导,同时为设计与之相关的新算法提供参考,本文将对RBM进行较为系统的介绍,详细阐述其基本模型、具有代表性的快速学习算法、参数设置、评估方法及其变形算法,最后对RBM在未来值得硏究的方向进行探讨。本文后续内容安排如下:第1节介绍受限波尔兹曼机RBM的基本模型,第2节详细阐述当前训练RBM的快速学习算法,第3节讨论RBM的参数设置,第4节给出评价RBM优劣的方法,第5节简单介绍几种具有代表性的RBM变形算法,第6是总结与展望,主要探讨RBM在未米值得研究的方向。1受限波尔兹曼机RBM的基本模型RBM也可以被视为一个无向图 undirected graph)模型,如图2所示。v为可见层,用于表示观测数据,h为隐层,可视为一些特征提取器( feature detectors),W为两层之间的连接权重。 Welling19指出,RBM中的隐单兀和可见单元可以为任意的指数族单元(即给定隐单元(可见单元,可见单元(隐单元)的分布可以为任意的指数族分布),如 softmax单元、高斯单元、泊松单元等等。这里,为了讨论方便起见,我们假设所有的可见单元和隐单元均为二值变量,即V,j,v∈{0,1},h;∈{0,1}如果一个RBM有m个可见单元和m个隐单元,用向量v和h分别表示可见单元和隐单元的状态。其中,v;表示第个可见单元的状态,h表示第个隐单元的状态。那么,对于一组给定的状国武技论义在线隐层h可见层v图2:RBM的图模型表示,层内单元之间无连接态(v,h,RBM作为一个系统所具备的能量定义为∑a"2-∑bh-∑∑上式中,O={Wn,a,b}是RBM的参数,它们均为实数。其中,W表示可见单元;与隐单元j之间的连接权重,;表小可见单元的煸置(bias),b;表小隐单元j的偏置。当参数确定时,基于该能量函数,我们可以得到(v,h)的联合概率分布,E(v, ho)P(v, h0Z(0)=∑e-E(v, h ez(6)(2其中z(0)为归一化因子(也称为配分函数, partition function)时于一个实际问题,我们最关心的是由RBM所定义的关于观测数据v的分布P(ve),即联合概率分布P(v,h)的边际分布,也称为似然函数( likelihood functionP(v8∑E(v, h 0)3)Z(0为了确定该分布,需要计算归一化因子z(),这需要2n+m次计算。因此,即使通过训练可以得到模型的参数W,α和b,我们仍旧无法有效地计算由这些参数所确定的分布。但是,由RBM的特殊结构(即层间有连接,层内无连接)可知:当给定可见单元的状态时,各隐单元的激活状态之间是条件独立的。此时,第j个隐单元的激活概率为P(h=1v,O)=o(+∑W其中,O(x)1+exp(-a)为 sigmoid激活函数由于RBM的结构是对称的,当给定隐单元的状态时,各可见单元的激活状态之间也是条件独立的,即第i个可见单元的激活概率为11)=o(a+∑Wh1国武技论义在线2基于对比散度的RBM快速学习算法学习RBM的任务是求出参数θ的值,以拟合给定的训练数据。参数0可以通过最大化RBM在训练集(偎设包含T个样本)上的对数似然函数学习得到,即A=arg max C(0)=arg max>log(6为了获得最优参数θ^,我们可以使用随札梯度上升法( stochastic gradient ascent)求C(6)=∑1lgP(vθ)的最大值。其中,关键步骤是计算logP(v()关于各个模型参数的偏导数由于c(O)=∑ log p(vo)=∑og∑Pv"),het=1∑1pBw,b∑pE(,hO)-lg∑∑oxp-E(,h)令0表示6中的某个参数,则对数似然函数关于的梯度为OC、、8(og>expl-E(v(t),h)-10e∑∑∑ep-E(v,hO)exp[-e(vo,ho) d(E(vo),h0))(Σ h exp[-E(v,hO>∑8+E(-E(v,h)06∑0(-E(v(0,hl(-E(v,h6)S06P(hv(t),0)P(v, h0)其中,()P表示求关于分布P的数学期望。P(hv),)表示在可见单元限定为已知的训练样本v()时,隐层的概率分布,故式()中的前一项比较容易计算。P(v,h0)表示可见单元与隐单元的联合分布,由于归一化因子z(θ)的存在,该分布很难获取,导致我们无法直接计算式(8)中的第二项,只能通过一些采样方法(如Gibs釆样)获取其近似值。值得指出的是,在最大化似然函数的过程中,为了加快计算速度,上述偏导数在每一迭代步中的计算一般只基于部分而非所有的训练样本进行,关丁这部分内容我们将在后面讨论RBM的参数设置时详细阐述。下面,假设只有一个训练样本,我们分别用“data”和“modl”来简记P(hv(),6)和P(v,h)这两个概率分布,则对数似然函数关于连接权重W、可见层单元的偏置a和隐层单山国科技论文在线元的偏置b;的偏导数分别为alog P(v8ihi idata -(ihi,modelalog P(v 0)datai ) modela log P(v 8=(hi)data(hi model2.1RBM中的 Gibbs采样Gibs采样( Gibbs sanpling)[20是一种基于马尔可夫链蒙特卡罗( Markov chain monteCarlo,MCMC)策略的采样方法。对于一个K维随机向量X=(X1,X2,……,Xk),假设我们无法求得关于X的联合分布P(X),但我们知道给定X的其他分量时,其第k个分量Xk的条件分布,即P(Xk|Xk),Xk-(X1,X2,…,Kk-1,Xk+1,…,Xk)。那么,我们可以从X的一个任意状态(比如{c1(0),x2(0),…,xk(O))开始,利用上述条件分布,迭代地对其分量依次采样,随着采样次数的增加,随机变量[r1(m),x2(m),…,xk(n)]的概率分布将以n的几何级数的速度收敛于X的联合概率分布P(X)。换句话说,我们可以在未知联合概率分布P(X)的条件下对其进行样。基于RBM模型的对称结构,以及其中神经元状态的条件独立性,我们可以使用 Gibbs采样方法得到服从RBM定义的分布的随机样本。在RBM中进行k步吉布斯采样的具体算法为:用一个训练样本(或可见层的任何随机化状态)初始化可见层的状态v,交替进行如下采样:ho w P(h vo), V1 P(v ho),h1 n P(hv1),P(vhP(v hk)在采样步数k足够大的情况下,我们可以得到服从RBM所定义的分布的样本。此外,使用Gib样我们也可以得到式(8)中第二项的一个近似。22基于对比散度的快速学习算法尽管利用吉布斯采样我们可以得到对数似然函数关于未知参数梯度的近似,但通常情况下需要使用较大的采样步数,这使得RBM的训练效率仍旧不高,尤其是当观测数据的特征维数较高时。2002年, Hinton7提出了RBM的一个快速学习算法,即对比散度( Contrastive DivergenceCD)。与吉布斯采样不同, Hinton指出当使用训练数据初始化vo时,我们仅需要使用k(通常k=1)步吉布斯采样使可以得到足够好的近似。在CD算法一开始,可见单元的状态被设置成个训练样本,并利用式(4)计算所有隐层单元的二值状态。在所有隐层单元的状态确定之后,根据式(5)来确定第个可见单元v;取值为1的概率,进而产生可见层的一个重构 reconstruction)国武技论义在线这样,在使用随杋悌度上升法最大化对数似然函数在训练数据上的值时,各参数的更新准则为△Wx=(vh;)ata-(vhrecondata这里,是学习率( (learning rate),{}reon表示一步重构后模型定义的分布在RBM中,可见单元数一般等于训练数据的特征维数,而隐单元数需要事先给定。为了与前文记号致,假设可见单元数和隐单元数分别为和m。令W表示可见层与隐层间的连接权重矩阵(m×m阶),a(n维列向量)和b(m维列向量分别表示可见层与隐层的偏置向量。RBM的基于CID的快速学习算法主要步骤可描述如下输入:一个训练样本xo;隐层单元个数灬m;学习率e;最大训练周期T●输出:连接权重矩阵W、可见层的偏置向量a、隐层的偏置向量b.·训练阶段初始化:令可见层单元的初始状态v1=x0;W、a和b为随机的较小数值。For t=1.2TFor j=1,2,…,m(对所有隐单元)计算P(h1=11),即P(h1;=1v1)=0(b+∑,从条件分布P(h1v)中抽取h∈{0,1}EndFor上ori=1,2,……,m(对所有可见单元计算P1h1,即P(v2=1h1)=0(a+∑,Wh1);从条件分布P(v2h1)中抽取v2∈{0,1}EndOForj=1.2,…,m(对所有隐单元)计算P(h2=1v2),即P(h2y=1lv2)=a(b;+∑;2:W7);Endfor按下式更新各个参数W←W+∈(P(h1.=1v1)lv2)V2);a←-a+((v1-v2);+c(P(h1=1v1)-P(h=1)v2)山国科技论文在线Endfor算法1.RBⅥ的基于CD的快速学习算法主要步骤在上述算法中,记号P(hk.=1|vk)(k=1,2)是m维列向量,其第个元素为P(h;=1vk)尽管上述基于CD的学习算法是针对RBM的可见单元和隐层单元均为二值变量的情形提出的,但很容易推广到可见层单元为高斯变量、可见层和隐层单元均为高斯变量等其他情形,关于这方面的研究具体可参见[2125此外,还有一些研究者在CD算法的基础上,对其作了进一步改进。例如, Tieleman②26提出了持续对比散度( Persistent contrastive divergence,PCD)算法,该算法与CD的区别在于首先,PCD不再使用训练数据初始化CD算法中的 Gibbs采样的马氏链;其次,PCD算法中的学习率较小且不断衰减。根据随机近似理论,尽管每次更新参数后模型都发生了改变(每次对于W,a和b的更新,RBM定义的分布都会发生改变),但由于学习率较小且不断衰减,则可认为那条马氏链产生的负样本是由当前RBM定义的分布的一个近似分布米样而来Tieleman和 Hinton[27进一步改进了PCD算法,他们通过引入一组辅助参数以加快PCD中的马氏链的混合率,提出了快速持续对比散度( Fast Persistent Contrastive Divergence,FPCD)算法。关于RBM的学习算法,除了上述提到的基于CD的一些方法之外,还有最大化拟似然函数( maximum pseudo- likelihood)、比率匹配方法 (ratio matching)等,有兴趣的读者可参阅[28]查找关于RBM学习算法比较详细的阐述。3RBM的参数设置RBM的训练通常是基于CD的方法(即算法1)进行的,但如何设置其中的些参数(如隐单元个数、学习率、参数的初始值等),是需要有一定经验的。近来,已有部分研究结果②29,30表明:对于特定的数据集和RBM结构,如果参数设置不合适,RBM将很难对真正的数据分布正确建模。因此,对实际使用者(尤其是初学者)米说,了解RBM中参数设置的一般规则是非常重要的。根据 Hinton{23]提供的建议以及我们进行数值试验所获部分经验,对RBM中的参数设置可参考以下规则。小批量数据及其容量对于连接权重、可见层和隐层偏置的更新,虽然可以基于一个训练样本进行(类似于在线学习的方式),但计算量将很大。将训练集事先分成包含几|或几百个样本的小批量数据(mini- batches)进行计算将更高效,这主要是可以利用图形处珥器GPU( graphicProcessing Unit)或 Matlab屮矩阵之间相乘运算的优势。同时,为了避免在小批量数据的样本容量发生改变时,学习率也必须做相应的修改,通常的做法是在参数的更新过程中,使用参数的平均梯度(即总梯度除以数据容量),即B(t+1(t+1)=0(+∑alog P(v(t)aB06t′=Bt+1这里,B表示小批量数据的容量,其值不应设得太大。B=1表示参数更新以在线学习的方式进行,而B一T则表示传统的批处理方式。一股而言,若训练集是包含来自不同类(具有同等概山国武技亡文在线率)的样本,理想的B应为总类数、使得每批数据屮都包含来自每个类的一个样本,以减小悌度估计的抽样误差。对于其他数据集.则可先随机化训练样本的次序,再将其分为容量为10的倍数的小批量数据。学习率学习率若过大,将导致重构误差急剧增加,权重也会变得异常大。设置学习率的一般做法是先做权重更新和权重的直方图,令权重更新量为权重的10-3倍左右。如果有一个单元的输入值很大,则权重更新应再小些,因为同·方向上较多小的波动很容易改变梯度的符号。相反地,对于偏置,其权重更新可以大一些。权重和偏置的初始值一般地、连接权重W可初始化为来自正态分布N(0,0.01)的随机数,隐单元的偏置b初始化为0。对于第讠个可见单元,其偏置az通常初始化为logP/(1-p),其中γ;表示训练样本中第讠个特征处于激活状态所占的比率。如果不这样做,在学习的早期阶段,RBM会利用隐单元使得第个特征以概率p处于激活状态。动量学习率学习率e的选择至关重要.ξ大收敛速度快,但过大可能引起算法不稳定;c小可避免不稳定情况的出现,但收敛速度较慢。为克服这一矛盾,一种具有代表性的思想是在参数更新式中增加动量项 momentum),使本次参数值修改的方向不完全由当前样本下的似然函数梯度方向决定,而采用上一次参数值修改方向与本次梯度方向的组合。在某些情况下,这可以避免算法过早地收敛到局部最优点。以连接权重参数W为例,其更新公式为W(+D)kw(t)aL(t)其中k为动量项学习率。开始时,k可设为0.5,在重构误差处于平稳增加状态时,k可取为0.9权衰减使用权衰减( weight- decay)策略的主要目的是避免学习过程出现过拟合( overfitting)现象,一般做法是在正常的梯度项后额外增加一项,以对较大的参数值作出惩罚。最简单的罚函数是2函数(M/2)>∑,W,即所有权重参数的平方和的1/2乘上一个正则化系数入入在RBM中又称为权损失( weight-cost)。重要的是,惩罚项关于权重参数的梯度必须乘上学习率,否则.学习率的改变将导致优化的目标函数也发生改变。在RBM中,若使用L2罚函数,贝权损失系数的取值可以取介于001与0.0001之间的任意值。值得指出的是,权衰减策略只需应用于连接权重参数W上,可见层和隐层偏置不需使用,因为它们不人可能导致过拟合。并且在某些情况下,偏置的值还必须较大才行隐单元个数如果我们关心的主要目标是避免过拟合而不是计算复杂度,则可以先估算一下用个好的模型描述一个数据所需的比特数,月其乘上训练集容量。基于所得的数,选择比其低个数量级的值作为隐元个数。如果训练数据是高度冗氽的(比如数据集容量非常大),则可以使用更少些的隐元。以上讨论的是RBM中的一些常用的参数设置,针对一个实际问题,应使用什么类型的可见单元和隐单元,在其中如何加入稀疏性使得隐单元只在少数情况下处于激活状态等问题的讨论,可参见文[23,31]山国科技论文在线4RBM的评估算法对于一个已经学习得到或正在学习中的RBM,应通过何种指标评价其优劣呢?显然,最简单的指标就是该RBM在训练数据上的似然度C()=∑1logP(v(()。但是,C(0)的计算涉及到归一化常数(),而这个值是无法通过数学方法直接解析得到的,但我们又不可能枚举RBM的所有状态。因此,只能采用近似方法对RBM进行评估。4.1重构误差所谓“重构误差”( reconstruction error),就是以训练数据作为初始状态,根据RBM的分布进行一次 Gibbs采样后所获样本与原数据的差异(一般用一范数或二范数来评估)Error=0初始化误差for all y(),t∈{1,2,…,T}do%对每个训练样本y(进行以下计算h N P(v()%对隐层采样ⅴ~P(h%对可见层采样Error=Eror+‖v-v)‖%累计当前误差end forreturn上mOP%返回总误差算法2.重构误差的计算.重构误差能够在一定程度上反映RBM对训练数据的似然度,不过并不完全可靠[23。但总的来说,重构误差的计算十分简单,因此在实践中非常有用。4.2退火式重要性采样退火式重要性采样”( Annealed Importance Sampling,AIS)图2是目前比较主流的RBM评估方法。它的想法非常直接,就是利用蒙特卡岁方法估计RBM对数据的似然度。只不过没有使用MCMC,而是通过一种叫做“重要性采样”( Importance Sampling)[20的算法进行逼近。这种算法的优点在于:当目标分布十分陡峭时,不直接对其进行采样,而是引入另一个简单的分布,在这个简单的分布上采样。然后,利用采样所获样本和两个分布之间的关系对原分布上的均值进行估算。“重要性抽样”的基本思想如下:假设我们要计算某个分布P4(x)的归一化常数ZA,那么,我们可以引入另一个状态空间相同,但更容易采样的分布PB(x),并且事先知道它的归化常数zB。这时,只要能计算出zA/zB的值,我们就可以算出原分布的归一化常数ZA。假
- 2020-12-04下载
- 积分：1
Finite-Dimensional Vector Spaces - P. Halmos (Springer, 1987)
在学习代数学之余，值得一看的代数学书籍。里面介绍了更为丰富的代数学概念和结论。PREFACEMy purpose in this book is to treat linear transformations on finite-dimensional vector spaces by the methods of more general theories. Theidea is to emphasize the simple geometric notions common to many partsof mathematics and its applications, and to do so in a language that givesaway the trade secrets and tells the student what is in the back of the mindsof people proving theorems about integral equations and Hilbert spaces.The reader does not, however, have to share my prejudiced motivationExcept for an occasional reference to undergraduate mathematics the bookis self-contained and may be read by anyone who is trying to get a feelingfor the linear problems usually discussed in courses on matrix theory orhigher"algebra. The algebraic, coordinate-free methods do not lose powerand elegance by specialization to a finite number of dimensions, and theyare, in my belief, as elementary as the classical coordinatized treatmentI originally intended this book to contain a theorem if and only if aninfinite-dimensional generalization of it already exists, The temptingeasiness of some essentially finite-dimensional notions and results washowever, irresistible, and in the final result my initial intentions are justbarely visible. They are most clearly seen in the emphasis, throughout, ongeneralizable methods instead of sharpest possible results. The reader maysometimes see some obvious way of shortening the proofs i give In suchcases the chances are that the infinite-dimensional analogue of the shorterproof is either much longer or else non-existent.A preliminary edition of the book (Annals of Mathematics Studies,Number 7, first published by the Princeton University Press in 1942)hasbeen circulating for several years. In addition to some minor changes instyle and in order, the difference between the preceding version and thisone is that the latter contains the following new material:(1) a brief dis-cussion of fields, and, in the treatment of vector spaces with inner productsspecial attention to the real case.(2)a definition of determinants ininvariant terms, via the theory of multilinear forms. 3 ExercisesThe exercises(well over three hundred of them) constitute the mostsignificant addition; I hope that they will be found useful by both studentPREFACEand teacher. There are two things about them the reader should knowFirst, if an exercise is neither imperative "prove that.., )nor interrogtive("is it true that...?" )but merely declarative, then it is intendedas a challenge. For such exercises the reader is asked to discover if theassertion is true or false, prove it if true and construct a counterexample iffalse, and, most important of all, discuss such alterations of hypothesis andconclusion as will make the true ones false and the false ones true. Secondthe exercises, whatever their grammatical form, are not always placed 8oas to make their very position a hint to their solution. Frequently exer-cises are stated as soon as the statement makes sense, quite a bit beforemachinery for a quick solution has been developed. A reader who tries(even unsuccessfully) to solve such a"misplaced"exercise is likely to ap-preciate and to understand the subsequent developments much better forhis attempt. Having in mind possible future editions of the book, I askthe reader to let me know about errors in the exercises, and to suggest im-provements and additions. (Needless to say, the same goes for the text.)None of the theorems and only very few of the exercises are my discovery;most of them are known to most working mathematicians, and have beenknown for a long time. Although i do not give a detailed list of my sources,I am nevertheless deeply aware of my indebtedness to the books and papersfrom which I learned and to the friends and strangers who, before andafter the publication of the first version, gave me much valuable encourage-ment and criticism. Iam particularly grateful to three men: J. L. Dooband arlen Brown, who read the entire manuscript of the first and thesecond version, respectively, and made many useful suggestions, andJohn von Neumann, who was one of the originators of the modern spiritand methods that I have tried to present and whose teaching was theinspiration for this bookP、R.HCONTENTS的 FAPTERPAGRI SPACESI. Fields, 1; 2. Vector spaces, 3; 3. Examples, 4;4. Comments, 55. Linear dependence, 7; 6. Linear combinations. 9: 7. Bases, 108. Dimension, 13; 9. Isomorphism, 14; 10. Subspaces, 16; 11. Calculus of subspaces, 17; 12. Dimension of a subspace, 18; 13. Dualspaces, 20; 14. Brackets, 21; 15. Dual bases, 23; 16. Reflexivity, 24;17. Annihilators, 26; 18. Direct sums, 28: 19. Dimension of a directsum, 30; 20. Dual of a direct sum, 31; 21. Qguotient spaces, 33;22. Dimension of a quotient space, 34; 23. Bilinear forms, 3524. Tensor products, 38; 25. Product bases, 40 26. Permutations41; 27. Cycles,44; 28. Parity, 46; 29. Multilinear forms, 4830. Alternating formB, 50; 31. Alternating forms of maximal degree,52II. TRANSFORMATIONS32. Linear transformations, 55; 33. Transformations as vectors, 5634. Products, 58; 35. Polynomials, 59 36. Inverses, 61; 37. Mat-rices, 64; 38. Matrices of transformations, 67; 39. Invariance,7l;40. Reducibility, 72 41. Projections, 73 42. Combinations of pro-jections, 74; 43. Projections and invariance, 76; 44. Adjoints, 78;45. Adjoints of projections, 80; 46. Change of basis, 82 47. Similarity, 84; 48. Quotient transformations, 87; 49. Range and null-space, 88; 50. Rank and nullity, 90; 51. Transformations of rankone, 92 52. Tensor products of transformations, 95; 53. Determinants, 98 54. Proper values, 102; 55. Multiplicity, 104; 56. Triangular form, 106; 57. Nilpotence, 109; 58. Jordan form. 112III ORTHOGONALITY11859. Inner products, 118; 60. Complex inner products, 120; 61. Innerproduct spaces, 121; 62 Orthogonality, 122; 63. Completeness, 124;64. Schwarz e inequality, 125; 65. Complete orthonormal sets, 127;CONTENTS66. Projection theorem, 129; 67. Linear functionals, 130; 68. P aren, gBCHAPTERtheses versus brackets, 13169. Natural isomorphisms, 138;70. Self-adjoint transformations, 135: 71. Polarization, 13872. Positive transformations, 139; 73. Isometries, 142; 74. Changeof orthonormal basis, 144; 75. Perpendicular projections, 14676. Combinations of perpendicular projections, 148; 77. Com-plexification, 150; 78. Characterization of spectra, 158; 79. Spec-ptral theorem, 155; 80. normal transformations, 159; 81. Orthogonaltransformations, 162; 82. Functions of transformations, 16583. Polar decomposition, 169; 84. Commutativity, 171; 85. Self-adjoint transformations of rank one, 172IV. ANALYSIS....17586. Convergence of vectors, 175; 87. Norm, 176; 88. Expressions forthe norm, 178; 89. bounds of a self-adjoint transformation, 17990. Minimax principle, 181; 91. Convergence of linear transformations, 182 92. Ergodic theorem, 184 98. Power series, 186APPENDIX. HILBERT SPACERECOMMENDED READING, 195INDEX OF TERMS, 197INDEX OF SYMBOLS, 200CHAPTER ISPACES§L. FieldsIn what follows we shall have occasion to use various classes of numbers(such as the class of all real numbers or the class of all complex numbers)Because we should not at this early stage commit ourselves to any specificclass, we shall adopt the dodge of referring to numbers as scalars. Thereader will not lose anything essential if he consistently interprets scalarsas real numbers or as complex numbers in the examples that we shallstudy both classes will occur. To be specific(and also in order to operateat the proper level of generality) we proceed to list all the general factsabout scalars that we shall need to assume(A)To every pair, a and B, of scalars there corresponds a scalar a+called the sum of a and B, in such a way that(1) addition is commutative,a+β=β+a,(2)addition is associative, a+(8+y)=(a+B)+y(3 there exists a unique scalar o(called zero)such that a+0= a forevery scalar a, and(4)to every scalar a there corresponds a unique scalar -a such that十(0(B)To every pair, a and B, of scalars there corresponds a scalar aBcalled the product of a and B, in such a way that(1)multiplication is commutative, aB pa(2)multiplication is associative, a(Br)=(aB)Y,( )there exists a unique non-zero scalar 1 (called one)such that al afor every scalar a, and(4)to every non-zero scalar a there corresponds a unique scalar a-1or-such that aaSPACES(C)Multiplication is distributive with respect to addition, a(a+n)If addition and multiplication are defined within some set of objectsscalars) so that the conditions(A),B), and (c)are satisfied, then thatset(together with the given operations) is called a field. Thus, for examplethe set Q of all rational numbers(with the ordinary definitions of sumand product)is a field, and the same is true of the set of all real numberaand the set e of all complex numbersHHXERCISIS1. Almost all the laws of elementary arithmetic are consequences of the axiomsdefining a field. Prove, in particular, that if 5 is field and if a, and y belongto 5. then the following relations hold80+a=ab )Ifa+B=a+r, then p=yca+(B-a)=B (Here B-a=B+(a)(d)a0=0 c=0.(For clarity or emphasis we sometimes use the dot to indi-cate multiplication.()(-a)(-p)(g).If aB=0, then either a=0 or B=0(or both).2.(a)Is the set of all positive integers a field? (In familiar systems, such as theintegers, we shall almost always use the ordinary operations of addition and multi-lication. On the rare occasions when we depart from this convention, we shallgive ample warningAs for "positive, "by that word we mean, here and elsewherein this book, "greater than or equal to zero If 0 is to be excluded, we shall say"strictly positive(b)What about the set of all integers?(c) Can the answers to these questiong be changed by re-defining addition ormultiplication (or both)?3. Let m be an integer, m2 2, and let Zm be the set of all positive integers lessthan m, zm=10, 1, .. m-1). If a and B are in Zmy let a +p be the leastpositive remainder obtained by dividing the(ordinary) sum of a and B by m, andproduct of a and B by m.(Example: if m= 12, then 3+11=2 and 3. 11=9)a) Prove that i is a field if and only if m is a prime.(b What is -1 in Z5?(c) What is囊izr?4. The example of Z, (where p is a prime)shows that not quite all the laws ofelementary arithmetic hold in fields; in Z2, for instance, 1 +1 =0. Prove thatif is a field, then either the result of repeatedly adding 1 to itself is always dif-ferent from 0, or else the first time that it is equal to0 occurs when the numberof summands is a prime. (The characteristic of the field s is defined to be 0 in thefirst case and the crucial prime in the second)SEC. 2VECTOR SPACES35. Let Q(v2)be the set of all real numbers of the form a+Bv2, wherea and B are rational.(a)Ie(√2) a field?(b )What if a and B are required to be integer?6.(a)Does the set of all polynomials with integer coefficients form a feld?(b)What if the coeficients are allowed to be real numbers?7: Let g be the set of all(ordered) pairs(a, b)of real numbers(a) If addition and multiplication are defined by(a月)+(,6)=(a+y,B+6)and(a,B)(Y,8)=(ary,B6),does s become a field?(b )If addition and multiplication are defined by(α,月)+⑦,b)=(a+%,B+6)daB)(,b)=(ay-6a6+的y),is g a field then?(c)What happens (in both the preceding cases)if we consider ordered pairs ofcomplex numbers instead?§2. Vector spaceWe come now to the basic concept of this book. For the definitionthat follows we assume that we are given a particular field s; the scalarsto be used are to be elements of gDEFINITION. A vector space is a set o of elements called vectors satisfyingthe following axiomsQ (A)To every pair, a and g, of vectors in u there corresponds vectora t y, called the aum of a and y, in such a way that(1)& ddition is commutative,x十y=y十a(2)addition is associative, t+(y+2)=(+y)+a(3)there exists in V a unique vector 0(called the origin) such thata t0=s for every vector and(4)to every vector r in U there corresponds a unique vector -rthat c+(-x)=o(B)To every pair, a and E, where a is a scalar and a is a vector in u,there corresponds a vector at in 0, called the product of a and a, in sucha way that(1)multiplication by scalars is associative, a(Bx)=aB)=, and(2 lz a s for every vector xSPACESSFC B(C)(1)Multiplication by scalars is distributive with respect to vectorddition, a(+y=a+ ag, and2)multiplication by vectors is distributive with respect to scalar ad-dition, (a B )r s ac+ Bc.These axioms are not claimed to be logically independent; they aremerely a convenient characterization of the objects we wish to study. Therelation between a vector space V and the underlying field s is usuallydescribed by saying that v is a vector space over 5. If S is the field Rof real number, u is called a real vector space; similarly if s is Q or if gise, we speak of rational vector spaces or complex vector space§3. ExamplesBefore discussing the implications of the axioms, we give some examplesWe shall refer to these examples over and over again, and we shall use thenotation established here throughout the rest of our work.(1) Let e(= e)be the set of all complex numbers; if we interpretr+y and az as ordinary complex numerical addition and multiplicatione becomes a complex vector space2)Let o be the set of all polynomials, with complex coeficients, in avariable t. To make into a complex vector space, we interpret vectoraddition and scalar multiplication as the ordinary addition of two poly-nomials and the multiplication of a polynomial by a complex numberthe origin in o is the polynomial identically zeroExample(1)is too simple and example (2)is too complicated to betypical of the main contents of this book. We give now another exampleof complex vector spaces which(as we shall see later)is general enough forall our purposes.3)Let en,n= 1, 2,. be the set of all n-tuples of complex numbers.Ix=(1,…,轨)andy=(m1,…,n) are elements of e, we write,,bdefinitionz+y=〔1+叽,…十物m)0=(0,…,0),-inIt is easy to verify that all parts of our axioms(a),(B), and (C),52, aresatisfied, so that en is a complex vector space; it will be called n-dimenaionalcomplex coordinate space
- 2020-12-05下载
- 积分：1
