Google word2vec算法 数学原理
文档是 word2vec 算法 数学原理详解。word2vec是google的一个开源工具，能够仅仅根据输入的词的集合计算出词与词直接的距离，既然距离知道了自然也就能聚类了，而且这个工具本身就自带了聚类功能，很是强大。32预备知识本节介绍word2v中将用到的一些重要知识点,包括 sigmoid函数、 Bccs公式和Huffman编码等821 sigmoid函数sigmoid函数是神经网络中常用的激活函数之一,其定义为1+e该函数的定义域为(-∞,+∞),值域为(0,1).图1给出了 sigmoid函数的图像0.56图1 sigmoid函数的图像sigmoid函数的导函数具有以下形式(x)=0(x)1-0(x)由此易得,函数loga(x)和log(1-0(x)的导函数分别为log a(a)-1 a(a),log(1 o(a))l-a(a),(2.1)公式(2.1)在后面的推导中将用到32.2逻辑回归生活中经常会碰到二分类问题,例如,某封电子邮件是否为垃圾邮件,某个客户是否为潜在客户,某次在线交易是否存在欺诈行为,等等设{(x;)}温1为一个二分类问题的样本数据,其中x∈Rn,∈{0,1},当v=1时称相应的样本为正例当v=0时称相应的样本为负例利用 sigmoid函数,对于任意样本x=(x1,x2,…,xn),可将二分类问题的 hypothesis函数写成h(x)=o(6o+b1x1+62+…+bnxn)其中θ=(0,61,…,On)为待定参数.为了符号上简化起见,引入x0=1将x扩展为(x0,x1,x2,……,xn),且在不引起混淆的情况下仍将其记为ⅹ.于是,he可简写为取阀值T=0.5,则二分类的判别公式为ho(x)≥0.5:X)=0,ha(x)6),可分别用000001、010、011、100、101对“A,E,R,T,F,D”进行编码发送,当对方接收报文时再按照三位一分进行译码显然编码的长度取决报文中不同字符的个数.若报文中可能出现26个不同字符,则固定编码长度为5(25=32>26).然而,传送报文时总是希望总长度尽可能短.在实际应用中各个字符的出现频度或使用次数是不相同的,如A、B、C的使用颗率远远高于X、Y、Z,自然会想到设计编码时,让使用频率高的用短码,使用频率低的用长码,以优化整个报文编码为使不等长编码为前缀编码(即要求一个字符的编码不能是另一个字符编码的前缀),可用字符集中的每个字符作为叶子结点生成一棵编码二叉树,为了获得传送报文的最短长度,可将每个字符的岀现频率作为字符结点的权值赋于该结点上,显然字使用频率越小权值起小,权值越小叶子就越靠下,于是频率小编码长,频率高编码短,这样就保证了此树的最小带权路径长度,效果上就是传送报文的最短长度.因此,求传送报文的最短长度问题转化为求由字符集中的所有字符作为叶子结点,由字符出现频率作为其权值所产生的 Huffman树的问题.利用 Huffman树设计的二进制前缀编码,称为 Huffman编码,它既能满足前缀编码的条件,又能保证报文编码总长最短本文将介绍的word2ve工具中也将用到 Huffman编码,它把训练语料中的词当成叶子结点,其在语料中岀现的次数当作权值,通过构造相应的 Huffman树来对每一个词进行Huffman编码图3给岀了例2.1中六个词的 Huffman编码,其中约定(词频较大的)左孩子结点编码为1,(词频较小的)右孩子编码为0.这样一来,“我”、“喜欢”、“观看”、“巴西”、“足球”、“世界杯”这六个词的 Huffman编码分别为0,111,110,101,1001和100000欢观有巴西足球图3 Huffman编码示意图注意,到目前为止关于 Huffman树和 Huffman编码,有两个约定:(1)将权值大的结点作为左孩子结点,权值小的作为右孩子结点;(②)左孩子结点编码为1,右孩子结点编码为0.在word2vee源码中将权值较大的孩子结点编码为1,较小的孩子结点编码为θ.为亐上述约定统一起见,下文中提到的“左孩子结点”都是指权值较大的孩子结点3背景知识word2vec是用来生成词向量的工具,而词向量与语言模型有着密切的关系,为此,不妨先来了解一些语言模型方面的知识83.1统计语言模型当今的互联网迅猛发展,每天都在产生大量的文本、图片、语音和视频数据,要对这些数据进行处理并从中挖掘出有价值的信息,离不开自然语言处理( Nature Language processingNIP)技术,其中统计语言模型( Statistical language model)就是很重要的一环,它是所有NLP的基础,被广泛应用于语音识别、机器翻译、分词、词性标注和信息检索等任务例3.1在语音识别亲统中,对于给定的语音段Voie,需要找到一个使概率p(Tcrt| Voice最大的文本段Tert.利用 Bayes公式,有P(Teact Voice)p(VoiceTert)p(Text)P(Veonce其中p( Voice Teat)为声学模型,而p(Tert)为语言模型(l8])简单地说,统计语言模型是用来计算一个句子的概率的概率模型,它通常基于一个语料库来构建那什么叫做一个句子的概率呢?假设W=m1:=(n1,w2,…,tr)表示由T个词1,2,…,ur按顺序构成的一个句子,则n,U2,…,wr的联合概率p(W)=p(u1)=p(u1,u2,…,r)就是这个句子的概率.利用 Baves公式,上式可以被链式地分解为1)=p(u1)·p(u2l1)·p(vai)…p(ur1-)3.1其中的(条件)概率p(1),p(U2mn1),p(u3),…,p(urln1-1)就是语言模型的参数,若这些参数巳经全部算得,那么给定一个句子1,就可以很快地算出相应的p(1)了看起来妤像很简单,是吧?但是,具体实现起来还是有点麻烦.例如,先来看看模型参数的个数.刚才是考虑一个给定的长度为T的句子,就需要计算T个参数.不妨假设语料库对应词典D的大小(即词汇量)为N,那么,如果考虑长度为T的任意句子,理论上就有N种可能,而每种可能都要计算T个参数,总共就需要计算TN个参数.当然,这里只是简单估算,并没有考虑重复参数,但这个量级还是有蛮吓人.此外,这些概率计算好后,还得保存下来,因此,存储这些信息也需要很大的內存开销此外,这些参数如何计算呢?常见的方法有 II-gram模型、决策树、最大熵模型、最大熵马尔科夫模型、条件随杋场、神经网络等方法.本文只讨论n-gram模型和神经网络两种方法.首先来看看n-gram模型32n-gram模型考虑pko4-)(k>1)的近似计算.利用 Baves公式,有p(wr wi)P(uP(w根据大数定理,当语料库足够大时,p(k4-1)可近似地表示为P(wwi)count(wi)(3.2)count(a其中 count(u4)和 count-)分别表示词串t和v-在语料中出现的次数,可想而知,当k很大时, count(o4)和 count(4-1)的统计将会多么耗时从公式(3.1)可以看出:一个词出现的慨率与它前面的所有词都相关.如果假定一个词出现的概率只与它前面固定数目的词相关呢?这就是n-gran模型的基本思想,它作了一个n-1阶的 Markov假设,认为一个词出现的概率就只与它前面的n-1个词相关,即-1)≈p(kk-1+),于是,(3.2)就变成了p(wxJuk-)count(n+1countri(3.3以〃=2为例,就有p(uk4-1)≈count(k-1, Wk)count(Wk-1)这样一简化,不仅使得单个参数的统计变得更容易(统计时需要匹配的词串更短),也使得参数的总数变少了那么, n-gran中的参数n取多大比较合适呢?一般来说,n的选取需要同时考虑计算复杂度和模型效果两个因素表1模型参数数量与n的关系模型参数数量1( ingram)2×1052(bigram)4×10103( trigram)8×10154(4grm)16×10在计算复杂度方面,表1给出了n-gram模型中模型参数数量随着n的逐渐增大而变化的情况,其中假定词典大小N=2000(汉语的词汇量大致是这个量级).事实上,模型参数的量级是N的指数函数(O(N"),显然n不能取得太大,实际应用中最多的是采用n=3的三元模型在模型效果方面,理论上是π越大,效果越奷.现如今,互联网的海量数据以及机器性能的提升使得计算更高阶的语言模型(如n>10)成为可能,但需要注意的是,当n大到一定程度时,模型效果的提升幅度会变小.例如,当n从1到2,再从2到3时,模型的效果上升显著,而从3到4时,效果的提升就不显著了(具体可参考吴军在《数学之美》中的相关章节).事实上,这里还涉及到一个可靠性和可区别性的问题,参数越多,可区别性越好,但同时单个参数的实例变少从而降低了可靠性,因此需要在可靠性和可区别性之间进行折中另外, n-gran模型中还有一个叫做平滑化的重要环节.回到公式(3.3),考虑两个问题:若 count(uk-n+1)=0,能否认为p(kln1-1)就等于0呢?若 count(kn+)= count(uk-+1,能否认为p(uur-)就等于1呢?显然不能!但这是一个无法回避的问题,哪怕你的语料库有多么大.平滑化技术就是用来处理这个问题的,这里不展开讨论,具体可参考[11总结起来,n-gram模型是这样一种模型,其主要工作是在语料中统计各种词串岀现的次数以及平滑化处理.概率值计算好之后就存储起来,下次需要计算一个句子的概率时,只需找到相关的概率参数,将它们连乘起来就好了然而,在机器学习领域有一种通用的招数是这样的:对所考虑的问题建模后先为其构造一个目标函数,然后对这个目标函数进行优化,从而求得一组最优的参数,最后利用这组最优参数对应的模型来进行预測对于统计语言模型而言,利用最大似然,可把目标函数设为plwlConteat(w))∈C其中C表示语料( Corpus), Context(u)表示词U的上下文( Context),即周边的词的集合.当 Context(u)为空时,就取p( Context(w)=p(u).特别地,对于前面介绍的 n-gran模型,就有 Context(mn)=2-n+1注3.1语料￠和词典仍的区别:词典仍是从语料￠中抽取岀来的,不存在重复的词;而语料C是指所有的文本內容,包括重复的词当然,实际应用中常采用最大对数似然,即把目标函数设为∑ logp(u( ontext(o)(3.4)然后对这个函数进行最大化从(3.4)可见,概率p( CONtex()已被视为关于和 Context()的函数,即p(w Context(w))= F(w, Conteact(w), 0)
- 2020-06-14下载
- 积分：1
abaqus script reference manual
ABAQUS脚本参考文件，适合二次开发，ABAQUS各个模块的语句参考ABAQUS Offices and RepresentativesABAQUS, IncABAQUS Europe BVRising Sun millsGaetano Martinolaan 95166 Valley streetP.O. BOx 1637Providence RI 02909-24996201 BP MaastrichtTel:+14012764400The netherlandsFax:+14012764408Tel:+31433566906E-mail:support(@abaqus.comFax:+31433566908http://www.abaqus.comE-mail:info.europe@abaqus.comSales, Support, and ServicesUNITED STATESABAQUS Central, IncABAQUS East LLC1440 Innovation place300 Centerville road. Suite 209WWest Lafayette, IN 47906-1000Warwick. RI 02886-0201Iel:+17654971373Tel:+14017393637Fax:+17654974444Fax:+14017393302E-mail:support(@abaqusCentral.comE-mail: support@ abaqus EastcomAbAqUS Erie. incABAQUS Great Lakes, Inc3601 Green Road. Suite 31614500 Sheldon road. Suite 160Beachwood. OH 44122Plymouth, MI 48170-2408Tel:+12163781070Tel:+17344510217ax:+1216378107F7344510458E-mail:support(@abaqusErie.comE-mail:support(@abaqusGreatlakes.comABAQUS South, IncABAQUS West, Inc3700 Forums Drive Suite 10139221 Paseo Padre Parkway Suite FFlower Mound. TX 75028remont, CA 94538-1611Tel:+12145131600Tel:+15107945891Fax:+12145131700Fax:+15107941194F-mail:support(@abaqusSouth.comF-mail:supportabaqusWest.comARGENTINAAUSTRALIAKB Engineering S.R. LWorley Advanced analysisFlorida 274-Oficina 35Level 17. 300 Flinders Street1005 Buenos airesMelbourne Vic 3000ArgentinaTel:+61386125132Iel:+541143269176/7542Fax:+61392050573Fax:+541143262424E-mail:abaqus@worley.com.auE-mail: sanchezsarmientoarnet com arAUSTRIABENELUXABAQUS AuStria GmbHABAQUS Benelux EⅤZinckgasse 20-22/2/13Huizermaatweg 576A-1150 Vienna1276 LN HuizenaustriaThe NetherlandTel:+4319291625-0Tcl:+31355258424Fax:+4319291625-20Fax:+31355244257E-mail: support @abaqus.atE-mail: support(abaqus. nlCHINACZECH REPUBLICABAQUS ChinaSynerma s.r.oRoom a-2703 Eagle plazaHuntirov 58No. 26 Xiao yun rd468 22 SkuhrovBeijing, 100016Czech republicP.R. ChinaTel:+420603145769Tel:+860184580366Fax:+420603181944Fax:+860184580360E-mail: abaqus asynerma czE-mail:abaqus(@abaqus.com.cnFRANCEGERMANY(Aachen)ABAQUS France SasABAQUS Deutschland gmbH7 rue jean MMermozBat. aTheaterstrabe 30-3278000 VersaillesD-52062 AachenIel:+330139241540lel:+49241474010Fax:+330139241545Fax:+492414090963E-mail: support (@abaqus. frEmail: info(abaqus.deGERMANY(Munich)INDIA(Chennai)ABAOUS Deutschland gmbhABAQUS Engineering India(Pvt. LtdSendlinger-Tor -platz 83M. Prince arcadeD-80336 Munchen22-A Cathedral roadTel:+498959991768Chennai. 600086Fax:+498959991767Tel:+914428114624E-mail: info(abaqus.deFax:+914428115087E-mail: abaqus(@abaqus.coinITALYJAPAN(Tokyo)ABAQUS Italia s r IABAQUs. IncVia domodossola. 173rd Floor, Akasaka Nihon Building20145 Milano(MD)5-24, Akasaka 9-chome, Minato-kuTel:+390239211211Tokyo,107-0052Fax:+390231800064Tcl:+81354745817E-mail: info(@abaqus.itFax:+81354745818E-mail: tokyo(abaqus. jpJAPAN (Osaka)KOREAABAQUS, IncABAQUS Korea. Inc9th Floor, Higobashi Watanabe BuildingSuite 306, Sambo Building6-10. Edobori 1-chome Nishi-ku13-2 Yoido-Dong, Youngdeungpo-kuOsaka.550-0002Seoul.150-010Tel:+81648035020Tel:+8227856707Fax:+81648035021Fax:+8227856709E-mail: osaka @abaqus. jpE-mail: info(@abaqus. co kIMALAYSIANEW ZEALANDWorley advanced AnalysisMatrix Applied Computing Ltd19th Floor, Empire TowerP.O. Box 56-316. AucklandCity square centreCourier: Unit 2-5, 72 Dominion road Mt eden182 Jalan Tun razakauckland50400 Kuala LumpurTel:+6496231223Tel:+6032161226Fax:+6496231134Fax:+60321614266E-mail: abaqus-tech(amatrix co nzE-mail:abaqusmy@worley.com.auPOLANDRUSSIA, BELARUS UKRAINEBudSoft sp. z0.0TESIS Ltd61-807 PoznanOffice701-703,Sw Marcin 58/64l8. Unnatoy strIe:+486l8508466127083 Moscow. RussiaFax:+486l8508467Tel:+7095212-44-22E-mail:budsoft@budsoft.com.plFax:+7095212-42-62E-mail:infoatesis.com.ruSINGAPORESOUTH AFRICAWorley Advanced AnalysisFinite Element analysis Services (Ply)Ltd491B River valley roadUnit 4, The Waverley#09-01Ⅴ alley pointWyecroft roadSingapore, 248373Mowbray 7700Tel:+6567358444Tel:+27214487608Fax:+6567357444Fax:+27214487679E-mail:abaqus.sg@worley.com.auE-mail: feas(afeas co zaSPAINSWEDENPrincipia Ingenieros Consultores, SAABAQUS Scandinaviavelazquez, 94FEM-Tech ABE-28006 MadridPilgatan &cIel:+34912091482SE-72130 VasterasFax:+34915751026Tcl:+4621126410E-mail: abaqus(principia. esFax:+4621181244E-mail: abaqus abaqus.seTAIWANTHAILANDAPICWorley Advanced Analysis11F, No. 71, Sung Chiang road333 Lao Peng Nguan l BuildingTaipei. 1042820th Floor Unit bIel:+8860225083066Soi ChaypuangFax:+8860225077185Vibhavadi-Rangsit roadE-mail:apicaapic.com.twLadyao, JatujakBangkok 10900Tel:+6626893000Fax:+6626188109E-mail:abaqusth(@worley.com.auTURKEYUNITED KINGDOM(Cheshire)A-Ztech ltdABAQUS UK LtdPerdemsac Plaza, Teknoloji eviThe Genesis CentreBayar Cad, Gulbahar Sok., No: 17Science park south BirchwoodKozyataWarrington Cheshire WA3 7BH34742 IstanbulTel:+441925810166TURKIYEFax:+441925810178Tel:+902163618850E-mail: hotline abaqus. co ukFax:+90216361885E-mail:info(aa-ztech.com.tiSales OnlUNITED STATESABAOUS EaSt llc. mid-Atlantic officeABAOUS SOUth. Inc. Southeast office1 1 4 Zachary Court484 Broadstone wayForest hilL. MD 21050Acworth.GA30101lel:+14104208587lel:+17707950960Fax:+14104208908Fax:+17707957614E-mail:support(@abaquseast.comE-mail:support(@AbaqusSouth.comABAQUS West, Inc, Southern CA and Az Office ABAQUS West, Inc, Rocky Mountains Office1100 Irvine boulevard #2486910 Cordwood CtTustin Ca 92780Boulder co 80301Iel:+17147315895Tel:+13036645444Fax:+17142427002Fax:+1303200948lE-mail:inFo(aabaqusWest.comE-mail:Into(aabaqusWest.comFINLANDINDIA (Pune)ABAQUS Finland OyABAQUS Engineering Analysis Solutions(Pvt. LtdTekniikantie 12C-9. 3rd FloorFIN-02150 EspooBramha Estate, Kondwa roadTel:+358925172973Pune-411040Fax:+358925172200Tel:+912030913739F-mail: abaqus(@abaqus.seF-mail: abaqus( abaqus. co inUNITED KINGDOM(Kent)ABAOUS UK LtdGreat hollander business centre. Unit aMill Lane. UnderriverSevenoaks, Kent tN15 OSQTel:+441732834930Fax:+441732834720E-mail: hotline(abaqus. co ukPrefaceThis section lists various resources that are available for help with using ABAQUS, including technicalengineering and systems support, training seminars, and documentationSupportABAQUS, Inc, offers both technical engineering support and systems support for ABAQUS. Technicalengineering and systems support are provided through the nearest local support office. You can contactour offices by telephone, fax, electronic mail, the ABAQUs web-based support system, or regular mailInformation on how to contact each office is listed in the front of each ABAQUS manual. The ABAQUSOnline Support System(AOSS) is accessible through the MY ABAQUS section of the ABAQUS HomePage(www.abaqus.com).Whencontactingyourlocalsupportofficespleasespecifywhetheryouwouldlike technical engineering support (you have encountered problems performing an ABAQUS analysis orcreating a model in ABAQUS) or systems support (ABAQUS will not install correctly, licensing does notwork correctly, or other hardware-related issues have arisenThe ABAQUS Online Support System has a knowledge database of ABAQUS Answers. The ABAQUSAns wers are solutions to questions that we have had to answer or guidelines on how to use ABAQUS. Wewelcome any suggestions for improvements to the support program or documentation. We will ensure thatany enhancement requests you make are considered for future releases. If you wish to file a complaint aboutthe service or products provided by aBaqUs, refer to the aBaQus Home PageTechnical engineering supportABAQUS technical support engineers can assist in clarifying ABAQUS features and checking errors by givingboth general information on using ABAQUS and information on its application to specific analyses. If youhave concerns about an analysis, we suggest that you contact us at an early stage, since it is usually easier tosolve problems at the beginning of a project rather than trying to correct an analysis at the end.Please have the following information ready before calling the technical engineering support hotline, andinclude it in any written contacts:Your site identifier, which can be obtained by typing abaqus whereami at your system prompt(or byselecting Help-On Version from the main menu bar in ABAQUS/CAE or ABAQUS/Viewer)· The version of△ BAQUS that are you usingThe version numbers for ABaQUS/Standard and ABAQUS/Explicit are given at the top of the data(. dat)fileThe version numbers for ABAQUS/CAE and ABAQUS/Viewer can be found by selectingHelp-On Version from the main menu bar.The version numbers for the abaqus Interface for MoldFLOW and the abaQus Interface forMSC ADAMS are output to the screenThe version number for ABAQUS for CAtIa V5 can be found by selecting Help-AboutABAQUS for CaTiA V5 from the main menu bar in either of the aBaqus for CAtIA V5workbenchesThe type of computer on which you are running ABAQUSThe symptoms of any problems, including the exact error messages, if anyWorkarounds or tests that you have already triedWhen calling for support about a specific problem, any available abaqus output files may be helpful inanswering questions that the support engineer may ask youThe support engineer will try to diagnose your problem from the model description and a description ofthe difficulties you are having. The support engineer may need model sketches, which can be sent via fae-mail, or regular mail. Plots of the final results or the results near the point that the analysis terminated mayalso be needed to understand what may have caused the problemIf the support engineer cannot diagnose your problem from this information, you may be asked tosupply the input data. The data can be attached to a support incident in the aBaQUS Online SupportSystem. It may also be sent by means of e-mail, tape, disk, or ftp Please check the ABAQUS Home Page(http://www.abaquscom)forthemediaformatsthatarecurrentlyacceptedAll support incidents are tracked in the abaQUs Online Support System. This enables you( as well asthe support engineer) to monitor the progress of a particular problem and to check that we are resolving supportissues efficiently. To use the ABAQUS Online Support System, you need to register with the system. Visit theMY ABAQUS Section of the ABAQUS Home Page for instructions on how to register. If you are contactingus by means outside the aoss to discuss an existing support problem and you know the incident number,please mention it so that we can consult the database to see what the latest action has been and, thus, giveyou more efficient support as well as avoid duplication of effort. In addition, please give the receptionist thesupport engineers name if contacting us via telephone or include it at the top of any e-mail correspondenceSystems supportABAQUS SyStems support engineers can help you resolve issues related to the installation and running ofABAQUS, including licensing difficulties, that are not covered by technical engineering supportYou should install ABaQUS by carefully following the instructions in the ABAQUS Installation andLicensing Guide If you are able to complete the installation, please make sure that the product verificationprocedure was run successfully at the end of the installation procedure. Successful verification for licensedproducts would indicate that you can run these products on your computer; unsuccessful verification forlicensed products indicates problems with the installation or licensing (or both ) If you encounter problemswith the installation, licensing, or verification, first review the instructions in the abaQus Installation andLicensing Guide to ensure that they have been followed correctly. If this does not resolve the problems,consult the ABAQUS Answers database in the ABaQUS Online Support System for information about knowninstallation problems. If this does not address your situation, please create an incident in the AOSS anddescribe your problem, including the output from abaqus info=support. If you call, mail, e-mail, or faxus about a problem(instead of using the Aoss), please provide the output from abaqus info=support. Itis important that you provide as much information as possible about your problem: error messages from anaborted analysis, output from the abaqus info=support command, etc
- 2020-12-10下载
- 积分：1
