| 引用本文: | 郭伟立,魏海坤,赵军圣,张侃健.多层感知器模型互反奇异性区域学习动态的理论分析[J].控制理论与应用,2014,31(2):140~147.[点击复制] |
| GUO Wei-li,WEI Hai-kun,ZHAO Jun-sheng,ZHANG Kan-jian.Theoretical analysis of learning dynamics near the opposite singularities in multilayer perceptrons[J].Control Theory and Technology,2014,31(2):140~147.[点击复制] |
|
| 多层感知器模型互反奇异性区域学习动态的理论分析 |
| Theoretical analysis of learning dynamics near the opposite singularities in multilayer perceptrons |
| 摘要点击 2482 全文点击 1891 投稿时间：2013-06-18 修订日期：2013-10-07 |
| 查看全文 查看/发表评论 下载PDF阅读器 |
| DOI编号 10.7641/CTA.2014.30600 |
| 2014,31(2):140-147 |
| 中文关键词 多层感知器 神经网络 学习动态 奇异性 互反 |
| 英文关键词 multilayer perceptrons neural networks learning dynamics singularity opposite |
| 基金项目 国家自然科学基金重大项目资助项目(11190015); 高等学校博士学科点专项科研基金资助项目(20100092110020); 国家自然科学基金资助项目(61374006). |
|
| 中文摘要 |
| 多层感知器神经网络(MLPs)的学习过程经常发生一些奇异性行为, 容易陷入平坦区, 这都和MLPs的参数
空间中存在的奇异性区域有直接关系. 当MLPs的两个隐节点的权值接近互反时, 置换对称性会导致学习困难. 对
MLPs的互反奇异性区域附近的学习动态进行分析. 本文首先得到了平均学习方程的解析表达式, 然后给出了互反
奇异性区域附近的理论学习轨迹, 并通过数值方法得到了其附近的实际学习轨迹. 通过仿真实验, 分别观察了MLPs
的平均学习动态, 批处理学习动态和在线学习动态, 并进行了比较分析. |
| 英文摘要 |
| Owing to the existence of singularities in the parameter space, multilayer perceptrons (MLPs) may behave
extremely slowly in learning or even be trapped in plateaus. When weights of two hidden units are nearly mutually opposite,
the learning process will encounter difficulties because of the permutation symmetry. We investigate the learning dynamics
of MLPs near opposite singularities, and derive the analytical expressions for averaged learning equations. Then, we obtain
the theoretical learning trajectories near the opposite singularities. Furthermore, real learning trajectories near the opposite
singularities are also calculated by using numerical methods. In simulations, we study the averaged learning dynamics, the
batch mode learning dynamics and the online learning dynamics, respectively. |
|
|
|
|
|