# 1. 訓練誤差與測試誤差

Remp(f^)=1Ni=1NL(yi,f^(xi))Remp(f^)=1N∑i=1NL(yi,f^(xi))

etest(f^)=1Ni=1NL(yi,f^(xi))etest(f^)=1N′∑i=1NL(yi,f^(xi))

# 2. 過擬合與模型選擇

T={(x1,y1),(x2,y2),,(xn,yn)}T={(x1,y1),(x2,y2),…,(xn,yn)}

fM(x,w)=w0+w1x+w2x2++wMxM=j=0MwjxjfM(x,w)=w0+w1x+w2x2+⋯+wMxM=∑j=0Mwjxj

L(w)=12i=1N(f(xi,w)yi)2L(w)=12∑i=1N(f(xi,w)−yi)2

import numpy as npimport matplotlib.pyplot as pltimport randomx = np.linspace(0,1,10)y = np.sin(2*np.pi*x)for i in range(0,10):        y[i] = y[i] + random.uniform(-0.4,0.4)p = np.polyfit(x,y,9)t = np.linspace(0,1.0,100)plt.plot(x,y,'o')plt.plot(t,np.sin(np.pi*2*t),label='$y=sin(x)$');plt.plot(t,np.polyval(p,t),label='$y = \sum_{i=0}^Mw_ix_i,M=9,x_0=0$');plt.legend()plt.show()

# 3. 正則化與交叉驗證

## 3.1 正則化

L(w)=1Ni=1N(f(xi;w)yi)2+λ2||w||2L(w)=1N∑i=1N(f(xi;w)−yi)2+λ2||w||2

L(w)=1Ni=1N(f(xi;w)yi)2+λ||w||1L(w)=1N∑i=1N(f(xi;w)−yi)2+λ||w||1

## 3.2 交叉驗證

### 3. 留一交叉驗證

S折交叉驗證的特征情形是S=NS=N，稱為留一交叉驗證，往往在數據缺乏的情況下使用。這里，N是給定數據集的容量。

# 4. 泛化能力

Rexp(f^)=EP[L(Y,f^(X))]=X×YL(y,f^(x))P(x,y)dxdyRexp(f^)=EP[L(Y,f^(X))]=∫X×YL(y,f^(x))P(x,y)dxdy

R(f)R^(f)+ε(d,N,δ)R(f)≤R^(f)+ε(d,N,δ)