基于遗传算法（GA）的深度卷积神经网络超参数搜索--在凯斯西储轴承诊断数据集上的测试（5）

• 1. 遗传算法整体框架
• 2. 超参数的编码
• 3. 搜索结果可视化
• 参考文献

遗传算法框架主要包括以下步骤：初始化-编码-交叉/变异（生成子代）- 子代评价（优胜劣汰）

def select(pop, fitness):
    '''
    根据适应度轮盘赌
    :param pop:
    :param fitness:是一个适应度列表，p=fitness / fitness.sum()可描述根据适应度好坏应该出现的概率
    :return: 返回的是 一个按概率应该出现的适应度好准确率高的二元数组[LR，opt]
    '''
    idx = np.random.choice(np.arange(POP_SIZE), size=POP_SIZE, replace=True, p=fitness / fitness.sum())
    return pop[idx]

def crossover(parent, pop, fitness, fitness_thresh):
    if fitness < fitness_thresh:
        if np.random.rand() < CROSS_RATE:
            i_ = np.random.randint(0, POP_SIZE, size=1)  # 和父代中一个随机的数组做交换[learningrate, opt]
            cross_points = np.random.randint(0, 2, size=DNA_SIZE).astype(np.bool)
            parent[cross_points] = pop[i_, cross_points]  # pop是 combination.copy() # 复制了这个父代种群二维数组【【LR1，OPt1】，【LR2，Opt2】,....】
    return parent

def mutate(child, fitness, fitness_thresh):
    if fitness < fitness_thresh:
        for point in range(DNA_SIZE):  # point = 0,1
            if np.random.rand() < MUTATION_RATE:  # 有MUTATION_RATE的变异机率
                if point == 0:
                    child[point] = np.random.randint(0, 5)  # 0的话学习率变异，变成另外当中一个 # randint(0, 4) 决定了一个基因的的子类有多少种
                else:
                    child[point] = np.random.randint(0, 5)  # 1的话优化器变异，...
    return child

combination = np.zeros(shape=(POP_SIZE, DNA_SIZE), dtype=np.int32)

for i in range(POP_SIZE):
    combination[i][0] = np.random.randint(0, 5)
    combination[i][1] = np.random.randint(0, 5)

for generation in range(N_GENERATIONS):
    fitness = np.zeros(POP_SIZE, )
    for i in range(POP_SIZE):
        fitness[i] = get_fitness(combination[i]) # 对第i代的种群个体计算适应度
        print('Gen:', generation, '| LearningRate: {}'.format(LearningRate[combination[i][0]]), '| Optimizer: {}'.format(Optimizers[combination[i][1]]), '| fitness: {}'.format(fitness[i]))
    best_idx = np.argmax(fitness)
    third_idx=fitness.argsort()[::-1][1]
    print('Gen:', generation, '| best fit: %.2f' % fitness[best_idx], '| LearningRate: {}'.format(LearningRate[combination[best_idx][0]]), '| Optimizer: {}'.format(Optimizers[combination[best_idx][1]]))

    combination = select(combination, fitness)  # 根据 select函数的轮盘赌法则，高适应度，大概率出现的二元数组[【LR，OPt】,[lR,opt],...]可重复的[lr,opt]
    combination_copy = combination.copy()  # 复制了这个父代种群二维数组【【LR1，OPt1】，【LR2，Opt2】,....】
    for i, parent in enumerate(combination):
        child = crossover(parent, combination_copy, fitness[i], fitness[third_idx])
        child = mutate(child, fitness[i], fitness[third_idx])
        parent = child

模型训练过程可视化结果

只做简单的框架测试，搜索空间很小

LearningRate = [0.001, 0.005, 0.01, 0.05, 0.08]
Optimizers = ['sgd', 'adam', 'rmsprop', 'adadelta', 'adagrad']

由于没清除matplotlib画布，所有图都打印在一起了⊙▃⊙

搜索过程由于搜索空间太小 也很快陷在局部最优解

修改下，增加搜索空间，再次测试！搜索到的效果较好的

LearningRate = [0.001, 0.003, 0.005, 0.01, 0.02, 0.05, 0.08]
Optimizers = ['sgd', 'adam', 'rmsprop', 'adadelta', 'adagrad', 'Adamax', 'Nadam']

[1] https://engineering.case.edu/bearingdatacenter
[2] https://zhuanlan.zhihu.com/p/448210993
[3]Esfahanian P, Akhavan M. Gacnn: Training deep convolutional neural networks with genetic algorithm[J]. arXiv preprint arXiv:1909.13354, 2019.