Distributed deep learning method and device, terminal equipment and storage medium

Abstract

The invention is suitable for the technical field of computers, and provides a distributed deep learning method and device, terminal equipment and a storage medium, and the method comprises the steps: obtaining at least two same sample sets, each sample set comprising a plurality of data blocks; associating each data block with a working node according to a preset rule to ensure that the data blocks associated with any preset target number of working nodes can form at least one sample set, performing model training on the basis of each associated data block by the working node to obtain a gradient corresponding to the data block, and sending the gradient to a parameter server; the parameter server receives the gradients corresponding to the data blocks sent by the working node, calculates a target gradient based on the received gradients after receiving the gradients corresponding to all the data blocks in the at least one sample set, and sends the target gradient to the working node; the method does not affect the model training, improves the training speed, guarantees the integrity of the model training, and improves the accuracy of the model. 本申请适用于计算机技术领域，提供了一种分布式深度学习方法、装置、终端设备及存储介质，该方法包括：获取相同的至少两个样本集，每个样本集中包括多个的数据块；将各个数据块按照预设规则与工作节点关联，以保证任意预设目标数量的工作节点关联的数据块能够组成至少一个所述样本集，工作节点基于关联的各个数据块进行模型训练得到数据块对应的梯度，并向参数服务器发送所述梯度；参数服务器接收工作节点发送的所述数据块对应的梯度，并在接收到至少一个样本集中所有数据块对应的梯度后，基于接收到的梯度计算目标梯度，并向工作节点发送目标梯度；本申请不会影响模型训练，提高了训练速度，保证了模型训练的完整性，进而提高了模型的准确度。