Baiduが'Ring Allreduce' Library で効率化したって

Vengineerの戯言 : Twitter
SystemVerilogの世界へようこそ、すべては、SystemC v0.9公開から始まった

Baidu's 'Ring Allreduce' Library Increases Machine Learning Efficiency Across Many GPU Nodes

リングでクルクル廻すのね。

Baiduのサイト：Bringing HPC Techniques to Deep Learning

引用
　The ring allreduce algorithm could speed up the training of an example neural network 
　by 31x across 40 GPUs, compared to using a single GPU.

単一GPUに対して、40GPUで31倍になった例もあったって。

引用
  Baidu’s SVAIL used the ring allreduce algorithm to train state 
  of the art speech recognition models, 
  and now it hopes that others will take advantage of the group’s implementation 
  to do even more interesting things. 

  The group released its ring allreduce implementation 
  as both a standalone C++ library as well as a patch for TensorFlow.

とあったので、Google君に聞いたら、ありました。

TensorFlow with MPI