Which framework is best for deep learning in 2020 ?
All major computer (Microsoft, Amazon, Facebook) have joined the race, resulting in their wake much enthusiastic, everyone from his contribution. But, in 2020, how to make a choice?
All major computer (Microsoft, Amazon, Facebook) have joined the race, resulting in their wake much enthusiastic, everyone from his contribution.
But, in 2019, how to make a choice?
Without mentioning them all, we will first separate the bookstores into two groups:
- Backends, independent libraries, capable of managing the details of implementation of neural networks
- Overlays, libraries complementing a backend to simplify its use and allow the developer to focus on the design part of learning and structuring the network
I will talk about the following main backend:
- TensorFlow made by google
- Torch developed by three researchers
- Apache MxNet, the library supported by Amazon
- Theano the library of the University of Montreal
- CNTK, the Cognitive Toolkit developed by Microsoft
For the overlays we will consider:
- Keras (François Chollet)
- PyTorch (Facebook)
- Gluon (Amazon)
- Caffe / Caffe2 (Berkeley Vision and Learning Center)
Of course, these lists are not exhaustive.
Each of these libraries could be evaluated with the following criteria:
- Ease of installation and deployment
- Development assistance tools / Debugging
- Popularity on GitHub / Sustainability
What about TensorFlow ?
- Most popular on GitHub
- Massively used by Google in all its services (gmail, youtube, google translate, ...)
- Open source and used by major brands like Airbnb, SAP, Nvidia or Coca Cola
- the most widespread support Python, but also available in C ++, Java and Go with bridges to C # and Julia
- Has an extensive official documentation, and an even more extensive documentation from the community
- Has graphical interfaces to visualize learning processes, like TensorBoard
- TensorFlow serving provides training distributed models, and their deployment
- TensorFlow maintains a "low level" approach favoring power, but requiring writing a lot of code
The structure of neurons is defined statically promoting performance, but involving focus more rigid item
What about PyTorch ?
- Very popular in the field of research
- Used by Facebook in its own services, but also Twitter and SalesForce
- The neurons graph is generated dynamically, unlike TensorFlow, favoring a dynamic design and a simplified debugging
- Few visualization tools as TensorBoard, not training distributed as the Google cloud
- It is possible to connect to Pytorch TensorBoard through third party libraries TensorBoardX
- Researchers appreciate PyTorch for its dynamic side, even if the production is less elaborate than TensorFlow
PyTorch can be combined with Caffe 2 for large deployments
Amazon and MxNet:
- Easily deployable on AWS (Amazon Web Services) and Azure (Microsoft cloud)
- Promotes portability and multi-language
- Very good performance and scalability almost linearly, when adding GPUs
- The Gluon overlay allows optimal integration with AWS; these are the tools used by Amazon internally
- Backend also providing high level libraries
- Very good integration with multiple tools promoting developer productivity
- Very easy to deploy, especially on Azure or AWS
- Works mainly with Windows
- Used by Microsoft internally, especially for Skype, XBox and Cortana
In addition to the technical criteria developed here, the choice will also be made according to the objectives of the system, scalability and deployment constraints, but mostly based on the rates charged by Google, Amazon or Microsoft for using their cloud need for training models