Tensorflow
Tensorflow is a popular ML framework developed by Google that makes implementing optimized deep learning models very simple. Popular high-level APIs include Keras and TF Probability.
On this page:
Tensorflow
Tensorflow (TF) is a powerful ML framework that lets us develop and train models that are computationally optimized and compatible with different processing units, including CPUs, GPUs, and the more recent TPUs.
Tensorflow 1.x
In TF 1.x, the computational graph, which maps out operation order and dependencies, is specified first, before any computations occur. This type of graph is commonly called a static graph since it doesn't change during a session, which is when the graph is executed and values are computed.
A TF computational graph is specified using a combination of these objects and operations:
TF Placeholders are objects that are given information at the time of computation. Ex: the input layer of a neural network.
TF Variables are the parameters that are evaluated during a session. Ex: weights and biases.
TF operations, which are performed on placeholders and variables, closely follow NumPy operations. TF also includes more ML specific functions, such as one-hot encoding (i.e. turning classifications into integer representation) and nonlinear activation functions.
In addition to the graph, we also need to specify the training method and any functions to execute during training:
Optimizer minimizes the specified loss function to iteratively update the TF variables.
Callback functions are functions that are executed during training after a certain number of iterations. Popular callbacks include early stopping (i.e. determining when to stop training), using TensorBoard, etc.
Finally, to perform the computation, we start a TF session and feed the placeholders with our dataset.
Tensorflow 2.x (Eager Execution)
With TF 2.x, there are no longer TF "sessions" and everything is executed eagerly.
TensorBoard (Visualization Tool)
Tensorboard is Tensorflow's visualization tool for viewing computational graphs, tracking variables & performance metrics, etc. Installation of Tensorflow includes TensorBoard. To launch the dashboard, run this on terminal and point the --logdir to the appropriate directory.
Open the link shown on the terminal (default: localhost:8888) on a web browser.
Keras
Keras is a popular high-level API for building deep neural networks using prebuilt layers (i.e. dense layers, convolutional layers, batch normalization layers, etc.). Keras is great for quickly building regular models, but can't accommodate custom operations like in Tensorflow/Pytorch.
Another benefit of Keras is the abundance of pre-trained model architectures that are available for transfer learning.
Tensorflow Probability
Tensorflow Probability is a library for probabilistic deep learning models and includes useful tools for Variational Inference, Bayesian Neural Networks, MCMC, etc.
Last updated
