Tensorflow

Tensorflow is a popular ML framework developed by Google that makes implementing optimized deep learning models very simple. Popular high-level APIs include Keras and TF Probability.

On this page:

1. Tensorflow

   1. Version 1.x

   2. Version 2.x

   3. Tensorboard

2. Keras

3. Tensorflow Probability

Tensorflow

Tensorflow (TF) is a powerful ML framework that lets us develop and train models that are computationally optimized and compatible with different processing units, including CPUs, GPUs, and the more recent TPUs.

Tensorflow 1.x

API Documentation  |  TensorFlow v1.15.0TensorFlow

In TF 1.x, the computational graph, which maps out operation order and dependencies, is specified first, before any computations occur. This type of graph is commonly called a static graph since it doesn't change during a session, which is when the graph is executed and values are computed.

Tensorflow graph visualized using Tensorboard

A TF computational graph is specified using a combination of these objects and operations:

• TF Placeholders are objects that are given information at the time of computation. Ex: the input layer of a neural network.

• TF Variables are the parameters that are evaluated during a session. Ex: weights and biases.

• TF operations, which are performed on placeholders and variables, closely follow NumPy operations. TF also includes more ML specific functions, such as one-hot encoding (i.e. turning classifications into integer representation) and nonlinear activation functions.

In addition to the graph, we also need to specify the training method and any functions to execute during training:

• Optimizer minimizes the specified loss function to iteratively update the TF variables.

• Callback functions are functions that are executed during training after a certain number of iterations. Popular callbacks include early stopping (i.e. determining when to stop training), using TensorBoard, etc.

Finally, to perform the computation, we start a TF session and feed the placeholders with our dataset.

Tensorflow 2.x (Eager Execution)

API Documentation  |  TensorFlow v2.16.1TensorFlow

With TF 2.x, there are no longer TF "sessions" and everything is executed eagerly.

TensorBoard (Visualization Tool)

Get started with TensorBoard  |  TensorFlowTensorFlow

Tensorboard is Tensorflow's visualization tool for viewing computational graphs, tracking variables & performance metrics, etc. Installation of Tensorflow includes TensorBoard. To launch the dashboard, run this on terminal and point the --logdir to the appropriate directory.

tensorboard --logdir=summaries

Open the link shown on the terminal (default: localhost:8888) on a web browser.

TensorBoard dashboard

Keras

Keras: Deep Learning for humanskeras.io

Keras is a popular high-level API for building deep neural networks using prebuilt layers (i.e. dense layers, convolutional layers, batch normalization layers, etc.). Keras is great for quickly building regular models, but can't accommodate custom operations like in Tensorflow/Pytorch.

Another benefit of Keras is the abundance of pre-trained model architectures that are available for transfer learning.

Tensorflow Probability

TensorFlow ProbabilityTensorFlow

Tensorflow Probability is a library for probabilistic deep learning models and includes useful tools for Variational Inference, Bayesian Neural Networks, MCMC, etc.