Introduction#
Why HoloViews?#
HoloViews is an open-source Python library for data analysis and visualization. Python already has excellent tools like numpy, pandas, and xarray for data processing, and bokeh and matplotlib for plotting, so why yet another library?
HoloViews helps you understand your data better, by letting you work seamlessly with both the data and its graphical representation.
HoloViews focuses on bundling your data together with the appropriate metadata to support both analysis and visualization, making your raw data and its visualization equally accessible at all times. This process can be unfamiliar to those used to traditional data-processing and plotting tools, and this getting-started guide is meant to demonstrate how it all works at a high level. More detailed information about each topic is then provided in the User Guide.
With HoloViews, instead of building a plot using direct calls to a plotting library, you first describe your data with a small amount of crucial semantic information required to make it visualizable, then you specify additional metadata as needed to determine more detailed aspects of your visualization. This approach provides immediate, automatic visualization that can be effortlessly requested at any time as your data evolves, rendered automatically by one of the supported plotting libraries (such as Bokeh or Matplotlib).
Tabulated data: subway stations#
To illustrate how this process works, we will demonstrate some of the key features of HoloViews using a collection of datasets related to transportation in New York City. First let’s run some imports to make numpy and pandas accessible for loading the data. Here we start with a table of subway station information loaded from a CSV file with pandas:
import pandas as pd
import numpy as np
import holoviews as hv
from holoviews import opts
hv.extension('bokeh')
This is the standard way to make the numpy and pandas libraries available in the namespace. We recommend always importing HoloViews as hv and if you haven’t already installed HoloViews, check out the install instructions on our homepage.
Note that after importing HoloViews as hv we run hv.extension('bokeh') to load the bokeh plotting extension, allowing us to generate visualizations with Bokeh. In the next section we will see how you can use other plotting libraries such as matplotlib and even how you can mix and match between them.
Now let’s load our subway data using pandas:
station_info = pd.read_csv('../assets/station_info.csv')
station_info.head()
| name | lat | lon | opened | services | service_names | ridership | |
|---|---|---|---|---|---|---|---|
| 0 | First Avenue | 40.730953 | -73.981628 | 1924 | 1 | ['L'] | 7.702110 |
| 1 | Second Avenue | 40.723402 | -73.989938 | 1936 | 1 | ['F'] | 5.847710 |
| 2 | Third Avenue | 40.732849 | -73.986122 | 1924 | 1 | ['L'] | 2.386533 |
| 3 | Fifth Avenue | 40.753821 | -73.981963 | 1920 | 6 | ['7', 'E', 'M', 'N', 'R', 'W'] | 16.220605 |
| 4 | Sixth Avenue | 40.737335 | -73.996786 | 1924 | 1 | ['L'] | 16.121318 |
We see that this table contains the subway station name, its latitude and longitude, the year it was opened, the number of services available from the station and their names, and finally the yearly ridership (in millions for 2015).
Elements of visualization#
We can immediately visualize some of the the data in this table as a scatter plot. Let’s view how ridership varies with the number of services offered at each station:
scatter = hv.Scatter(station_info, 'services', 'ridership')
scatter
Here we passed our dataframe to hv.Scatter to create an object called scatter, which is independent of any plotting library but here is visualized using bokeh. HoloViews provides a wide range of Element types, all visible in the Reference Gallery.
In this example, scatter is a simple wrapper around our dataframe that knows that the ‘services’ column is the independent variable, normally plotted along the x-axis, and that the ‘ridership’ column is a dependent variable, plotted on the y-axis. These are our dimensions which we will describe in more detail a little later.
Given that we have the handle scatter on our Scatter object, we can show that it is indeed an object and not a plot by printing it:
print(scatter)
:Scatter [services] (ridership)
The bokeh plot above is simply the rich, visual representation of scatter which is plotted automatically by HoloViews and displayed automatically in the Jupyter notebook. Although HoloViews itself is independent of notebooks, this convenience makes working with HoloViews easiest in the notebook environment.
Compositional Layouts#
The class Scatter is a subclass of Element. As shown in our element gallery, Elements are the simplest viewable components in HoloViews. Now that we have a handle on scatter, we can demonstrate the composition of these components:
layout = scatter + hv.Histogram(np.histogram(station_info['opened'], bins=24), kdims=['opened'])
layout
In a single line using the + operator, we created a new, compositional object called a Layout built from our scatter visualization and a Histogram that shows how many subway stations opened in Manhattan since 1900. Note that once again, all the plotting is happening behind the scenes. The layout is not a plot, it’s a new object that exists independently of any given plotting system:
print(layout)
:Layout
.Scatter.I :Scatter [services] (ridership)
.Histogram.I :Histogram [opened] (Frequency)
Array data: taxi dropoffs#
So far we have visualized data in a pandas DataFrame but HoloViews is as agnostic to data formats as it is to plotting libraries; see Applying Customizations for more information. This means we can work with array data as easily as we can work with tabular data. To demonstrate this, here are some numpy arrays relating to taxi dropoff locations in New York City:
taxi_dropoffs = {hour:arr for hour, arr in np.load('../assets/hourly_taxi_data.npz').items()}
print('Hours: {hours}'.format(hours=', '.join(taxi_dropoffs.keys())))
print('Taxi data contains {num} arrays (one per hour).\nDescription of the first array:\n'.format(num=len(taxi_dropoffs)))
np.info(taxi_dropoffs['0'])
Hours: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23
Taxi data contains 24 arrays (one per hour).
Description of the first array:
class: ndarray
shape: (256, 256)
strides: (1024, 4)
itemsize: 4
aligned: True
contiguous: True
fortran: False
data pointer: 0x120338000
byteorder: little
byteswap: False
type: float32
As we can see, this dataset contains 24 arrays (one for each hour of the day) of taxi dropoff locations (by latitude and longitude), aggregated over one month in 2015. The array shown above contains the accumulated dropoffs for the first hour of the day.
Compositional Overlays#
Once again, we can easily visualize this data with HoloViews by passing our array to hv.Image to create an object named image. This object has the spatial extent of the data declared as the bounds, in terms of the corresponding range of latitudes and longitudes.
bounds = (-74.05, 40.70, -73.90, 40.80)
image = hv.Image(taxi_dropoffs['0'], ['lon','lat'], bounds=bounds)
HoloViews supports numpy, xarray, and dask arrays when working with array data (see Gridded Datasets). We can also compose elements containing array data with those containing tabular data. To illustrate, let’s pass our tabular station data to a Points element, which is used to mark positions in two-dimensional space:
points = hv.Points(station_info, ['lon','lat']).opts(color="red")
image + image * points
On the left, we have the visual representation of the image object we declared. Using + we put it into a Layout together with a new compositional object created with the * operator called an Overlay. This particular overlay displays the station positions on top of our image, which works correctly because the data in both elements exists in the same space, namely New York City.
The .opts() method call for specifying the visual style is part of the HoloViews options system, which is described in the next ‘Getting started’ section.
This overlay on the right lets us see the location of all the subway stations in relation to our midnight taxi dropoffs. Of course, HoloViews allows you to visually express more of the available information with our points. For instance, you could represent the ridership of each subway by point color or point size. For more information see Applying Customizations.
Effortlessly exploring data#
You can keep composing datastructures until there are more dimensions than can fit simultaneously on your screen. For instance, you can visualize a dictionary of Images (one for every hour of the day) by declaring a HoloMap:
dictionary = {int(hour):hv.Image(arr, ['lon','lat'], bounds=bounds)
for hour, arr in taxi_dropoffs.items()}
hv.HoloMap(dictionary, kdims='Hour')
This is yet another object that is rendered by the HoloViews plotting system with Bokeh behind the scenes:
holomap = hv.HoloMap(dictionary, kdims='Hour')
print(holomap)
:HoloMap [Hour]
:Image [lon,lat] (z)
As this HoloMap is a container for our Image elements, we can use the methods it offers to return new containers. For instance, in the next cell we select three different hours of the morning from the HoloMap and display them as a Layout:
holomap.select(Hour={3,6,9}).layout()
Here the select method picks values from the specified ‘Hour’ dimension. The various Elements like Scatter and Image all accept two types of dimensions: key dimensions (i.e., indexing dimensions or independent variables), and value dimensions (resulting data or dependent variables). These attributes are named kdims and vdims, respectively, and can be passed as the second and third positional argument for all Elements other than Histogram. As you can see above, the HoloMap of Images also has a kdims argument, allowing it to be indexed over those dimensions. The kdims and vdims accept either single dimensions or lists of dimensions, and let you conveniently express the full space in which your data lives.
Note how the Image elements where the holomap is constructed are declared using key dimensions of ['lat','lon'], which describes the fact that New York City is being viewed in terms of longitude and latitude. This semantic information is automatically mapped to our visualization by the HoloViews plotting system, which sets the x-axis and y-axis labels accordingly. In the case of the HoloMap we used a key dimension of 'Hour' to declare that the interactive slider ranges over the hours of the day.
Data as visualization#
Holomaps are able to compose with elements and other holomaps into overlay and layouts just as easily as you compose two elements together. Here is one such composition where we select a range of longitudes and latitudes from our Points before we overlay them:
hotspot = points.select(lon=(-73.99, -73.96), lat=(40.75,40.765))
composition = holomap * hotspot
composition.opts(
opts.Image(xrotation=90),
opts.Points(color='red', marker='v', size=6))
In the cell above we created and styled a composite object with a few short lines of code. Furthermore, this composite object relates tabular and array data and is immediately presented in a way that can be explored interactively. This way of working enables highly productive exploration, allowing new insights to be gained easily. For instance, after exploring with the slider we notice a hotspot of taxi dropoffs at 7am, which we can select as follows:
composition.select(Hour=7)
We can now see that the slice of subway locations was chosen in relation to the hotspot in taxi dropoffs around 7am in the morning. This area of Manhattan just south of Central Park contains many popular tourist attractions, including Times Square, and we can infer that tourists often take short taxi rides from the subway stations into this area. Notice how the customizations applied to this new plot, just as for the original one, which is only possible because of how HoloViews lets you capture the data and its attributes independently of any particular visualization.
At this point it may appear that HoloViews is about easily generating explorative, interactive visualizations from your data. In fact, as we have been building these visualizations we have actually been working with our data, as we can show by examining the .data attribute of our sliced subway locations:
hotspot.data
| name | lat | lon | opened | services | service_names | ridership | |
|---|---|---|---|---|---|---|---|
| 3 | Fifth Avenue | 40.753821 | -73.981963 | 1920 | 6 | ['7', 'E', 'M', 'N', 'R', 'W'] | 16.220605 |
| 5 | Seventh Avenue | 40.762862 | -73.981637 | 1919 | 7 | ['B', 'D', 'E', 'N', 'Q', 'R', 'W'] | 12.013107 |
| 14 | 42nd Street | 40.757308 | -73.989735 | 1932 | 7 | ['A', 'B', 'C', 'D', 'E', 'F', 'M'] | 66.359208 |
| 15 | 49th Street | 40.759901 | -73.984139 | 1919 | 4 | ['N', 'Q', 'R', 'W'] | 8.029988 |
| 16 | 51st Street | 40.757107 | -73.971920 | 1918 | 2 | ['4', '6'] | 20.479923 |
| 17 | 57th Street | 40.764664 | -73.980658 | 1968 | 1 | ['F'] | 4.720245 |
| 18 | 59th Street | 40.762526 | -73.967967 | 1918 | 3 | ['4', '5', '6'] | 25.566655 |
| 46 | Grand Central–42nd Street | 40.751776 | -73.976848 | 1904 | 5 | ['4', '5', '6', '7', 'S'] | 46.737564 |
| 49 | Lexington Avenue | 40.762660 | -73.967258 | 1920 | 7 | ['E', 'F', 'M', 'N', 'Q', 'R', 'W'] | 21.407792 |
| 55 | Times Square–42nd Street | 40.754672 | -73.986754 | 1917 | 7 | ['1', '2', '3', 'N', 'Q', 'R', 'W'] | 66.359208 |
We see that slicing the HoloViews Points object in the visualization sliced the underlying data, with the structure of the table left intact. We can see that the Times Square 42nd Street station is indeed one of the subway stations surrounding our taxi dropoff hotspot. This seamless interplay and exchange between the raw data and easy-to-generate visualizations of it is crucial to how HoloViews helps you understand your data.
Saving and rendering objects#
HoloViews renders objects using a variety of backends, making it easy to export plots to a variety of formats. The hv.save utility makes it trivial to save the rendered output of a HoloViews object to a file. For instance, if we wanted to save the composition object from above (including the widgets) to a standalone HTML file, we would simply run:
hv.save(composition, 'holomap.html')
0%| | 0/24 [00:00<?, ?it/s]
8%|▊ | 2/24 [00:00<00:02, 10.85it/s]
17%|█▋ | 4/24 [00:00<00:01, 12.57it/s]
25%|██▌ | 6/24 [00:00<00:01, 14.29it/s]
33%|███▎ | 8/24 [00:00<00:01, 14.45it/s]
42%|████▏ | 10/24 [00:00<00:00, 14.18it/s]
50%|█████ | 12/24 [00:00<00:00, 14.23it/s]
58%|█████▊ | 14/24 [00:00<00:00, 14.42it/s]
67%|██████▋ | 16/24 [00:01<00:00, 14.50it/s]
75%|███████▌ | 18/24 [00:01<00:00, 14.56it/s]
83%|████████▎ | 20/24 [00:01<00:00, 14.80it/s]
92%|█████████▏| 22/24 [00:01<00:00, 15.49it/s]
100%|██████████| 24/24 [00:01<00:00, 15.59it/s]
Or to save a single frame with no widget:
hv.save(composition.last, 'image.html')
hv.save defaults to the currently selected backend, which can be overridden by adding an argument like backend='matplotlib'. The Bokeh backend supports png output, and the Matplotlib backend supports png, svg, gif, or mp4, selected by using the appropriate extension instead of .html. In each case, rendering to a given format may require additional libraries to be installed.
Onwards#
The next getting-started section shows how to do Customization of the visual appearance of your data, allowing you to highlight the most important features and change the look and feel. Other related topics for deeper study:
The above plots did not require any special geographic-data support, but when working with larger areas of the Earth’s surface (for which curvature becomes significant) or when overlaying data with geographic features, the separate GeoViews library provides convenient geo-specific extensions to HoloViews.
The taxi array data was derived from a very large tabular dataset and rasterized using datashader, an optional add-on to HoloViews and Bokeh that makes it feasible to work with very large datasets in a web browser.