Streams¶

In the DynamicMap tutorial we saw how DynamicMap allows you to explore high dimensional data using the widgets in the same style as HoloMaps. Although suitable for unbounded exploration of large parameter spaces, the DynamicMaps described in that notebook support exactly the same mode of interaction as HoloMaps. In particular, the key dimensions are used to specify a set of widgets that when manipulated apply the appopriate indexing to invoke the user-supplied callable.

In this tutorial we will explore the HoloViews streams system that allows any sort of value to be supplied from anywhere. This system opens a huge set of new possible visualization types, including continuously updating plots that reflect live data as well as dynamic visualizations that can be interacted with directly, as described in the Linked Streams tutorial.

A simple DynamicMap¶

Before introducing streams, let us declare a simple DynamicMap of the sort discussed in the DynamicMap tutorial. This example consists of an Curve element showing a Lissajous curve with VLine and HLine annotation to form a crosshair:

As expected, the declared key dimension (kdims) has turned into slider widgets that let us move the crosshair along the curve. Now let's see how to position the crosshair using streams.

Introducing streams¶

The core concept behind a stream is simple: it is a parameter that can change over time that automatically refreshes code depending on those parameter values.

Like all objects in HoloViews, these parameters are declared using param and you can define streams as a parameterized subclass of the holoviews.streams.Stream. A more convenient way is to use the Stream.define classmethod:

This results in a Time class with a numeric t parameter that defaults to zero. As this object is parameterized, we can use hv.help to view it's parameters:

This parameter is a param.Number as we supplied a float, if we have supplied an integer it would have been a param.Integer. Notice that there is no docstring in the help output above but we can add one by explicit defining the parameter as follows:

Now we have defined this Time stream class, we can make of an instance of it and looks at its parameters:

As with all parameterized classes, we can instantiate our parameters with a suitable value of our choice instead of relying on defaults.

For more information on defining Stream classes this way, use hv.help(Stream.define).

Simple streams example¶

We can now supply this streams object to a DynamicMap using the same lissajous_curve callback above by adding it to the streams list:

Immediately we see that the crosshair position of the DynamicMap reflects the t parameter values we set on the Time stream. This means that the t parameter was supplied as the argument to the lissajous_curve callback. As we now have no key dimensions, there is no widgets for the t dimensions.

Although we have what looks like a static plot, it is in fact dynamic and can be updated in place at any time. To see this, we can call the event method on our DynamicMap:

Running this cell will have updated the crosshair from its original position where $t=\frac{\pi}{4}$ to a new position where t=0.2. Try running the cell above with different values of t and watch the plot update!

This event method is the recommended way of updating the stream parameters on a DynamicMap but if you have a handle on the relevant stream instance, you can also call the event method on that:

Running the cell above also move the crosshair to a new position. As there are no key dimensions, there is only a single valid (empty) key that can be accessed with dmap[()] or dmap.select() making event the only way to explore new parameters.

We will examine the event method and the machinery that powers streams in more detail later in the tutorial after we have looked at more examples of how streams are used in practice.

Working with multiple streams¶

The previous example showed a curve parameterized by a single dimension t. Often you will have multiple stream parameters you would like to declare as follows:

You can update both x and y by passing multiple keywords to the event method:

Note that the definition above behaves the same as this definition where we define separate X and Y stream classes:

X = Stream.define('X',x=0.0)
Y = Stream.define('Y',y=0.0)
hv.DynamicMap(crosshairs, streams=[X(),Y()])

The reason why you might want to list multiple streams instead of always defining a single stream containing all the required stream parameters will be made clear in the Linked Streams tutorial.

Combining streams and key dimensions¶

All the DynamicMap examples above can't be indexed with anything other than dmap[()] or dmap.select() as none of them had any key dimensions. This was to focus exclusively on the streams system at the start of the tutorial and not because you can't combine key dimensions and streams:

In this example, you can drag the slider to see a numeric approximation to the integral on the left side on the VLine.

As 'limit' is declared as a key dimension, it is given a normal HoloViews slider. As we have also defined a time stream, we can update the displayed curve for any time value:

We now see how to control the time argument of the integral function by triggered an event with a new time value, and how to control the limit argument by moving a slider. Controlling limit with a slider this way is valid but also a little unintuitive: what if you could control limit just by hovering over the plot?

In the Linked Streams tutorial, we will see how we can do exactly this by switching to the bokeh backend and using the linked streams system.

Matching names to arguments¶

Note that in the example above, the key dimension names and the stream parameter names match the arguments to the callable. This must be true for stream parameters but this isn't a requirement of key dimensions: if you replace the word 'radius' with 'size' in the example above after XY is defined, the example still works.

Here are the rules regarding the callback argument names:

• If your key dimensions and stream parameters match the callable argument names, the definition is valid.
• If your callable accepts mandatory positional arguments and their number matches the number of key dimensions, the names don't need to match and these arguments will be passed key dimensions values.

As stream parameters always need to match the argument names, there is a method to allow them to be easily renamed. Let's say you imported a stream class as shown in the Linked Streams tutorial or for this example, reuse the existing XY stream class. You can then use the rename method allowing the following definition:

Overlapping stream and key dimensions¶

In the above example above, the stream parameters do not overlap with the declared key dimension. What happens if we add 'time' to the declared key dimensions?

First you might notice that the 'time' value is now shown in the title but that there is no corresponding time slider as its value is supplied by the stream.

The 'time'parameters is now what is called 'dimensioned streams' which renables indexing of these dimensions:

In A, we supply our own values for the 'time and 'limit' parameters. This doesn't change the values of the 'time' parameters on the stream itself but it does allow us to see what would happen when the time value is one. Note the use of None in C as a way of leaving an explicit value unspecified, allowing the current stream value to be used.

This is one good reason to use dimensioned streams - it restores access to convenient indexing and selecting operation as a way of exploring your visualizations. The other reason it is useful is that if you keep all your parameters dimensioned, it re-enables the DynamicMap cache described in the DynamicMap tutorial, allowing you to record your interaction with streams and allowing you to cast to HoloMap for export:

One use of this would be to have a simulator drive a visualization forward using event in a loop. You could then stop your simulation and retain the a recent history of the output as long as the allowed DynamicMap cache.

Generators and argument-free callables¶

In addition to callables, Python supports generators that can be defined with the yield keyword. Calling a function that uses yield returns a generator iterator object that accepts no arguments but returns new values when iterated or when next() is applied to it.

HoloViews supports Python generators for completeness and generator expressions can be a convenient way to define code inline instead of using lambda functions. As generators expressions don't accept arguments and can get 'exhausted' we recommend using callables with DynamicMap - exposing the relevant arguments also exposes control over your visualization.

Callables that have arguments, unlike generators allow you to re-visit portions of your parameter space instead of always being forced in one direction via calls to next(). With this caveat in mind, here is an example of a generator and the corresponding generator iterator that returns a BoxWhisker element:

This returns two box whiskers representing samples from two Gaussian distributions of 10 samples. Iterating over this generator simply resamples from these distributions using an additional sample each time.

As with a callable, we can pass our generator iterator to DynamicMap:

Without using streams, we now have a problem as there is no way to trigger the generator to view the next distribution in the sequence. We can solve this by defining a stream with no parameters:

Stream event update loops¶

Now we can simply use event() to drive the generator forward and update the plot, showing how the two Gaussian distributions converge as the number of samples increase.

Note that there is a better way to run loops that drive dmap.event() which supports a period (in seconds) between updates and a timeout argument (also in seconds):

In this generator example, event does not require any arguments but you can set the param_fn argument to a callable that takes an iteration counter and returns a dictionary for setting the stream parameters. In addition you can use block=False to avoid blocking the notebook using a threaded loop. This can be very useful although there can have two downsides 1. all running visualizations using non-blocking updates will be competing for computing resources 2. if you override a variable that the thread is actively using, there can be issues with maintaining consistent state in the notebook.

Generally, the periodic utility is recommended for all such event update loops and it will be used instead of explicit loops in the rest of the tutorials involving streams.

Using next()¶

The approach shown above of using an empty stream works in an exactly analogous fashion for callables that take no arguments. In both cases, the DynamicMap next() method is enabled:

Next steps¶

The streams system allows you to update plots in place making it possible to build live visualizations that update in response to incoming live data or any other type of event. As we have seen in this tutorial, you can use streams together with key dimensions to add additional interactivity to your plots while retaining the familiar widgets.

This tutorial used examples that work with either the matplotlib or bokeh backends. In the Linked Streams tutorial, you will see how you can directly interact with dynamic visualizations when using the bokeh backend.

[Advanced] How streams work¶

This optional section is not necessary for users who simply want to use the streams system, but it does describes how streams actually work in more detail.

A stream class is one that inherits from Stream that typically defines some new parameters. We have already seen one convenient way of defining a stream class:

This is equivalent to the following definition which would be more appropriate in library code or for complex stream class requiring lots of parameters that need to be documented:

As we have already seen, we can make an instance of XY with some initial values for x and y.

However, trying to modify these parameters directly will result in an exception as they have been declared constant (e.g xy.x=4 will throw an error). This is because there are two allowed ways of modifying these parameters, the simplest one being update:

This shows how you can update the parameters and also shows the correct way to view the stream parameter values via the contents property as this will apply any necessary renaming.

So far, using update has done nothing but forced us to access parameter a certain way. What makes streams work are the side-effects you can trigger when changing a value via the event method. The relevant side-effect is to invoke callables called 'subscribers'

Subscribers¶

Without defining any subscribes, the event method is identical to update:

Now let's add a subscriber:

As we can see, now when you call event, our subscriber is called with the updated parameter values, renamed as appropriate. The event method accepts the original parameter names and the subscriber receives the new values after any renaming is applied. You can add as many subscribers as you want and you can clear them using the clear method:

When you define a DynamicMap using streams, the HoloViews plotting system install the necessary callbacks as subscibers to update the plot when the stream parameters change. The above example clears all subscribers (it is equivalent to clear('all'). To clear only the subscribers you define yourself use clear('user') and to clear any subscribers installed by the HoloViews plotting system use clear('internal').

When using linked streams as described in the Linked Streams tutorial, the plotting system recognizes the stream class and registers the necessary machinery with Bokeh to update the stream values based on direct interaction with the plot.