Introducing plotly.py 3.0.0

Last October, at our New York City Dash Workshops, an attendee showed us something really special. Jon Mease ( @jmmease ), a data scientist and software developer from the Johns Hopkins Applied Physics Laboratory, had spent three months working on a fork of plotly.py , our open source Python graphing library:

I’ve spent years working in scientific and engineering domains and I’m really excited about the benefits of bringing the best of the Python open data science ecosystem to these areas.

Data visualization is a critical component of analysis in these domains, and even though there are literally dozens of libraries in Python’s Visualization Landscape , I couldn’t find one that supported the full set of features that I need (or at least that I want :smiley:). These include: Support for a wide range of plot types covering statistical, 3D, and geographic use cases; efficient GPU acceleration to handle realistically large data sets; offline export of high-quality static images; two-way interactivity in the Jupyter Notebook; and stand-alone dashboarding support.

So I decided to start with the technology that I felt was the closest (plotly.js with plotly.py and Dash), and try my hand at filling in some of the gaps. As you’ll see in this post, I got a bit carried away :relieved:

— Jon

So Jon dug deep into plotly.js and started creating his own Pythonic interface to the library. It started with typeahead support and grew and grew to include first class Jupyter widgets, fast typed arrays, deep figure validation, offline static image export, and more. Now, one year after his first commit and nine months after our workshops, we’ve integrated Jon’s fork into the official plotly.py library and are excited to announce our biggest release yet: plotly.py 3.0.0 :tada:

Eager to get started? 3.0.0 hasn’t landed yet but we have a release candidate available: Download and installation instructions .

First Class Jupyter Widget Support

3.0.0 introduces a new Jupyter widget class: plotly.graph_objs.FigureWidget

FigureWidget has the same call signature as our existing Figure, and it’s made specifically for Jupyter Notebook and JupyterLab environments.

Print the widget, see the figure. Just like every other object in Jupyter. plotly.offline.init_notebook_mode and plotly.offline.iplot are no longer necessary.

This figure widget is synced up with the plotly.js JavaScript figure object, so you can access all of the properties of the figure, including the dynamic default properties that were previously only available in the JavaScript context.

Introspect all of the default properties of the plotly figure. Thanks to the Jupyter widget framework, these properties are accessed directly from the JavaScript context.

This widget is compatible with the rest of the widget framework and has event listeners for hovering, clicking, and selecting points and zooming into regions.

Tying together the Plotly FigureWidget with the Text IPywidget to display Walmart Store openings over time. Tying together Plotly and Datashader with the new FigureWidget’s “property change” callbacks. Whenever the axes ranges change, the callback gets fired, recomputing the server-rendered image with Datashader. View the source.

An Imperative Alternative

plotly.js graphs are declarative. This means that every aspect of the charts (the colors of the bars, the axes grid lines, the position of annotations) are completely described through a set of attributes. This set of attributes is the “Plotly JSON Chart Schema,” and it describes every aspect of over 33 chart types ( and counting )!

Our interface to this library has always been declarative. You describe what the graph should look like, not how to draw it .

plotly 3.0.0 introduces a set of imperative methods for manipulating and exploring the figure in Jupyter’s interactive computing environments. It’s easier than ever to traverse the figure structure and manipulate the graph. The imperative methods are one-to-one with the declarative attributes, so it’s easy to switch between describing your graph entirely upfront ( declarative method ) and editing it on-the-fly in Jupyter ( imperative method).

Interactively editing the attributes of a rendered Plotly `GraphWidget` in Jupyter Notebook.

Tab-Complete Support

With plotly.py, it’s easier than ever to discover the attributes of your graphs without leaving the notebook. plotly.py generates rich Python classes from the chart schema with great docstrings and perfect tab-complete support.

For any instantiated figure, you can traverse through all attributes of the figure and view their full descriptions:

Tab complete all the way down.

And the docstrings are beautiful. They’re pulled directly from the plotly.js schema so when plotly.js updates the underlying source code, plotly.py will automatically get the new doc strings.

>>> fig.layout.barmode?
Determines how bars at the same location coordinate are
displayed on the graph. With *stack*, the bars are stacked on
top of one another With *relative*, the bars are stacked on top
of one another, with negative values below the axis, positive
values above With *group*, the bars are plotted next to one
another centered around the shared location. With *overlay*,
the bars are plotted over one another, you might need to an
*opacity* to see multiple bars.

The 'barmode' property is an enumeration that may be specified as:
  - One of the following enumeration values:
        ['stack', 'group', 'overlay', 'relative']

All of the traces are accessible under the ` add_* ` methods of a figure. In Jupyter, type shift+tab to pull up the inline-documentation and shift+tab a second time to expand it.

Typeahead also works for the plotly.graph_objs , the pythonic wrappers around each of the chart types and groups of nested attributes.

Declarative graph objects with input argument tab complete, all the way down!

Calling help on these objects displays rich descriptions, all of the available keyword arguments, and descriptions of each argument . Complete and explicit.

>>> go.Carpet?

Init signature: go.Carpet(
    a=None, a0=None, aaxis=None, asrc=None,
    b=None, b0=None, baxis=None, bsrc=None,
    carpet=None, cheaterslope=None, color=None,
    customdata=None, customdatasrc=None,
    da=None, db=None, font=None, hoverinfo=None,
    hoverinfosrc=None, hoverlabel=None,
    ids=None, idssrc=None, legendgroup=None,
    name=None, opacity=None, selectedpoints=None,
    showlegend=None, stream=None,
    uid=None, visible=None, x=None, xaxis=None, xsrc=None,
    y=None, yaxis=None, ysrc=None)

Construct a new Carpet object

The data describing carpet axis layout is set in `y` and
(optionally) also `x`. If only `y` is present, `x` the plot is
interpreted as a cheater plot and is filled in using the `y`
values. `x` and `y` may either be 2D arrays matching with each
dimension matching that of `a` and `b`, or they may be 1D
arrays with total length equal to that of `a` and `b`.

Parameters
----------
a
An array containing values of the first parameter value

a0
Alternate to `a`. Builds a linear space of a
coordinates. Use with `da` where `a0` is the starting
coordinate and `da` the step.

aaxis
plotly.graph_objs.carpet.Aaxis instance or dict with
compatible properties

b
A two dimensional array of y coordinates at each carpet
point.

b0
Alternate to `b`. Builds a linear space of a
coordinates. Use with `db` where `b0` is the starting
coordinate and `db` the step.

baxis
plotly.graph_objs.carpet.Baxis instance or dict with
compatible properties

carpet
An identifier for this carpet, so that `scattercarpet`
and `scattercontour` traces can specify a carpet plot
on which they lie

cheaterslope
The shift applied to each successive row of data in
creating a cheater plot. Only used if `x` is been
ommitted.

color
Sets default for all colors associated with this axis
all at once: line, font, tick, and grid colors. Grid
color is lightened by blending this with the plot
background Individual pieces can override this.

[...]

These nice tab-complete docstrings also work with traditional code editors like PyCharm:

Deep typeahead attribute documentation across all plotly graph properties in PyCharm.

I’ve been spoiled over the years by the level of auto-completion and inline documentation that powerful IDEs like IntelliJ and Visual Studio provide for statically typed languages like Java and C#. My goal was to recreate this experience when working with the plotly.py object hierarchy. After several iterations on the design of the generated classes and their associated docstrings, I think I’ve gotten pretty close.

Code generation can be a lot of fun when you have a clearly defined schema!

— Jon

Deep Validation and Helpful Error Messages

The plotly.js chart schema provides strict types for each attribute. There are 14 data types and plotly.py now provides strict type validators and type-specific exceptions for each of them.

The validators are strict and helpful. For example, they’ll raise errors if your strings aren’t valid CSS color strings:

>>> go.scatter.Marker(color='fuschia')

Invalid value of type 'builtins.str' received for the 'color' property of scatter.marker
        Received value: 'fuschia'

The 'color' property is a color and may be specified as:
      - A hex string (e.g. '#ff0000')
      - An rgb/rgba string (e.g. 'rgb(255,0,0)')
      - An hsl/hsla string (e.g. 'hsl(0,100%,50%)')
      - An hsv/hsva string (e.g. 'hsv(0,100%,100%)')
      - A named CSS color:
            aliceblue, antiquewhite, aqua, aquamarine, azure,
            beige, bisque, black, blanchedalmond, blue,
            blueviolet, brown, burlywood, cadetblue,
            chartreuse, chocolate, coral, cornflowerblue,
            cornsilk, crimson, cyan, darkblue, darkcyan,
            darkgoldenrod, darkgray, darkgrey, darkgreen,
            darkkhaki, darkmagenta, darkolivegreen, darkorange,
            darkorchid, darkred, darksalmon, darkseagreen,
            darkslateblue, darkslategray, darkslategrey,
            darkturquoise, darkviolet, deeppink, deepskyblue,
            dimgray, dimgrey, dodgerblue, firebrick,
            floralwhite, forestgreen, fuchsia, gainsboro,
            ghostwhite, gold, goldenrod, gray, grey, green,
            greenyellow, honeydew, hotpink, indianred, indigo,
            ivory, khaki, lavender, lavenderblush, lawngreen,
            lemonchiffon, lightblue, lightcoral, lightcyan,
            lightgoldenrodyellow, lightgray, lightgrey,
            lightgreen, lightpink, lightsalmon, lightseagreen,
            lightskyblue, lightslategray, lightslategrey,
            lightsteelblue, lightyellow, lime, limegreen,
            linen, magenta, maroon, mediumaquamarine,
            mediumblue, mediumorchid, mediumpurple,
            mediumseagreen, mediumslateblue, mediumspringgreen,
            mediumturquoise, mediumvioletred, midnightblue,
            mintcream, mistyrose, moccasin, navajowhite, navy,
            oldlace, olive, olivedrab, orange, orangered,
            orchid, palegoldenrod, palegreen, paleturquoise,
            palevioletred, papayawhip, peachpuff, peru, pink,
            plum, powderblue, purple, red, rosybrown,
            royalblue, saddlebrown, salmon, sandybrown,
            seagreen, seashell, sienna, silver, skyblue,
            slateblue, slategray, slategrey, snow, springgreen,
            steelblue, tan, teal, thistle, tomato, turquoise,
            violet, wheat, white, whitesmoke, yellow,
            yellowgreen
      - A number that will be interpreted as a color
        according to scatter.marker.colorscale
      - A list or array of any of the above

Or if a number is outside of the valid range:

>>> go.scatter.Marker(size=-1)

ValueError: 
    Invalid value of type 'builtins.int' received for the 'size' property of scatter.marker
        Received value: -1

    The 'size' property is a number and may be specified as:
      - An int or float in the interval [0, inf]
      - A tuple, list, or one-dimensional numpy array of the above

Or if the string isn’t one of the enumerated values:

>>> go.Scatter(mode='marker')

Invalid value of type 'builtins.str' received for the 'mode' property of scatter
        Received value: 'marker'

The 'mode' property is a flaglist and may be specified
    as a string containing:
      - Any combination of ['lines', 'markers', 'text'] joined with '+' characters
        (e.g. 'lines+markers')
        OR exactly one of ['none'] (e.g. 'none')

Performance Improvements

Plotting large data interactively is now faster than ever:

• plotly.py figure specifications are now serialized and transferred to plotly.js using the Widget Message Protocol over the Jupyter Comms infrastructure.
• Thanks to the great work of the ipywidgets team , Python numpy arrays are now transferred into JavaScript TypedArrays without any intermediate UTF-8 encoding. We’d especially like to thank Maarten Breddels, Sylvain Corlay, and Jason Grout for laying very thoughtful foundations for this. We’re standing on the shoulders of giants!
• plotly.js provides direct support for TypedArrays ( added in v0.35.0 ) and it provides a super fast WebGL pathway for many trace types.

Plotting 1M points in Jupyter used to take 35 seconds. Now it takes only three. And the result is a responsive, completely interactive chart. :rocket:

Plotting an interactive chart with 1M data points in 1… 2… 3! seconds :racehorse: Fast client-side interactivity with 1M scatter points thanks to plotly.js’s WebGL renderer .

I’ve worked with a lot of data visualization libraries over the years, both proprietary and open source, and I don’t know of any that can both display a million points this quickly, and support zooming, panning, and selection at interactive speed.

(And now that I’ve I said this, I look forward to learning about some in the Hacker News discussion of this post :wink:)

— Jon

Context Manager for Animations

Two years ago, we introduced an animation interface in plotly.js . Previously, it’s been possible to create animations in plotly.py by defining a set of transitions up-front ( frames ). Now, in plotly.py 3.0.0 , you can define the animation transitions on-the-fly with a new ContextManager .

For example, here’s how to animate smooth transitions of a line chart with ipywidgets.IntSlider .

def animate_figure(change):
    with fig.batch_animate(duration=500):
        trace.y = np.sin(x) * change['new']

slider.observe(animate_figure, names='value')

Using the animation context manager to provide smooth transitions of updates in plotly.js. plotly.py animations use plotly.js’s underlying Plotly.animate method.

JupyterLab Support

plotly.py 3.0.0 ’s FigureWidget is completely compatible with the new JupyterLab environment.

Interactive coding in the new JupyterLab environment with the FigureWidget. The FigureWidget is within an output cell of a notebook, pulled out as a separate view in the pane on the right. The `fig.add_histogram2dcontour` method is one of the new imperative API interfaces, designed specifically for the interactive Jupyter computing environments.

Static Image Export

plotly.js graphs are rendered with JavaScript in a browser context. In order to create static images of these charts, without opening a browser, we needed a way to render the graphs in some type of headless browser.

In May, we open-sourced Orca , a command-line application that generates static images of plotly graphs. Orca has been used in production at Plotly for about ninemonths, creating images for tens of millions of charts.

Orca is the technology for creating static images of Plotly graphs. We’re working on bundling Orca as an optional dependency in the official plotly.py package. For now, follow the instructions in the Orca repository .

Backwards Compatibility

At Plotly, we take backwards compatibility pretty seriously.

Over the last five years of developing plotly.js , we’ve published over 100 releases , all without a single breaking change to the API.

plotly.py graphs are rendered with plotly.js , so this release does not change the way that your graphs are rendered. The attributes of the graphs remain one-to-one with plotly.js, so the declarative interface to the figures (data, layout, and all of the nested properties within) has not changed either.

What has changed is the structure of the Python classes that wrap these attributes: plotly.graph_objs (“ go ”). We’re now placing our nested properties, like go.Marker , in a namespace underneath their parent attribute. That is, go.Marker is now accessible as go.scatter.Marker or go.bar.Marker or whatever trace it is nested within. By providing unique objects under the parent-trace namespace, we can provide better validation (the properties for a marker object within a scatter trace may be different than the properties of a marker object within a bar trace). Although deprecated, the previous objects are still supported, they just won’t provide the same level of validation as our new objects.

Of course, you’re not required to use these higher level objects. You can still write your graphs as straight dictionaries and lists. Or, you can mix-and-match. Here are four equivalent representations of a trace:

# New method
go.Scatter(
    x=[1, 2, 3],
    y=[3, 1, 2],
    marker=go.scatter.Marker(color='blue')
)

# Old method - deprecated go.Marker object
go.Scatter(
    x=[1, 2, 3],
    y=[3, 1, 2],
    marker=go.Marker(color='blue')
)

# Mix-and-match dicts and lists
go.Scatter(
    x=[1, 2, 3],
    y=[3, 1, 2],
    marker={'color': 'blue'}
)

# Or use primitive types all the way down
{
    'x': [1, 2, 3],
    'y': [3, 1, 2],
    'type': 'scatter',
    'marker': {'color': 'blue'}
}

We have a more detailed document in our plotly 3.0.0 migration guide . Please try out the prerelease and let us know if we missed anything!

Looking Forward and Further Resources

• 3.0.0 is a major release which means there are some breaking changes . Get ahead of the release by trying out our release candidate with our migration guide . Questions, issues? Please make an issue :bow:
• Jon will be speaking about v3.0.0 at SciPy 2018 in Austin. If you’re there, come say hello! :wave:
• Jon also wrote a paper for SciPy 2018 outlining these features and, more technically, the underlying architecture of the code. Check out the draft .
• Can’t make it to SciPy? Jon will be giving a webinar about plotly.py 3.0.0 on July 25. Sign up here .
• Dash , our open source Python library for standalone analytic apps, uses plotly.py and is compatible with plotly.py v3.0.0 .
• The underlying graphing library that powers plotly.py (and other libraries like Dash) is plotly.js. This library is in active development. V iew our latest releases on GitHub .
• More of an R person? Carson has been making a lot of fantastic improvements in the Plotly R library lately.
• Follow our latest work with Twitter: @plotlygraphs
• Much of our open source work is funded directly from organizations that have budgets for software. Many thanks to everyone that has supported us so far :heart:. Interested in sponsoring a particular feature? Please get in touch .
• Plotly, Inc. also provides public, private, and on-premise platforms for publishing graphs, datasets, and interactive Dash applications. More than half of our revenue directly funds open source libraries like these. Learn more about our commercial products .

Much love —

Team Plotly