Visualize large time-series data in plotly

Overview

Plotly-Resampler logo

PyPI Latest Release codecov Code quality PRs Welcome Documentation Testing

plotly_resampler enables visualizing large sequential data by adding resampling functionality to Plotly figures.

example demo

In this Plotly-Resampler demo over 110,000,000 data points are visualized!

Installation

pip pip install plotly-resampler

Usage

To add dynamic resampling to your plotly Figure, you should;

  1. wrap the constructor of your plotly Figure with FigureResampler
  2. call .show_dash() on the Figure

(OPTIONAL) add the trace data as hf_x and hf_y (for faster initial loading)

Minimal example

import plotly.graph_objects as go; import numpy as np
from plotly_resampler import FigureResampler

x = np.arange(1_000_000)
noisy_sin = (3 + np.sin(x / 200) + np.random.randn(len(x)) / 10) * x / 1_000

fig = FigureResampler(go.Figure())
fig.add_trace(go.Scattergl(name='noisy sine', showlegend=True), hf_x=x, hf_y=noisy_sin)

fig.show_dash(mode='inline')

Features

  • Convenient to use:
    • just add the FigureResampler decorator around a plotly Figure consructor and call .show_dash()
    • allows all other ploty figure construction flexibility to be used!
  • Environment-independent
    • can be used in Jupyter, vscode-notebooks, Pycharm-notebooks, as application (on a server)
  • Interface for various downsampling algorithms:
    • ability to define your preffered sequence aggregation method

Important considerations & tips

  • When running the code on a server, you should forward the port of the FigureResampler.show_dash method to your local machine.
  • In general, when using downsamplingm one should be aware of (possible) aliasing effects.
    The [R] in the legend indicates when the corresponding trace is being resampled (and thus possibly distorted) or not.

Future work 🔨

  • Add downsampler methods that take aliasing into account
  • Parallelize the resampling


👤 Jonas Van Der Donckt, Jeroen Van Der Donckt, Emiel Deprost

Comments
  • FigureWidget() update

    FigureWidget() update

    Hi, very useful project, all my career I dream about such thing. It seems that it can make plotly usable in real life, not only in the iris dataset.

    Is there a way to dynamic update the resampled FigureWidget instance? For example, in the Jupyter lab: image

    The last cell causes an update of the data in the chart if fig is an FigureWidget instance, but does not update if the instance is a FigureResampler(go.FigureWidget())

    Test case:

    import numpy as np
    from plotly_resampler import FigureResampler
    
    x = np.arange(1_000_000)
    noisy_sin = (3 + np.sin(x / 15000) + np.random.randn(len(x)) / 10) * x / 1_000
    
    fig = FigureResampler(go.FigureWidget())
    fig.add_scattergl(name='noisy sine', showlegend=True, x=x, y=noisy_sin)
    
    fig.update_layout(autosize=True, height=300, template=None, legend=dict(x=0.1, y=1, orientation="h"),
                      margin=dict(l=45, r=15, b=20, t=30, pad=3))
    fig.show()
    
    # does not update chart if fig is FigureResampler instance
    with fig.batch_update():
        fig.data[0].y = -fig.data[0].y
    

    PS: It seems that resampling only works in dash, but not in jupyterlab?

    opened by zxweed 26
  • After upgrading from 0.3 to 0.8.1, one of my notebook cells with resampler runs indefinitely

    After upgrading from 0.3 to 0.8.1, one of my notebook cells with resampler runs indefinitely

    I have several figures in a notebook. All other figures plot correctly and I can wrap PlotlyResampler around and show them. However, one particular figure plots just fine, but when I wrap it in PlotlyResampler my cell keeps running indefinitely. This unfortunately blocks my update to 0.8.1. Do you have any idea @jonasvdd ?

    #32

    FigureResampler(fig, default_n_shown_samples=MAX_POINTS).show_dash(mode="inline") image

    Other observations:

    • if I downgrade to 0.3.0 it still does not work,
    • if I downgrade to 0.3.0 and remove the dash show, it works fine image
    opened by Alexander-Serov 25
  • Trying to make the resampler work with dynamic graphs

    Trying to make the resampler work with dynamic graphs

    So I made this minimal example but I can not figure out why I can't get the callbacks to work.

    `

    """
    Minimal dynamic dash app example.
    """
    
    import numpy as np
    import plotly.graph_objects as go
    import trace_updater
    from dash import Dash, Input, Output, State, dcc, html
    from plotly_resampler import FigureResampler
    
    x = np.arange(1_000_000)
    noisy_sin = (3 + np.sin(x / 200) + np.random.randn(len(x)) / 10) * x / 1_000
    
    app = Dash(__name__)
    
    fig = FigureResampler(go.Figure(go.Scatter(x=x, y=noisy_sin)))
    
    
    app.layout = html.Div(
        [
            html.Div(
                children=[
                    html.Button("Add Chart", id="add-chart", n_clicks=0),
                ]
            ),
            html.Div(id="container", children=[]),
        ]
    )
    
    
    @app.callback(
        Output("container", "children"),
        Input("add-chart", "n_clicks"),
        State("container", "children"),
    )
    def display_graphs(n_clicks: int, div_children: list[html.Div]) -> list[html.Div]:
        """
        This function is called when the button is clicked. It adds a new graph to the div.
        """
        figure = fig
        figure.register_update_graph_callback(
            app=app,
            graph_id=f"graph-id-{n_clicks}",
            trace_updater_id=f"trace-updater-id-{n_clicks}",
        )
    
        new_child = html.Div(
            children=[
                dcc.Graph(id=f"graph-id-{n_clicks}", figure=fig),
                trace_updater.TraceUpdater(
                    id=f"trace-updater-id-{n_clicks}", gdID=f"graph-id-{n_clicks}"
                ),
            ],
        )
        div_children.append(new_child)
        return div_children
    
    
    if __name__ == "__main__":
        app.run_server(debug=True)
    

    `

    question 
    opened by prokie 23
  • Unable to install `plotly-resampler` on some linux distributions

    Unable to install `plotly-resampler` on some linux distributions

    I am using plotly-resampler, which installs correctly on my local Windows machine and on another Linux machine I have access to (by just using pip install plotly-resampler). However, we also run Gitlab-CI tests in a controlled environment and installing with pip kept failing in that environment. The exact error was

    Building wheel for lttbc (setup.py): started
      Building wheel for lttbc (setup.py): finished with status 'error'
      ERROR: Command errored out with exit status 1:
       command: /usr/local/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-jgc_t9dg/lttbc_[494](https://gitlab.edf-sf.com/optim/statistical_analysis/-/jobs/131076#L494)9a59daf574371b0f97218e19bdac5/setup.py'"'"'; __file__='"'"'/tmp/pip-install-jgc_t9dg/lttbc_4949a59daf574371b0f97218e19bdac5/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-6yato32h
           cwd: /tmp/pip-install-jgc_t9dg/lttbc_4949a59daf574371b0f97218e19bdac5/
      Complete output (16 lines):
      running bdist_wheel
      running build
      running build_ext
      building 'lttbc' extension
      creating build
      creating build/temp.linux-x86_64-3.9
      gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -I/usr/local/lib/python3.9/site-packages/numpy/core/include -I/tmp/pip-install-jgc_t9dg/lttbc_4949a59daf574371b0f97218e19bdac5 -I/usr/local/lib/python3.9/site-packages/numpy/core/include -I/tmp/pip-install-jgc_t9dg/lttbc_4949a59daf574371b0f97218e19bdac5 -I/usr/local/include/python3.9 -c lttbc.c -o build/temp.linux-x86_64-3.9/lttbc.o
      In file included from /usr/lib/gcc/x86_64-linux-gnu/10/include/syslimits.h:7,
                       from /usr/lib/gcc/x86_64-linux-gnu/10/include/limits.h:34,
                       from /usr/local/include/python3.9/Python.h:11,
                       from lttbc.c:2:
      /usr/lib/gcc/x86_64-linux-gnu/10/include/limits.h:195:15: fatal error: limits.h: No such file or directory
        195 | #include_next <limits.h>  /* recurse down to the real one */
            |               ^~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/gcc' failed with exit code 1
      ----------------------------------------
      ERROR: Failed building wheel for lttbc
      Running setup.py clean for lttbc
     Running setup.py install for lttbc: started
        Running setup.py install for lttbc: finished with status 'error'
        ERROR: Command errored out with exit status 1:
         command: /usr/local/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-lgmdp697/lttbc_3817a2fad4f7468ea159ce68739ae851/setup.py'"'"'; __file__='"'"'/tmp/pip-install-lgmdp697/lttbc_3817a2fad4f7468ea159ce68739ae851/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-uwdc83fp/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.9/lttbc
             cwd: /tmp/pip-install-lgmdp697/lttbc_3817a2fad4f7468ea159ce68739ae851/
        Complete output (16 lines):
        running install
        running build
        running build_ext
        building 'lttbc' extension
        creating build
        creating build/temp.linux-x86_64-3.9
        gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -I/usr/local/lib/python3.9/site-packages/numpy/core/include -I/tmp/pip-install-lgmdp697/lttbc_3817a2fad4f7468ea159ce68739ae851 -I/usr/local/lib/python3.9/site-packages/numpy/core/include -I/tmp/pip-install-lgmdp697/lttbc_3817a2fad4f7468ea159ce68739ae851 -I/usr/local/include/python3.9 -c lttbc.c -o build/temp.linux-x86_64-3.9/lttbc.o
        In file included from /usr/lib/gcc/x86_64-linux-gnu/10/include/syslimits.h:7,
                         from /usr/lib/gcc/x86_64-linux-gnu/10/include/limits.h:34,
                         from /usr/local/include/python3.9/Python.h:11,
                         from lttbc.c:2:
        /usr/lib/gcc/x86_64-linux-gnu/10/include/limits.h:195:15: fatal error: limits.h: No such file or directory
          195 | #include_next <limits.h>  /* recurse down to the real one */
              |               ^~~~~~~~~~
        compilation terminated.
        error: command '/usr/bin/gcc' failed with exit code 1
        ----------------------------------------
    ERROR: Command errored out with exit status 1: /usr/local/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-lgmdp697/lttbc_3817a2fad4f7468ea159ce68739ae851/setup.py'"'"'; __file__='"'"'/tmp/pip-install-lgmdp697/lttbc_3817a2fad4f7468ea159ce68739ae851/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-uwdc83fp/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.9/lttbc Check the logs for full command output.
    

    I am posting it here in case it helps other folks who might encounter the same problem.

    I have played around with the test environment and was able to install all packages by executing

    apt update && apt install -yqq --no-install-recommends gcc musl-dev linux-headers-amd64 libc-dev
    

    before the pip command. This allowed me to install the apparently missing linux header, lttbc and ploty-resampler. However, for some reason resulted in an incompatibility with numpy:

    ------------------------------- Captured stderr --------------------------------
    RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
    ___________________ ERROR collecting pv/test_pv_generator.py ___________________
    ImportError while importing test module '/builds/--YGsyLe/2/optim/statistical_analysis/tests/pv/test_pv_generator.py'.
    Hint: make sure your test modules/packages have valid Python names.
    Traceback:
    /usr/local/lib/python3.9/importlib/__init__.py:127: in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
    tests/test.py:12: in <module>
        from plotly_resampler import FigureResampler
    /usr/local/lib/python3.9/site-packages/plotly_resampler/__init__.py:8: in <module>
        from .figure_resampler import FigureResampler
    /usr/local/lib/python3.9/site-packages/plotly_resampler/figure_resampler.py:28: in <module>
        from .downsamplers import AbstractSeriesDownsampler, LTTB
    /usr/local/lib/python3.9/site-packages/plotly_resampler/downsamplers/__init__.py:5: in <module>
        from .downsamplers import LTTB, EveryNthPoint, AggregationDownsampler
    /usr/local/lib/python3.9/site-packages/plotly_resampler/downsamplers/downsamplers.py:8: in <module>
        import lttbc
    E   ImportError: numpy.core.multiarray failed to import
    

    So I abandoned.

    As I said, I am publishing this info here in case someone stumbles on a similar issue, so feel free to close. However, I saw that lttbc is a top-level dependency of plotly-resampler and is still in early stages (version <1) and has not been updated since 2020. So there is little chance its python wheels will be changed anytime soon. So I wonder, whether on the plotly-resampler side we could add a try-except for lttbc import and fall back onto another resampler if lttbc is unavailable for import? Or, perhaps, if you have any idea of how to install the lttbc dependency without gcc compiling, it would be much appreciated!

    I understand this is not directly related to ploty-resampler. I have thought about posting in lttbc instead, but the repo does not seem to be actively maintained. Thanks again for the resampler. Great idea!

    installation 
    opened by Alexander-Serov 13
  • adding requirements.txt for example_folder

    adding requirements.txt for example_folder

    adding requirements for example_folder will help us to run the application more easily ( before we have to install required modules separately) we can now just do pip install -r requirements.txt and pip will install dependencies for us.

    documentation enhancement examples 
    opened by someshfengde 11
  • when install plotly-resampler Collecting lttbc==0.2.0 always fail

    when install plotly-resampler Collecting lttbc==0.2.0 always fail

    Collecting lttbc==0.2.0 Downloading lttbc-0.2.0.tar.gz (91 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 91.4/91.4 kB 247.6 MB/s eta 0:00:00 Preparing metadata (setup.py): started Preparing metadata (setup.py): finished with status 'error' error: subprocess-exited-with-error

    × python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [6 lines of output] Traceback (most recent call last): File "", line 2, in File "", line 34, in File "/tmp/pip-install-j1wawuxs/lttbc_df1b4fc03c7946fd893244304b8faa7f/setup.py", line 7, in import numpy ModuleNotFoundError: No module named 'numpy' [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed

    × Encountered error while generating package metadata. ╰─> See above for output.

    note: This is an issue with the package mentioned above, not pip. hint: See above for details.

    bug duplicate help wanted installation 
    opened by clemente0420 10
  • Using plotly-resampler with dashapp?

    Using plotly-resampler with dashapp?

    I'm having some issues when rendering this figure with dashapp.

    Firstly, I make a dashapp with the following controls:

    controls = [
                dcc.Graph(
                    id='uptime-graph',
                   ''' some additional styling"""
                    }
                ),
                dcc.Graph(
                    id='timeseries-graph',
                    figure={
                        'data': []
                        
                    }
                )
            ]
    
    

    I'm using an uptime graph to select specific trace segments I want to look at. then, I update 'timeseries-graph' with a callback upon selection within the uptime graph:

    def update_timeseries(relayoutData):
        if new_coords is None or 'autosize' in new_coords.keys() or 'xaxis.autorange' \
            in new_coords.keys():
                return None
        start = new_coords['xaxis.range[0]']
        end   = new_coords['xaxis.range[1]']
        dict_frame = self.model.get_timeseries(start,end)
        n_titles, plotting_dict = self._restructure_data(dict_frame)
    
        fig = FigureResampler(
                            make_subplots(
                                rows=len(plotting_dict.keys()),
                                cols=1,
                                row_titles=n_titles,
                                vertical_spacing=0.001,
                                shared_xaxes=True),
                                default_n_shown_samples=5_000,
                                    verbose=False,
                        )
        fig['layout'].update(height=1700)
        row_iterator = 1
        has_legend = {'ex':False,'ey':False,'hx':False,'hy':False,'hz':False}
        for station_key in plotting_dict.keys():
            for trace_data in plotting_dict[station_key]:
                color = self._get_trace_color(station_key)
                name, showlegend =self._legend_name_parser(has_legend,station_key)
                fig.add_trace(go.Scattergl(name=name, showlegend=showlegend,connectgaps=False,
                                                       line={'color':color,'dash':'solid'}), 
                                hf_x=trace_data.time, hf_y=trace_data['value'],row=row_iterator,col=1)
            row_iterator+=1
        print('updated timeseries figure')
        fig.show_dash(mode='inline')
        return fig
    
    
    @dashapp.callback(
        Output('timeseries-graph', 'figure'),
        Input('uptime-graph', 'relayoutData'))
    def uptime_data_select(relayoutData):
        fig = controller.update_timeseries_daterange(relayoutData)
        return fig
    

    It kinda works, then begins to spit the same error every four seconds, preventing any further interaction with the webapp

    
    Traceback (most recent call last):
      File "/Users/.../miniconda3/envs/mtpytest/lib/python3.9/site-packages/flask/app.py", line 2077, in wsgi_app
        response = self.full_dispatch_request()
      File "/Users.../miniconda3/envs/mtpytest/lib/python3.9/site-packages/flask/app.py", line 1525, in full_dispatch_request
        rv = self.handle_user_exception(e)
      File "/Users/.../miniconda3/envs/mtpytest/lib/python3.9/site-packages/flask/app.py", line 1523, in full_dispatch_request
        rv = self.dispatch_request()
      File "/Users/.../miniconda3/envs/mtpytest/lib/python3.9/site-packages/flask/app.py", line 1509, in dispatch_request
        return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
      File "/Users/.../miniconda3/envs/mtpytest/lib/python3.9/site-packages/dash/dash.py", line 1382, in dispatch
        raise KeyError(msg.format(output)) from missing_callback_function
    KeyError: "Callback function not found for output 'timeseries-graph.figure', perhaps you forgot to prepend the '@'?"
    2022-05-11T15:10:10 [line 1455] local_app.log_exception - ERROR: Exception on /_dash-update-component [POST]
    Traceback (most recent call last):
      File "/Users/kevinmendoza/miniconda3/envs/mtpytest/lib/python3.9/site-packages/dash/dash.py", line 1344, in dispatch
        cb = self.callback_map[output]
    KeyError: 'timeseries-graph.figure'
    
    

    I suppose its possible i'm not using it correctly, but if I am, there appears to be an error with resampling hooking back into the graph.

    documentation question 
    opened by k-a-mendoza 10
  • Cannot assing hf_data series when initial size is small

    Cannot assing hf_data series when initial size is small

    I make a graph that initially has a small amount of data (or no data at all), and then they are incrementally added. I have encountered that if I don't set hf_y at all (or pass less than 1000 points), then the hf_data property is not created (because there is no need to resample, I guess.) Is there a way to create it later, or create it even in case of small amount of data)? image

    Testcase: resampler.debug.zip

    bug documentation discussion 
    opened by zxweed 9
  • Python 3.11 not supported

    Python 3.11 not supported

    Hi, I've tired to run the "working examples" in Python3.11 and get the following error: image When running the same example in Python3.9, it works fine. really cool package! thx for all the support!

    opened by ekreate 8
  • Improve docs

    Improve docs

    Add more docs (+ examples) about:

    • [x] How to integrate plotly-resampler with your custom dash app
    • [x] Plotly-resampler & non hf-traces
    • [x] Tips & tricks:
      • optimizing your code for faster figure construction
      • how to (not) add hf-traces and why you should do so
      • Aliasing
    • [x] Position ploty-resampler to other tools (Plotjuggler, DataShader, (Holoviews), FigureWidgets + interaction ...) see plotly-resampler benchmarks
    opened by jonasvdd 8
  • :package: improve docs

    :package: improve docs

    This PR aims at improving the documentation of this package, as we have had several issues about the lacking documentation (#99, #91, #102)

    • [x] create FAQ
    • [x] update Readme.md
    • [x] add Contributing.md
    • [ ] add Changelog.md
    • [x] add requirements.txt to example folder
    • [x] improve dash integration docs
    • [x] add very minimal dash integration example
    • [x] update example notebooks
    • [x] add more documentation to the C code
    • [x] incorporate figure serialization into the docs

    ##Other stuff this PR does;

    • [x] enable numerically unstable test (see #93)

    incorporate Plotly-resampler's LTTBc bindings

    TODO:

    • [x] test lttb_core_py
    • [x] test the EfficientLTTB method when LTTB_core_c is not available note: this method is tested for equality with the lttbc method
    documentation enhancement 
    opened by jvdd 7
  • Linking zoom between dynamically generated plots

    Linking zoom between dynamically generated plots

    Hi,

    I am making a dashboard where I want to visualize a large timeseries dataset. Currently the user can upload a datafile and plots are generated sorted by physical quantity (e.g. plot all temperatures together, plot all pressures together). This works perfectly with the resampler!

    Now I want to add the functionality where the x-axis of all plots zoom when the user zooms in one of the plots. I created the following (non-)working example

    from uuid import uuid4
    
    
    from dash import dcc, ctx, ALL, MATCH, no_update
    from dash import html
    from dash_extensions.enrich import Dash, ServersideOutput, Output, Input, State, Trigger, DashProxy, TriggerTransform, ServersideOutputTransform, MultiplexerTransform 
    
    import pandas as pd
    import numpy as np
    
    import plotly.io as pio
    import plotly.graph_objects as go
    
    from plotly_resampler import FigureResampler
    from trace_updater import TraceUpdater
    
    pio.renderers.default='browser'
    pd.options.plotting.backend = "plotly"
    
    external_stylesheets = ['https://codepen.io/chriddyp/pen/bWLwgP.css']
    app = Dash(__name__,external_stylesheets=external_stylesheets)\
        
    app = DashProxy(
        __name__,
        suppress_callback_exceptions=True,
        external_stylesheets=external_stylesheets,
        transforms=[ServersideOutputTransform(), TriggerTransform() ,MultiplexerTransform()],
    )
    app.layout = html.Div([
        html.Button("plot", id="btn-plot"),
        dcc.Store(id="store-temp"),
        html.Div(id='graph-container'),
        ])
    
    
    @app.callback(
    
        ServersideOutput("store-temp", "data"),
        Input("btn-plot", "n_clicks"),
        )
    def store_data(click):
        print('store data')
        n=10000
        x = np.arange(n)
        df=pd.DataFrame()
        y1 = (np.sin(x / 200) * 1 + np.random.randn(n) / 10 * 1 )
        y2 = (np.sin(x / 100) * 1 + np.random.randn(n) / 20 * 1 )
        df['y1']=y1
        df['y2']=y2
        return df
    
    @app.callback(
        Output("graph-container", "children"),
        State("graph-container", "children"),
        Input("store-temp", "data"),
        prevent_initial_call=True
        )
    def create_graphs(gc_children,df):
        print('creating graphs')
        gc_children = [] if gc_children is None else gc_children
        
        uid = str(uuid4())
        diff_container = html.Div(
            children=[
                # The graph and its needed components to serialize and update efficiently
                # Note: we also add a dcc.Store component, which will be used to link the
                #       server side cached FigureResampler object
                dcc.Graph(id={"type": "dynamic-graph", "index": uid}, figure=go.Figure()),
                dcc.Store(id={"type": "store", "index": uid}),
                dcc.Store(id={"type": "store-columns", "index": uid},data=['y1','y2']),
                TraceUpdater(id={"type": "dynamic-updater", "index": uid}, gdID=f"{uid}"),
                # This dcc.Interval components makes sure that the `construct_display_graph`
                # callback is fired once after these components are added to the session
                # its front-end
                dcc.Interval(
                    id={"type": "interval", "index": uid}, max_intervals=1, interval=1
                ),
            ],
        )
        gc_children.append(diff_container)
        
        uid = str(uuid4())
        diff_container = html.Div(
            children=[
                dcc.Graph(id={"type": "dynamic-graph", "index": uid}, figure=go.Figure()),
                dcc.Store(id={"type": "store", "index": uid}),
                dcc.Store(id={"type": "store-columns", "index": uid},data=['y2', 'y1']),
                TraceUpdater(id={"type": "dynamic-updater", "index": uid}, gdID=f"{uid}"),
                dcc.Interval(
                    id={"type": "interval", "index": uid}, max_intervals=1, interval=1
                ),
            ],
        )
        gc_children.append(diff_container)
            
        print('store data cb finish')
        return gc_children
    
    
    @app.callback(
        ServersideOutput({"type": "store", "index": MATCH}, "data"),
        Output({"type": "dynamic-graph", "index": MATCH}, "figure"),
        State("store-temp", "data"),
        State({"type": "store-columns", "index": MATCH}, "data"),
        Trigger({"type": "interval", "index": MATCH}, "n_intervals"),
        prevent_initial_call=True,
    )
    def construct_display_graph(df,columns) -> FigureResampler:
        df2=df[columns]
        fr = FigureResampler(go.Figure(), verbose=True)
        for col in df2.columns:
            fr.add_trace(go.Scattergl(name=col, mode='lines'),hf_y=df2[col],hf_x=df2.index)
        fr.update_traces(connectgaps=True)
        return fr, fr
    
    
    # @app.callback(
    #     Output({"type": "dynamic-updater", "index": MATCH}, "updateData"),
    #     Input({"type": "dynamic-graph", "index": MATCH}, "relayoutData"),
    #     State({"type": "store", "index": MATCH}, "data"),
    #     prevent_initial_call=True,
    #     memoize=True,
    # )
    # def update_fig(relayoutdata: dict, fig: FigureResampler):
    #     print(fig)
    #     if fig is not None:
    #         return fig.construct_update_data(relayoutdata)
    #     return no_update
    
    @app.callback(
        Output({"type": "dynamic-updater", "index": ALL}, "updateData"),
        Input({"type": "dynamic-graph", "index": ALL}, "relayoutData"),
        State({"type": "dynamic-graph", "index": ALL}, "id"),
        State({"type": "store", "index": ALL}, "data"),
        prevent_initial_call=True,
        memoize=True,
    )
    def update_fig(relayoutdata: list[dict],ids: list[dict], figs: list[FigureResampler]):
        figure_updated = ctx.triggered_id # get the id of the figure that triggered the callback
        triggered_index=ids.index(figure_updated) # get the index of the figure in the Input/Output lists of the callback
        # print(figure_updated)
        # print(relayoutdata)
        print(ids)
        # print('index : '+ str(triggered_index))
        zoomdata=dict(relayoutdata[triggered_index]) # get the relayoutdata of the figure that triggered the callback
        
        new_relayoutdata = []
        for i, data in enumerate(relayoutdata): # loop over current relayoutdata
            if i == triggered_index:
                new_relayoutdata.append(zoomdata) # keep relayoutdata of figure that triggered callback
            else:
                if 'xaxis.range[0]' in zoomdata:
                    data = dict(relayoutdata[i])
                    data['xaxis.range[0]'] = zoomdata ['xaxis.range[0]']
                    data['xaxis.range[1]'] = zoomdata ['xaxis.range[1]']
                    data['xaxis.autorange'] = False
                    new_relayoutdata.append(data)
                else:
                    new_relayoutdata.append(zoomdata)
        print(zoomdata)
        updatedata = []
        print(figs)
        for i,fig in enumerate(figs):
            if fig is None:
                return [no_update, no_update]
            else:
                updatedata.append(fig.construct_update_data(new_relayoutdata[i]))
        return updatedata
    
    
    
    if __name__ == '__main__':
        app.run_server(debug=True, port=9023)
    

    Like in the 11_sine_generator.py example plots are generated with a unique id. In the update_fig() callback I want to update all graphs if one of them updates. In the example (commented out in my code) MATCH is used. To get all FigureResampler object I replaced it with ALL. However, State({"type": "store", "index": ALL}, "data") produces a list with hashes(?) like ['b4de3743d91d23d6b85d2bcdd11cb531', '7d9efd7bb10eafd6cfcffe27b9ec566c']where State({"type": "store-columns", "index": MATCH}, "data") gives me the single FigureResampler object. With ALL I would expect a list of FigureResampler objects.

    What am I missing here, how can I obtain the FigureResampler objects so I can modify them with construct_update_data()?

    question examples 
    opened by Wout-S 2
  • FigureResampler not working when using backend parameter on ServersideOutputTransform

    FigureResampler not working when using backend parameter on ServersideOutputTransform

    Hello and thank you for this awesome project.

    I'm trying to specify backend parameter on ServersideOutputTransform in order to set threshold parameter on FileSystemCache, but then if I do that my callback with fig.construct_update_data(relayoutdata) stop working. Any idea about why this is happening?

    Code:

    from dash import html, dcc, Output, Input, ctx, State, no_update
    import dash_bootstrap_components as dbc
    import plotly.graph_objects as go
    
    from dash_extensions.enrich import (
        DashProxy,
        ServersideOutput,
        ServersideOutputTransform
    )
    
    from plotly_resampler import FigureResampler
    from trace_updater import TraceUpdater
    from flask_caching.backends import FileSystemCache
    
    from settings import DROPDOWN_OPTIONS
    from preprocess import Preprocess
    # ----------------- Defining Global Variable with all data ---------------------
    
    data = Preprocess()
    data.preprocess_all_data()
    
    
    def get_plot(plot_domain, acc, file):
        return data.get_plot(plot_domain, acc, file)
    
    
    def get_stats(plot_domain, acc, file):
        return data.get_stats(plot_domain, acc, file)
    
    # ---------------------- Create App ----------------------------
    
    backend = FileSystemCache(cache_dir='file_system_store',
                               threshold=3)
    
    
    app = DashProxy(__name__, external_stylesheets=[dbc.themes.BOOTSTRAP],
                    transforms=[ServersideOutputTransform(backend=backend)])
    
    
    def create_figure(files_list=['a_normal__2000rpm'], plot_domain='time', acc=0):
    
        fig = FigureResampler(go.Figure(), default_n_shown_samples=10_000)
    
        if plot_domain == 'fft':
    
            for file in files_list:
    
                fig.add_trace(get_plot(plot_domain, acc, file))
    
            fig.update_layout(
                yaxis_title='Amplitude',
                xaxis_title='Frequency (Hz)',
                width=1400,
                height=600)
    
        elif plot_domain == 'time':
    
            for file in files_list:
    
                fig.add_trace(get_plot(plot_domain, acc, file))
    
            fig.update_layout(
                yaxis_title='Amplitude (g)',
                xaxis_title='Time (s)',
                width=1400,
                height=600)
    
        return fig
    
    
    app.layout = html.Div(
        [
            dbc.Row(
                dbc.Col([
                    html.H1('Medições de Bancada')
                ], width={'size': 6, 'offset': 4}
                ),
                align='center'
            ),
            dbc.Row(
                [
                    dbc.Col(width={'size': 3, 'offset': 1},
                            children=[
                            html.H3('Medições:'),
                            dcc.Dropdown(id='Arquivo',
                                            options=DROPDOWN_OPTIONS,
                                            multi=True,
                                            optionHeight=50
                                         ),
                                html.H3('Tipo de Gráfico:'),
                                dcc.RadioItems(
                                    id='plot_domain',
                                    options=[
                                        {'label': 'FFT', 'value': 'fft'},
                                        {'label': 'Tempo', 'value': 'time'}
                                    ],
                                    labelStyle={'display': 'block'},
                                    value='time'
                            )
                            ]
                            ),
                    dbc.Col(width={'size': 5, 'offset': 2},
                            children=[
                                html.H4('Tempo de Aquisição: 30s'),
                                html.H4(
                                    'Frequência de Amostragem: 25600 Hz'),
                                html.H3('Spot:'),
                                dcc.RadioItems(
                                    id='accel',
                                    options=[
                                        {'label': 'Drive End Bearing',
                                            'value': 0},
                                        {'label': 'Non Drive End Bearing',
                                            'value': 1},
                                        {'label': 'Drive  End Motor',
                                            'value': 2},
                                        {'label': 'Fan End Motor',
                                            'value': 3}
                                    ],
                                    labelStyle={'display': 'block'},
                                    value=0
                                )
                    ]
                    )
                ]
            ),
            dbc.Row(
                children=[
                    dcc.Graph(id="plot"),
                    dcc.Loading(dcc.Store(id='storage-data')),
                    TraceUpdater(id='dynamic-updater', gdID='plot')
                ]
            ),
            dbc.Row(
                id='stats-card-row'
            )
    
    
        ]
    )
    
    
    # ----------------- App Callbacks -----------------------
    
    
    @app.callback(
        ServersideOutput('storage-data', 'data'),
        Output('plot', 'figure'),
        Input('Arquivo', 'value'),
        Input('plot_domain', 'value'),
        Input('accel', 'value'),
        prevent_initial_call=True
    
    
    )
    def create_new_plot(dropdown_selection, plot_domain, acc):
    
        fig = create_figure(
            dropdown_selection, plot_domain=plot_domain, acc=acc)
    
        return fig, fig
    
    
    @app.callback(
        Output('stats-card-row', 'children'),
        Input('Arquivo', 'value'),
        Input('plot_domain', 'value'),
        Input('accel', 'value'),
        prevent_initial_call=True
    )
    def create_stats_card(dropdown_selection, plot_domain, acc):
    
        card = dbc.Card(
            [
                dbc.CardHeader(html.H2('Estatísticas das medições')),
                dbc.CardBody(dbc.ListGroup(
                    [
                        dbc.ListGroupItem(children=[
                            html.H3(stats["name"]),
                            html.H4(f'Média: {stats["mean"]}'),
                            html.H4(f'Valor RMS: {stats["rms"]}'),
                            html.H4(f'Skewness: {stats["skewness"]}')
                        ]
                        )
                        for stats in [get_stats(plot_domain, acc, file) for file in dropdown_selection]
                    ],
                    flush=True,
                ),
                    style={"width": "100rem"},
                )])
    
        return card
    
    
    @app.callback(
        Output("dynamic-updater", "updateData"),
        Input("plot", "relayoutData"),
        State("storage-data", "data"),
        prevent_initial_call=True,
        memoize=True,
    )
    def update_fig(relayoutdata: dict, fig: FigureResampler):
        if fig is not None:
            return fig.construct_update_data(relayoutdata)
        return no_update
    
    
    # ----------- Run App --------------
    
    if __name__ == '__main__':
    
        app.run_server(debug=True)
    
    opened by VictorBauler 1
  • Validate whether orjson casting still is necessary with more recent orjson versions

    Validate whether orjson casting still is necessary with more recent orjson versions

    https://github.com/predict-idlab/plotly-resampler/blob/5df40fd0575db62bd06d20c129d8643d75dba558/plotly_resampler/figure_resampler/figure_resampler_interface.py#L703-L705

    Maybe also make a test which validates this? (expected behavior -> casting is needed)

    opened by jonasvdd 1
Releases(v0.8.3)
  • v0.8.3(Dec 2, 2022)

    Main changes:

    • Try to parse the object dtype of the hf_x property in plotly-resampler, see #116 #120 #115
    • Add the check_nan option to the add_trace(s) methods. Setting this variable to True allows for graph construction speedups when no Nans are present in your data.

    What's Changed

    • :pen: add contributing guide + changelog by @jvdd in https://github.com/predict-idlab/plotly-resampler/pull/111
    • 🔧 Tweaks - improve code quality, fix type-checking bug when IPywidgets is not installed & loosen up plotly-version by @jonasvdd in https://github.com/predict-idlab/plotly-resampler/pull/114
    • :bug: update layout axes range bug by @jvdd in https://github.com/predict-idlab/plotly-resampler/pull/126
    • ✨ fix + test for #124 by @jonasvdd in https://github.com/predict-idlab/plotly-resampler/pull/127
    • :dash: making orjson non-option and fixating werkzeug #123 by @jonasvdd in https://github.com/predict-idlab/plotly-resampler/pull/128
    • 💪🏼 making orjson serialization more robust, see #118 by @jonasvdd in https://github.com/predict-idlab/plotly-resampler/pull/131
    • Resample bug, see #137 by @jonasvdd in https://github.com/predict-idlab/plotly-resampler/pull/138
    • :sparkles: add check_nans to add_trace(s) by @jvdd in https://github.com/predict-idlab/plotly-resampler/pull/140
    • :bug: parse object arrays for hf_x by @jvdd in https://github.com/predict-idlab/plotly-resampler/pull/116

    Full Changelog: https://github.com/predict-idlab/plotly-resampler/compare/v0.8.0...v0.8.3

    Source code(tar.gz)
    Source code(zip)
  • v0.8.0(Aug 15, 2022)

    Major changes

    Faster aggregation 🐎

    the lttbc dependency is removed; and we added our own (faster) lttb C implementation. Additionally we provide a Python fallback when this lttb-C building fails. In the near future, we will look into CIBuildWheels to build the wheels for the major OS & Python matrix versions.
    A well deserved s/o to dgoeris/lttbc, who heavily inspired our implementation!

    Figure Output serialization 📸

    Plotly-resampler now also has the option to store the output figure as an Image in notebook output. As long the notebook is connected, the interactive plotly-resampler figure is shown; but once the figure / notebook isn't connected anymore, a static image will be rendered in the notebook output.

    What's Changed (generated)

    • :bug: return self when calling add_traces by @jvdd in https://github.com/predict-idlab/plotly-resampler/pull/75
    • :fire: add streamlit integration example by @jvdd in https://github.com/predict-idlab/plotly-resampler/pull/80
    • ✨ adding convert_traces_kwargs by @jonasvdd in https://github.com/predict-idlab/plotly-resampler/pull/81
    • Fix numeric hf_y input as dtype object by @jonasvdd in https://github.com/predict-idlab/plotly-resampler/pull/90
    • :fire: add support for figure dict input + propagate _grid_str by @jvdd in https://github.com/predict-idlab/plotly-resampler/pull/92
    • :pray: fix tests for all OS by @jvdd in https://github.com/predict-idlab/plotly-resampler/pull/95
    • Add python3dot10 by @jvdd in https://github.com/predict-idlab/plotly-resampler/pull/96
    • :sunrise: FigureResampler display improvements by @jvdd in https://github.com/predict-idlab/plotly-resampler/pull/97
    • :package: serialization support + :level_slider: update OS & python version in test-matrix by @jvdd in https://github.com/predict-idlab/plotly-resampler/pull/87
    • Lttbv2 🍒 ⛏️ branch by @jonasvdd in https://github.com/predict-idlab/plotly-resampler/pull/103
    • :robot: hack together output retention in notebooks by @jvdd in https://github.com/predict-idlab/plotly-resampler/pull/105
    • :package: improve docs by @jvdd in https://github.com/predict-idlab/plotly-resampler/pull/104

    & some other minor bug fixes :see_no_evil:

    Full Changelog: https://github.com/predict-idlab/plotly-resampler/compare/v0.7.0...v0.8.0

    Source code(tar.gz)
    Source code(zip)
  • v0.7.0(Jun 17, 2022)

    What's Changed

    You can register plotly_resampler; this adds dynamic resampling functionality under the hood to plotly.py! 🥳 As a result, you can stop wrapping plotly figures with a plotly-resampler decorator (as this all happens automatically)

    You only need to call the register_plotly_resampler method and all plotly figures will be wrapped (under the hood) according to that method's configuration.

    -> More info in the README and docs!

    Aditionally, all resampler Figures are now composable; implying that they can be decorated by themselves and all other types of plotly-(resampler) figures. This eases the switching from a FigureResampler to FigureWidgetResampler and vice-versa.

    What's Changed (PR's)

    • 🦌 Adding reset-axes functionality by @jonasvdd in https://github.com/predict-idlab/plotly-resampler/pull/48
    • 🐛 Small bugfixes by @jonasvdd in https://github.com/predict-idlab/plotly-resampler/pull/52
    • 🔍 investigating gap-detection methodology by @jonasvdd in https://github.com/predict-idlab/plotly-resampler/pull/53
    • :mag: fix float index problem of #63 by @jonasvdd in https://github.com/predict-idlab/plotly-resampler/pull/64
    • :wrench: hotfix for rounding error by @jonasvdd in https://github.com/predict-idlab/plotly-resampler/pull/66
    • 🗳️ Compose figs by @jonasvdd in https://github.com/predict-idlab/plotly-resampler/pull/72
    • :sparkles: register plotly-resampler by @jvdd in https://github.com/predict-idlab/plotly-resampler/pull/70
    • :robot: update dependencies + new release by @jvdd in https://github.com/predict-idlab/plotly-resampler/pull/74

    Full Changelog: https://github.com/predict-idlab/plotly-resampler/compare/v0.6.0...v0.7.0

    Source code(tar.gz)
    Source code(zip)
  • v0.6.0(May 6, 2022)

    What's Changed

    Dynamically adjusting raw data 🔧

    The hf_data property now allows adjusting the hf_traces their data; documentation 📖

    fig.hf_data[-1]["y"] = - sin ** 2
    

    FigureWidget support 👀

    plotly-resampler can now wrap plotly's FigureWidget graph-object with the FigureWidgetResampler (see #47).

    This has several advantages

    1. ✔️ Able to use the on_click callback and thus create annotation app 👉🏼 see this example notebook.
    2. ✔️ No web-application with dash callbacks need to be started

    You can just seamlessly use plolty-resampler within your jupyter environment, remote or local.

    Source code(tar.gz)
    Source code(zip)
Owner
PreDiCT.IDLab
Repositories of the IDLab PreDiCT group
PreDiCT.IDLab
Blender addon that creates a temporary window of any type from the 3D View.

CreateTempWindow2.8 Blender addon that creates a temporary window of any type from the 3D View. Features Can the following window types: 3D View Graph

3 Nov 27, 2022
📊📈 Serves up Pandas dataframes via the Django REST Framework for use in client-side (i.e. d3.js) visualizations and offline analysis (e.g. Excel)

📊📈 Serves up Pandas dataframes via the Django REST Framework for use in client-side (i.e. d3.js) visualizations and offline analysis (e.g. Excel)

wq framework 1.2k Jan 01, 2023
eoplatform is a Python package that aims to simplify Remote Sensing Earth Observation by providing actionable information on a wide swath of RS platforms and provide a simple API for downloading and visualizing RS imagery

An Earth Observation Platform Earth Observation made easy. Report Bug | Request Feature About eoplatform is a Python package that aims to simplify Rem

Matthew Tralka 4 Aug 11, 2022
paintable GitHub contribute table

githeart paintable github contribute table how to use: Functions key color select 1,2,3,4,5 clear c drawing mode mode on turn off e print paint matrix

Bahadır Araz 27 Nov 24, 2022
erdantic is a simple tool for drawing entity relationship diagrams (ERDs) for Python data model classes

erdantic is a simple tool for drawing entity relationship diagrams (ERDs) for Python data model classes. Diagrams are rendered using the venerable Graphviz library.

DrivenData 129 Jan 04, 2023
Write python locally, execute SQL in your data warehouse

RasgoQL Write python locally, execute SQL in your data warehouse ≪ Read the Docs · Join Our Slack » RasgoQL is a Python package that enables you to ea

Rasgo 265 Nov 21, 2022
Seismic Waveform Inversion Toolbox-1.0

Seismic Waveform Inversion Toolbox (SWIT-1.0)

Haipeng Li 98 Dec 29, 2022
nvitop, an interactive NVIDIA-GPU process viewer, the one-stop solution for GPU process management

An interactive NVIDIA-GPU process viewer, the one-stop solution for GPU process management.

Xuehai Pan 1.3k Jan 02, 2023
DALLE-tools provided useful dataset utilities to improve you workflow with WebDatasets.

DALLE tools DALLE-tools is a github repository with useful tools to categorize, annotate or check the sanity of your datasets. Installation Just clone

11 Dec 25, 2022
Generate visualizations of GitHub user and repository statistics using GitHub Actions.

GitHub Stats Visualization Generate visualizations of GitHub user and repository statistics using GitHub Actions. This project is currently a work-in-

JoelImgu 3 Dec 14, 2022
Make visual music sheets for thatskygame (graphical representations of the Sky keyboard)

sky-python-music-sheet-maker This program lets you make visual music sheets for Sky: Children of the Light. It will ask you a few questions, and does

21 Aug 26, 2022
Drag’n’drop Pivot Tables and Charts for Jupyter/IPython Notebook, care of PivotTable.js

pivottablejs: the Python module Drag’n’drop Pivot Tables and Charts for Jupyter/IPython Notebook, care of PivotTable.js Installation pip install pivot

Nicolas Kruchten 512 Dec 26, 2022
A small script written in Python3 that generates a visual representation of the Mandelbrot set.

Mandelbrot Set Generator A small script written in Python3 that generates a visual representation of the Mandelbrot set. Abstract The colors in the ou

1 Dec 28, 2021
This plugin plots the time you spent on a tag as a histogram.

This plugin plots the time you spent on a tag as a histogram.

Tom Dörr 7 Sep 09, 2022
LabGraph is a a Python-first framework used to build sophisticated research systems with real-time streaming, graph API, and parallelism.

LabGraph is a a Python-first framework used to build sophisticated research systems with real-time streaming, graph API, and parallelism.

MLH Fellowship 7 Oct 05, 2022
Flexitext is a Python library that makes it easier to draw text with multiple styles in Matplotlib

Flexitext is a Python library that makes it easier to draw text with multiple styles in Matplotlib

Tomás Capretto 93 Dec 28, 2022
Python package to visualize and cluster partial dependence.

partial_dependence A python library for plotting partial dependence patterns of machine learning classifiers. The technique is a black box approach to

NYU Visualization Lab 25 Nov 14, 2022
Create 3d loss surface visualizations, with optimizer path. Issues welcome!

MLVTK A loss surface visualization tool Simple feed-forward network trained on chess data, using elu activation and Adam optimizer Simple feed-forward

7 Dec 21, 2022
flask extension for integration with the awesome pydantic package

Flask-Pydantic Flask extension for integration of the awesome pydantic package with Flask. Installation python3 -m pip install Flask-Pydantic Basics v

249 Jan 06, 2023
Resources for teaching & learning practical data visualization with python.

Practical Data Visualization with Python Overview All views expressed on this site are my own and do not represent the opinions of any entity with whi

Paul Jeffries 98 Sep 24, 2022