web application for flight log analysis & review

Overview

Flight Review

Build Status

This is a web application for flight log analysis. It allows users to upload ULog flight logs, and analyze them through the browser.

It uses the bokeh library for plotting and the Tornado Web Server.

Flight Review is deployed at https://review.px4.io.

Plot View

3D View

3D View

Installation and Setup

Requirements

Ubuntu

sudo apt-get install sqlite3 fftw3 libfftw3-dev

Note: Under some Ubuntu and Debian environments you might have to install ATLAS

sudo apt-get install libatlas3-base

macOS

macOS already provides SQLite3. Use Homebrew to install fftw:

brew install fftw

Installation

# After git clone, enter the directory
git clone --recursive https://github.com/PX4/flight_review.git
cd flight_review/app
pip install -r requirements.txt
# Note: preferably use a virtualenv

Setup

  • By default the app will load config_default.ini configuration file
  • You can override any setting from config_default.ini with a user config file config_user.ini (untracked)
  • Any setting on config_user.ini has priority over config_default.ini
  • Run setup_db.py to initialize the database.

Note: setup_db.py can also be used to upgrade the database tables, for instance when new entries are added (it automatically detects that).

Usage

For local usage, the server can be started directly with a log file name, without having to upload it first:

cd app
./serve.py -f <file.ulg>

To start the whole web application:

cd app
./serve.py --show

The plot_app directory contains a bokeh server application for plotting. It can be run stand-alone with bokeh serve --show plot_app (or with cd plot_app; bokeh serve --show main.py, to start without the html template).

The whole web application is run with the serve.py script. Run ./serve.py -h for further details.

Interactive Usage

The plotting can also be used interative using a Jupyter Notebook. It requires python knowledge, but provides full control over what and how to plot with immediate feedback.

  • Start the notebook
  • Locate and open the test notebook file testing_notebook.ipynb.
# Launch jupyter notebook
jupyter notebook testing_notebook.ipynb

Implementation

The web site is structured around a bokeh application in app/plot_app (app/plot_app/configured_plots.py contains all the configured plots). This application also handles the statistics page, as it contains bokeh plots as well. The other pages (upload, browse, ...) are implemented as tornado handlers in app/tornado_handlers/.

plot_app/helper.py additionally contains a list of log topics that the plot application can subscribe to. A topic must live in this list in order to be plotted.

Tornado uses a single-threaded event loop. This means all operations should be non-blocking (see also http://www.tornadoweb.org/en/stable/guide/async.html). (This is currently not the case for sending emails).

Reading ULog files is expensive and thus should be avoided if not really necessary. There are two mechanisms helping with that:

  • Loaded ULog files are kept in RAM using an LRU cache with configurable size (when using the helper method). This works from different requests and sessions and from all source contexts.
  • There's a LogsGenerated DB table, which contains extracted data from ULog for faster access.

Caching

In addition to in-memory caching there is also some on-disk caching: KML files are stored on disk. Also the parameters and airframes are cached and downloaded every 24 hours. It is safe to delete these files (but not the cache directory).

Notes about python imports

Bokeh uses dynamic code loading and the plot_app/main.py gets loaded on each session (page load) to isolate requests. This also means we cannot use relative imports. We have to use sys.path.append to include modules in plot_app from the root directory (Eg tornado_handlers.py). Then to make sure the same module is only loaded once, we use import xy instead of import plot_app.xy. It's useful to look at print('\n'.join(sys.modules.keys())) to check this.

Docker usage

This section explains how to work with docker.

Arguments

Edit the .env file according to your setup:

  • PORT - The number of port, what listen service in docker, default 5006
  • USE_PROXY - The set his, if you use reverse proxy (Nginx, ...)
  • DOMAIN - The address domain name for origin, default = *
  • CERT_PATH - The SSL certificate volume path
  • EMAIL - Email for challenging Let's Encrypt DNS

Paths

  • /opt/service/config_user.ini - Path for config
  • /opt/service/data - Folder where stored database
  • .env - Environment variables for nginx and app docker container

Build Docker Image

cd app
docker build -t px4flightreview -f Dockerfile .

Work with docker-compose

Run the following command to start docker container. Please modify the .env and add app/config_user.ini with respective stages.

Uncomment the BOKEH_ALLOW_WS_ORIGIN with your local IP Address when developing, this is for the bokeh application's websocket to work.

Development

docker-compose -f docker-compose.dev.yml up

Test Locally

Test locally with nginx:

docker-compose up

Remember to Change NGINX_CONF to use default_ssl.conf and add the EMAIL for production.

Production

htpasswd -c ./nginx/.htpasswd username
# here to create a .htpasswd for nginx basic authentication
chmod u+x init-letsencrypt.sh
./init-letsencrypt.sh

Contributing

Contributions are welcome! Just open a pull request with detailed description why the changes are needed, or open an issue for bugs, feature requests, etc...

Owner
PX4 Drone Autopilot
Professional Open Source Autopilot Stack
PX4 Drone Autopilot
A simple project on Data Visualization for CSCI-40 course.

Simple-Data-Visualization A simple project on Data Visualization for CSCI-40 course - the instructions can be found here SAT results in New York in 20

Hugo Matousek 8 Oct 27, 2021
A Graph Learning library for Humans

A Graph Learning library for Humans These novel algorithms include but are not limited to: A graph construction and graph searching class can be found

Richard Tjörnhammar 1 Feb 08, 2022
Generate "Jupiter" plots for circular genomes

jupiter Generate "Jupiter" plots for circular genomes Description Python scripts to generate plots from ViennaRNA output. Written in "pidgin" python w

Robert Edgar 2 Nov 29, 2021
Smarthome Dashboard with Grafana & InfluxDB

Smarthome Dashboard with Grafana & InfluxDB This is a complete overhaul of my Raspberry Dashboard done with Flask. I switched from sqlite to InfluxDB

6 Oct 20, 2022
Example scripts for generating plots of Bohemian matrices

Bohemian Eigenvalue Plotting Examples This repository contains examples of generating plots of Bohemian eigenvalues. The examples in this repository a

Bohemian Matrices 5 Nov 12, 2022
📊 Extensions for Matplotlib

📊 Extensions for Matplotlib

Nico Schlömer 519 Dec 30, 2022
Plotly Dash Command Line Tools - Easily create and deploy Plotly Dash projects from templates

🛠️ dash-tools - Create and Deploy Plotly Dash Apps from Command Line | | | | | Create a templated multi-page Plotly Dash app with CLI in less than 7

Andrew Hossack 50 Dec 30, 2022
Interactive Dashboard for Visualizing OSM Data Change

Dashboard and intuitive data downloader for more interactive experience with interpreting osm change data.

1 Feb 20, 2022
a plottling library for python, based on D3

Hello August 2013 Hello! Maybe you're looking for a nice Python interface to build interactive, javascript based plots that look as nice as all those

Mike Dewar 1.4k Dec 28, 2022
Show Data: Show your dataset in web browser!

Show Data is to generate html tables for large scale image dataset, especially for the dataset in remote server. It provides some useful commond line tools and fully customizeble API reference to gen

Dechao Meng 83 Nov 26, 2022
Lumen provides a framework for visual analytics, which allows users to build data-driven dashboards from a simple yaml specification

Lumen project provides a framework for visual analytics, which allows users to build data-driven dashboards from a simple yaml specification

HoloViz 120 Jan 04, 2023
Pebble is a stat's visualization tool, this will provide a skeleton to develop a monitoring tool.

Pebble is a stat's visualization tool, this will provide a skeleton to develop a monitoring tool.

Aravind Kumar G 2 Nov 17, 2021
Visualize data of Vietnam's regions with interactive maps.

Plotting Vietnam Development Map This is my personal project that I use plotly to analyse and visualize data of Vietnam's regions with interactive map

1 Jun 26, 2022
Collection of data visualizing projects through Tableau, Data Wrapper, and Power BI

Data-Visualization-Projects Collection of data visualizing projects through Tableau, Data Wrapper, and Power BI Indigenous-Brands-Social-Movements Pyt

Jinwoo(Roy) Yoon 1 Feb 05, 2022
The open-source tool for building high-quality datasets and computer vision models

The open-source tool for building high-quality datasets and computer vision models. Website • Docs • Try it Now • Tutorials • Examples • Blog • Commun

Voxel51 2.4k Jan 07, 2023
A Jupyter - Three.js bridge

pythreejs A Python / ThreeJS bridge utilizing the Jupyter widget infrastructure. Getting Started Installation Using pip: pip install pythreejs And the

Jupyter Widgets 844 Dec 27, 2022
A streamlit component for bi-directional communication with bokeh plots.

Streamlit Bokeh Events A streamlit component for bi-directional communication with bokeh plots. Its just a workaround till streamlit team releases sup

Ashish Shukla 123 Dec 25, 2022
This is a Boids Simulation, written in Python with Pygame.

PyNBoids A Python Boids Simulation This is a Boids simulation, written in Python3, with Pygame2 and NumPy. To use: Save the pynboids_sp.py file (and n

Nik 17 Dec 18, 2022
This is simply repo for line drawing rendering using freestyle in Blender.

blender_freestyle_line_drawing This is simply repo for line drawing rendering using freestyle in Blender. how to use blender2935 --background --python

MaxLin 3 Jul 02, 2022
Streamlit component for Let's-Plot visualization library

streamlit-letsplot This is a work-in-progress, providing a convenience function to plot charts from the Lets-Plot visualization library. Example usage

Randy Zwitch 9 Nov 03, 2022