JSON Schema validation library

Overview

jsonschema

ci codecov Crates.io docs.rs gitter

A JSON Schema validator implementation. It compiles schema into a validation tree to have validation as fast as possible.

Supported drafts:

  • Draft 7 (except optional idn-hostname.json test case)
  • Draft 6
  • Draft 4 (except optional bignum.json test case)
# Cargo.toml
jsonschema = "0.13"

To validate documents against some schema and get validation errors (if any):

use jsonschema::JSONSchema;
use serde_json::json;

fn main() {
    let schema = json!({"maxLength": 5});
    let instance = json!("foo");
    let compiled = JSONSchema::compile(&schema)
        .expect("A valid schema");
    let result = compiled.validate(&instance);
    if let Err(errors) = result {
        for error in errors {
            println!("Validation error: {}", error);
            println!(
                "Instance path: {}", error.instance_path
            );
        }
    }
}

Each error has an instance_path attribute that indicates the path to the erroneous part within the validated instance. It could be transformed to JSON Pointer via .to_string() or to Vec via .into_vec().

If you only need to know whether document is valid or not (which is faster):

use jsonschema::is_valid;
use serde_json::json;

fn main() {
    let schema = json!({"maxLength": 5});
    let instance = json!("foo");
    assert!(is_valid(&schema, &instance));
}

Or use a compiled schema (preferred):

use jsonschema::JSONSchema;
use serde_json::json;

fn main() {
    let schema = json!({"maxLength": 5});
    let instance = json!("foo");
    // Draft is detected automatically
    // with fallback to Draft7
    let compiled = JSONSchema::compile(&schema)
        .expect("A valid schema");
    assert!(compiled.is_valid(&instance));
}

Output styles

jsonschema supports basic & flag output styles from Draft 2019-09, so you can serialize the validation results with serde:

use jsonschema::{Output, BasicOutput, JSONSchema};

fn main() {
    let schema_json = serde_json::json!({
        "title": "string value",
        "type": "string"
    });
    let instance = serde_json::json!{"some string"};
    let schema = JSONSchema::options()
        .compile(&schema_json)
        .expect("A valid schema");
    
    let output: BasicOutput = schema.apply(&instance).basic();
    let output_json = serde_json::to_value(output)
        .expect("Failed to serialize output");
    
    assert_eq!(
        output_json, 
        serde_json::json!({
            "valid": true,
            "annotations": [
                {
                    "keywordLocation": "",
                    "instanceLocation": "",
                    "annotations": {
                        "title": "string value"
                    }
                }
            ]
        })
    );
}

Status

This library is functional and ready for use, but its API is still evolving to the 1.0 API.

Bindings

  • Python - See the ./bindings/python directory
  • Ruby - a crate by @driv3r
  • NodeJS - a package by @ahungrynoob

Performance

There is a comparison with other JSON Schema validators written in Rust - jsonschema_valid==0.4.0 and valico==3.6.0.

Test machine i8700K (12 cores), 32GB RAM.

Input values and schemas:

Case Schema size Instance size
OpenAPI 18 KB 4.5 MB
Swagger 25 KB 3.0 MB
Canada 4.8 KB 2.1 MB
CITM catalog 2.3 KB 501 KB
Fast (valid) 595 B 55 B
Fast (invalid) 595 B 60 B

Here is the average time for each contender to validate. Ratios are given against compiled JSONSchema using its validate method. The is_valid method is faster, but gives only a boolean return value:

Case jsonschema_valid valico jsonschema (validate) jsonschema (is_valid)
OpenAPI - (1) - (1) 4.717 ms 4.279 ms (x0.90)
Swagger - (2) 83.357 ms (x12.47) 6.681 ms 4.533 ms (x0.67)
Canada 32.987 ms (x31.38) 141.41 ms (x134.54) 1.051 ms 1.046 ms (x0.99)
CITM catalog 4.735 ms (x2.00) 13.222 ms (x5.58) 2.367 ms 535.07 us (x0.22)
Fast (valid) 2.00 us (x3.85) 3.18 us (x6.13) 518.39 ns 97.91 ns (x0.18)
Fast (invalid) 339.28 ns (x0.50) 3.34 us (x5.00) 667.55 ns 5.41ns (x0.01)

Notes:

  1. jsonschema_valid and valico do not handle valid path instances matching the ^\\/ regex.

  2. jsonschema_valid fails to resolve local references (e.g. #/definitions/definitions).

You can find benchmark code in benches/jsonschema.rs, Rust version is 1.57.

Support

If you have anything to discuss regarding this library, please, join our gitter!

Comments
  • Add a `path` field to `ValidationError`

    Add a `path` field to `ValidationError`

    Summary

    On validation error make the path to the problematic field available to the client. Currently the path is only a field in ValidationError but later it can be used to improve the error messages (related to #100, #144)

    Example

    Schema
    {
       "type":"object",
       "properties":{
          "foo":{
             "type":"object",
             "properties":{
                "bar":{
                   "type":"integer"
                }
             }
          }
       }
    }
    
    Input
    {
      "foo": {
        "bar": "some string"
      }
    }
    
    Produced Error

    path: [foo, bar]

    opened by gavadinov 16
  • patternProperties interferes with properties

    patternProperties interferes with properties

    It seems like the presence of "patternProperties" interferes with validation that should happen in "properties".

    This replit shows an example of an instance that should fail validation (and does fail in jsonschemavalidator.net). If you comment out the "patternProperties" portion of the schema, it fails as expected.

    I don't have a good enough understanding of the jsonschema internals, but it seems like this could be related to changes from #173 .

    UPDATE: I updated the replit and the jsonschemavalidator link to provide a more minimal example.

    opened by duckontheweb 16
  • [FEATURE] User-defined validation for the `format` keyword

    [FEATURE] User-defined validation for the `format` keyword

    At the moment, jsonschema supports the format keyword for formats defined in Drafts 4, 6, 7. However, JSON Schema allows custom values for the format keyword.

    It will be awesome to provide a way for the end-user to validate their own formats.

    As all validators are "compiled" to some structs (to avoid some run-time overhead e.g., map lookup, etc.), I think that the public API might look like this:

    use jsonschema;
    use std::collections::HashMap;
    use serde_json::{json, Value};
    
    struct CustomFormatValidator {}
    
    // New trait somehow derived from `Validate`?
    impl jsonschema::SomeNewTrait for CustomFormatValidator {
        fn is_valid(&self, _: &jsonschema::JSONSchema, instance: &Value) -> bool {
            if let Value::String(item) = instance {
                item == "bar"  // Format matching logic goes here
            } else {
                true
            }
        }
    }
    
    fn main() -> Result<(), jsonschema::CompilationError> {
        let schema = json!({"format": "custom"});
        let instance = json!("foo");
        let mut formats = HashMap::new();
        formats.insert(
            "custom": CustomFormatValidator
        );
        // Shortcuts
        jsonschema::is_valid(&schema, &instance, Some(formats))
        // Compiled schema
        let compiled = jsonschema::JSONSchema::options()
            .formats(formats)
            .compile(&schema)
            .expect("Invalid schema");
        compiled.is_valid(&instance)
    }
    

    It will be nice to avoid the boilerplate that extracts the actual string from Value::String, so the user won't write this code for each format.

    Anticipated changes to implement this:

    • Take extra argument for the formats map in jsonschema::is_valid;
    • Extend CompilationOptions builder to handle formats;
    • Store the formats map as a part of CompilationContext;
    • Inside format.rs take a look at this mapping before going to take a look at the built-in validators;
    • Make the Validate trait public and implement some trait on top of it to avoid boilerplate?

    For now, I think it is OK to keep the scope on the Rust code only. A similar feature for bindings could be implemented later.

    help wanted good first issue Priority: Medium Type: Feature Difficulty: Medium 
    opened by Stranger6667 16
  • HTTP Resolver Not Working?

    HTTP Resolver Not Working?

    Hi,

    Sorry, I've been trying to work this out myself, but I'm not sure what's going wrong.

    Validation error: failed to resolve https://schema.rawkode.dev/organization: error sending request for url (https://schema.rawkode.dev/organization): error trying to connect: invalid URL, scheme is not http
    Instance path:
    Validation error: failed to resolve https://schema.rawkode.dev/person: error sending request for url (https://schema.rawkode.dev/person): error trying to connect: invalid URL, scheme is not http
    Instance path:
    

    These URLs are definitely OK and I'm not sure what I've done wrong.

    I've tried using http instead of https in my schema, but the problem is the same.

    Any help appreciated.

    Priority: High Type: Bug 
    opened by rawkode 15
  • Allow Schema Value to be Owned

    Allow Schema Value to be Owned

    In the current form it is impossible to retain a non-static reference to JSONSchema until the end of the program's execution. The below block demonstrates the problem.

    #[actix_web::main]
    async fn main() -> std::io::Result<()> {
        let schema_file = fs::File::open("config/schema.json").unwrap();
    
        let schema: Value = serde_json::from_reader(schema_file).unwrap();
        let validator = JSONSchema::compile(&schema, None).unwrap();
    
        let ad = AppData{
            validator: Arc::new(validator)
        };
    
        HttpServer::new(move ||{
            App::new()
                .data(ad.clone())
                .service(echo)
        })
        .bind("localhost:9070")?
        .run()
        .await
    }
    
    #[derive(Clone)]
    pub struct AppData<'a> {
        pub validator: Arc<JSONSchema<'a>>
    }
    

    The above code results in the below compiler error:

    error[E0597]: `schema` does not live long enough
      --> src/main.rs:15:41
       |
    15 |     let validator = JSONSchema::compile(&schema, None).unwrap();
       |                     --------------------^^^^^^^-------
       |                     |                   |
       |                     |                   borrowed value does not live long enough
       |                     argument requires that `schema` is borrowed for `'static`
    ...
    31 | }
       | - `schema` dropped here while still borrowed
    

    One(or the only?) way to get this fixed is to let JSONSchema own the Value containing schema.

    opened by kayyagari 12
  • fix: convert python tuples into lists

    fix: convert python tuples into lists

    We use tuples for geographic coordinates and bounding boxes which doesn't seem to have a direct mapping in JSON Schema.

    This pull requests converts the tuple into a list to fix ValueError: Unsupported type: 'tuple' when validating a object that contains a tuple.

    { "bbox": (1,2,3,4) }
    
    opened by blacha 11
  • Add support for look-around patterns using fancy-regex

    Add support for look-around patterns using fancy-regex

    Uses fancy-regex in place of regex for matching against all patterns in JSON schemas.

    regex is still used when converting a RegEx for ECMA 262 to take advantage of the Regex::replace_all method, which is not included in fancy-regex. Since fancy-regex depends on regex anyway, this does not affect the final size of the compiled binary (thanks to this commit for the idea).

    I also removed the test from #214, since that pattern is supported in this PR.

    opened by duckontheweb 10
  • Report schema paths in validation errors

    Report schema paths in validation errors

    When a validation error is reported, it contains only instance_path that points to the erroneous part of the input, but there is no information about what part of the schema caused this error.

    The gist of the implementation is to keep a stack of strings while the input schema is compiled. The stack should be convertible to paths::JSONPointer.

    The process for regular, non-composite validators (more on this below):

    • When the compilation process enters a validator, push its relevant keyword name to the stack (e.g. properties);
    • When the validator struct is created, clone this stack and store it within this struct;
    • When the validator is compiled - pop from the stack

    I think that pushing/popping could be implemented similarly to instance_path - a linked list without heap allocations. This way, the actual popping step is not needed at all.

    Composite validators are slightly more complicated - they are single structs that implement the logic for multiple different keywords or their specific variants (e.g. this one handles patternProperties and additionalProperties: false) which is done due to performance reasons.

    To handle them, the stack should not be pushed during compilation, but rather during validation - if a validation error happens, the stored stack should be cloned, and the exact keyword variant should be pushed to it. And finally, it is moved to ValidationError.

    Validators live in keywords.

    A lot of work in this direction was done by @gavadinov in https://github.com/gavadinov/jsonschema-rs/commit/40149322bfdf5549a0150122a00fa66dfe770828, and I think that it could be a great base for implementing this feature.

    @gavadinov, let me know if you are OK if somebody can base their work on your commit, or if you want to continue it yourself :)

    help wanted good first issue Priority: High Type: Feature Difficulty: Medium 
    opened by Stranger6667 10
  • Replace `ValidationError::schema` with custom errors

    Replace `ValidationError::schema` with custom errors

    Hi there, bump into here via hacktoberfest mark.

    Honestly most likely it doesn't address JSONPointer correctly in errors. And I was not sure which type of error better use for things like as64 error or regex syntax error but I did used ValidationError::format.

    Note: there might be an issue with cargo fmt.

    closes #235

    Thank you for the library.

    opened by zhiburt 9
  • Use the original value in error formatting for some numeric validators

    Use the original value in error formatting for some numeric validators

    Validating 3.0 against {"exclusiveMaximum": 3.0} will give 3.0 is greater than or equal to the maximum of 3. But the last number in the error message should be 3.0. It affects exclusiveMaximum, exclusiveMinimum, minimum, maximum.

    Priority: Low Type: Bug Difficulty: Easy 
    opened by Stranger6667 9
  • Use a better structure instead of `Map` in validators

    Use a better structure instead of `Map` in validators

    E.g. in AdditionalPropertiesNotEmptyValidator. Because during the validation we use self.properties.contains_key(*property) which will be O(logN) (because of BTreeMap). We can use HashMap to have O(1).

    As perf showed, significant time is spent on it

    Priority: Low Type: Enhancement Topic: Performance 
    opened by Stranger6667 8
  • Don't depend on a particular ahash rng

    Don't depend on a particular ahash rng

    I'm not very familiar with ahash, or WASM in general, but it seems like ahash's default randomness source panics when run inside a WASM environment. This change should make it so that jsonschema uses the runtime randomness source by default, but will not require it when included with default-features = false.

    opened by SamWilsn 3
  • Try compile tauri in wasm target , encountered some problems

    Try compile tauri in wasm target , encountered some problems

    Hi, First of all, I'm not the maintainer of tauri.

    I find the tauri-cli crate used jsonschema, and I am trying run it with wasm , When I run here, an error is reported.

    so, I changed it like this, Because tauri does not use this part when parsing json. From the perspective of jsonschema, this change should be inappropriate. Is there a more reasonable solution ?

    opened by tu6ge 0
  • multipleOf logic produces incorrect results

    multipleOf logic produces incorrect results

    The logic for multipleOf compares the remainder of a division with the f64 EPSILON constant to decide if a number is a multiple or not. This approach doesn't make sense. EPSILON is simply the difference between 1.0 and the next larger representable number. With regards to the multipleOf calculation this is just an arbitrary constant and comparing remainders to it gives us no guarantees.

    EG:

    1e-15 is a multiple of 1e-16. But (1e-15 / 1e-16) % 1.0 = 1.7763568394002505e-15 which is greater than epsilon so it is incorrectly said to not be a multiple.

    1.3e-16 is not a multiple of 1.3. But (1.3e-16 / 1.3) % 1.0 = 9.999999999999999e-17 which is less than epsilon so it is incorrectly said to be a multiple.

    opened by DanielBauman88 1
  • feat: Custom keyword validation

    feat: Custom keyword validation

    Add support for user defined custom keyword validation. The user provides and registers custom validator functions when configuring a JSONSchema.

    Custom keyword validators may be used when the user wants to enforce constraints that can't, or can't easily, be expressed in JSON schema.

    This PR addresses at least some of the features requested in #379 by @tamasfe as well as my own. The same pros and cons described in #379 apply. The PR also follows the structure of #354, but of course uses validation functions as opposed to sub-schema for validation.

    We could collapse the two user implementable validation functions into a single trait, mirroring the internal Validate trait. Implementations would have to sit behind an Arc.

    Example usage

    use jsonschema::{CustomKeywordDefinition, ErrorIterator, JSONSchema, paths::JSONPointer};
    use serde_json::{json, Value};
    use std::sync::Arc;
    
    fn validate(
        instance: &Value,
        instance_path: JSONPointer,
        schema: Arc<Value>,
        schema_path: JSONPointer,
    ) -> ErrorIterator {
        // ... validate instance ... 
        Box::new(None.into_iter())
    }
    fn is_valid(instance: &Value, schema: &Value) -> bool {
        // ... determine if instance is valid ... 
        return true;
    }
    let definition = CustomKeywordDefinition::Validator { validate, is_valid };
    assert!(JSONSchema::options()
        .with_custom_keyword("my-type", definition)
        .compile(&json!({ "my-type": "my-schema"}))
        .expect("A valid schema")
        .is_valid(&json!({ "a": "b"})));
    
    opened by bnjt 8
  • Async SchemaResolver

    Async SchemaResolver

    I'm using an async S3 client to load schemas from a bucket. I've hit a wall in that SchemaResolver is not async, but I need to call an async method within it. Is AsyncSchemaResolver on the roadmap or is there a known work around?

    ///
    /// Download an object from the S3 bucket and turn the result as Bytes
    /// 
    pub async fn download_object(client: &Client, bucket_name: &str, key: &str) -> Result<Bytes, S3Error> {
        let resp = client
            .get_object()
            .bucket(bucket_name)
            .key(key)
            .send()
            .await?;
        let data = match resp.body.collect().await {
            Ok(data) => data,
            Err(e) => return Err(S3Error::GetObjectError{ bucket_name: bucket_name.to_string(), key: key.to_string(), msg: e.to_string() }),
        };
        let data = data.into_bytes();
    
        Ok(data)
    }
    
    ///
    /// Download and object from the S3 bucket and return the results as a json Value
    /// 
    pub async fn download_json(client: &Client, bucket_name: &str, path: &str) -> Result<serde_json::Value, JsonError> {
        let data = download_object(client, bucket_name, path).await?;
        let json: serde_json::Value = serde_json::from_slice(&data)?;
        Ok(json)
    }
    
    struct S3SchemaResolver {
        client: Client,         // S3 client
        bucket_name: String     // name of bucket that holds schemas
    }
    
    impl SchemaResolver for S3SchemaResolver {
        fn resolve(&self, root_schema: &Value, url: &Url, original_reference: &str) -> Result<Arc<Value>, SchemaResolverError> {
            //
            // add the base path to the reference
            //
            match download_json(&self.client, &self.bucket_name, original_reference).await {
                Ok(schema_value) => Ok(Arc::new(schema_value)),
                Err(json_error) => {
                    let msg = format!("{}", json_error);
                    log::error!("{}", msg);
                    Err(anyhow::anyhow!(msg))
                }
            }
        }
    }
    
    opened by Ezward 0
Releases(python-v0.16.0)
  • python-v0.16.0(May 12, 2022)

    Added

    • Python 3.10 support

    Fixed

    • Installation error due to pyo3-built incompatibility
    • Memory leak in iter_errors. #325

    Changed

    • Update pyo3 to 0.16.

    Removed

    • Support for Python 3.6
    Source code(tar.gz)
    Source code(zip)
  • rust-v0.16.0(Apr 21, 2022)

Owner
Dmitry Dygalo
Building a platform for effortless API testing
Dmitry Dygalo
Make JSON serialization easier

Make JSON serialization easier

4 Jun 30, 2022
A fast JSON parser/generator for C++ with both SAX/DOM style API

A fast JSON parser/generator for C++ with both SAX/DOM style API Tencent is pleased to support the open source community by making RapidJSON available

Tencent 12.6k Dec 30, 2022
Fileson - JSON File database tools

Fileson is a set of Python scripts to create JSON file databases

Joonas Pihlajamaa 2 Feb 02, 2022
Marshall python objects to and from JSON

Pymarshaler - Marshal and Unmarshal Python Objects Disclaimer This tool is in no way production ready About Pymarshaler allows you to marshal and unma

Hernan Romer 9 Dec 20, 2022
Small python wrapper around the valico rust library to provide fast JSON schema validation.

Small python wrapper around the valico rust library to provide fast JSON schema validation.

Simon J Knibbs 5 Jul 12, 2019
Json GUI for No Man's Sky save file

NMS-Save-Parser Json GUI for No Man's Sky save file GUI python NMS_SAVE_PARSER.py [optional|save.hg] converter only python convert.py usage: conver

2 Oct 19, 2022
Wikidot-forum-dump - Simple Python script that dumps a Wikidot wiki forum into JSON structures.

wikidot-forum-dump Script is partially based on 2stacks by bluesoul: https://github.com/scuttle/2stacks To dump a Wiki's forum, edit config.py and put

ZZYZX 1 Jun 29, 2022
Random JSON Key:Pair Json Generator

Random JSON Key:Value Pair Generator This simple script take an engish dictionary of words and and makes random key value pairs. The dictionary has ap

Chris Edwards 1 Oct 14, 2021
This open source Python project allow you to create JSON data trees using Minmup.com

This open source Python project allow you to create JSON data trees using Minmup.com. I try to develop this project all the time. But feel free to use :).

Arttu Väisänen 1 Jan 30, 2022
Ibmi-json-beautify - Beautify json string with python

Ibmi-json-beautify - Beautify json string with python

Jefferson Vaughn 3 Feb 02, 2022
A Python application to transfer Zeek ASCII (not JSON) logs to Elastic/OpenSearch.

zeek2es.py This Python application translates Zeek's ASCII TSV logs into ElasticSearch's bulk load JSON format. For JSON logs, see Elastic's File Beat

Corelight, Inc. 28 Dec 22, 2022
Editor for json/standard python data

Editor for json/standard python data

1 Dec 07, 2021
Creates fake JSON files from a JSON schema

Use jsf along with fake data generators to provide consistent and meaningful fake data for your system.

Andy Challis 86 Jan 03, 2023
A python library to convert arbitrary strings representing business opening hours into a JSON format that's easier to use in code

A python library to convert arbitrary strings representing business opening hours into a JSON format that's easier to use in code

Adrian Edwards 9 Dec 02, 2022
JSON Schema validation library

jsonschema A JSON Schema validator implementation. It compiles schema into a validation tree to have validation as fast as possible. Supported drafts:

Dmitry Dygalo 309 Jan 01, 2023
With the help of json txt you can use your txt file as a json file in a very simple way

json txt With the help of json txt you can use your txt file as a json file in a very simple way Dependencies re filemod pip install filemod Installat

Kshitij 1 Dec 14, 2022
Package to Encode/Decode some common file formats to json

ZnJSON Package to Encode/Decode some common file formats to json Available via pip install znjson In comparison to pickle this allows having readable

ZINC 2 Feb 02, 2022
RedisJSON - a JSON data type for Redis

RedisJSON is a Redis module that implements ECMA-404 The JSON Data Interchange Standard as a native data type. It allows storing, updating and fetching JSON values from Redis keys (documents).

3.4k Dec 29, 2022
JsonParser - Parsing the Json file by provide the node name

Json Parser This project is based on Parsing the json and dumping it to CSV via

Ananta R. Pant 3 Aug 08, 2022
A Cobalt Strike Scanner that retrieves detected Team Server beacons into a JSON object

melting-cobalt 👀 A tool to hunt/mine for Cobalt Strike beacons and "reduce" their beacon configuration for later indexing. Hunts can either be expans

Splunk GitHub 150 Nov 23, 2022