Full text search for flask.

Overview

flask-msearch

https://img.shields.io/badge/pypi-v0.2.9-brightgreen.svg https://img.shields.io/badge/python-2/3-brightgreen.svg https://img.shields.io/badge/license-BSD-blue.svg

Installation

To install flask-msearch:

pip install flask-msearch
# when MSEARCH_BACKEND = "whoosh"
pip install whoosh blinker
# when MSEARCH_BACKEND = "elasticsearch", only for 6.x.x
pip install elasticsearch==6.3.1

Or alternatively, you can download the repository and install manually by doing:

git clone https://github.com/honmaple/flask-msearch
cd flask-msearch
python setup.py install

Quickstart

from flask_msearch import Search
[...]
search = Search()
search.init_app(app)

# models.py
class Post(db.Model):
    __tablename__ = 'post'
    __searchable__ = ['title', 'content']

# views.py
@app.route("/search")
def w_search():
    keyword = request.args.get('keyword')
    results = Post.query.msearch(keyword,fields=['title'],limit=20).filter(...)
    # or
    results = Post.query.filter(...).msearch(keyword,fields=['title'],limit=20).filter(...)
    # elasticsearch
    keyword = "title:book AND content:read"
    # more syntax please visit https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html
    results = Post.query.msearch(keyword,limit=20).filter(...)
    return ''

Config

# when backend is elasticsearch, MSEARCH_INDEX_NAME is unused
# flask-msearch will use table name as elasticsearch index name unless set __msearch_index__
MSEARCH_INDEX_NAME = 'msearch'
# simple,whoosh,elaticsearch, default is simple
MSEARCH_BACKEND = 'whoosh'
# table's primary key if you don't like to use id, or set __msearch_primary_key__ for special model
MSEARCH_PRIMARY_KEY = 'id'
# auto create or update index
MSEARCH_ENABLE = True
# logger level, default is logging.WARNING
MSEARCH_LOGGER = logging.DEBUG
# SQLALCHEMY_TRACK_MODIFICATIONS must be set to True when msearch auto index is enabled
SQLALCHEMY_TRACK_MODIFICATIONS = True
# when backend is elasticsearch
ELASTICSEARCH = {"hosts": ["127.0.0.1:9200"]}

Usage

from flask_msearch import Search
[...]
search = Search()
search.init_app(app)

class Post(db.Model):
    __tablename__ = 'basic_posts'
    __searchable__ = ['title', 'content']

    id = db.Column(db.Integer, primary_key=True)
    title = db.Column(db.String(49))
    content = db.Column(db.Text)

    def __repr__(self):
        return '<Post:{}>'.format(self.title)

if raise sqlalchemy ValueError,please pass db param to Search

db = SQLalchemy()
search = Search(db=db)

Create_index

search.create_index()
search.create_index(Post)

Update_index

search.update_index()
search.update_index(Post)
# or
search.create_index(update=True)
search.create_index(Post, update=True)

Delete_index

search.delete_index()
search.delete_index(Post)
# or
search.create_index(delete=True)
search.create_index(Post, delete=True)

Custom Analyzer

only for whoosh backend

from jieba.analyse import ChineseAnalyzer
search = Search(analyzer=ChineseAnalyzer())

or use __msearch_analyzer__ for special model

class Post(db.Model):
    __tablename__ = 'post'
    __searchable__ = ['title', 'content', 'tag.name']
    __msearch_analyzer__ = ChineseAnalyzer()

Custom index name

If you want to set special index name for some model.

class Post(db.Model):
    __tablename__ = 'post'
    __searchable__ = ['title', 'content', 'tag.name']
    __msearch_index__ = "post111"

Custom schema

from whoosh.fields import ID

class Post(db.Model):
    __tablename__ = 'post'
    __searchable__ = ['title', 'content', 'tag.name']
    __msearch_schema__ = {'title': ID(stored=True, unique=True), 'content': 'text'}

Note: if you use hybrid_property, default field type is Text unless set special __msearch_schema__

Custom parser

from whoosh.qparser import MultifieldParser

class Post(db.Model):
    __tablename__ = 'post'
    __searchable__ = ['title', 'content']

    def _parser(fieldnames, schema, group, **kwargs):
        return MultifieldParser(fieldnames, schema, group=group, **kwargs)

    __msearch_parser__ = _parser

Note: Only for MSEARCH_BACKEND is whoosh

Custom index signal

flask-msearch uses flask signal to update index by default, if you want to use other asynchronous tools such as celey to update index, please set special MSEARCH_INDEX_SIGNAL

# app.py
app.config["MSEARCH_INDEX_SIGNAL"] = celery_signal
# or use string as variable
app.config["MSEARCH_INDEX_SIGNAL"] = "modulename.tasks.celery_signal"
search = Search(app)

# tasks.py
from flask_msearch.signal import default_signal

@celery.task(bind=True)
def celery_signal_task(self, backend, sender, changes):
    default_signal(backend, sender, changes)
    return str(self.request.id)

def celery_signal(backend, sender, changes):
    return celery_signal_task.delay(backend, sender, changes)

Relate index(Experimental)

for example

class Tag(db.Model):
    __tablename__ = 'tag'

    id = db.Column(db.Integer, primary_key=True)
    name = db.Column(db.String(49))

class Post(db.Model):
    __tablename__ = 'post'
    __searchable__ = ['title', 'content', 'tag.name']

    id = db.Column(db.Integer, primary_key=True)
    title = db.Column(db.String(49))
    content = db.Column(db.Text)

    # one to one
    tag_id = db.Column(db.Integer, db.ForeignKey('tag.id'))
    tag = db.relationship(
        Tag, backref=db.backref(
            'post', uselist=False), uselist=False)

    def __repr__(self):
        return '<Post:{}>'.format(self.title)

You must add msearch_FUN to Tag model,or the tag.name can’t auto update.

class Tag....
  ......
  def msearch_post_tag(self, delete=False):
      from sqlalchemy import text
      sql = text('select id from post where tag_id=' + str(self.id))
      return {
          'attrs': [{
              'id': str(i[0]),
              'tag.name': self.name
          } for i in db.engine.execute(sql)],
          '_index': Post
      }
Owner
honmaple
风落花语风落天,花落风雨花落田.
honmaple
Python script for finding duplicate images within a folder.

Python script for finding duplicate images within a folder.

194 Dec 31, 2022
An image inline search telegram bot.

Image-Search-Bot An image inline search telegram bot. Note: Use Telegram picture bot. That is better. Not recommending to deploy this bot. Made with P

Fayas Noushad 24 Oct 21, 2022
A play store search application programming interface ( API )

Play-Store-API A play store search application programming interface ( API ) Made with Python3

Fayas Noushad 8 Oct 21, 2022
Home for Elasticsearch examples available to everyone. It's a great way to get started.

Introduction This is a collection of examples to help you get familiar with the Elastic Stack. Each example folder includes a README with detailed ins

elastic 2.5k Jan 03, 2023
High level Python client for Elasticsearch

Elasticsearch DSL Elasticsearch DSL is a high-level library whose aim is to help with writing and running queries against Elasticsearch. It is built o

elastic 3.6k Dec 30, 2022
Inverted index creation and query search mechanism on Wikipedia pages.

WikiPedia Search Engine Step 1 : Installing Requirements Install "stemming" module for python using pip. Step 2 : Parsing the Data To parse the data,

Piyush Atri 1 Nov 27, 2021
Google Project: Search and auto-complete sentences within given input text files, manipulating data with complex data-structures.

Auto-Complete Google Project In this project there is an implementation for one feature of Google's search engines - AutoComplete. Autocomplete, or wo

Hadassah Engel 10 Jun 20, 2022
Pythonic Lucene - A simplified python impelementaiton of Apache Lucene

A simplified python impelementaiton of Apache Lucene, mabye helps to understand how an enterprise search engine really works.

Mahdi Sadeghzadeh Ghamsary 2 Sep 12, 2022
Es-schema - Common Data Schemas for Elasticsearch

Common Data Schemas for Elasticsearch The Common Data Schema for Elasticsearch i

Tim Schnell 2 Jan 25, 2022
Yuno is context based search engine for anime.

Yuno yuno.mp4 Table of Contents Introduction Power Of Yuno Try Yuno How Yuno was created? References Introduction Yuno is a context based search engin

IAmParadox 354 Dec 19, 2022
txtai executes machine-learning workflows to transform data and build AI-powered semantic search applications.

txtai executes machine-learning workflows to transform data and build AI-powered semantic search applications.

NeuML 3.1k Dec 31, 2022
esguard provides a Python decorator that waits for processing while monitoring the load of Elasticsearch.

esguard esguard provides a Python decorator that waits for processing while monitoring the load of Elasticsearch. Quick Start You need to launch elast

po3rin 5 Dec 08, 2021
Wagtail CLIP allows you to search your Wagtail images using natural language queries.

Wagtail CLIP allows you to search your Wagtail images using natural language queries.

Matt Segal 10 Dec 21, 2022
Jina allows you to build deep learning-powered search-as-a-service in just minutes

Cloud-native neural search framework for any kind of data

Jina AI 17k Dec 31, 2022
A Python web searcher library with different search engines

Robert A simple Python web searcher library with different search engines. Install pip install roberthelper Usage from robert import GoogleSearcher

1 Dec 23, 2021
基于RSSHUB阅读器实现的获取P站排行和P站搜图,使用时需使用代理

基于RSSHUB阅读器实现的获取P站排行和P站搜图

34 Dec 05, 2022
rclip - AI-Powered Command-Line Photo Search Tool

rclip is a command-line photo search tool based on the awesome OpenAI's CLIP neural network.

Yurij Mikhalevich 394 Dec 12, 2022
TG-searcherBot - Search any channel/chat from keyword

TG-searcherBot Search any channel/chat from keyword. Commands /start - Starts th

TechiError 12 Nov 04, 2022
A web search server for ParlAI, including Blenderbot2.

Description A web search server for ParlAI, including Blenderbot2. Querying the server: The server reacting correctly: Uses html2text to strip the mar

Jules Gagnon-Marchand 119 Jan 06, 2023
An open source, non-profit search engine implemented in python

Mwmbl: No ads, no tracking, no cruft, no profit Mwmbl is a non-profit, ad-free, free-libre and free-lunch search engine with a focus on useability and

639 Jan 04, 2023