Cải thiện Elasticsearch trong bài toán semantic search sử dụng phương pháp Sentence Embeddings

Last update: Nov 25, 2022

Related tags

Overview

Cải thiện Elasticsearch trong bài toán semantic search sử dụng phương pháp Sentence Embeddings

Trong bài viết này mình sẽ sử dụng pretrain model SimCSE_Vietnamese để cải thiện elastic search trong bài toán semantic search.

Mọi người có thể xem bài viết hướng dẫn đầy đủ tại đây.

Cài đặt:

git clone https://github.com/vovanphuc/elastic_simCSE.git
cd elastic_simCSE
pip install -r requirements.txt

Đánh index cho toàn bộ data

python3 index_elastic.py

Search keyword và so sánh giữa BM25 (elasticsearch thường) và simCSE:

streamlit run main.py

Kết quả tìm kiếm:

Kết quả khi sử dụng elasticsearch bình thường.

Kết quả khi sử dụng SimCSE_VietNamese.

Contact:

Email: [email protected]

Facebook: facebook.com/vovanphucc

Linkedin: linkedin.com/in/vovanphuc

Owner

Vo Van Phuc

KEEP IT SIMPLE STUPID!

GitHub Repository

State-of-the-art NLP through transformer models in a modular design and consistent APIs.

Trapper (Transformers wRAPPER) Trapper is an NLP library that aims to make it easier to train transformer based models on downstream tasks. It wraps h

42 Sep 21, 2022

Open-source offline translation library written in Python. Uses OpenNMT for translations

Open source neural machine translation in Python. Designed to be used either as a Python library or desktop application. Uses OpenNMT for translations and PyQt for GUI.

1.6k Jan 01, 2023

This is a Prototype of an Ai ChatBot "Tea and Coffee Supplier" using python.

Ai-ChatBot-Python A chatbot is an intelligent system which can hold a conversation with a human using natural language in real time. Due to the rise o

1 Oct 30, 2021

Sequence modeling benchmarks and temporal convolutional networks

Sequence Modeling Benchmarks and Temporal Convolutional Networks (TCN) This repository contains the experiments done in the work An Empirical Evaluati

3.5k Jan 03, 2023

NumPy String-Indexed is a NumPy extension that allows arrays to be indexed using descriptive string labels

NumPy String-Indexed NumPy String-Indexed is a NumPy extension that allows arrays to be indexed using descriptive string labels, rather than conventio

1 Jan 08, 2022

pkuseg多领域中文分词工具; The pkuseg toolkit for multi-domain Chinese word segmentation

pkuseg：一个多领域中文分词工具包 (English Version) pkuseg 是基于论文[Luo et. al, 2019]的工具包。其简单易用，支持细分领域分词，有效提升了分词准确度。目录主要亮点编译和安装各类分词工具包的性能对比使用方式论文引用作者常见问题及解答主要

6k Dec 29, 2022

In this project, we compared Spanish BERT and Multilingual BERT in the Sentiment Analysis task.

Applying BERT Fine Tuning to Sentiment Classification on Amazon Reviews Abstract Sentiment analysis has made great progress in recent years, due to th

5 Jan 03, 2022

Code for EMNLP20 paper: "ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training"

ProphetNet-X This repo provides the code for reproducing the experiments in ProphetNet. In the paper, we propose a new pre-trained language model call

394 Dec 17, 2022

NeurIPS'21: Probabilistic Margins for Instance Reweighting in Adversarial Training (Pytorch implementation).

source code for NeurIPS21 paper robabilistic Margins for Instance Reweighting in Adversarial Training

9 Dec 20, 2022

Collection of scripts to pinpoint obfuscated code

Obfuscation Detection (v1.0) Author: Tim Blazytko Automatically detect control-flow flattening and other state machines Description: Scripts and binar

230 Nov 26, 2022

pyMorfologik MorfologikpyMorfologik - Python binding for Morfologik.

Python binding for Morfologik Morfologik is Polish morphological analyzer. For more information see http://github.com/morfologik/morfologik-stemming/

18 Dec 29, 2021

Code for paper Multitask-Finetuning of Zero-shot Vision-Language Models

2 Jul 15, 2022

A music comments dataset, containing 39,051 comments for 27,384 songs.

Music Comments Dataset A music comments dataset, containing 39,051 comments for 27,384 songs. For academic research use only. Introduction This datase

2 Jan 10, 2022

Extracting Summary Knowledge Graphs from Long Documents

GraphSum This repo contains the data and code for the G2G model in the paper: Extracting Summary Knowledge Graphs from Long Documents. The other basel

10 Oct 21, 2022

Create a machine learning model which will predict if the mortgage will be approved or not based on 5 variables

Mortgage-Application-Analysis Create a machine learning model which will predict if the mortgage will be approved or not based on 5 variables: age, in

1 Jan 29, 2022

Visual Automata is a Python 3 library built as a wrapper for Caleb Evans' Automata library to add more visualization features.

55 Nov 17, 2022

This project uses unsupervised machine learning to identify correlations between daily inoculation rates in the USA and twitter sentiment in regards to COVID-19.

4 Oct 15, 2022

Cải thiện Elasticsearch trong bài toán semantic search sử dụng phương pháp Sentence Embeddings

Related tags

Overview

Cải thiện Elasticsearch trong bài toán semantic search sử dụng phương pháp Sentence Embeddings

Cài đặt:

Đánh index cho toàn bộ data

Search keyword và so sánh giữa BM25 (elasticsearch thường) và simCSE:

Kết quả tìm kiếm:

Contact:

Owner

Vo Van Phuc

State-of-the-art NLP through transformer models in a modular design and consistent APIs.

Open-source offline translation library written in Python. Uses OpenNMT for translations

This is a Prototype of an Ai ChatBot "Tea and Coffee Supplier" using python.

Sequence modeling benchmarks and temporal convolutional networks

NumPy String-Indexed is a NumPy extension that allows arrays to be indexed using descriptive string labels

pkuseg多领域中文分词工具; The pkuseg toolkit for multi-domain Chinese word segmentation

In this project, we compared Spanish BERT and Multilingual BERT in the Sentiment Analysis task.

Code for EMNLP20 paper: "ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training"

NeurIPS'21: Probabilistic Margins for Instance Reweighting in Adversarial Training (Pytorch implementation).

Collection of scripts to pinpoint obfuscated code

pyMorfologik MorfologikpyMorfologik - Python binding for Morfologik.

Code for paper Multitask-Finetuning of Zero-shot Vision-Language Models

A music comments dataset, containing 39,051 comments for 27,384 songs.

Extracting Summary Knowledge Graphs from Long Documents

Create a machine learning model which will predict if the mortgage will be approved or not based on 5 variables

Visual Automata is a Python 3 library built as a wrapper for Caleb Evans' Automata library to add more visualization features.

This project uses unsupervised machine learning to identify correlations between daily inoculation rates in the USA and twitter sentiment in regards to COVID-19.

Task-based datasets, preprocessing, and evaluation for sequence models.

LightSeq: A High-Performance Inference Library for Sequence Processing and Generation

A PyTorch-based model pruning toolkit for pre-trained language models