使用深度学习框架提取视频硬字幕；docker容器免安装深度学习库，使用本地api接口使得界面和后端识别分离；

Related tags

Deep Learningdockersubtittles
Overview

extract-video-subtittle

使用深度学习框架提取视频硬字幕；

本地识别无需联网；

CPU识别速度可观；

容器提供API接口；

运行环境

本项目运行环境非常好搭建，我做好了docker容器免安装各种深度学习包；

提供windows界面操作；

容器为CPU版本；

视频演示

https://www.bilibili.com/video/BV18Q4y1f774/

程序说明

1、先启动后端容器实例

docker run -d -p 6666:6666 m986883511/extract_subtitles

image-20210801214757813

2、启动程序

简单介绍页面

1：点击左边按钮连接第一步启动的容器；

2：视频提取字幕的总进度

3：当前视频帧显示的位置，就是视频进度条

4：识别出来的文字会在这里显示一下

image-20210801215010179

image-20210801215258761

3、点击选择视频确认字幕位置

点击选择视频按钮，这时你可以拖动进度条到有字幕的位置；然后点击选择字幕区域；在视频中画一个矩形；

image-20210801215258761

4、点击测试连接API

image-20210801220206554

后端没问题的话，会显示已连通；此时所有步骤准备就绪

5、开始识别

点击请先完成前几步按钮，内部分为这几个步骤

  1. 本地通过ffmpeg提取视频声音保存到temp目录（0%-10%）
  2. api通信将声音文件发送到容器内，容器内spleeter库提取声音中人声，结果保存在容器内temp目录，很耗时间，吃CPU和内存（10%-30）
  3. api通信，将人声根据停顿分片，返回分片结果，耗较短的时间（30%-40%）
  4. 根据说话分片时间开始识别字幕（40-%100%）

当100%的时候查看temp目录就生成了和视频同名的srt字幕文件

运行后台

后端接口容器地址Docker Hub

此过程可能时间较长，您需要预先安装好好docker，并配置好docker加速器，你可能需要先docker login

docker run -d -p 6666:6666 m986883511/extract_subtitles

本项目缺少文件

因网速墙的问题，大文件推送不上去，可以参考.gitignore中写的

其他

视频提取

# 视频片段提取
ffmpeg -ss 00:15:45 -t 00:02:15 -i test/three_body_3_7.mp4 -vcodec copy -acodec copy test/3body.mp4
# 打包界面程序
C:/Python/Python38-32/Scripts/pyinstaller.exe main.spec

参考资料

本项目中深度学习源代码为/docker/backend

原作者为：https://github.com/YaoFANGUK/video-subtitle-extractor

You might also like...
Comments
  • 提取人声一直没结果

    提取人声一直没结果

    image 视频是40多分钟的连续剧。CPU版本。之前用YaoFANGUK/video-subtitle-extractor提取字幕很成功也准确，但时间比较长。看到作者用音频分析减少了识别的帧数，所以试了一下。但在提取人声时，已经等待了近50分钟没有结果。而且CPU的占用只有1%左右，这明显不正常。用YaoFANGUK/video-subtitle-extractor整个的耗时可能都没有这么久。另外autosub也是提取音频来语音识别字幕，识别人声也很快，同样的视频几分钟就完了。麻烦作者看看是出了什么问题呢。

    opened by royzengyi 2
  • 项目咨询

    项目咨询

    Hello，我尝试了一下这个软件，感觉还是不错的，不过在实际使用中还是会有不少问题。

    我是一个独立开发者，这边愿意付费或者合作来完善一下，让这个项目更具实用性，不知道你有没有兴趣呢?

    没有找到联系方式，只好通过issue来试一下，你可以在看到之后删除，谢谢。

    我的邮箱是yedaxia#foxmail.com

    opened by YeDaxia 1
Releases(0.2.0)
Owner
歌者
失去人性，失去很多；失去兽性，失去一切；活着才能燃烧自己。
歌者
GitHub Repository
[CVPR2021 Oral] UP-DETR: Unsupervised Pre-training for Object Detection with Transformers

UP-DETR: Unsupervised Pre-training for Object Detection with Transformers This is the official PyTorch implementation and models for UP-DETR paper: @a

dddzg 430 Dec 23, 2022
Deep ViT Features as Dense Visual Descriptors

dino-vit-features [paper] [project page] Official implementation of the paper "Deep ViT Features as Dense Visual Descriptors". We demonstrate the effe

Shir Amir 113 Dec 24, 2022
SAS: Self-Augmentation Strategy for Language Model Pre-training

SAS: Self-Augmentation Strategy for Language Model Pre-training This repository

Alibaba 5 Nov 02, 2022
Person Re-identification

Person Re-identification Final project of Computer Vision Table of content Person Re-identification Table of content Students: Proposed method Dataset

Nguyễn Hoàng Quân 4 Jun 17, 2021
This is a virtual picture dragging application. Users may virtually slide photos across the screen. The distance between the index and middle fingers determines the movement. Smaller distances indicate click and motion, whereas bigger distances indicate only hand movement.

Virtual_Image_Dragger This is a virtual picture dragging application. Users may virtually slide photos across the screen. The distance between the ind

Randrita Sarkar 17 Dec 17, 2022
Polyp-PVT: Polyp Segmentation with Pyramid Vision Transformers (arXiv2021)

Polyp-PVT by Bo Dong, Wenhai Wang, Deng-Ping Fan, Jinpeng Li, Huazhu Fu, & Ling Shao. This repo is the official implementation of "Polyp-PVT: Polyp Se

Deng-Ping Fan 102 Jan 05, 2023
Implementation of the bachelor's thesis "Real-time stock predictions with deep learning and news scraping".

Real-time stock predictions with deep learning and news scraping This repository contains a partial implementation of my bachelor's thesis "Real-time

David Álvarez de la Torre 0 Feb 09, 2022
A rule learning algorithm for the deduction of syndrome definitions from time series data.

README This project provides a rule learning algorithm for the deduction of syndrome definitions from time series data. Large parts of the algorithm a

0 Sep 24, 2021
Implementation for the EMNLP 2021 paper "Interactive Machine Comprehension with Dynamic Knowledge Graphs".

Interactive Machine Comprehension with Dynamic Knowledge Graphs Implementation for the EMNLP 2021 paper. Dependencies apt-get -y update apt-get instal

Xingdi (Eric) Yuan 19 Aug 23, 2022
Simple-System-Convert--C--F - Simple System Convert With Python

Simple-System-Convert--C--F REQUIREMENTS Python version : 3 HOW TO USE Run the c

Jonathan Santos 2 Feb 16, 2022
Generative Art Using Neural Visual Grammars and Dual Encoders

Generative Art Using Neural Visual Grammars and Dual Encoders Arnheim 1 The original algorithm from the paper Generative Art Using Neural Visual Gramm

DeepMind 231 Jan 05, 2023
Graph Representation Learning via Graphical Mutual Information Maximization

GMI (Graphical Mutual Information) Graph Representation Learning via Graphical Mutual Information Maximization (Peng Z, Huang W, Luo M, et al., WWW 20

93 Dec 29, 2022
Prototypical Networks for Few shot Learning in PyTorch

Prototypical Networks for Few shot Learning in PyTorch Simple alternative Implementation of Prototypical Networks for Few Shot Learning (paper, code)

Orobix 835 Jan 08, 2023
A fast, dataset-agnostic, deep visual search engine for digital art history

imgs.ai imgs.ai is a fast, dataset-agnostic, deep visual search engine for digital art history based on neural network embeddings. It utilizes modern

Fabian Offert 5 Dec 14, 2022
MAUS: A Dataset for Mental Workload Assessment Using Wearable Sensor - Baseline system

MAUS: A Dataset for Mental Workload Assessment Using Wearable Sensor - Baseline system Getting started To start working on this assignment, you should

2 Aug 06, 2022
Official PyTorch implementation of the paper "Graph-based Generative Face Anonymisation with Pose Preservation" in ICIAP 2021

Contents AnonyGAN Installation Dataset Preparation Generating Images Using Pretrained Model Train and Test New Models Evaluation Acknowledgments Citat

Nicola Dall'Asen 10 May 24, 2022
Project NII pytorch scripts

project-NII-pytorch-scripts By Xin Wang, National Institute of Informatics, since 2021 I am a new pytorch user. If you have any suggestions or questio

Yamagishi and Echizen Laboratories, National Institute of Informatics 184 Dec 23, 2022
Physics-Informed Neural Networks (PINN) and Deep BSDE Solvers of Differential Equations for Scientific Machine Learning (SciML) accelerated simulation

NeuralPDE NeuralPDE.jl is a solver package which consists of neural network solvers for partial differential equations using scientific machine learni

SciML Open Source Scientific Machine Learning 680 Jan 02, 2023
Pip-package for trajectory benchmarking from "Be your own Benchmark: No-Reference Trajectory Metric on Registered Point Clouds", ECMR'21

Map Metrics for Trajectory Quality Map metrics toolkit provides a set of metrics to quantitatively evaluate trajectory quality via estimating consiste

Mobile Robotics Lab. at Skoltech 31 Oct 28, 2022
Histocartography is a framework bringing together AI and Digital Pathology

Documentation | Paper Welcome to the histocartography repository! histocartography is a python-based library designed to facilitate the development of

155 Nov 23, 2022