Proyecto - Análisis de texto de eventos históricos

Overview

Acceder al código desde Google Colab para poder ver de manera adecuada todas las visualizaciones y poder interactuar con ellas.

Link de acceso: https://colab.research.google.com/drive/1XqDm6szrNG8ZdH37EVITPCSw7BDZFFQ5?usp=sharing

Corto video explicativo: https://youtu.be/ZDPXc56jOj4

Proyecto Big Data - Análisis de texto de eventos históricos

Declaración del conjunto de datos

Contamos con un dataset en formato JSON proveniente del repositorio 'awesome-json-datasets' en la sección 'Historical Events' sobre eventos históricos (disponible en: https://github.com/jdorfman/awesome-json-datasets). Este dataset cuenta con información desde el año 299 A.C. hasta el año 2013. Recopila sucesos importantes en el mundo a lo largo de este periodo señalado.

La estrucutra de cada recopilación es la siguiente:

{
    "date": "fecha del acontecimiento",
    "description": "descripción del evento en cuestión",
    "lang": "lenguaje de la descripción",
    "category1": "catergoría interna del dataset",
    "granularity": "granularidad"
}

Como se puede ver, no cuenta con una estructura compleja, y sus campos más importantes son 'date' que nos indica la fecha del suceso y 'description' donde se encuentran todos los detalles del evento. Este dataset cuenta con 20.330 registros diferentes.

Planteamiento de la problemática y diseño de la solución (tecnologías a implementar)

Se plantea realizar un análisis descriptivo de esta información a nivel de país, agrupando sus eventos históricos y ver qué palabras son recurrentes en estos eventos. Así nos podemos dar una rápida percepción de la historia de un país en concreto. También se plantea analizar palabras clave en los eventos históricos como lo son 'guerra', 'atentados', 'ataque', 'muertos', 'descubrimiento', 'invención' y ver que tan concurrentes son a lo largo de la historia.

Para esta labor, nos apoyaremos de la herramienta MongoDB en su entorno de Python Pymongo. Este sistema de base de datos NoSQL nos ayudará a manejar adecuadamente el formato de este dataset (JSON) y más importante aún con el tratamiento de textos. Para esto último nos apoyaremos en dos funcionalidades de MongoDB: En el uso de expresiones regulares para busqueda en campos de texto y en las operaciones Map-Reduce. Junto con MongoDB, nos apoyaremos en las librerías propias de analítica de datos de Python. Con esto se pretenderá alcanzar los objetivos de este proyecto.

Python 101 Forever

🚀 Python 101 Forever 🚀 Official Python 101 Forever GitHub repository. START HERE - CHECK README SUBSCRIBE FOR UPDATES HERE Sponsors Contac

Hack Bulgaria 58 Nov 30, 2022
Audio-analytics for music-producers! Automate tedious tasks such as musical scale detection, BPM rate classification and audio file conversion.

Click here to be re-directed to the Beat Inspect Streamlit Web-App You are a music producer? Let's get in touch via LinkedIn Fundamental Analytics for

Stefan Rummer 11 Dec 27, 2022
Markov Chain Composer

Markov Chain Composer Using Markov Chain to represent relationships between words in song lyrics and then generating new lyrics.. ahem interpretive po

Kylie 85 Dec 09, 2022
A web app for presenting my research in BEM(building energy model) simulation

BEM(building energy model)-SIM-APP The is a web app presenting my research in BEM(building energy model) calibration. You can play around with some pa

8 Sep 03, 2021
A Tandy Color Computer 1, 2, and 3 assembler written in Python

CoCo Assembler and File Utility Table of Contents What is it? Requirements License Installing Assembler Assembler Usage Input File Format Print Symbol

Craig Thomas 16 Nov 03, 2022
Pyjiting is a experimental Python-JIT compiler, which is the product of my undergraduate thesis

Pyjiting is a experimental Python-JIT compiler, which is the product of my undergraduate thesis. The goal is to implement a light-weight miniature general-purpose Python JIT compiler.

Lance.Moe 10 Apr 17, 2022
berisi kodingan kodingan python umum yang kubuat.

python-codevault berisi kodingan kodingan python umum yang kubuat. untuk memudahkan transisi dan menjadi refrensi tutorial. daily challange for myself

Agung Zon Blade 1 Dec 19, 2021
JD-backup is an advanced Python script, that will extract all links from a jDownloader 2 file list and export them to a text file.

JD-backup is an advanced Python script, that will extract all links from a jDownloader 2 file list and export them to a text file.

Kraken.snv 3 Jun 07, 2022
A step-by-step tutorial for how to work with some of the most basic features of Nav2 using a Jupyter Notebook in a warehouse environment to create a basic application.

This project has a step-by-step tutorial for how to work with some of the most basic features of Nav2 using a Jupyter Notebook in a warehouse environment to create a basic application.

Steve Macenski 49 Dec 22, 2022
Experiments with Tox plugin system

The project is an attempt to add to the tox some missing out of the box functionality. Basically it is just an extension for the tool that will be loa

Volodymyr Vitvitskyi 30 Nov 26, 2022
pyinsim is a InSim module for the Python programming language.

PYINSIM pyinsim is a InSim module for the Python programming language. It creates socket connection with LFS and provides many classes, functions and

2 May 12, 2022
A simple countdown timer in eazy code to show timer with python

Countdown_Timer The simple CLI countdown timer in eazy code to show timer How Work First you fill the input by int-- (Enter the time in Seconds:) for

Yasin Rezvani 3 Nov 15, 2022
Excel cell checker with python

excel-cell-checker Description This tool checks a given .xlsx file has the struc

Paul Aumann 1 Jan 04, 2022
Change your Windows background with this program safely & easily!

Background_Changer Table of Contents: About the Program Features Requirements Preview Credits Reach Me See Also About the Program: You can change your

Sina.f 0 Jul 14, 2022
A framework to create reusable Dash layout.

dash_component_template A framework to create reusable Dash layout.

The TolTEC Project 4 Aug 04, 2022
A python script to get your activity

activities A python script to get your activity Not complete Requirements Python (=3.7) Pip (for python = 3.7) Git Pip packages psutil asyncio aioht

StarNumber 3 Nov 07, 2021
Control your gtps with gtps-tools!

Note Please give credit to me! Do not try to sell this app, because this app is 100% open source! Do not try to reupload and rename the creator app! S

Jesen N 6 Feb 16, 2022
Simple tools for the Horse Reality webgame

Realtools (Web Tools for Horse Reality) These tools were made on request from a close friend of mine who plays this game. A live instance can be found

shay 0 Sep 06, 2022
Python Commodore BBS multi-client

python-cbm-bbs-petscii Python Commodore BBS multi-client This is intended for commodore 64, c128 and most commodore compatible machines (as the new Co

7 Sep 16, 2022
A few of my adventures with Devito.

Devito-playbox A few of my adventures with Devito. This repository contains a few notebooks and scripts that will lead me in the road of learning this

Átila Saraiva Quintela Soares 1 Feb 08, 2022