bigdata_analyse 大数据分析项目

Overview

bigdata_analyse

大数据分析项目

wish

采用不同的技术栈,通过对不同行业的数据集进行分析,期望达到以下目标:

  • 了解不同领域的业务分析指标
  • 深化数据处理、数据分析、数据可视化能力
  • 增加大数据批处理、流处理的实践经验
  • 增加数据挖掘的实践经验

tip

  • 项目主要使用的编程语言是 python、sql、hql
  • .ipynb 可以用 jupyter notebook 打开,如何安装, 可以参考 jupyter notebook

jupyter notebook 是一种网页交互形式的 python 编辑器,直接通过 pip 安装,也支持 markdown,很适合用来做数据分析可视化以及写文章、写示例代码等。

list

主题 处理方式 技术栈 数据集下载
1 亿条淘宝用户行为数据分析 离线处理 清洗 hive + 分析 hive + 可视化 echarts 阿里云 或者 百度网盘 提取码:5ipq
1000 万条淘宝用户行为数据实时分析 实时处理 数据源 kafka + 实时分析 flink + 可视化(es + kibana) 百度网盘 提取码:m4mc
300 万条《野蛮时代》的玩家数据分析 离线处理 清洗 pandas + 分析 mysql + 可视化 pyecharts 百度网盘 提取码:paq4
130 万条深圳通刷卡数据分析 离线处理 清洗 pandas + 分析 impala + 可视化 dbeaver 百度网盘 提取码:t561
10 万条厦门招聘数据分析 离线处理 清洗 pandas + 分析 hive + 可视化 ( hue + pyecharts ) + 预测 sklearn 百度网盘 提取码:9wx0
7000 条租房数据分析 离线处理 清洗 pandas + 分析 sqlite + 可视化 matplotlib 百度网盘 提取码:9en3
6000 条倒闭企业数据分析 离线处理 清洗 pandas + 分析 pandas + 可视化 (jupyter notebook + pyecharts) 百度网盘 提取码:xvgm

refer

  1. https://tianchi.aliyun.com/dataset/
  2. https://opendata.sz.gov.cn/data/api/toApiDetails/29200_00403601
  3. https://www.kesci.com/home/dataset
In this tutorial, raster models of soil depth and soil water holding capacity for the United States will be sampled at random geographic coordinates within the state of Colorado.

Raster_Sampling_Demo (Resulting graph of this demo) Background Sampling values of a raster at specific geographic coordinates can be done with a numbe

2 Dec 13, 2022
Tkinter Izhikevich Neuron Model With Python

TKINTER IZHIKEVICH NEURON MODEL WITH PYTHON Hodgkin-Huxley Model It is a mathematical model for the generation and transmission of action potentials i

Rabia KOÇ 8 Jul 16, 2022
Average time per match by division

HW_02 Unzip matches.rar to access .json files for matches. Get an API key to access their data at: https://developer.riotgames.com/ Average time per m

11 Jan 07, 2022
Pipeline and Dataset helpers for complex algorithm evaluation.

tpcp - Tiny Pipelines for Complex Problems A generic way to build object-oriented datasets and algorithm pipelines and tools to evaluate them pip inst

Machine Learning and Data Analytics Lab FAU 3 Dec 07, 2022
Minimal working example of data acquisition with nidaqmx python API

Data Aquisition using NI-DAQmx python API Based on this project It is a minimal working example for data acquisition using the NI-DAQmx python API. It

Pablo 1 Nov 05, 2021
A meta plugin for processing timelapse data timepoint by timepoint in napari

napari-time-slicer A meta plugin for processing timelapse data timepoint by timepoint. It enables a list of napari plugins to process 2D+t or 3D+t dat

Robert Haase 2 Oct 13, 2022
DaCe is a parallel programming framework that takes code in Python/NumPy and other programming languages

aCe - Data-Centric Parallel Programming Decoupling domain science from performance optimization. DaCe is a parallel programming framework that takes c

SPCL 330 Dec 30, 2022
Galvanalyser is a system for automatically storing data generated by battery cycling machines in a database

Galvanalyser is a system for automatically storing data generated by battery cycling machines in a database, using a set of "harvesters", whose job it

Battery Intelligence Lab 20 Sep 28, 2022
Tokyo 2020 Paralympics, Analytics

Tokyo 2020 Paralympics, Analytics Thanks for checking out my app! It was built entirely using matplotlib and Tokyo 2020 Paralympics data. This applica

Petro Ivaniuk 1 Nov 18, 2021
A multi-platform GUI for bit-based analysis, processing, and visualization

A multi-platform GUI for bit-based analysis, processing, and visualization

Mahlet 529 Dec 19, 2022
Data science/Analysis Health Care Portfolio

Health-Care-DS-Projects Data Science/Analysis Health Care Portfolio Consists Of 3 Projects: Mexico Covid-19 project, analyze the patient medical histo

Mohamed Abd El-Mohsen 1 Feb 13, 2022
Catalogue data - A Python Scripts to prepare catalogue data

catalogue_data Scripts to prepare catalogue data. Setup Clone this repo. Install

BigScience Workshop 3 Mar 03, 2022
pyETT: Python library for Eleven VR Table Tennis data

pyETT: Python library for Eleven VR Table Tennis data Documentation Documentation for pyETT is located at https://pyett.readthedocs.io/. Installation

Tharsis Souza 5 Nov 19, 2022
A library to create multi-page Streamlit applications with ease.

A library to create multi-page Streamlit applications with ease.

Jackson Storm 107 Jan 04, 2023
AWS Glue ETL Code Samples

AWS Glue ETL Code Samples This repository has samples that demonstrate various aspects of the new AWS Glue service, as well as various AWS Glue utilit

AWS Samples 1.2k Jan 03, 2023
Learn machine learning the fun way, with Oracle and RedBull Racing

Red Bull Racing Analytics Hands-On Labs Introduction Are you interested in learning machine learning (ML)? How about doing this in the context of the

Oracle DevRel 55 Oct 24, 2022
A variant of LinUCB bandit algorithm with local differential privacy guarantee

Contents LDP LinUCB Description Model Architecture Dataset Environment Requirements Script Description Script and Sample Code Script Parameters Launch

Weiran Huang 4 Oct 25, 2022
Data processing with Pandas.

Processing-data-with-python This is a simple example showing how to use Pandas to create a dataframe and the processing data with python. The jupyter

1 Jan 23, 2022
Statistical package in Python based on Pandas

Pingouin is an open-source statistical package written in Python 3 and based mostly on Pandas and NumPy. Some of its main features are listed below. F

Raphael Vallat 1.2k Dec 31, 2022