BIGDATA SIMULATION ONE PIECE WORLD CENSUS
=================
Solution Architecture
Description
ONE PIECE is a Japanese manga of great international success. The story turns inhabited in a fictional world, tells the adventures of a young man whose body gained rubber properties after accidentally eating a devil fruit (AKUMA NO MI). In this universe there are three types of AKUMA NO MI; Logia, Zoan and Paramecia. Each has a characteristic. The Logia type are elements that can modify the body, the Zoan are of the animal type (and can be extinct or not) and Paramecia are of the object type. These powers may or may not represent a danger to society, all that pose a danger to society are considered criminals and, varying the type of crime, are announced with a reward. The government always seeks to collect its taxes. So in this BigData project we explore the census of this population. Imagining a population of at least 100,000.00 inhabitants, we wrote a project that has MONGODB as its final repository, a non-relational database that organizes its documents by Collections. Below is a glossary of data.
Glossary of Data
| Fields | Type | Description |
|---|---|---|
| _id | string | undescore ID |
| region_birth | string | region of birth |
| country_birth | string | country of birth |
| city_birth | string | city_birth |
| current_region | string | current region |
| current_country | string | current country |
| current_city | string | current city |
| street | string | current street |
| number | string | number of house |
| postalcode | string | postal code |
| mailer | string | mailer |
| street | string | street name informed |
| number | string | number of street name informed |
| register_data | string | date your data was entered into the record |
| type_of_fruit | string | type of fruit |
| fruit_name | string | fruit name |
| fruit_category | string | fruit hazard level |
| number_times_resurrected | string | number of times that fruit was resurrected |
| job | string | occupation |
| current_job | string | current job |
| contracting_company | string | name of contracting company |
| start_date | string | start date in job company |
| year_working_time | string | time in year working in company |
| initial_salary | string | initial salary |
| current_wage | string | current wage |
| first_name | string | first name |
| last_name | string | last name |
| gender | string | gender |
| race | string | race of person |
| birthday | string | date of birthday |
| age | string | age |
| has_disability | string | have or do not have a disability |
| security_social_number | string | security social number |
| phone | string | phone |
| sketch | string | sketch |
| has_tatoo | string | have or do not have a tatoo |
| has_scar | string | have or do not have a scar |
| has_rewards | string | have or do not have a rewards |
| devil_fruit_user | string | whether or not you are an akuma no mi user |
| color_hair | string | color of hair |
| color_skill | string | color of skill |
| type_of_tatoo | string | type of tatoo |
| where_in_body | string | where in body is the tatoo |
| color_of_tatoo | string | color of tatoo |
| scar | string | where in body is the scar |
| color_eyes | string | color of eyes |
| main_crime | string | If the person is a criminal. main crime |
| code_crime | string | code of crime |
| tax_collected_government | string | tax collected by government |
| debt_with_government | string | debt with government |
| rewards | string | rewards |
Description
For a better view of the world of ONE PIECE, its regions, cities and islands, we put the map created for the world.
Map
Start the Project
To run the project, you need to install the dependencies located in the "dependencies" folder and in the root of the project, run the shell_script "run_script.sh".
Sample of Payload in Stagin
address
{
"_id":"2W1159879A",
"region_birth":"East Blue",
"country_birth":"Warship Island",
"city_birth":"North Wayne",
"current_region":"East Blue",
"current_country":"Warship Island",
"current_city":"East Joshua",
"street":"Christine Fields",
"number":"4104",
"postalcode":"04650",
"mailer":"m[email protected]",
"register_data":"20210423"
}
fruit
{
"_id":"3Y6898825C",
"type_of_fruit":"it does not have",
"fruit_name":"it does not have",
"fruit_category":"it does not have",
"number_times_resurrected":"0",
"register_data":"20210622"
}
job
{
"_id":"2W1159879A",
"job":"Freight forwarder",
"current_job":"YES",
"contracting_company":"Robinson, Simon and Hernandez",
"start_date":"1981/11/02",
"year_working_time":40,
"initial_salary":4904.0,
"current_wage":5345.36,
"register_data":"20210423"
}
persona
{
"_id":"7P1521176A",
"first_name":"Kristin",
"last_name":"Smith",
"gender":"F",
"race":"Minks",
"birthday":"1967-03-26",
"age":"54",
"devil_fruit_user":"it does not have",
"has_job":"has",
"has_tatoo":"it does not have",
"has_scar":"has",
"has_disability":"no deficiency",
"security_social_number":"575-40-5565",
"phone":"001-985-833-8626x33224",
"has_rewards":"has",
"sketch":"https://www.lorempixel.com/350/215",
"register_data":"20210816"
}
physical_characteristics
{
"_id":"1S6151128X",
"color_hair":"SeaShell",
"color_skill":"BLUISH",
"type_of_tatoo":"it does not have",
"where_in_body":"it does not have",
"color_of_tatoo":"it does not have",
"scar":"Left arm",
"color_eyes":"SeaShell",
"register_data":"20210828"
}
rewards
{
"_id":"2W1159879A",
"ssn_people":"165-53-1723",
"main_crime":"female violence",
"code_crime":13,
"tax_collected_government":37824.56,
"debt_with_government":31503.56,
"rewards":961679.94,
"register_data":"20210423"
}
Sample of Payload in Datalake
one_piece
collection not_fruit_user
> db.not_fruit_user.findOne()
{
"_id" : ObjectId("61a80938f9fae20940d6d7a9"),
"payload" : {
"personal_information" : {
"first_name" : "Kimberly",
"last_name" : "Thompson",
"gender" : "F",
"race" : "Dwarf",
"birthday" : "1996-11-11",
"age" : "25"
},
"physical_characteristics" : {
"has_disability" : "no deficiency",
"color_hair" : "Blue",
"color_skill" : "WHITE",
"scar" : "Back",
"color_eyes" : "Blue"
},
"social_characteristics" : {
"security_social_number" : "740-38-7150",
"phone" : "+1-705-306-4346x28383",
"sketch" : "https://dummyimage.com/716x261"
}
}
}
collection fruit_user
> db.fruit_user.findOne()
{
"_id" : ObjectId("61a8143e22cbec6d05f38f4e"),
"payload" : {
"personal_characteristics" : {
"first_name" : "Kenneth",
"last_name" : "Brady",
"gender" : "M",
"race" : "Skypiea",
"birthday" : "2000-05-28",
"age" : "21"
},
"fruit_characteristics" : {
"type_of_fruit" : "Logia",
"fruit_name" : "Bismuth\t Bismuth\t no Mi",
"fruit_category" : "Dangerous",
"number_times_resurrected" : "2"
},
"job_characteristics" : {
"job" : "Swordsman",
"current_job" : "YES",
"contracting_company" : "Williams, Wilson and Patterson",
"start_date" : "1954/09/01",
"year_working_time" : "67",
"initial_salary" : "4058.0",
"current_wage" : "4423.22"
},
"physical_characteristics" : {
"type_of_tatoo" : "it does not have",
"where_in_body" : "it does not have",
"color_of_tatoo" : "it does not have",
"color_eyes" : "Red",
"color_hair" : "Red",
"has_disability" : "no deficiency"
},
"social_characteristics" : {
"security_social_number" : "151-48-5282",
"phone" : "+1-842-853-5857",
"sketch" : "https://dummyimage.com/428x136"
},
"rewards_informations" : {
"main_crime" : "Tax evasion",
"code_crime" : "9",
"tax_collected_government" : 29491.37,
"debt_with_government" : "25393.37",
"rewards" : "968090.23"
}
}
}
2 Oct 14, 2021
3 Nov 16, 2021
1 Nov 16, 2021
20 Dec 22, 2022
1 Jan 21, 2022
3 Jul 02, 2022
1 Feb 18, 2022
16 Nov 04, 2022
5 Apr 26, 2022
1 Jan 19, 2022
81 Dec 26, 2022
70 Dec 14, 2022
1k Jan 05, 2023
581 Jan 07, 2023
3 Aug 24, 2022
2.2k Dec 28, 2022
2 Jul 01, 2022
3 Nov 03, 2022
759 Jan 07, 2023
1.6k Jan 07, 2023