Hitchhikers-guide - The Hitchhiker's Guide to Data Science for Social Good

Overview

Welcome to the Hitchhiker's Guide to Data Science for Social Good.

What is the Data Science for Social Good Fellowship?

The Data Science for Social Good Fellowship (DSSG) is a hands-on and project-based summer program that launched in 2013 at the University of Chicago and has now expanded to multiple locations globally. It brings data science fellows, typically graduate students, from across the world to work on machine learning, artificial intelligence, and data science projects that have a social impact. From a pool of typically over 800 applicants, 20-40 fellows are selected from diverse computational and quantitative disciplines including computer science, statistics, math, engineering, psychology, sociology, economics, and public policy.

The fellows work in small, cross-disciplinary teams on social good projects spanning education, health, energy, transportation, criminal justice, social services, economic development and international development in collaboration with global government agencies and non-profits. This work is done under close and hands-on mentorship from full-time, dedicated data science mentors as well as dedicated project managers, with industry experience. The result is highly trained fellows, improved data science capacity of the social good organization, and a high quality data science project that is ready for field trial and implementation at the end of the program.

In addition to hands-on project-based training, the summer program also consists of workshops, tutorials, and ethics discussion groups based on our data science for social good curriculum designed to train the fellows in doing practical data science and artificial intelligence for social impact.

Who is this guide for?

The primary audience for this guide is the set of fellows coming to DSSG but we want everything we create to be open and accessible to larger world. We hope this is useful to people beyond the summer fellows coming to DSSG.

If you are applying to the program or have been accepted as a fellow, check out the manual to see how you can prepare before arriving, what orientation and training will cover, and what to expect from the summer.

If you are interested in learning at home, check out the tutorials and teach-outs developed by our staff and fellows throughout the summer, and to suggest or contribute additional resources.

*Another one of our goals is to encourage collaborations. Anyone interested in doing this type of work, or starting a DSSG program, to build on what we've learned by using and contributing to these resources.

What is in this guide?

Our number one priority at DSSG is to train fellows to do data science for social good work. This curriculum includes many things you'd find in a data science course or bootcamp, but with an emphasis on solving problems with social impact, integrating data science with the social sciences, discussing ethical implications of the work, as well as privacy, and confidentiality issues.

We have spent many (sort of) early mornings waxing existential over Dunkin' Donuts while trying to define what makes a "data scientist for social good," that enigmatic breed combining one part data scientist, one part consultant, one part educator, and one part bleeding heart idealist. We've come to a rough working definition in the form of the skills and knowledge one would need, which we categorize as follows:

  • Programming, because you'll need to tell your computer what to do, usually by writing code.
  • Computer science, because you'll need to understand how your data is - and should be - structured, as well as the algorithms you use to analyze it.
  • Math and stats, because everything else in life is just applied math, and numerical results are meaningless without some measure of uncertainty.
  • Machine learning, because you'll want to build predictive or descriptive models that can learn, evolve, and improve over time.
  • Social science, because you'll need to know how to design experiments to validate your models in the field, and to understand when correlation can plausibly suggest causation, and sometimes even do causal inference.
  • Problem and Project Scoping, because you'll need to be able to go from a vague and fuzzy project description to a problem you can solve, understand the goals of the project, the interventions you are informing, the data you have and need, and the analysis that needs to be done.
  • Project management, to make progress as a team, to work effectively with your project partner, and work with a team to make that useful solution actually happen.
  • Privacy and security, because data is people and needs to be kept secure and confidential.
  • Ethics, fairness, bias, and transparency, because your work has the potential to be misused or have a negative impact on people's lives, so you have to consider the biases in your data and analyses, the ethical and fairness implications, and how to make your work interpretable and transparent to the users and to the people impacted by it.
  • Communications, because you'll need to be able to tell the story of why what you're doing matters and the methods you're using to a broad audience.
  • Social issues, because you're doing this work to help people, and you don't live or work in a vacuum, so you need to understand the context and history surrounding the people, places and issues you want to impact.

All material is licensed under CC-BY 4.0 License: CC BY 4.0

Table of Contents

The links below will help you find things quickly.

DSSG Manual

Summer Overview

This sections covers general information on projects, working with partners, presentations, orientation information, and the following schedules:

Conduct, Culture, and Communications

This section details the DSSG anti-harassment policy, goals of the fellowship, what we hope fellows get out of the experience, the expectations of the fellows, and the DSSG environment. A slideshow version of this can also be found here.

Curriculum

This section details the various topics we will be covering throughout the summer. This includes:

Wiki

In the wiki, you will find a bunch of helpful information and instructions that people have found helpful along the way. It covers topics like:

  • Accessing S3 from the command line
  • Creating an alias to make Python3 your default (rather than python2)
  • Installing RStudio on your EC2
  • Killing your query
  • Creating a custom jupyter setup
  • Mounting box from ubuntu
  • Pretty Print psql and less output
  • Remotely editing text files in your favorite text editor
  • SQL Server to Postgres
  • Using rpy2
  • VNC Viewer
Owner
Data Science for Social Good
Data Science for Social Good
One Ansible Module for using LINE notify API to send notification. It can be required in the collection list.

Ansible Collection - hazel_shen.line_notify Documentation for the collection. ansible-galaxy collection install hazel_shen.line_notify --ignore-certs

Hazel Shen 4 Jul 19, 2021
Buffer overflow example for python

Buffer overflow example for python

Mehmet 1 Jan 04, 2022
Graphene Metanode is a locally hosted node for one account and several trading pairs, which uses minimal RAM resources.

Graphene Metanode is a locally hosted node for one account and several trading pairs, which uses minimal RAM resources. It provides the necessary user stream data and order book data for trading in a

litepresence 5 May 08, 2022
A OBS service to package a published repository into a tar.gz file

OBS Source Service obs-service-publish_tar obs-service-publish_tar will create a archive.tar[.tar compression] archive containing the published repo

Erico Mendonca 1 Feb 16, 2022
Collection of system-wide scripts that I use on my Gentoo

linux-scripts Collection of scripts that I use on my Gentoo machine. I tend to put all scripts in /scripts directory. It is not likely that you would

Xoores 1 Jan 09, 2022
Writeup and scripts for the 2021 malwarebytes crackme

Malwarebytes Crackme 2021 Tools and environment setup We will be doing this analysis in a Windows 10 VM with the flare-vm tools installed. Most of the

Jerome Leow 9 Dec 02, 2022
Google Scholar App Using Python

Google Scholar App Watch the tutorial video How to build a Google Scholar App | Streamlit #30 Demo Launch the web app: Reproducing this web app To rec

Chanin Nantasenamat 4 Jun 05, 2022
Projeto para ajudar no aprendizado da linguagem Pyhon

Economize Este projeto tem o intuito de criar desáfios para a codificação em Python, fazendo com que haja um maior entendimento da linguagem em seu to

Lucas Cunha Rodrigues 1 Dec 16, 2021
A platform for developers 👩‍💻 who wants to share their programs and projects.

Fest-Practice-2021 This project is excluded from Hacktoberfest 2021. Please use this as a testing repo/project. A platform for developers 👩‍💻 who wa

Mayank Choudhary 40 Nov 07, 2022
List of short Codeforces problems with a statement of 1000 characters or less. Python script and data files included.

Shortest problems on Codeforces List of Codeforces problems with a short problem statement of 1000 characters or less. Sorted for each rating level. B

32 Dec 24, 2022
A Python script to delete movies with a certain tag after a certain amount of days.

radarr_autodelete Simple script, which deletes movies with a specific tag after a certain amount of days Pip Packages pip3 install pyarr python-dotenv

7 Dec 06, 2022
Set named timers for cooking, watering plants, brewing tea and more.

Timer Set named timers for cooking, watering plants, brewing tea and more. About Use Mycroft when your hands are messy or you need more that the one t

OpenVoiceOS 3 Nov 02, 2022
This repository containing cross-section cut and fill calculations using Python programming language.

cross-section This repository is containing cut and fill calculations for cross-section using Python programming language. This codes is made to calcu

3 Jun 15, 2022
Pseudometa's dotfiles

pseudometa's dotfiles Table of Contents Why this repository? How this Repository works Special Explanations Got an idea for an improvement? Contact Wh

pseudometa 23 Dec 27, 2022
A lightweight Python module to interact with the Mitre Att&ck Enterprise dataset.

enterpriseattack - Mitre's Enterprise Att&ck A lightweight Python module to interact with the Mitre Att&ck Enterprise dataset. Built to be used in pro

xakepnz 7 Jan 01, 2023
Scripts for hosting urbit in production-ish

Urbit Sysops Contains some helpful scripts for hosting Urbit. There are two variants included in this repo: one using docker, and one using plain syst

Jōshin 12 Sep 25, 2022
🍏 Make Thinc faster on macOS by calling into Apple's native Accelerate library

🍏 Make Thinc faster on macOS by calling into Apple's native Accelerate library

Explosion 81 Nov 26, 2022
Student Management System Built With Python

Student-Management-System Group Members 19BCE183 - Patel Sarthak 19BCE195 - Patel Jinil 19BCE220 - Rana Yash Project Description In our project Studen

Sarthak Patel 6 Oct 20, 2022
Buggy script to play with GPOs

GPOwned /!\ This is a buggy PoC I made just to play with GPOs in my lab. Don't use it in production! /!\ The script uses impacket and ldap3 to update

45 Dec 15, 2022
Test reproducibility of leiden/umap on different systems

Demonstrate that UMAP and Leiden analysis is not reproducible between different cpu architectures.

Gregor Sturm 2 Oct 16, 2021