• Introduction
• Families In the Wild Database
• Publications
• Organization
• To Do
• License
• Getting Involved

Python tools for automatic kinship recognition in images and videos.Python tools for automatic kinship recognition in images and videos.

• This is the documentation of the visual kinship recognition toolbox and tools specific to the FIW dataset (i.e., FIW data development kit). If you want the Places-CNN models instead of the training data, please refer to the FIW-models.
• Demos for Recognizing Families In the Wild (RFIW) 5th Edition:
  • Track 1, Verification;
  • Track 2, Tri-Subject Verification;
  • Track 3, Search & Retrieval.

This repo is a work in progress and all contributions and requests are welcome.

In summary, the following items are available:

• Version and contact information, download links, along with a brief description of the different download options.

• Overview of the API, its purpose, contents, and premiere features.

• Image data details for FIW-Standard and RFIW-Challenge.

  1. Image list and annotations
  2. Submission format
  3. Evaluation routines

• Overview of the FIW data development kit.

• List of action items (tentative; open to requests and PRs)

This repo serves as the main set of tools for kinship recognition effort, including the FIW database. Besides, the next section is detailed description of database (i.e., data and label) structure.

Please contact Joseph Robinson robinson.jo@northeastern.edu for questions, comments, or bug reports.

Download data and learn more about it here https://web.northeastern.edu/smilelab/fiw/.

FIW can be obtained from two primary locations: the main dataset (i.e., raw data, experimental splits, and more) downloads page, along with task-specific data splits on codalab (i.e., Task 1, Task 2, and Task 3), which were at one time used for data challenge (i.e., 2020 RFIW in conjunction with the IEEE FG Conference). Oncce download, we suggest to decompress the files in the data to their own folder.

This documentation describes FIW DB and (working) development kit. This is work in progress (i.e., still to come are FIW-CNN models, updated benchmarks, more in README (this), and more).

Check out FIW project page

Download here

• FIW_PIDs.csv: Photo lookup table. Each row is an image instance, containing the following fields:

  • PID: Photo ID
  • Name: Surname.firstName (root reference for given family)
  • URL: Photo URL on web
  • Metadata: Text caption for photo

• FIW_FIDs.csv: FID (family)/ Surname lookup table.

  • FID: Unique ID key assigned to each family.
  • Surname: Family Name corresponding to FID key.

• FIW_RIDs.csv: Relationship lookup table with keys [1-9] assigned to relationship types.

• FIDs/

  • FID####/ Contains labels and cropped facial images for members of family (1-1000)
    • MID#/: Face images of family member with ID key , i.e., MID #.
    • F####.csv: File containing member information of each family:
      • relationships matrix: representation of relationships.
      • name: First name of family member.
      • gender: gender of family member.

For example:

FID0001.csv

	MID     1     2     3     Name    Gender
	 1      0     4     5     name1     F
	 2      1     0     1     name2     F
	 3      5     4     0     name3     M

Here we have 3 family members, as listed under the MID column (far-left). Each MID reads acorss its row.

We can see that MID1 is related to MID2 by 4->1 (Parent->Sibling), which of course can be viewed as the inverse, i.e., MID2->MID1 is 1->4. It can also be seen that MID1 and MID3 are Spouses of one another, i.e., 5->5. And so on, and so forth.

Papers on FIW describe the data collection processes and details; supplemental to this is the FIW Data Card below. Note that the Latex source file for the datasheet could be borrowed as a tempalate for another dataset of similar structure. Check out repo, as well as DatasheetForFiw/main.pdf.

@ARTICLE{robinsonSurvey2021,
  author={Robinson, Joseph Peter and Shao, Ming and Fu, Yun},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
  title={Survey on the Analysis and Modeling of Visual Kinship: A Decade in the Making},
  year={2021},
  pages={1-1},
  doi={10.1109/TPAMI.2021.3063078},
  }

@ARTICLE{robinsonfiwmm,
  author={Robinson, Joseph Peter and Yin, Yu and Khan, Zaid and Shao, Ming and Fu, Yun},
  journal={IEEE Transactions on Multimedia (TMM)},
  title={Families In Wild Multimedia (FIW MM): A Multimodal Database for Recognizing Kinship},
  year={2021},
  }

@article{robinson2020recognizing,
    title={Recognizing Families In the Wild (RFIW): The 4th Edition},
    author={Robinson, Joseph P and Yin, Yu and Khan, Zaid and Shao, Ming and Xia, Siyu and
            Stopa, Michael and Timoner, Samson and Turk, Matthew A and Chellappa, Rama and Fu, Yun},
    journal={arXiv preprint arXiv:2002.06303},
    year={2020}
}

@article{robinson2018visulkinship,
	title={Visual Kinship Recognition of Families in the Wild},
	author={Robinson, Joseph P and Shao, Ming and Wu, Yue and Liu, Hongfu and Gillis, Timothy and Fu, Yun},
	journal={IEEE Transactions on pattern analysis and machine intelligence (TPAMI) Special Issue: Computational Face},
	year={2020}
}

@article{robinson2016families,
  title="Visual Kinship Recognition of Families in the Wild",
  author="Robinson, Joseph P and Shao, Ming and Wu, Yue and Liu, Hongfu and Gillis, Timothy and Fu, Yun",
  journal="ACM on Multimedia Conference",
  year="2016"
}

A more complete list of references can be found here.

By downloading the image data you agree to the following terms:

1. You will use the data only for non-commercial research and educational purposes.
2. You will NOT distribute the above images.
3. Northeastern University makes no representations or warranties regarding the data, including but not limited to warranties of non-infringement or fitness for a particular purpose.
4. You accept full responsibility for your use of the data and shall defend and indemnify Northeastern University, including its employees, officers and agents, against any and all claims arising from your use of the data, including but not limited to your use of any copies of copyrighted images that you may create from the data.

See Download links (and Terms and Conditions) here.

0.1.0 Created: 16 January 2020

• Joseph Robinson - Github - web
• Zaid Khan - Github

Please bring up any questions, comments, bugs, PRs, etc.

• Finish documentation
• Demo for Track 1
• Demo for Track 2
• Demo for Track 3
• Dataloader
• Publish results (all baselines)