Towards Understanding Quality Challenges of the Federated Learning: A First Look from the Lens of Robustness

Overview

FL Analysis

This repository contains the code and results for the paper "Towards Understanding Quality Challenges of the Federated Learning: A First Look from the Lens of Robustness" submitted to EMSE journal.

Replication

Main experiment

All experiments are done using python 3.8 and TensorFlow 2.4

Steps to run the experiments are as follows:

  1. The options for each configuration are set in JSON file which should be in the root directory by default. However, this can be changed using the environment variable CONFIG_PATH.

  2. The paths for the output and the processed ADNI dataset is set using the environment variables RESULTS_ROOT and ADNI_ROOT respectively. If these variables are not set the mentioned paths will use "./results" and "./adni" as default.

  3. Run the main program by python test.py

  • Note that the results will be overwritten if same config is run for multiple time. To avoid that RESULTS_ROOT can be changed at each run.

Config details

The config file can have the following options:

    "dataset": one of the following 
      "adni"
      "mnist"
      "cifar"
    "aggregator": one of the following 
      "fed-avg"
      "median"
      "trimmed-mean"
      "krum"
      "combine"
    "attack": one of the following
      "label-flip"
      "noise-data"
      "overlap-data"
      "delete-data"
      "unbalance-data"
      "random-update"
      "sign-flip"
      "backdoor"
    "attack-fraction": a float between 0 and 1
    "non-iid-deg": a float between 0 and 1
    "num-rounds": an integer value

Notes:

  1. attack field is optional. If it is not present, no attack will be applied and attack-fraction is not necessary.
  2. If dataset is set to adni, non-iid-deg field is not necessary
  3. The aggregator field is optional and if it is not present it will use the default fed-avg.
  4. All configurations used in our experiments are available in configs folder

ADNI dataset

ADNI dataset is not included in the repository due to user agreements, but information about it is available in www.adni-info.org.

Once the dataset is available, data can be processed with extract_central_axial_slices_adni.ipynb

Results Visualization

Results can be visualized using the visualizer.ipynb.

  • The root folder of the results should be set in the notebook before running.
  • Visualizations will be saved in the root folder under 0images folder.
  • The visualizer expects the root sub folders to be the results of the different runs.

An example:


_root
├── _run1
│   ├── cifar-0--fedavg--clean
│   └── cifar-0--krum--clean
├── _run2
│   ├── cifar-0--fedavg--clean
│   └── cifar-0--krum--clean
└── _run3
    ├── cifar-0--fedavg--clean
    └── cifar-0--krum--clean


Results

All results are available in the results folder (ADNI, CIFAR, Fashion MNIST, Ensemble). Each sub folder that represents a dataset contains the details of runs, plus processed visualizations and raw csv files in a folder called 0images.

Generalized Random Forests

generalized random forests A pluggable package for forest-based statistical estimation and inference. GRF currently provides non-parametric methods fo

GRF Labs 781 Dec 25, 2022
Learning To Have An Ear For Face Super-Resolution

Learning To Have An Ear For Face Super-Resolution [Project Page] This repository contains demo code of our CVPR2020 paper. Training and evaluation on

50 Nov 16, 2022
Multi-task head pose estimation in-the-wild

Multi-task head pose estimation in-the-wild We provide C++ code in order to replicate the head-pose experiments in our paper https://ieeexplore.ieee.o

Roberto Valle 26 Oct 06, 2022
🔀 Visual Room Rearrangement

AI2-THOR Rearrangement Challenge Welcome to the 2021 AI2-THOR Rearrangement Challenge hosted at the CVPR'21 Embodied-AI Workshop. The goal of this cha

AI2 55 Dec 22, 2022
Classic Papers for Beginners and Impact Scope for Authors.

There have been billions of academic papers around the world. However, maybe only 0.0...01% among them are valuable or are worth reading. Since our limited life has never been forever, TopPaper provi

Qiulin Zhang 228 Dec 18, 2022
Code for CVPR2019 Towards Natural and Accurate Future Motion Prediction of Humans and Animals

Motion prediction with Hierarchical Motion Recurrent Network Introduction This work concerns motion prediction of articulate objects such as human, fi

Shuang Wu 85 Dec 11, 2022
CountDown to New Year and shoot fireworks

CountDown and Shoot Fireworks About App This is an small application make you re

5 Dec 31, 2022
A Streamlit demo demonstrating the Deep Dream technique. Adapted from the TensorFlow Deep Dream tutorial.

Streamlit Demo: Deep Dream A Streamlit demo demonstrating the Deep Dream technique. Adapted from the TensorFlow Deep Dream tutorial How to run this de

Streamlit 11 Dec 12, 2022
Generic U-Net Tensorflow implementation for image segmentation

Tensorflow Unet Warning This project is discontinued in favour of a Tensorflow 2 compatible reimplementation of this project found under https://githu

Joel Akeret 1.8k Dec 10, 2022
Transferable Unrestricted Attacks, which won 1st place in CVPR’21 Security AI Challenger: Unrestricted Adversarial Attacks on ImageNet.

Transferable Unrestricted Adversarial Examples This is the PyTorch implementation of the Arxiv paper: Towards Transferable Unrestricted Adversarial Ex

equation 16 Dec 29, 2022
From Fidelity to Perceptual Quality: A Semi-Supervised Approach for Low-Light Image Enhancement (CVPR'2020)

Under-exposure introduces a series of visual degradation, i.e. decreased visibility, intensive noise, and biased color, etc. To address these problems, we propose a novel semi-supervised learning app

Yang Wenhan 117 Jan 03, 2023
DSTC10 Track 2 - Knowledge-grounded Task-oriented Dialogue Modeling on Spoken Conversations

DSTC10 Track 2 - Knowledge-grounded Task-oriented Dialogue Modeling on Spoken Conversations This repository contains the data, scripts and baseline co

Alexa 51 Dec 17, 2022
BADet: Boundary-Aware 3D Object Detection from Point Clouds (Pattern Recognition 2022)

BADet: Boundary-Aware 3D Object Detection from Point Clouds (Pattern Recognition

Rui Qian 17 Dec 12, 2022
This is the repository of our article published on MDPI Entropy "Feature Selection for Recommender Systems with Quantum Computing".

Collaborative-driven Quantum Feature Selection This repository was developed by Riccardo Nembrini, PhD student at Politecnico di Milano. See the websi

Quantum Computing Lab @ Politecnico di Milano 10 Apr 21, 2022
Escaping the Gradient Vanishing: Periodic Alternatives of Softmax in Attention Mechanism

Period-alternatives-of-Softmax Experimental Demo for our paper 'Escaping the Gradient Vanishing: Periodic Alternatives of Softmax in Attention Mechani

slwang9353 0 Sep 06, 2021
GUI for a Vocal Remover that uses Deep Neural Networks.

GUI for a Vocal Remover that uses Deep Neural Networks.

4.4k Jan 07, 2023
Project page for our ICCV 2021 paper "The Way to my Heart is through Contrastive Learning"

The Way to my Heart is through Contrastive Learning: Remote Photoplethysmography from Unlabelled Video This is the official project page of our ICCV 2

36 Jan 06, 2023
Implementation of Transformer in Transformer, pixel level attention paired with patch level attention for image classification, in Pytorch

Transformer in Transformer Implementation of Transformer in Transformer, pixel level attention paired with patch level attention for image c

Phil Wang 272 Dec 23, 2022
The code used for the free [email protected] Webinar series on Reinforcement Learning in Finance

Reinforcement Learning in Finance [email protected] Webinar This repository provides the code f

Yves Hilpisch 62 Dec 22, 2022
This repository contains the code for EMNLP-2021 paper "Word-Level Coreference Resolution"

Word-Level Coreference Resolution This is a repository with the code to reproduce the experiments described in the paper of the same name, which was a

79 Dec 27, 2022