Towards Understanding Quality Challenges of the Federated Learning: A First Look from the Lens of Robustness

Overview

FL Analysis

This repository contains the code and results for the paper "Towards Understanding Quality Challenges of the Federated Learning: A First Look from the Lens of Robustness" submitted to EMSE journal.

Replication

Main experiment

All experiments are done using python 3.8 and TensorFlow 2.4

Steps to run the experiments are as follows:

  1. The options for each configuration are set in JSON file which should be in the root directory by default. However, this can be changed using the environment variable CONFIG_PATH.

  2. The paths for the output and the processed ADNI dataset is set using the environment variables RESULTS_ROOT and ADNI_ROOT respectively. If these variables are not set the mentioned paths will use "./results" and "./adni" as default.

  3. Run the main program by python test.py

  • Note that the results will be overwritten if same config is run for multiple time. To avoid that RESULTS_ROOT can be changed at each run.

Config details

The config file can have the following options:

    "dataset": one of the following 
      "adni"
      "mnist"
      "cifar"
    "aggregator": one of the following 
      "fed-avg"
      "median"
      "trimmed-mean"
      "krum"
      "combine"
    "attack": one of the following
      "label-flip"
      "noise-data"
      "overlap-data"
      "delete-data"
      "unbalance-data"
      "random-update"
      "sign-flip"
      "backdoor"
    "attack-fraction": a float between 0 and 1
    "non-iid-deg": a float between 0 and 1
    "num-rounds": an integer value

Notes:

  1. attack field is optional. If it is not present, no attack will be applied and attack-fraction is not necessary.
  2. If dataset is set to adni, non-iid-deg field is not necessary
  3. The aggregator field is optional and if it is not present it will use the default fed-avg.
  4. All configurations used in our experiments are available in configs folder

ADNI dataset

ADNI dataset is not included in the repository due to user agreements, but information about it is available in www.adni-info.org.

Once the dataset is available, data can be processed with extract_central_axial_slices_adni.ipynb

Results Visualization

Results can be visualized using the visualizer.ipynb.

  • The root folder of the results should be set in the notebook before running.
  • Visualizations will be saved in the root folder under 0images folder.
  • The visualizer expects the root sub folders to be the results of the different runs.

An example:


_root
├── _run1
│   ├── cifar-0--fedavg--clean
│   └── cifar-0--krum--clean
├── _run2
│   ├── cifar-0--fedavg--clean
│   └── cifar-0--krum--clean
└── _run3
    ├── cifar-0--fedavg--clean
    └── cifar-0--krum--clean


Results

All results are available in the results folder (ADNI, CIFAR, Fashion MNIST, Ensemble). Each sub folder that represents a dataset contains the details of runs, plus processed visualizations and raw csv files in a folder called 0images.

PICARD - Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models

This is the official implementation of the following paper: Torsten Scholak, Nathan Schucher, Dzmitry Bahdanau. PICARD - Parsing Incrementally for Con

ElementAI 217 Jan 01, 2023
Semi-SDP Semi-supervised parser for semantic dependency parsing.

Semi-SDP Semi-supervised parser for semantic dependency parsing. This repo contains the code used for the semi-supervised semantic dependency parser i

12 Sep 17, 2021
Instance Semantic Segmentation List

Instance Semantic Segmentation List This repository contains lists of state-or-art instance semantic segmentation works. Papers and resources are list

bighead 87 Mar 06, 2022
Flower classification model that classifies flowers in 10 classes made using transfer learning (~85% accuracy).

flower-classification-inceptionV3 Flower classification model that classifies flowers in 10 classes. Training and validation are done using a pre-anot

Ivan R. Mršulja 1 Dec 12, 2021
FedTorch is an open-source Python package for distributed and federated training of machine learning models using PyTorch distributed API

FedTorch is a generic repository for benchmarking different federated and distributed learning algorithms using PyTorch Distributed API.

Machine Learning and Optimization Lab @PennState 136 Dec 23, 2022
The audio-video synchronization of MKV Container Format is exploited to achieve data hiding

The audio-video synchronization of MKV Container Format is exploited to achieve data hiding, where the hidden data can be utilized for various management purposes, including hyper-linking, annotation

Maxim Zaika 1 Nov 17, 2021
Code for "Reconstructing 3D Human Pose by Watching Humans in the Mirror", CVPR 2021 oral

Reconstructing 3D Human Pose by Watching Humans in the Mirror Qi Fang*, Qing Shuai*, Junting Dong, Hujun Bao, Xiaowei Zhou CVPR 2021 Oral The videos a

ZJU3DV 178 Dec 13, 2022
A cool little repl-based simulation written in Python

A cool little repl-based simulation written in Python planned to integrate machine-learning into itself to have AI battle to the death before your eye

Em 6 Sep 17, 2022
aka "Bayesian Methods for Hackers": An introduction to Bayesian methods + probabilistic programming with a computation/understanding-first, mathematics-second point of view. All in pure Python ;)

Bayesian Methods for Hackers Using Python and PyMC The Bayesian method is the natural approach to inference, yet it is hidden from readers behind chap

Cameron Davidson-Pilon 25.1k Jan 02, 2023
PyTorch implementation of paper "StarEnhancer: Learning Real-Time and Style-Aware Image Enhancement" (ICCV 2021 Oral)

StarEnhancer StarEnhancer: Learning Real-Time and Style-Aware Image Enhancement (ICCV 2021 Oral) Abstract: Image enhancement is a subjective process w

IDKiro 133 Dec 28, 2022
Tensorforce: a TensorFlow library for applied reinforcement learning

Tensorforce: a TensorFlow library for applied reinforcement learning Introduction Tensorforce is an open-source deep reinforcement learning framework,

Tensorforce 3.2k Jan 02, 2023
Graph Convolutional Networks for Temporal Action Localization (ICCV2019)

Graph Convolutional Networks for Temporal Action Localization This repo holds the codes and models for the PGCN framework presented on ICCV 2019 Graph

Runhao Zeng 318 Dec 06, 2022
Adaptive Dropblock Enhanced GenerativeAdversarial Networks for Hyperspectral Image Classification

This repo holds the codes of our paper: Adaptive Dropblock Enhanced GenerativeAdversarial Networks for Hyperspectral Image Classification, which is ac

Feng Gao 17 Dec 28, 2022
Python parser for DTED data.

DTED Parser This is a package written in pure python (with help from numpy) to parse and investigate Digital Terrain Elevation Data (DTED) files. This

Ben Bonenfant 12 Dec 18, 2022
Code for "LoFTR: Detector-Free Local Feature Matching with Transformers", CVPR 2021

LoFTR: Detector-Free Local Feature Matching with Transformers Project Page | Paper LoFTR: Detector-Free Local Feature Matching with Transformers Jiami

ZJU3DV 1.4k Jan 04, 2023
A project that uses optical flow and machine learning to detect aimhacking in video clips.

waldo-anticheat A project that aims to use optical flow and machine learning to visually detect cheating or hacking in video clips from fps games. Che

waldo.vision 542 Dec 03, 2022
Building a real-time environment using webcam frame division in OpenCV and classify cropped images using a fine-tuned vision transformers on hybryd datasets samples for facial emotion recognition.

Visual Transformer for Facial Emotion Recognition (FER) This project has the aim to build an efficient Visual Transformer for the Facial Emotion Recog

Mario Sessa 8 Dec 12, 2022
A pytorch implementation of Pytorch-Sketch-RNN

Pytorch-Sketch-RNN A pytorch implementation of https://arxiv.org/abs/1704.03477 In order to draw other things than cats, you will find more drawing da

Alexis David Jacq 172 Dec 12, 2022
MG-GCN: Scalable Multi-GPU GCN Training Framework

MG-GCN MG-GCN: multi-GPU GCN training framework. For more information, please read our paper. After cloning our repository, run git submodule update -

Translational Data Analytics (TDA) Lab @GaTech 6 Oct 24, 2022
Spatial-Location-Constraint-Prototype-Loss-for-Open-Set-Recognition

Spatial Location Constraint Prototype Loss for Open Set Recognition Official PyTorch implementation of "Spatial Location Constraint Prototype Loss for

Xia Ziheng 12 Jun 24, 2022