Skip to content
This repository has been archived by the owner on Oct 31, 2021. It is now read-only.

c0nn3r/pytorch_highway_networks

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

44 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

PyTorch Highway Networks

Highway networks implemented in PyTorch.

Highway Equation

Just the MNIST example from PyTorch hacked to work with Highway layers.

Todo

  • Make the Highway nn.Module reuseable and configurable.
  • Why does softmax work better than sigmoid? This shouldn't be the case...
  • Make training graphs on the MNIST dataset.
  • Add convolutional highway networks.
  • Add recurrent highway networks.
  • Experiment with convolutional highway networks for character embeddings.

Notes

  • ELU doesn't work better than RELU for the layer activation.
  • Softmax seems to work better than sigmoid for the gate function?!

Releases

No releases published

Packages

No packages published

Languages