- As it is a code that implements only the model structure, the training code is not provided. Therefore, in order for you to actually train the model, you have to train the model by writing the train code yourself.
- I wrote the variable names in detail to help you study.
- If you want to set the same as the library environment in which I developed this code, you can use
pip3 install -r requirements.txt - You can do a simple test by running the
ALBERT-testing.ipynborALBERT-testing.pycode.
ALBERT.pyTransformer.pyMultiHeadAttention.pyEmbeddings.pyALBERT-testing.ipynbALBERT-testing.pyrequirements.txt