31,870,464 non-embedding params, 38,598,913 embedding params, 70,469,377 total.
31,870,464
38,598,913
70,469,377
"n_embd": 768 "n_head": 6 "n_inner": 1920 "n_layer": 6 "n_positions": 4096
