ChatGLM

ChatGLM-130B is an AI programming assistant, "ChatGLM-130B.CodeGeexThe Wisdom Spectrum AI team behind "Wisdom Spectrum" has developed a 130 billion parameter open-source, bilingual conversational language model.

GLM-130B is an open-source and open-access bilingual (Chinese and English) bidirectional dense model with 130 billion parameters and a model architecture using the Generalized Language Model (GLM1). It is designed to support reasoning on models with hundreds of billions of parameters on a single A100 (40G * 8) or V100 (32G * 8) server. As of July 3, 2022, GLM-130B has been trained on 400 billion text identifiers (200 billion in Chinese and 200 billion in English) and has the following unique advantages:

• Bilingual: Both Chinese and English are supported.
• High Accuracy (English): Better than GPT-3 175B (+4.0%), OPT-175B (+5.5%), and BLOOM-176B (+13.0%) on LAMBADA and slightly better than GPT-3 175B (+0.9%) on MMLU.
• High accuracy (Chinese): significantly better than ERNIE TITAN 3.0 260B on 7 zero-sample CLUE dataset (+24.26%) and 5 zero-sample FewCLUE dataset (+12.75%).
• Fast Inference: Supports fast inference with SAT and FasterTransformer on a single A100 server (up to 2.5x speedup).
• Reproducibility: All results (more than 30 tasks) can be easily reproduced with our open source code and model parameters.
• Cross-platform: supports training and inference on NVIDIA, Hygon DCU, Ascend 910 and Sunway processors.