PLaMo community license to download PLaMo 2 8B

To download PLaMo 2 8B, you have to agree to our license. PLaMo 2 8B is released PLaMo community license. For commercial use, please contact us via this form.

PLaMo Community License Agreement

The PLaMo Community License Agreement outlines the terms and conditions for utilizing the large-scale language foundation model PLaMo and its derivatives provided by Preferred Networks, Inc., as well as any obligations the USER must observe. The USER’s access to the PLaMo model and its derivatives is subject to this AGREEMENT. By agreeing to or utilizing this MODEL, the USER shall be bound by the terms of this AGREEMENT.

Article 1 (Definitions)

(1) "AGREEMENT" refers to the PLaMo Community License Agreement.
(2) "PFN" refers to Preferred Networks, Inc.
(3) "MODEL" denotes the model weights, model code, tokenizer, training scripts, and other supplementary materials provided by PFN under the name "PLaMo" as separately specified.
(4) "USER" means an individual or legal entity utilizing the MODEL.
(5) "DERIVED MODEL" refers to models created through modification or use of the MODEL, including their model weights, code, and other associated materials.
(6) "OUTPUT" means the results produced by the MODEL or DERIVED MODEL.
(7) " MODEL MATERIALS" collectively refers to the MODEL, DERIVED MODEL, and OUTPUT.
(8) "LICENSE" means PFN's authorization to grant the USER permission to use the MODEL MATERIALS under this AGREEMENT.
(9) "COMMERCIAL PURPOSE" means any use beyond personal or academic use, including business applications or commercial purposes. Notably, COMMERCIAL PURPOSE also includes usage for developing, modifying, or providing products, services, or business operations (including through hosting services or APIs), and usage within the USER's organizational frameworks.

Article 2 (User Requirements)

The USER must be 18 years of age or older, or have reached the age of legal capacity in the USER’s country of residence, unless the USER's legal guardian or legal representative has consented to this AGREEMENT on the USER’s behalf.

Article 3 (This License)

(1) PFN grants the USER permission to use the MODEL MATERIALS under the terms and conditions and scope specified in this AGREEMENT, provided that the USER agrees to and comply with this AGREEMENT.
(2) This LICENSE is non-exclusive, worldwide, non-transferable, and royalty-free.
(3) The USER may utilize the MODEL MATERIALS for COMMERCIAL PURPOSE only if the USER satisfies all of the following conditions. If the USER fails to meet any of these conditions, the USER may no longer use the MODEL MATERIALS for COMMERCIAL PURPOSE and must obtain a new commercial license from PFN if the USER wishes to use the MODEL MATERIALS for COMMERCIAL PURPOSE.

(i) Prior registration on PFN's official registration page (https://forms.gle/mTL8tBLrMYXKNZD56).

(ii) The USER's or the USER’s affiliated company's annual revenue or sales must not exceed 1 billion yen (converted to the local currency of the USER's jurisdiction).

Article 4 (Redistribution and Attribution Requirements)

(1) When providing the MODEL MATERIALS to third parties, the USER must meet the following requirements:

(i) Provide a copy of this AGREEMENT and ensure the recipients comply with its terms.

(ii) Explicitly indicate "Built with PLaMo" in related websites, user interfaces, blog posts, product information pages, or product documentation.

(iii) Include "PLaMo" in the names of AI models created using the MODEL MATERIALS.

Article 5 (Usage of Outputs)

(1) The USER may publicly distribute the OUTPUT provided that the USER clearly indicates that it is the output of the MODEL or DERIVED MODEL.
(2) If models are trained using the OUTPUT, those models shall be treated as DERIVED MODEL subject to this AGREEMENT's terms, and may only be used, distributed, and commercialized under the same license conditions.

Article 6 (Additional Usage Terms)

The USER must not engage in any of the following activities regarding the use of the MODEL MATERIALS:
(1) Actions violating laws or public order and morals
(2) Criminal activities or any actions that threaten, participate in, or facilitate such activities
(3) Actions infringing upon the rights or interests of PFN or third parties
(4) Actions that damage the reputation or credibility of PFN or third parties
(5) Dissemination of information that could create the false impression that the OUTPUT represents PFN's official views, etc.
(6) Dissemination of false information
(7) Any other actions that PFN reasonably determines to be inappropriate

Article 7 (Warranty Disclaimer)

(1) The MODEL and the OUTPUT are provided "as is" without any warranties from PFN regarding accuracy, truthfulness, merchantability, quality, performance, fitness for specific purposes, non-infringement, or any other matter.
(2) The USER must not use the OUTPUT as the sole evidence, evaluation, or opinion regarding legal matters, medical diagnoses, financial decisions, or important personal assessments.
(3) The USER assumes all responsibility for the use of the MODEL MATERIALS and any consequences arising therefrom.

Article 8 (Limitation of Liability)

(1) Regardless of whether liability arises from contract, tort, product liability, or other legal grounds, PFN's maximum liability to the USER regarding this AGREEMENT and the MODEL MATERIALS is limited to direct and ordinary damages only (including loss of profits, special damages, indirect damages, and other damages, regardless of foreseeability). The maximum liability amount shall be 500 yen. This limitation does not apply if PFN is found to have acted with intent or gross negligence.
(2) Notwithstanding the preceding paragraph, if the USER utilizes the MODEL MATERIALS for business purposes, PFN shall assume no liability whatsoever, including any damages or other obligations, regarding this AGREEMENT and the MODEL MATERIALS.

Article 9 (User Responsibility)

(1) The USER shall comply with all applicable laws and regulations (including those related to import/export and trade) and this AGREEMENT regarding the acquisition and use of the MODEL MATERIALS.
(2) If the USER causes PFN any damages due to breach of this AGREEMENT or use of the MODEL MATERIALS, the USER must compensate PFN for those damages.
(3) If PFN receives any claims for damages or other demands from third parties due to USER's use of the MODEL MATERIALS, the USER shall indemnify PFN against such claims and ensure PFN is not harmed.

Article 10 (Ownership Rights)

(1) All rights to the MODEL belong to PFN or any third party licensed by PFN to license the MODEL.
(2) For DERIVED MODEL, rights pertaining to the portions modified by the USER belong to the USER, while all other rights belong to PFN.
(3) All rights to the OUTPUT belong to the USER.

Article 11 (Contract Term and Termination)

(1) This AGREEMENT shall be effective from the time the USER agrees to this AGREEMENT or access the MODEL MATERIALS and continue until termination.
(2) If the USER violates any terms of this AGREEMENT, PFN may immediately terminate this AGREEMENT, and the USER must delete all copies of the MODEL MATERIALS and cease usage immediately.

Article 12 (Contract Modification)

PFN may modify this AGREEMENT (including rules and regulations related to the MODEL MATERIALS; the same applies hereinafter). When modifying this AGREEMENT, PFN shall notify the USER of the modification details and the effective date of the change through the method specified by PFN by the effective date.

Article 13 (Governing Law and Jurisdiction)

(1) The laws of Japan shall govern this AGREEMENT.
(2) Any disputes arising from the MODEL MATERIALS or AGREEMENT shall be subject to the exclusive jurisdiction of the Tokyo District Court.

PLaMo 2 8B

Model Description

PLaMo 2 8B is a 8B model pre-trained on English and Japanese datasets, developed by Preferred Elements, Inc.

PLaMo 2 models adapt the hybrid architecture like Samba rather than the Transformer architecture. Samba integrates Mamba, a selective State Space Model (SSM), with sliding window attention, combining their strengths for improved efficiency and performance. The major differences between Samba and PLaMo 2 are 1) adding normalization layers to improve training stability, and 2) using Mamba2 kernel for computational efficiency.

PLaMo 2 8B is released under PLaMo community license. Please check the following license and agree to this before downloading.

NOTE: This model has NOT been instruction-tuned for chat dialog or other downstream tasks.

For commercial users

Please check the PLaMo community license and contact us via the following form to use commercial purpose.

(EN/JA) https://forms.gle/mTL8tBLrMYXKNZD56

Usage

Requirements

numpy>=1.26.4
numba>=0.60.0
torch<=2.5.1
transformers>=4.44.2,<=4.57.1
mamba_ssm>=2.2.2,<=2.2.4
causal_conv1d>=1.4.0

Use a pipeline as a high-level helper

import transformers
pipeline = transformers.pipeline("text-generation", model="pfnet/plamo-2-8b", trust_remote_code=True)
print(pipeline("The future of artificial intelligence technology is ", max_new_tokens=32))

Load model directly

from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("pfnet/plamo-2-8b", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("pfnet/plamo-2-8b", trust_remote_code=True)
text = "これからの人工知能技術は"
input_ids = tokenizer(text, return_tensors="pt").input_ids
generated_tokens = model.generate(
    inputs=input_ids,
    max_new_tokens=32,
    do_sample=True,
    top_k=50,
    top_p=0.95,
    temperature=1.0,
)[0]
generated_text = tokenizer.decode(generated_tokens)
print(generated_text)

Model Details

Model size: 8B
Trained tokens: 6T tokens
Developed by: Preferred Elements, Inc.
Model type: Causal decoder-only
Language(s): English, Japanese
License: PLaMo community license

Training Dataset

We trained PLaMo 2 8B in two phases, phase 1 with 5.25T tokens and phase 2 with 0.75T tokens. The percentage of datasets in each phase is shown in the following table.

	5.25T (phase 1)	0.75T (phase 2)	Tokens
English	45 %	35 %	2.625 T
Japanese	30 %	40 %	1.875 T
Coding	15 %	15 %	0.9 T
Other	10 %	10 %	0.6 T

Tokenizer

PLaMo 2 8B tokenizer is optimized by numba, which is JIT compiler for numerical functions. The tokenizer is trained on a subset of the datasets for model pre-training.

Tech Blog

Bias, Risks, and Limitations

PLaMo 2 8B is a new technology that carries risks with use. Testing conducted to date has been in English and Japanese, and has not covered, nor could it cover all scenarios. For these reasons, as with all LLMs, PLaMo 2 8B’s potential outputs cannot be predicted in advance, and the model may in some instances produce inaccurate, biased or other objectionable responses to user prompts. Therefore, before deploying any applications of PLaMo 2 8B, developers should perform safety testing and tuning tailored to their specific applications of the model.

Acknowledgement

This model is trained under the project, “Research and Development Project of the Enhanced Infrastructures for Post 5G Information and Communication System” (JPNP 20017), subsidized by the New Energy and Industrial Technology Development Organization (NEDO).

AI policies for Preferred Networks, Inc. group

Downloads last month: 122

Safetensors

Model size

9B params

Tensor type

F32

Model tree for pfnet/plamo-2-8b

Finetunes

7 models

Quantizations

4 models

Collection including pfnet/plamo-2-8b

PLaMo 2x

Collection

4 items • Updated Sep 17