lccc

Tài liệu tham khảo:

lớn

Sử dụng lệnh sau để tải tập dữ liệu này trong TFDS:

ds = tfds.load('huggingface:lccc/large')

Sự miêu tả :

LCCC: Large-scale Cleaned Chinese Conversation corpus (LCCC) is a large corpus of Chinese conversations.
A rigorous data cleaning pipeline is designed to ensure the quality of the corpus.
This pipeline involves a set of rules and several classifier-based filters.
Noises such as offensive or sensitive words, special symbols, emojis,
grammatically incorrect sentences, and incoherent conversations are filtered.

Giấy phép : MIT
Phiên bản : 1.0.0
Chia tách :

Tách ra	Ví dụ
`'train'`	12007759

Đặc trưng :

{
    "dialog": [
        {
            "dtype": "string",
            "id": null,
            "_type": "Value"
        }
    ]
}

căn cứ

Sử dụng lệnh sau để tải tập dữ liệu này trong TFDS:

ds = tfds.load('huggingface:lccc/base')

Sự miêu tả :

LCCC: Large-scale Cleaned Chinese Conversation corpus (LCCC) is a large corpus of Chinese conversations.
A rigorous data cleaning pipeline is designed to ensure the quality of the corpus.
This pipeline involves a set of rules and several classifier-based filters.
Noises such as offensive or sensitive words, special symbols, emojis,
grammatically incorrect sentences, and incoherent conversations are filtered.

Giấy phép : MIT
Phiên bản : 1.0.0
Chia tách :

Tách ra	Ví dụ
`'test'`	10000
`'train'`	6820506
`'validation'`	20000

Đặc trưng :

{
    "dialog": [
        {
            "dtype": "string",
            "id": null,
            "_type": "Value"
        }
    ]
}